Lab assignment

Lab Assignment SW542
Interpretation

For this section, we still use the data from Experimental Evaluation of a Youth Dating Violence Prevention Program in New York City. Researchers randomly assigned a school-based intervention to 6th and 7th grade classes (over 2,500 students) in 30 public middle schools in New York City to one of four conditions: (1) a classroom-based intervention; (2) a school-wide intervention; (3) interventions that included both classroom and school-wide components; or (4) a (no treatment) control group. Pencil and paper surveys were distributed to students at three different times: (1) immediately before the assignment to one of the four study conditions, (2) immediately after the treatment (or control condition) was completed, and (3) between five and six months after assignment to one of the four study conditions. There are two SPSS outputs, please interpret the outputs and answer the questions.

According to the output “what is your race”:
1. In the school-wide intervention group (or building only group), how many students are African American? What is the percentage of African American students in the building only group?

2. In classroom only group, which ethnicity has the most participants? How many of them?

3. Among four groups, which group has the most White students? What is the percentage of White students in that group?

4. According to the output “outcomes by treatment groups”, I have completed the information for building only group, please fill in the means and standard deviations for other groups in the table. (T1=pre-test, T2=immediately after treatment, T3=5-6 months after treatment)

Treatment groups
Means (Standard Deviation)

Knowledge T1
Knowledge T2
Knowledge T3
Attitude T1
Attitude T2
Attitude T3
Building only
3.83 (1.12)
3.83 (1.15)
3.87(1.01)
3.20(0.52)
3.28(0.53)
3.23 (0.62)
Classroom only

Both

Control

Outcomes by treatment groups
TREATMENT
Knowledge pre-test
Knowledge immediately after treatment
Knowledge 5-6 months after treatment
Attitude pre-test
Attitude immediately after treatment
Attitude 5-6 months after treatment
Building only
N
Valid
30
30
30
30
30
30
Missing
0
0
0
0
0
0
Mean
3.83
3.83
3.87
3.2000
3.2750
3.2250
Std. Deviation
1.117
1.147
1.008
.51862
.53478
.62059
Classroom only
N
Valid
30
30
30
30
30
30
Missing
0
0
0
0
0
0
Mean
3.90
4.07
4.33
3.2333
3.3667
3.3833
Std. Deviation
.845
.868
.802
.50827
.58991
.54825
Both school & classroom
N
Valid
30
30
30
30
30
30
Missing
0
0
0
0
0
0
Mean
4.03
4.50
4.53
3.2667
3.2083
3.2750
Std. Deviation
.999
.777
.571
.65631
.54569
.51003
Neither – control
N
Valid
30
30
30
30
30
30
Missing
0
0
0
0
0
0
Mean
4.20
4.27
4.07
3.2750
3.3750
3.1583
Std. Deviation
.714
.907
.907
.45177
.43426
.47563

This Vignette will help you answer Question #2.

Laketha works as a head correctional counselor in a residential treatment center for delinquent youth ages 12 to 17. All the children at the treatment center eat, attend programming and group counseling together, and go to school together, but sleep in four different dorms and have free time in their respective dorms <one boy dorm age 15 and under, one girl dorm age 15 and under, one boy dorm over age 15, and one girl dorm over age 15 >.

Laketha’s boss (the director of the facility) receives a grant to study a new psycho-educational treatment program to help improve the behavioral and emotional problems in the lives of at-risk youth.

Because each of the four dorms has a different therapist, Laketha has to train each of them on the new psycho-educational program. She runs out of time though, and so she trains the first two therapists in person, and the last two she just has read the training manuals on the psycho-educational program she is implementing.

Laketha strategically selects two dorms to receive the psycho-educational program, which will be taught by the respective therapist for those dorms. She chooses the two therapists that only read the manuals for their training, assuming the extra practice in instructing the program will make up for the training on the program they did not receive. It turns out that those chosen are the two counselors for the boy dorms. The other two dorms (girl dorms) will receive the regular residential treatment without the additional program (control group).

To test the effectiveness of her new program, Laketha first has each of the four therapists rate the overall emotional health of each of the youth in their dorms in a survey about each youth. After, the program is taught for three months. At the end of the three-month treatment, 55% of the children (boys dorms) who started the psycho-educational program also finished the treatment. Of the boys who did not complete the program, many were asked to leave because they were doing well and it was felt the program was a waste of time for them. Another group of boys dropped out in protest after the favorite therapist in the program was fired halfway through! Laketha stepped in and finished the program for the counselor who was fired.

To examine change, Laketha has the therapists who administered the program rate the emotional and behavioral wellbeing of the boys who completed the program. (She completes the post-test scales for the therapist who got fired). She also has the therapists from the “control” dorms (girls’ therapists) rate a second time the emotional and behavioral wellbeing of the girls who reside in their dorms. Laketha finds no statistical difference in the emotional and behavioral wellbeing of the experimental and control group based on the therapist ratings. Laketha concludes that her program was an ineffective waste of time and money!!!

A. First, Laketha’s boss has no idea about what experimental design is. Laketha explains to her boss that an experimental design is the best method in this evaluation because it’s the “gold standard.” Explain why Laketha says this.
B. How does a quasi-experimental design differ from an experimental design? What is an example of a type of quasi-experimental design? Is Laketha’s design above a true experimental design or a quasi-experimental?
C. Discuss threats to internal validity. Then discuss at least 5 examples of concerns with threats to internal validity in experimental design that may be accounting for Laketha’s findings in this study. (What are some of the methodological flaws) Explain why they are a threat in this specific example.
D. What type of evaluation is Laketha completing? If she were to complete a process evaluation instead, how would it be different, or what are a couple things she could look at.

Question #3

A national organization just awarded a local agency (Teens with a Future, Inc.) 1 million dollars to implement a program for high school students who use substances. The Foundation officer contacted you to evaluate this new program. The program consists of providing teens in a local high school with substance disorders 16 weeks of individual and group counseling completed by a psychotherapist around their substance use. The organization wants to see if after those 16 weeks of treatment, teens with substance disorders are “better”

1. Design an evaluation using a true experimental design for this scenario.
2. Conceptualize what “better” will look like.
3. Explain how you will sample your groups and be sure to discuss steps to ensure each of the three aspects of true experimental designs
4. Draw out a diagram of how your experimental design will look (see pg. 386 of the Grinnell text for an exemplar)
5. Discuss the statistical analysis (very briefly that you could use to assess change for your two groups, which will depend on how you conceptualize “better”

Writing the Results
Types
You Should Report
Examples
Correlation
– correlation coefficient
– p value
– direction and strength of the relationship

To examine the relationship between job satisfaction and opportunities for promotion, the Pearson product-moment correlation coefficient was utilized. The results show a significant positive relationship (r = .21, p < .05), indicating that social workers who have greater opportunities for promotion tend to experience higher levels of job satisfaction than those who have fewer opportunities.
T-tests
– t value
– degrees of freedom (df)
– p value
The independence t-test examined levels of burnout on gender. The results show a significant difference between male and female social workers (t(df=216) = 2.59; p <.05). In this study, male social workers reported significantly higher levels of burnout (X̄ = 21.08). The mean difference was 2.95

Chi-Square
– test statistics (X2)
– degree of freedom
– p value
A chi-square test of association was run to examine the association between levels of education and levels of depression. The results show a significant association between the two variables (X2(df=1) = 6.11; p<.05). In other words, these results indicate that levels of education and levels of depression are related in the population.

ANOVA

The one-way ANOVA was conducted to examine the posttest depression scores among all groups. The results of the one-way ANOVA show an overall significant difference between the posttest depression scores among the groups (F(df=3,46) = 56.86, p < .05)

Because overall significant results were found, the Bonferroni post hoc test was run to determine which groups were significantly different.

The results show that the first experiment group (X̄ = 13.60) and the second experiment group (X̄ = 13.40) have significantly lower levels of depression than the first control group (X̄ = 22.20) and the second control group (X̄ = 22.40). These results show no significant differences between the two experiment groups or between the two control groups.

The results of the on-way ANOVA and the Bonferroni post hoc tests indicate that the levels of depression of subjects who participated in therapy are significantly lower than the levels of depression of subjects who did not participate in therapy.

These results provide statistical evidence that therapy is effective in reducing the levels of depression of clinically depressed subjects.

Regression
– number of factors that best predict the criterion
– proportion of variance each contributes to the criterion
– each partial correlation coefficient
– level of significance
A stepwise multiple regression analysis was conducted to estimate a model that best predicts levels of job satisfaction among social workers. The results of the stepwise analysis revealed that four of the six factors emerged as significant predictors of job satisfaction (F = 13.50; p <. 05). With a beta of.26 (p<.05), qualify of supervision emerged as the strongest predictor of job satisfaction, accounting for 10.5% of the variance in job satisfaction. The second strongest factor was region of employment (β = .23; p < .05) accounting for an additional 5.5% of the variance in job satisfaction.

These results indicate that higher job satisfaction is a function of better quality of supervision, being employed in the south region, higher levels of education, and better collegial relationships. Overall the model explains almost 21% of the variance in job satisfaction (R =.45). On the other hand, almost 80% of the variance in job satisfaction is unaccounted for by this model.

Source: Abu-Bader S.H. (2006). Using Statistical Methods in Social Work Practice: A Complete SPSS Guide.Lyceum Books, Inc.