School-Based Violence Prevention Programs: Systematic Review of Secondary Prevention Trials

Objective  To quantify the effectiveness of school-based violence prevention programsfor children identified as at risk for aggressive behavior.

Design  Systematic review and meta-analysis of randomized controlled trials.Electronic databases and bibliographies were systematically searched and authorsand organizations were contacted to identify randomized controlled trials.Standardized, weighted mean effect sizes were assessed by meta-analysis.

Setting  Elementary, middle, and high schools.

Participants  Children at risk for aggressive behavior.

Main Outcome Measures  Violent injuries, observed or reported aggressive or violent behaviors,and school or agency responses to aggressive behaviors.

Results  Of the 44 trials identified, none reported data on violent injuries.For the 28 trials that assessed aggressive behaviors, the pooled differencebetween study groups was −0.36 (95% confidence interval, −0.54to −0.19) in favor of a reduction in aggression with intervention. Forthe 9 trials that reported data on school or agency responses to aggression,the pooled difference was −0.59 (95% confidence interval, −1.18to 0.01). Subgroup analyses suggested greater effectiveness in older studentsand when administered to mixed-sex groups rather than to boys alone.

Conclusions  School-based violence prevention programs may produce reductions inaggressive and violent behaviors in children who already exhibit such behavior.These results, however, need to be confirmed in large, high-quality trials.

IN 1998 in the United States, 43 of every 1000 children were victims of nonfatal violent crime while at school or on their way to and from school.¹ More than 250 000 serious violent crimes (1% of all schoolchildren), including rape, sexual assault, and aggravated assault, were committed against students at school or while going to or from school. Teachers are also victims of school violence, with 31 per 1000 teachers reporting violent crime victimization in 1998. Youth violence in the school has become an increasing concern in the United States and other nations.^2,3

Many schools have implemented prevention programs that attempt to address this problem. According to a US Surgeon General's report, "Hundreds of youth violence prevention programs are being used in schools and communities throughout the country, yet little is known about the actual effects of many of them."^2,4 Although previous reviews have attempted to identify model programs and best practices,^2,4 informed decision making by policymakers and school professionals about violence prevention programs requires ready access to information about the effectiveness of such programs based on systematic and comprehensive review and synthesis of all available literature. The most reliable evidence for effectiveness comes from randomized controlled trials.⁵ We therefore conducted a systematic review and meta-analysis of such trials to explore and quantify the effect of school-based violence prevention programs on aggressive and violent behaviors in children at high risk for violent behavior.

Studies were included if (1) participants were randomly assigned to intervention and control groups; (2) outcome data were collected concurrently in the 2 groups; (3) the study population was composed of children in grades kindergarten (K) through 12 (or their international equivalent) identified by author-defined criteria as exhibiting or at risk for aggressive behavior; (4) the experimental intervention was designed, either wholly or largely, to reduce aggression and violence; (5) the intervention was primarily school based, although it could contain additional components; and (6) outcome measures included aggressive behavior, school and agency responses to acts of aggression, or violent injuries. We defined outcome measures as follows. Aggressive behavior was defined as scores on standardized tests that assess aggressive behavior (eg, Achenbach Child Behavior Checklist, Miller School Behavior Checklist) or actual counts of aggressive behaviors, such as fights or bullying (eg, via classroom observation and videotapes). School or agency actions were defined as any school or agency actions, such as detention, suspension, or court contact, recorded in official records that were taken in response to aggressive behaviors (eg, fighting and bullying). When school or agency records did not differentiate between responses to aggressive behaviors and responses to nonaggressive misbehaviors, such as truancy, all types of misbehaviors were included. The last outcome measure was violent injuries (eg, emergency department attendances).

For studies with multiple outcome measures, 1 aggressive behavior and 1 school or agency action outcome were chosen on the basis of a predefined hierarchy of factors (in order, data availability, measure specificity, quality assurance of measure, outcome assessor, data completeness and validation of the measure) and random choice if none of these applied (details available from the authors). We did not assess outcomes indirectly related to violence, such as school achievement, knowledge about or attitudes toward violence, mental health outcomes, and measures of aggressive responses to artificial stimuli or experimental tasks. We excluded cluster randomized trials with only 2 randomized schools or classes in which confounding factors cannot be effectively eliminated by randomization.

To identify relevant trials, electronic databases were searched using content terms such as aggress*, violen*, and fight*, with terms such as school*, educat*, and student*. The results were combined with the Cochrane Collaboration's optimally sensitive search strategy to identify controlled trials, adapted as required for each database (full search strategy available from the authors). We searched the Cochrane Controlled Trials Register (1998, issue 1), MEDLINE (1994–June 1998), EMBASE (1980–January 1998), PsycLIT (1887–March 1998), ERIC (Educational Resource Information Centre) (1970–September 1997), CINAHL (Cumulative Index to Nursing and Allied Health Literature) (1982–April 1998), Dissertation Abstracts (1861–March 1998), IBSS (International Bibliography of Social Sciences) (1952-1998), and NCJRS (National Criminal Justice Reference Service) (1970–May 1999) and the bibliographies of published reviews^6-8 and relevant trials. Aggression and Violent Behavior (Issue 1, 1996–Issue 3, 1998) were hand searched. We contacted relevant international organizations and experts and attempted to contact the authors of relevant studies to identify unpublished and internal reports. No language or date restrictions were applied.

Titles, abstracts, and keywords of identified records were screened to exclude ineligible trials (if specified in sufficient detail). Full texts of remaining reports were reviewed and additional ineligible trials excluded. Authors were contacted for clarification where necessary.

From eligible reports, 2 of us (J.A.M. and C.D.) independently extracted detailed data on study participants, interventions, outcomes, follow-up, results, methods of group assignment and allocation concealment, blinding of outcomes assessment, and loss to follow-up. A third author (D.A.G.) also independently extracted data on participants, interventions, and outcomes. Differences were resolved by discussion. We attempted to contact all authors of eligible trials to confirm study details, obtain missing data, and identify relevant unpublished outcomes.

We compared results of any intervention to no intervention (ie, control or placebo group) immediately after intervention and at the 12-month follow-up in the subsample for which these data were collected. We also assessed the effect of different types of interventions, grouping them according to the predominant training focus: (1) skills of nonresponse, either managed (eg, conflict resolution) or not (eg, anger control); or (2) relationship skills and other interventions of social context (eg, family or social relationships, peer mediation).

Study-specific differences between intervention and control groups for each of these comparisons were pooled using meta-analysis⁹ (RevMan 4.1; The Nordic Cochrane Centre, Copenhagen, Denmark) to produce an overall estimate of effect. Pooled results are expressed as standardized mean differences (with 95% confidence intervals [CIs]). In trials with multiple intervention or control groups, weighted, pooled means and SDs were used in the meta-analysis to avoid statistical problems with nonindependence of data that would result from including multiple intervention groups as separate trials. Studies comparing different intervention groups or different intensities of the same intervention, with no placebo or control group, were excluded from the meta-analysis but are described in Table 1.^10-13

Trial heterogeneity was explored with a χ² test using a significance level of .05.⁹ If there was statistical evidence of heterogeneity, a random-effects model was used. We used formal statistical testing (using Stata statistical software, version 6; Stata Corp, College Station, Tex) and funnel plot analysis (in which study size is plotted against intervention effect)¹⁴ to examine effects of study size and bias. In funnel plots, results from small studies tend to scatter widely at the bottom, with the spread narrowing among larger studies, so that the plot should represent a symmetrical inverted funnel. Asymmetry or gaps in the funnel plot may indicate that some studies or study data exist that may not have been published or located.¹⁴ It may also indicate poor methodologic quality of smaller studies (which tend to show larger effect sizes [ESs]) or true heterogeneity in the results.^14,15 Exploration of heterogeneity by meta-regression, using covariates indicative of study quality such as allocation concealment, use of blinding, and type of intervention, was planned but could not be conducted because of inadequate reporting of such data.

When SDs were not published and could not be obtained, they were imputed by standard statistical methods¹⁶ or derived from trials reporting the outcome in a similar population, where possible. Sensitivity analyses were performed on the effect of imputing SDs.

Six trials^11,17-21 used cluster randomization at the level of the class or school. Of these, 3 trials^11,19,20 did not report data suitable for inclusion. One trial¹⁸ analyzed results by cluster. We analyzed the results of the other 2 trials^17,21 using a published intraclass correlation coefficient of 0.02 to take into account the cluster randomization.^22,23 As sensitivity analyses, we used more extreme values for the intraclass correlation coefficient (ie, 0.0001 and 0.1); there was no effect on results to one decimal place (data not shown).

Subgroup analyses specified a priori included assessing differences in intervention effects by whether the program was administered to primary or secondary school students and boys-only intervention groups vs mixed (or girls-only) groups. "Primary schools" included elementary schools (grades K through 5 or K through 6) or students of equivalent ages if grade was unspecified or study was international. "Secondary schools" included middle, junior high, and high schools (grades 6 through 12 or 7 through 12) or students of equivalent ages.

We identified 9286 unduplicated electronic records, from which 274 potentially relevant reports were identified and the full texts examined. Thirty-five eligible trials^{12,13,18-21,24-52} were found. An additional 3 trials^10,11,17 were identified from authors, 2^33,53 from bibliographies, and 4^54-57 from expert contacts, giving 44 randomized controlled trials of secondary violence prevention programs. Five authors provided additional data.^{26,37,42,52,53} We received responses from 66% of all trial authors contacted. No trials reported data on violent injuries. Trial details are shown in Table 1.

Thirty-eight trials compared the intervention to a no intervention or placebo (ie, alternative classroom activity) control group and measured aggressive behavior. Twenty-three had complete data, 5 reported group means alone but the SD could be imputed, and 10 reported only partial or no data, giving 28 trials for analysis. Of these, 5 adjusted posttest results for pretest scores. Combining these 28 trials (Figure 1), aggressive behavior was reduced in intervention compared with control groups after intervention (ES, −0.36; 95% CI, −0.54 to −0.19). A test for heterogeneity was significant (P<.001), indicating variation in results across trials. Results were similar for 6 trials that collected data to 12 months after intervention (ES, −0.35; 95% CI, −0.79 to 0.09, with significant heterogeneity; P<.001), suggesting that effects were maintained.

Results were similar for different types of training. Training in skills of nonresponse produced beneficial effects on aggressive behavior immediately after intervention (ES, −0.38; 95% CI, −0.65 to −0.11, with significant heterogeneity; P = .001) and at 12 months (ES, −0.56; 95% CI, −1.08 to −0.05, without heterogeneity; P = .38). Interventions to improve relationship skills or social context also reduced aggressive behavior after intervention (ES, −0.65; 95% CI, −1.05 to −0.25, without heterogeneity; P = .24). One trial measured this outcome at 12 months (ES, −0.50; 95% CI, −0.97 to −0.04).

Among 15 trials that measured effects on school or agency actions, 6 published no data and the authors either could not provide data or could not be contacted. The pooled ES among 9 included trials was −0.59 (95% CI, −1.18 to 0.01), indicating a reduction in school or agency actions with intervention (Figure 2). There was significant heterogeneity (P<.001). In 2 trials, there was no apparent effect at 12 months (ES, 0.05; 95% CI, −0.45 to 0.55).

Training in skills of nonresponse produced less conclusive beneficial effects on school or agency actions (ES, −0.32; 95% CI, −0.90 to 0.26, with heterogeneity; P = .003). Training to improve relationship skills or social context produced more favorable effects (ES, −0.69; 95% CI, −1.26 to −0.13), although only 2 trials were included.

The immediate postintervention effect on aggressive behavior was similar for trials in primary schools (ES, −0.33; 95% CI, −0.54 to −0.12) and secondary schools (ES, −0.43; 95% CI, −0.75 to −0.11). Significant heterogeneity persisted within each subgroup (Figure 1).

There was no evidence of benefit from interventions in primary schools on school or agency actions (ES, 0.11; 95% CI, −0.48 to 0.70, without heterogeneity; P = .05). The remaining 7 studies, in secondary schools, showed a stronger positive effect of the intervention (ES, −0.82; 95% CI, −1.56 to −0.09, with significant heterogeneity; P<.001).

Most students identified as aggressive or violent were boys. Twelve trials^{10,25,26,29,35-39,43,44,52} studied boys alone, 1 trial¹³ studied girls alone, and the remaining 31 studied mixed groups in which most students were boys. Violence prevention programs delivered to boys alone had a modest effect on aggressive behavior (ES, −0.18; 95% CI, −0.47 to 0.1, without heterogeneity; P = .18), which may have been due to chance. The ES was greater for interventions delivered to mixed groups or girls alone (ES, −0.44; 95% CI, −0.66 to −0.23), although results were heterogeneous (P<.001). Two trials evaluating the effect of interventions on school or agency actions among male students alone reported no evidence of a benefit (ES, 0.22; 95% CI, −0.20 to 0.63, without heterogeneity; P = .25). Among 7 studies^{17,18,24,28,32,45,51} of mixed-sex groups, there was a substantial reduction in school or agency actions after intervention (ES, −0.85; 95% CI, −1.59 to −0.10, with significant heterogeneity; P<.001).

The SD was imputed for 5 trials with incomplete reporting.^{27,30,39,41,46} Omitting these data, the ES is −0.24 (95% CI, −0.40 to –0.08), a more modest effect than that found in the main analysis. The 5 studies with imputed data showed a strong effect (ES, −1.19; 95% CI, −1.62 to –0.76).

The Begg test (P = .07) and Eggar test (bias coefficient P<.001) suggested a relationship between study size and reported effects on aggressive behavior (ie, larger studies reported smaller effects). This is illustrated in the funnel plot (Figure 3). The plot also demonstrates asymmetry, with fewer trials showing a positive standardized mean difference, suggesting that small studies showing harm or no benefit were not included.

School-based violence prevention programs for high-risk children modestly reduced both aggressive behaviors and school or agency actions. Training in nonresponse skills and in relationship skills both showed beneficial effects. Although the ESs appear small, the overall benefits may be substantial when implemented on a schoolwide or districtwide basis.

Effects on aggressive behavior were similar in primary and secondary schools, whereas effects on school or agency actions were greater in secondary schools, although this difference may have been due to chance or to the way such actions are implemented at different ages. Although most programs focused solely or largely on boys, program effects appeared to be greater among mixed groups. Whether programs are more effective when delivered to mixed-sex groups or schools or whether such programs have greater effects on girls than on boys cannot be determined from these data.

Both subgroup analyses were prespecified, but, given the relatively small number of studies in each subcategory and the possibility of confounding by other trial characteristics, they should be interpreted with caution.

Tests for heterogeneity highlighted wide variation in the magnitude of results across trials. Funnel plot asymmetry is often used as an indication of publication bias, but it can also be explained by true heterogeneity among studies, resulting from differences in type or intensity of intervention, underlying risk, and study quality.⁵⁸ We found that differences in age group, sex, and training focus contributed to, but did not fully explain, the substantial heterogeneity. Trials in this review varied substantially in their quality, execution, and reporting. Most sample sizes were small, and many studies provided limited methodologic information (Table 1). Data on indicators of methods, such as allocation concealment, use of blinding, and type of intervention, were so poorly reported that exploration of heterogeneity by meta-regression was not possible. Therefore, we cannot make firm conclusions about the impact of these factors on the apparent heterogeneity.

All the trials selected students at high risk for violent or aggressive behavior. However, selection processes varied considerably. Because of inadequate information on these processes for most trials and difficulty in establishing population rates for the service actions on which selection was based, we could not explore whether program effectiveness differed according to how the study determined "high-risk" status.

Meta-analyses were performed on posttest results. Although randomized, most trials were relatively small. Therefore, differences in the pretest scores and other baseline characteristics (eg, age) may have occurred by chance. Unfortunately, the difference between the posttest and pretest means was not available for most studies. Posttest scores adjusted for pretest differences were used where available, otherwise unadjusted posttest scores were used.

The 5 studies^{27,30,39,41,46} in which the SD was imputed showed a greater beneficial effect from intervention than did those reporting full data. This may have been due to quality differences. Methodologically weaker studies, which may be less likely to report complete data, tend to show larger ESs.⁵ Exclusion of trials with imputed values reduced the ES but did not change its direction. Excluding such trials could introduce bias, however. Of 3 trials that reported group means but were excluded from the meta-analysis because SDs could not be imputed, 2^34,35 reported that the intervention had beneficial effects in reducing the outcomes measured, whereas 1⁴³ reported no effect. Their exclusion may have biased our results toward the null.

Publication bias is an important threat to validity in systematic reviews. Authors may choose not to submit results that are negative or not significant, and journals may not publish such studies.^58,59 Publication bias may also operate within individual studies when investigators selectively report outcomes with "significant" results.⁵⁹ We attempted to contact each author to establish whether any unpublished data were available and whether they knew of any further studies, published or unpublished. In addition, we contacted experts and organizations for unpublished studies. Despite such efforts, many trials did not provide data for relevant outcomes known to have been measured, which may have affected these results. The funnel plot suggests that negative studies may exist that were not retrieved and included.

Violence within our society has been a public concern in recent years. Substantial resources have been allocated to preventing violent injuries and crime. Schoolchildren have been intensively targeted. We identified 44 randomized controlled trials that evaluated school-based programs, of which 28 (64%) provided outcome data. Many trials were small and reported insufficient information to assess quality. Nevertheless, pooled results suggest that these interventions may reduce aggressive and violent behavior and school or agency actions in response to such behavior. However, analysis also suggests the possibility of bias toward studies with positive results and genuine differences among trials, which may mean that the true effect is smaller than that indicated. Larger, better controlled trials, with improved reporting of methods, use of adequate methods for allocation, blinding of outcome assessment, and more complete reporting of results, appear warranted to determine whether the apparent benefit is real. In addition, unexpected results emerged regarding differential effects by sex, which warrant further research.

Accepted for publication February 27, 2002.

This study was funded by the Systematic Reviews Training Unit, Institute of Child Health (Dr Mytton), and Camden and Islington Health Authority, London, England (partial funding to Dr DiGuiseppi).

We thank Ian Roberts, MBBCh, MRCP, PhD, Public Health Intervention Research Unit, London School of Hygiene and Tropical Medicine, London, for his advice throughout this project, and Frances Bunn, BSc, MSc, coordinator of the Cochrane Injuries Group, London.

Each year, 1 in 25 US schoolchildren are victims of violent crime while at school or on the way to and from school. The aggregate effects of the hundreds of school-based youth violence prevention programs currently being implemented have not been adequately evaluated.

School-based violence prevention programs for high-risk children modestly reduced both aggressive behaviors and school or agency actions in response to aggressive behavior. Effects on aggressive behavior were similar regardless of whether the programs focused on training in skills of nonresponse (eg, conflict resolution or anger control) or on training in social skills or social context changes. The benefits of violence prevention programs were similar in programs introduced in both primary and secondary schools, but appeared to be greater among mixed-sex groups. Additional large, well-controlled trials are needed to determine whether the apparent benefits are real.

Corresponding author: Julie A. Mytton, MBBS, MRCGP, MSc (e-mail: jmytton@doctors.org.uk).