eTable 1. Risk Factors Prioritized and Targeted by Intervention Communities
eTable 2. Adjusted Odds Ratios for Cumulative Abstinence from Substance Use and Delinquency by Grade 12 Among Baseline Noninitiators Comparing Intervention to Control Communities
eTable 3. Adjusted Odds Ratios for Cumulative Incidence of Substance Use and Violence by Grade 12 Among Baseline Noninitiators Comparing Intervention to Control Communities
eTable 4. Adjusted Odds Ratios for Grade 12 Prevalence of Past-Month and Past-Year Substance Use, Delinquency, and Violence Comparing Intervention to Control Communities
Hawkins JD, Oesterle S, Brown EC, Abbott RD, Catalano RF. Youth Problem Behaviors 8 Years After Implementing the Communities That Care Prevention SystemA Community-Randomized Trial. JAMA Pediatr. 2014;168(2):122-129. doi:10.1001/jamapediatrics.2013.4009
Copyright 2014 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Community-based efforts to prevent adolescent problem behaviors are essential to promote public health and achieve collective impact community wide.
To test whether the Communities That Care (CTC) prevention system reduced levels of risk and adolescent problem behaviors community wide 8 years after implementation of CTC.
Design, Setting, and Participants
A community-randomized trial was performed in 24 small towns in 7 states, matched within state, assigned randomly to a control or intervention group in 2003. All fifth-grade students attending public schools in study communities in 2003-2004 who received consent from their parents to participate (76.4% of the eligible population) were included. A panel of 4407 fifth graders was surveyed through 12th grade, with 92.5% of the sample participating at the last follow-up.
A coalition of community stakeholders received training and technical assistance to install CTC, used epidemiologic data to identify elevated risk factors and depressed protective factors for adolescent problem behaviors in the community, and implemented tested and effective programs for youths aged 10 to 14 years as well as their families and schools to address their community’s elevated risks.
Main Outcomes and Measures
Levels of targeted risk; sustained abstinence, and cumulative incidence by grade 12; and current prevalence of tobacco, alcohol, and other drug use, delinquency, and violence in 12th grade.
By spring of 12th grade, students in CTC communities were more likely than students in control communities to have abstained from any drug use (adjusted risk ratio [ARR] = 1.32; 95% CI, 1.06-1.63), drinking alcohol (ARR = 1.31; 95% CI, 1.09-1.58), smoking cigarettes (ARR = 1.13; 95% CI, 1.01-1.27), and engaging in delinquency (ARR = 1.18; 95% CI, 1.03-1.36). They were also less likely to ever have committed a violent act (ARR = 0.86; 95% CI, 0.76-0.98). There were no significant differences by intervention group in targeted risks, the prevalence of past-month or past-year substance use, or past-year delinquency or violence.
Conclusions and Relevance
Using the CTC system continued to prevent the initiation of adolescent problem behaviors through 12th grade, 8 years after implementation of CTC and 3 years after study-provided resources ended, but did not produce reductions in current levels of risk or current prevalence of problem behavior in 12th grade.
clinicaltrials.gov Identifier: NCT01088542
Community-based efforts to prevent substance use, delinquency, and violence are an essential component of promoting health during adolescence and later life.1,2 Communities That Care (CTC) is a prevention system that activates a coalition of stakeholders to develop and implement a science-based approach to prevention in the community to achieve collective impact on youth development community wide.3,4 The CTC prevention system seeks to achieve this goal by increasing the use of tested, effective preventive interventions that address risk factors for adolescent problem behaviors prioritized by the community. This is expected to produce community-wide reductions in targeted risk factors and, in turn, decreased adolescent substance use, delinquency, and violence.3,5
The CTC system is different from other efforts to mobilize communities for the prevention of adolescent problem behaviors (eg, the Midwestern Prevention Project,6- 8 Project Northland,9 Communities Mobilizing for Change on Alcohol,10 the Community Trials Intervention to Reduce High Risk Drinking,11,12 and PROSPER13). It does not focus exclusively on the prevention of alcohol use but rather on reducing shared risk factors for multiple behavior problems. It does not prescribe specific programs but trains the local coalition to choose programs from a menu of tested programs that best address the community’s unique profile of risk and protection. In contrast to PROSPER, CTC does not prescribe who leads the prevention efforts but encourages stakeholders from a variety of organizations in the community to take leadership.
Results from a community-randomized trial of CTC support the CTC theory,3,5 including increased adoption of a science-based approach to prevention14- 16 and implementation of a greater number of tested and effective prevention programs.17 The trial also found that CTC lowered targeted risks for problem behavior and reduced the incidence and prevalence of seventh- and eighth-grade delinquency and substance use in a panel of youths followed up since fifth grade, 3 and 4 years after initial implementation of CTC.18,19 These reductions continued to be observed 2 years later in 10th grade, 6 years after initial implementation of CTC and 1 year after support for the implementation of CTC had ended in the trial.20
This study tested the enduring effects of CTC on risk exposure and youth problem behaviors in 12th grade, 3 years after study-provided resources ended and 8 years after initial implementation of CTC in the trial. Although most CTC coalitions continued during the unsupported period,21,22 very few of them used tested and effective prevention programs targeting high school students. The primary outcomes expected to be affected by CTC and examined in this study are targeted risk factors, substance use, delinquency, and violence.23
The Community Youth Development Study (CYDS)5 is a community-randomized trial of CTC. Twenty-four communities in Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington were matched in pairs within state on population size, racial and ethnic diversity, economic indicators, and crime rates. One community from within each matched pair was assigned randomly by a coin toss to either the intervention (CTC) or control group.5 The CYDS communities are small to moderate-sized incorporated towns with their own governmental, educational, and law enforcement structures, ranging from 1500 to 50 000 residents.
Beginning in summer 2003, intervention communities received CTC training over 6 to 12 months by certified trainers. The CTC coalition members were trained to use data from cross-sectional CTC Youth Surveys of public school students in the community to prioritize risk factors to be targeted by tested and effective preventive actions.24,25 Although CTC is designed for children and youths ages 0 to 18 years, CYDS communities were asked to focus their prevention plans on programs for youths aged 10 to 14 years and their families and schools so that possible effects on drug use and delinquency could be observed within the initial 5-year study period. Starting with the 2004-2005 school year and annually thereafter, community coalitions implemented between 1 and 5 preventive programs to address their prioritized risk factors. These included universal school-, family-, and community-based programs and selective school and community programs targeted at youths at elevated risk. The CYDS staff provided technical assistance and support for preventive interventions throughout the 5-year efficacy trial but stopped after the fifth year of the study. Control communities received data from CTC Youth Surveys administered in their schools every 2 years but received no resources, training, or technical assistance from the study.
The University of Washington Human Subjects Review Committee approved the protocol. Data were from a longitudinal panel of public school students in the 24 CYDS communities followed up from grade 5 through grade 12 (N = 4407).23 Students were surveyed annually (2004-2011), except in 11th grade when students were tracked but not surveyed. The sample is sex balanced (50% male). Twenty percent of students identified as Hispanic or Latino, 64% were non-Hispanic white, 3% were non-Hispanic African American, 5% were non-Hispanic Native American, 1% were non-Hispanic Asian American, and 6% were of other ethnicities. All students in fifth-grade classrooms during the 2003-2004 school year in the 24 CYDS communities were eligible for participation in the study. Recruitment continued in grade 6 to increase the overall participation rate. Parents of 4420 students provided written informed consent to their participation in the study (76.4% of the total eligible population; 76.1% in CTC communities and 76.7% in control communities). The first wave of data collection (fifth grade, 2004) was a preintervention baseline assessment. The seventh wave of data was collected in spring 2011 when panel students progressing normally were in grade 12. At this point, 10 of the original 12 CTC coalitions were still active but had not received any support from the study for 3 years.21,22 Tested and effective programs that were still being implemented in CTC communities during this unsupported period continued to be aimed primarily at middle school–aged adolescents (grades 5-9). Only 4 CTC communities implemented 1 of 3 substance abuse prevention programs aimed at high school–aged youths (Project Toward No Drug Abuse, Class Action, or Communities Mobilizing for Change on Alcohol) during this period. Therefore, few panel students were exposed during the high school years to tested and effective prevention programs selected through the CTC process.
The longitudinal panel consists of 4407 students who completed a wave 1 or wave 2 survey. Students in the longitudinal panel who remained in the intervention or control communities for at least 1 semester were tracked and surveyed, even if they left the community, moved schools, or dropped out.23 Seven students were deceased by the 12th-grade data collection and 2 students were permanently excluded from the sample owing to disability that precluded them from filling out the survey, leaving an active, still-living sample of 4398 students. Of the still-living sample members, 4068 (92.5%) completed the survey in 12th grade (2236 [93.2%] in CTC communities and 1832 [91.6%] in control communities) (Figure).
Students completed the Youth Development Survey,26 a self-administered paper-and-pencil questionnaire designed to be completed in a class period. In 12th grade, 25.5% of participants completed the survey online because they were no longer attending school. Identification numbers but no names or other identifying information were included on the surveys. Participants received a $10 incentive check after completing the survey.
The CTC coalitions prioritized 2 to 5 risk factors that were elevated in their community based on anonymous cross-sectional surveys of all assenting sixth- and eighth-grade students in their community.27,28 Data used for targeting decisions were different from those used in the present analysis to evaluate intervention effects on risk factors. The cohort of fifth graders followed up in the trial did not participate in the cross-sectional surveys.
A targeted risk factor score was calculated for panel students in CTC communities by averaging the community-specific set of targeted risk factors. Items composing each risk factor scale were standardized within each year, and each scale was then standardized across years to facilitate pre-post comparisons. Because control communities did not prioritize risk factors using the CTC process, the average risk factor score in control communities was calculated using the set of targeted risk factors identified in its matched CTC community. For example, for students in community pair A, the targeted risk factor score was the average of scale scores for family conflict, antisocial friends, peer rewards for antisocial behavior, attitudes favorable to antisocial behavior, and rebelliousness; for students in community pair B, the targeted risk factor score was calculated based on scale scores for low commitment to school, family conflict, and antisocial friends (eTable 1 in Supplement shows the community-specific sets of targeted risk factors for all intervention communities).
Students reported their lifetime and past-month use of substances in grades 5 through 12 and past-year substance use in grade 12. Based on these prospective data, we examined sustained abstinence from any substance use, use of gateway drugs (alcohol, cigarettes, or marijuana), and binge drinking (having ≥5 drinks in 1 occasion) through grade 12 to assess the overall effect of CTC on preventing substance use. Cumulative incidence was examined for substances where use by grade 12 was less common than nonuse (ie, ≥50% of the sample reported never using by grade 12). The 12th-grade prevalence rates in the past month and the past year were computed for individual substances as well as composite indices of any substance use and gateway drugs (alcohol, cigarettes, or marijuana).
Each year, students reported participation in 7 delinquent and violent acts (stealing, damaging property, shoplifting, attacking someone with intent to harm, carrying a handgun, being arrested, and beating up someone so badly that he or she probably needed medical attention). A subset of the delinquency items (attacking someone with intent to harm, carrying a handgun, and beating up someone) was used to measure violent behavior. We computed sustained abstinence from delinquency and cumulative incidence of violence through spring of grade 12 as well as the past-year prevalence of both outcomes in grade 12. We also examined the number of different delinquent acts (ranging from 0-7) and different violent behaviors (ranging from 0-3) in the past year in grade 12.
Student-level covariates included age, sex, race (white vs nonwhite), Hispanic ethnicity (Hispanic vs non-Hispanic), parental education, attendance at religious services during grade 5 (on a scale of 1 [never] to 4 [about once a week or more]), and rebelliousness in grade 5 (mean of 3 items; Cronbach α = .69). Community-level covariates included the total population of students in the community (mean [SD], 2628 ) and the percentage of students eligible for free or reduced-price school lunch (mean [SD], 38.2% [13.8%]).
Overall, 4021 students (91.4%) in the active, still-living sample participated in at least 6 of the 7 waves of data collection, and item nonresponse was small (<1%). Based on validity criteria (eg, reporting not being honest and using a fictitious drug), 78 students were excluded from the analysis sample in grade 12 (35 students in control and 43 students in CTC communities), resulting in valid data from 3990 students in 12th grade (90.7% of the active, still-living sample; 1797 students [89.8%] in control communities and 2193 students [91.5%] in CTC communities).
Of all the data points involved in the analysis (sample size times number of variables),29 11.8% were missing (10.8% in the CTC group and 13.0% in the control group). Missing data were imputed using multiple imputations to obtain unbiased estimates of model parameters and their standard errors, assuming that data are missing at random.30 Using NORM version 2.03 software (Pennsylvania State University), 40 data sets including data from all 7 waves were imputed separately by intervention group.31 Analyses were conducted within each imputed data set and then averaged using Rubin’s rules.32
Because communities rather than students were randomized within matched community pairs, the effect of CTC was estimated as the mean difference between intervention groups in community-level sustained abstinence, cumulative incidence, prevalence, and means. Because community randomization does not guarantee equivalent student populations or that community pairs will remain similar over time, all analyses were adjusted for student and community characteristics and the respective preintervention baseline measure of the outcome to improve the precision of estimated intervention effects.23,33,34 All covariates were grand-mean centered.
Sustained abstinence and cumulative incidence were assessed among students who had not yet engaged in the behavior at baseline (grade 5). Current prevalence in the 12th grade and targeted risk factor scores were examined in the full sample.
Generalized linear mixed models35,36 with random effects for intercepts were used to model variability in outcomes across 4407 students, 24 communities, and 12 community pairs. Linear regression was used to estimate mean differences between CTC and control communities in average levels of targeted risk factors in grade 12, adjusting for baseline levels of targeted risk. Poisson regression with a log link and binomial error distribution was used to estimate adjusted risk ratios for sustained abstinence, cumulative incidence, and current prevalence.37,38 Adjusted odds ratios estimated using logistic regression can be found in eTables 2, 3, and 4 in the Supplement.
The statistical significance of intervention effects was tested with 9 df (number of community-matched pairs  minus the number of community-level covariates , minus 1) and a type I error rate of .05 (2-tailed). All analyses were conducted using HLM 7 software (Scientific Software International), and population-average results are reported.39
There were no statistically significant baseline differences by intervention group in levels of average targeted risk factors, the incidence and prevalence of substance use, delinquency, violence, or the mean number of delinquent and violent acts.18,23 Accretion and attrition were similar in both intervention groups.
The adjusted mean difference between intervention groups in the targeted risk factor score in grade 12, adjusting for baseline levels of targeted risk and student and community characteristics, was not statistically significant (adjusted mean difference = 0.07; 95% CI, −0.03 to 0.18; P = .16).
Youths in CTC communities were significantly more likely than youths in control communities to have abstained from any substance use and the use of gateway drugs through the spring of 12th grade (Table 1). The proportion of 12th graders who had never used alcohol and who had never smoked cigarettes was significantly higher in CTC communities than in control communities, but there was no statistically significant difference by intervention group in sustained abstinence or in cumulative incidence of other substances (Table 1 and Table 2). Youths in CTC communities were also significantly more likely than youths in control communities to avoid ever engaging in delinquent (Table 1) or violent (Table 2) behavior through the spring of 12th grade.
The proportion of students in control and CTC communities who used drugs in the past month or the past year did not differ significantly, with the exception of ecstasy use (Table 3). Students in CTC communities were almost twice as likely to use ecstasy in the past month as students in control communities. There were no significant differences by intervention group in past-year prevalence of delinquency and violence (Table 3) or the number of different delinquent behaviors (adjusted risk ratio = 1.03; 95% CI, 0.89-1.19; P = .67) and the number of different violent acts (adjusted risk ratio = 0.98; 95% CI, 0.78-1.22; P = .81).
The results of this study indicate that 8 years after implementation of CTC in communities and 3 years after study-provided technical assistance and resources ended, CTC continued to prevent initiation of alcohol and tobacco use, delinquency, and violence through 12th grade in a panel of students followed up from grade 5. However, as implemented in this study, CTC did not produce reductions in levels of risk or the prevalence of current drug use, delinquent behavior, or violent behavior in grade 12.
Communities chosen for this randomized trial of CTC were towns of 50 000 or fewer residents and do not include urban or suburban populations. Findings of this study may not generalize to larger communities. Another limitation is that the effect of CTC was evaluated in only 12 matched community pairs, which may have limited power to detect smaller intervention effects. However, the study detected substantively meaningful risk reductions or increases in abstinence between 12% and 32%. Youths in CTC communities were 32% more likely than youths in control communities to abstain from any drug use through 12th grade; they were 31% more likely to avoid ever using any of 3 gateway drugs (alcohol, cigarettes, or marijuana). They were 18% more likely to have avoided delinquent behavior and 14% less likely to have engaged in violence. Twelfth graders in CTC communities also had a 31% higher probability than students in control communities of having never drank alcohol and were 12% more likely to have never smoked cigarettes. These effect sizes are similar to those found earlier when the panel was in 8th grade and when the benefit to cost ratio was estimated to be $5.30 per $1.00 invested in CTC based on the prevention of smoking and delinquency.40
Another possible threat to the internal validity of the study is that all analyses are based on self-report data, which carry the risk of social desirability bias or dishonesty. It is important to note that although this study was not a blinded trial, communities, not students, were randomized into intervention groups. It is highly unlikely that students in the longitudinal panel were aware of the intervention group to which their community belonged; thus, it is unlikely that there was differential self-report bias by intervention group that might account for any observed trial benefits. Further, we used validity checks to exclude a small number of students each year (<2% of the sample) deemed to have provided inaccurate reports of their behavior. This exclusion rate did not differ by intervention group. Additionally, the prevalence of substance use in this study is comparable to national data for the same cohort of 12th graders in the Monitoring the Future study.41
The enduring effects of CTC through 12th grade were observed with little preventive programming targeting the high school years. Because CTC communities were asked to focus their prevention plans on programs for youths in grades 5 through 9, and continued to do so following study support, few students in the longitudinal panel were exposed to tested and effective programs beyond ninth grade. It is noteworthy that initiation of alcohol use, tobacco use, delinquency, and violence in the panel was prevented through 12th grade in CTC communities.
Targeting preventive interventions during middle school, a developmentally sensitive time for drug use and delinquency initiation, appears to have prevented the onset of alcohol and tobacco use, delinquency, and violence in the panel through high school. However, the present findings suggest that continued preventive interventions during high school may be needed to lower the current prevalence of substance use, delinquency, and violence among those who have initiated these behaviors. This suggestion is consistent with results of the randomized trial of Project Northland, a school- and community-based approach to preventing adolescent alcohol use. Perry et al9 found significant positive effects of Project Northland during the active intervention phase in middle school, but alcohol use grew faster among youths in intervention communities than in control communities in grades 9 and 10 when little programming took place. Positive effects in reducing alcohol use were found again, however, after preventive interventions were introduced in grades 11 and 12.
The higher prevalence of past-month use of ecstasy among 12th-grade students in CTC communities compared with control communities is the only significant negative effect associated with CTC observed in this panel.18- 20 This result should be interpreted with caution as the estimation of this intervention effect is based on small numbers of students reporting ecstasy use. In 11 of the 12 control communities and in 7 of the 12 CTC communities, no more than 4 students reported past-month ecstasy use in 12th grade. When community pairs were compared, the prevalence of past-month ecstasy use was higher in the CTC community than in the control community in 8 of 12 pairs and lower in the CTC community than in the control community in 4 pairs. In the absence of specific hypotheses or other evidence that would explain a negative intervention effect, it is unclear whether the higher prevalence of ecstasy use in grade 12 in CTC communities is an iatrogenic effect attributable to the intervention.
Sustained effects of CTC on preventing the initiation of alcohol use, tobacco use, delinquency, and violence through 12th grade are important. These effects were sustained with little preventive programming targeted at high school students during a period in which communities experienced economic stress likely to threaten prevention efforts.42 Lack of a developmental focus on preventive intervention during the high school years may explain why CTC communities did not reduce current levels of targeted risk factors or the current prevalence of drug use, delinquency, or violence in the panel in grade 12. It is possible that communities using the CTC system could affect these behaviors if they expanded the use of tested and effective preventive interventions developmentally through the high school years, although research is needed to confirm this suggestion.
Corresponding Author: J. David Hawkins, PhD, Social Development Research Group, School of Social Work, University of Washington, 9725 Third Ave NE, Ste 401, Seattle, WA 98115 (firstname.lastname@example.org).
Accepted for Publication: August 15, 2013.
Published Online: December 9, 2013. doi:10.1001/jamapediatrics.2013.4009.
Author Contributions: Dr Hawkins had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition of data: Hawkins.
Analysis and interpretation of data: Oesterle, Brown, Abbott.
Drafting of the manuscript: Hawkins, Oesterle, Brown.
Critical revision of the manuscript for important intellectual content: Hawkins, Oesterle, Abbott, Catalano.
Statistical analysis: Oesterle, Brown, Abbott.
Obtained funding: Hawkins.
Study supervision: Hawkins.
Conflict of Interest Disclosures: Dr Catalano is a board member of Channing Bete Co, distributor of Supporting School Success and Guiding Good Choices. These programs were used in some communities in the study that produced the data set used in this article. No other disclosures were reported.
Funding/Support: This work was supported by research grant R01 DA015183-03 from the National Institute on Drug Abuse, with cofunding from the National Cancer Institute, the National Institute of Child Health and Human Development, the National Institute of Mental Health, the Center for Substance Abuse Prevention, and the National Institute on Alcohol Abuse and Alcoholism.
Role of the Sponsor: The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.
Additional Contributions: David M. Murray, PhD, Ohio State University, provided paid statistical consultation on this project, but the authors are responsible for all analyses and results. Tanya Williams provided editorial help. We acknowledge the contributions of the communities participating in the Community Youth Development Study and the collaborating state offices of drug abuse prevention in Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington.