School-Community Partnerships: A Cluster-Randomized Trial of an After-School Soccer Program

Importance Identifying community-based programs that increase physical activity among diverse youth could yield sustainable tools to reduce obesity and obesity disparities.

Objective To evaluate the impact of a community-based after-school soccer and youth development program, America SCORES, on students' physical activity, weight status, and fitness.

Design Cluster-randomized trial. Study measures were collected in the fall (baseline), winter (midpoint), and spring (end point) of the 2009-2010 school year.

Setting After-school programs in 6 schools within a large urban school district.

Participants All 4th and 5th grade students in after-school programs at the study schools were eligible.

Intervention Three schools were randomized to receive the SCORES after-school program, delivered via the train-the-trainer model.

Main Outcome Measures Change in minutes of after-school moderate-to-vigorous physical activity (MVPA), fitness (maximal oxygen consumption), and body mass index over 1 school year.

Results Participants (n = 156) were diverse (42% Latino, 32% Asian, and 12% African American) and 76 (49%) had a body mass index at or above the 85th percentile. There were no significant group differences in the change in physical activity, fitness, or weight status among all students. However, among students with a body mass index at or above the 85th percentile, SCORES significantly increased MVPA after school (3.4 min/d; 95% CI, 0.3-6.5) and on Saturdays (18.5 minutes; 95% CI, 3.4-33.6).

Conclusions and Relevance Existing community-based programs such as SCORES can increase physical activity among low-income youth, particularly those most at risk for weight-related comorbidities. While evaluating existing programs presents special challenges, partnerships between communities, schools, and researchers are an important component of translational research to address obesity.

Trial Registration clinicaltrials.gov Identifier: NCT01156103

The alarming increase in pediatric obesity^1,2 has been accompanied by a rise in the prevalence of diabetes mellitus and hypertension in youth.^3,4 Physical activity protects against the development of obesity and is closely linked to decreased cardiovascular risk,^5,6 yet youth are far from meeting recommended activity levels.^7-9 Low-income and minority youth are disproportionately less active and more obese than their high-income or white peers.^8-12

The after-school hours are a promising venue in which to increase physical activity for diverse youth.¹³ While recent demonstration projects in the after-school setting have shown promising trends in physical activity and fitness in youth of color,^14,15 when funding ends for these research-driven programs, new sources of financial support must be identified to continue programming. A train-the-trainer model, in which after-school staff are trained to implement a physical activity curriculum, offers greater promise for sustainability after research funding is gone. A handful of recent studies have used a train-the-trainer model in the after-school setting with some promising results,^16-18 although none were randomized, longitudinal trials.

Using a cluster-randomized controlled design, the present study investigated the impact of the America SCORES after-school program on physical activity levels, weight status, and cardiovascular fitness among diverse elementary school students. America SCORES is an organization that provides after-school soccer and literacy programs in 15 cities across the country to promote child health and positive youth development. It has refined its curriculum based on 20 years of experience working in low-income schools. A pilot study of SCORES demonstrated that participants significantly improved their fitness, with a trend toward improved weight status among obese students after 1 school year.¹⁹ The present study examined the impact of SCORES, delivered via a train-the-trainer model.

This cluster-randomized controlled trial examined the impact of the SCORES program on physical activity levels, weight status, and cardiovascular fitness during 1 school year among 4th and 5th grade students. The Committee on Human Research at the University of California at San Francisco approved this research. Parents or guardians provided written consent and students gave verbal assent.

Sample size calculations suggested 64 participants would be required to see a 5-minute difference in moderate-to-vigorous physical activity (MVPA), assuming student-level randomization within 2 schools. Subsequently, it was decided to randomize at the school level in response to principals' concerns that randomizing students within schools would be unfair. Given expected clustering by school, the sample size was increased to 6 schools.

This study took place in a large, diverse, urban school district, with an enrollment of 56 000 students. Of 72 schools with kindergarten through 5th grade enrollment, 60 schools that had not offered SCORES in the year prior to the study were eligible to participate. The study was presented at a regularly scheduled principals' meeting, at which 14 eligible schools were represented and 7 schools agreed to participate. At study schools, 61% of students were eligible for free or reduced-price meals (range, 44%-89%). There were no statistically significant differences between the study schools and other eligible schools (n = 53) in the percentage of free or reduced-price meals (P = .96), percentage of nonwhite students (P = .39), or school enrollment (P = .23).

All 4th and 5th grade students enrolled in the after-school program at participating schools were eligible for the study. After-school programs could accommodate approximately 25% of the total student body and preferentially enrolled students who qualified for free or reduced-price meals.

America SCORES is an after-school soccer program started by an elementary school teacher in 1994 in Washington, DC, to provide a safe after-school environment and to promote literacy among her inner-city students. SCORES differs from typical after-school sports programs in that it uses soccer as a hook to engage youth, with a primary goal of building competencies and skills that will support students' overall development, including teamwork, leadership, and academic commitment. The program, which has expanded to 15 major cities throughout the country, offers soccer, creative writing, and service learning experiences to youth who would otherwise have limited access to extracurricular activities. SCORES uses soccer and writing coaches who are trained with a standard curriculum to lead the SCORES program in the after-school setting. Students spend 2 to 3 days per week in soccer drills or games for up to 2 hours each day. The 2 nonsoccer days each week are dedicated to creative writing and performance in the 12-week fall session and to community service projects in the 12-week spring session. Historically, schools pay up to half of the cost of operating the SCORES program, while SCORES has raised the balance through grants and private donations.

While the initial plan was to study the traditional SCORES model previously described, owing to significant budget cuts in the district in 2009-2010, schools had fewer resources to contract staff from outside agencies, such as SCORES, to deliver their programs. SCORES responded by moving to a train-the-trainer model in which SCORES trained the district's after-school staff to operate the program. SCORES had already implemented this model with good success in several schools in the district that had a strong history with the SCORES program, but its impact and dissemination to new schools had not been tested.

Under the train-the-trainer model, the 3 SCORES schools received the SCORES curriculum, 6 hours of training in the fall before the program began, and an additional 6-hour training in the spring. Training included lesson planning and execution, student soccer and poetry skill development, and behavior management. SCORES provided coaches with soccer coaching manuals featuring more than 100 soccer practice activities and games and a writing program curriculum with examples and activities. Additionally, SCORES staff visited each of the intervention school sites multiple times during the course of the study to provide technical assistance. After-school staff were encouraged to use the SCORES curriculum to provide a minimum of 1 hour of soccer on 2 afternoons each week and 1 hour of creative writing on 2 afternoons each week during each 12-week session (fall and spring). SCORES organized game days on Saturdays, providing 1 additional hour of soccer each week. While SCORES activities were open to all 4th and 5th grade students at intervention sites, students were not required to participate in the program and could, instead, participate in other scheduled activities.

Study measures were collected in the fall (baseline), winter (midpoint), and spring (end point) of the 2009-2010 school year.

The primary outcome was change in after-school minutes of MVPA from baseline to both midpoint and end point. The after-school period was designated as the time from the end of the school day (based on each school's after-school program schedule) until the end of the after-school program (mean, 3.0 hours; range, 2.3-4.3 hours). Students wore a GT1M or GT3X accelerometer (Actigraph LLC), which yield equivalent activity counts when programmed in uniaxial mode,²⁰ for 7 consecutive days (including 5 school days) at all 3 points. Accelerometer data were collected from intervention and control schools in alternating weeks in an attempt to minimize differences of the weather's impact on physical activity between the 2 groups. Accelerometer data were recorded in 15-second epochs. Periods with more than 20 minutes (80 epochs) of continuous zero values were considered nonwear time²¹ and were omitted from analyses. A composite method of accelerometer data reduction, which uses more available accelerometer data compared with standard approaches and is more representative of student activity across days than imputation,²² was used to determine average minutes spent in MVPA. First, each epoch was categorized as MVPA if activity counts were 574 or greater based on Evenson cutpoints.²³ Second, each time-stamped epoch was averaged across school days, yielding the proportion of days for which a given epoch was considered MVPA. For example, if for the 15-second epoch ending at 3:00:15, 2 of 5 school days were considered MVPA, 40% (6 seconds) of the composite epoch for 3:00:15 would be considered MVPA. Finally, total time in MVPA was calculated by summing the number of seconds of MVPA accumulated in each epoch of the composite day.

Secondary outcomes were change in body mass index (BMI; calculated as weight in kilograms divided by height in meters squared) z score and cardiorespiratory fitness. With children in indoor clothes and shoes off, trained researchers measured height to the nearest tenth of a centimeter using the 420 Measure-All Portable Measuring Board (KWS Medical Supplies LLC) and weight to the nearest 10th of a kilogram using the Tanita Model BWB 800 digital scale (Tanita Corporation of America). Body mass index z scores were calculated using the 2000 Centers for Disease Control and Prevention growth reference data.²⁴ Cardiorespiratory fitness was assessed using the validated 20-m shuttle test.²⁵ A parent or guardian of each student provided parents' highest level of education (used as a proxy for socioeconomic status) and student's race/ethnicity.

As part of a process evaluation, school staff reported their SCORES schedules and student attendance. Researchers conducted semistructured interviews with after-school staff at each school (N = 11). Interview transcriptions were reviewed by 2 researchers, and barriers to implementing the curriculum were identified and enumerated.

The 7 study schools were randomly allocated to intervention or control status (Figure). Researchers informed schools and SCORES of the assignments. Neither schools nor researchers were blinded to assignment. Prior to the study start date, leadership at one of the intervention schools changed, and the new principal withdrew the school from the study.

To accommodate the cluster design and repeated measures, linear mixed-effect models (with random effects for school and student) were used to analyze differences between groups in change in after-school minutes of MPVA, BMI, and fitness from baseline to midpoint and end point, adjusting for baseline values as well as sex and baseline weight status. A test for interaction demonstrated that weight status (BMI ≥ 85th percentile) modified the impact of SCORES on after-school MVPA (P = .04) but no other outcomes. All analyses were performed using Stata/MP version 11 (StataCorp).

Of 88 eligible students in the 3 intervention schools, 82 (93%) enrolled in the study; 74 of 86 eligible students (86%) enrolled in the study in control schools. Follow-up data were available on all but 3 students in each group at end point (96% retention; Figure); all 6 of these students had left the school. Students were diverse and most parents had a high school education or less (Table 1). At baseline (Table 1), a smaller proportion of intervention students were overweight or obese than control students (40% vs 58%; 95% CI for difference, 2-34), with a trend toward lower BMI z scores among intervention students (mean difference, −0.28; 95% CI, −0.60 to 0.05). Students in intervention schools did more MVPA after school than students in control schools, both overall (mean difference, 6.4 minutes; 95% CI, 3.3-9.4) and among students with a BMI at or above the 85th percentile at baseline (mean difference, 4.6 minutes; 95% CI, 0.8-8.3) with no other differences between groups. Overall, students did more MVPA on weekdays than on Saturdays (mean difference overall, 14.5 minutes; 95% CI, 8.8-20.3), and overweight and obese students did less MVPA than nonoverweight students (mean difference, −4.5 minutes; 95% CI, −7.6 to −1.4).

In adjusted models, there was no difference between groups in change in after-school MVPA (Table 2), nor were there differences in change in BMI z score or fitness (or any changes within groups). Among students with a BMI at or above the 85th percentile at baseline (n = 76), SCORES significantly increased MVPA after school by 3.4 minutes (95% CI, 0.3-6.5) and on Saturdays by 18.5 minutes (95% CI, 3.4-33.6), although there were no differences in change in BMI z score or fitness between groups.

Interviews with 8 after-school staff at intervention schools suggested that staff enjoyed leading SCORES. All interviewees (n = 8) reported receiving excellent 1-on-1 help from SCORES staff during the school year, but noted they would have benefitted from longer trainings (n = 4) and additional training midseason (n = 4). Staff at all 3 intervention schools noted barriers to implementing SCORES, including: not enough time to implement the curriculum because of competing priorities including homework time and other enrichment activities (3 schools); space limitations (2 schools); and lack of student motivation to participate (2 schools). Participation in SCORES was optional and overall attendance was 48% (range, 0%-100%), with overweight and obese students attending more than normal weight students (60% vs 39%; 95% CI for difference, 2-38). While staff almost met the goal of offering 12 weeks of SCORES programming in the fall (mean, 11.3 weeks), in the spring, only 7 weeks of SCORES were offered on average (this was driven by low compliance at 1 school). Based on average attendance rates, students in intervention schools were exposed to an average of 1.4 hours of soccer each week.

There is a clear need for programs and policies that effectively increase physical activity among diverse youth. In the present study, the SCORES program significantly increased MVPA among overweight and obese youth, both after school and on the weekend. A unique strength of this study was its focus on a program that has demonstrated portability and scalability under its traditional model, where the program is delivered by SCORES staff. Like other long-standing programs, SCORES has specific advantages in implementing the train-the-trainer model, including extensive expertise working with school districts and existing infrastructure to readily disseminate its curriculum.

While the adjusted relative increase in after-school MVPA of 3.4 minutes in SCORES schools was small, it is consistent with the mean effect (4 minutes) reported in a recent meta-analysis of physical activity interventions conducted across multiple settings.²⁶ Taken together, the increased activity demonstrated after school and on Saturday among overweight and obese youth in the present study equated to an additional 35 minutes per week of MVPA.

Three recent studies have used train-the-trainer models across different after-school settings. One cluster-randomized study, conducted in a single school district over 3 years, trained after-school staff to implement the Catch Kids Club curriculum²⁷ in intervention schools.¹⁶ While the nested cross-sectional design could not account for significant between-group differences in baseline MVPA, in the first year of the intervention, overweight and obese students engaged in 6 more minutes of MVPA daily than did students in the baseline year.¹⁶ One quasiexperimental study implemented the Catch Kids Club curriculum in YMCAs and Boys and Girls Clubs, with no increase in MVPA over 1 school year.¹⁸ However, the training for staff was minimal and staff noted mixed reception of the program by youth. A third study offered multiday trainings for YMCA staff using a 10-unit curriculum to support the provision of at least 30 minutes of physical activity daily in YMCA after-school programs.¹⁷ This study demonstrated an increase in after-school MVPA of 10 minutes over 1 school year. While promising, it is unclear whether the increased MVPA seen in that quasiexperimental study, which had retention rates just under 50%, would be replicable in a randomized controlled trial.

To our knowledge, the present study of SCORES is the first randomized longitudinal study of after-school physical activity delivered via the train-the-trainer model. With retention rates of 96% in both arms and an intention-to-treat analysis, this study reflects real world effects. Differences in study design make it difficult to compare effect sizes between the present study and other recent studies.^16-18 While randomization in the community setting is difficult, future randomized trials comparing the effectiveness of various curricula under the train-the-trainer model would be of value.

In the present study, the largest increase in MVPA was seen on Saturdays, when the program was delivered by SCORES staff, rather than after-school staff. Under the intended design for the present study, the program was to have been delivered entirely by SCORES staff. However, school district budget constraints precluded this approach. It would be of interest to compare students' physical-activity levels in programs delivered by staff of existing programs, such as SCORES, vs via the train-the-trainer model, exploring cost-effectiveness as well. Both approaches, which take advantage of the expertise and infrastructure of existing community programs, could lead more quickly to solutions to the obesity epidemic than traditional clinical research models that may not be scalable or sustainable.

This study had several limitations. While we controlled for baseline characteristics, the significant between-group differences at baseline still pose a threat to internal validity. Accelerometers captured only 1 week of data at each point, which may not represent activity at other times, and we did not account for program attendance during accelerometer data collection or for the time at which children left the after-school program (parents could pick their child up early). Thus, after-school MVPA may have occurred outside the after-school program, although this would affect both groups equally. Finally, the intervention was not delivered with as much fidelity as we would have expected if SCORES staff, as opposed to trained after-school staff, had led the program, which may underestimate the program's effects.

America SCORES is a promising program that increases physical activity among overweight and obese youth. While research with community partners presents special challenges, it provides the most accurate assessment of intervention effectiveness in the real world setting where programs are actually delivered.

Correspondence: Kristine Madsen, MD, MPH, University of California at Berkeley School of Public Health, 219 University Hall, Berkeley, CA 94720-7360 (madsenk@berkeley.edu).

Accepted for Publication: October 11, 2012.

Published Online: February 25, 2013. doi:10.1001/jamapediatrics.2013.1071

Author Contributions:Study concept and design: Madsen, Thompson, Adkins, and Crawford. Acquisition of data: Thompson and Crawford. Analysis and interpretation of data: Madsen and Thompson. Drafting of the manuscript: Madsen and Thompson. Critical revision of the manuscript for important intellectual content: Madsen, Thompson, Adkins, and Crawford. Statistical analysis: Madsen and Thompson. Obtained funding: Madsen. Administrative, technical, and material support: Adkins and Crawford. Study supervision: Madsen and Thompson.

Funding/Support: This work was supported by grants K23HD054470 from the National Institutes of Health/Eunice Kennedy Shriver National Institute of Child Health and Human Development and 0865005F from the American Heart Association.

Additional Contributions: We appreciate the willingness of the children and their families to participate in this research, as well as the support of Colin Schmidt, MBA, Yuri Morales, BA, Emily Queliza, BA, Robert Joseph, JD, and Eva Sippola, MPH, at America SCORES Bay Area, without whom this research would not have been possible.