Promoting Physical Activity With the Out of School Nutrition and Physical Activity (OSNAP) Initiative: A Cluster-Randomized Controlled Trial

Importance  Millions of children attend after-school programs in the United States. Increasing physical activity levels of program participants could have a broad effect on children’s health.

Objective  To test the effectiveness of the Out of School Nutrition and Physical Activity (OSNAP) Initiative in increasing children’s physical activity levels in existing after-school programs.

Design, Setting, and Participants  Cluster-randomized controlled trial with matched program pairs. Baseline data were collected September 27 through November 12, 2010, with follow-up data collected April 25 through May 27, 2011. The dates of our analysis were March 11, 2014, through August 18, 2015. The setting was 20 after-school programs in Boston, Massachusetts. All children 5 to 12 years old in participating programs were eligible for study inclusion.

Interventions  Ten programs participated in a series of three 3-hour learning collaborative workshops, with additional optional opportunities for training and technical assistance.

Main Outcomes and Measures  Change in number of minutes and bouts of moderate to vigorous physical activity, vigorous physical activity, and sedentary activity and change in total accelerometer counts between baseline and follow-up.

Results  Participants with complete data were 402 racially/ethnically diverse children, with a mean age of 7.7 years. Change in the duration of physical activity opportunities offered to children during program time did not differ between conditions (−1.2 minutes; 95% CI, −14.2 to 12.4 minutes; P = .87). Change in moderate to vigorous physical activity minutes accumulated by children during program time did not differ significantly by intervention status (−1.0; 95% CI, −3.3 to 1.3; P = .40). Total minutes per day of vigorous physical activity (3.2; 95% CI, 1.8-4.7; P < .001), vigorous physical activity minutes in bouts (4.1; 95% CI, 2.7-5.6; P < .001), and total accelerometer counts per day (16 894; 95% CI, 5101-28 686; P = .01) increased significantly during program time among intervention participants compared with control participants.

Conclusions and Relevance  Although programs participating in the OSNAP Initiative did not allot significantly more time for physical activity, they successfully made existing time more vigorously active for children receiving the intervention.

Trial Registration  clinicaltrials.gov Identifier: NCT01396473

Participation in daily physical activity is associated with better cardiovascular and metabolic disease risk profiles and improved mental and physical health among children and youth.¹ However, most children and adolescents do not meet nationally recommended levels of physical activity.² While evidence indicates that interventions can promote physical activity during the school day,³ limited conclusive research has been performed to determine how best to increase physical activity in the hours after school.^4-6 Several randomized clinical trials have examined the effect of creating new physical activity–focused after-school programs on physical activity and weight status outcomes.^4-7 An alternate approach, the one used in this study, focuses on implementing physical activity–promoting strategies in existing after-school programs.⁸

More than 10 million children in the United States participate in after-school programs for a mean of 7.4 hours per week, with children of color and low-income children participating at higher rates.⁹ This time represents an important opportunity to engage participants in the physical activity necessary to meet the national recommendations by adding time into the program schedule for physical activity or by making available time more physically active. Prior randomized clinical trials in existing programs have provided single-sport or single-activity curriculum and intervention materials and training for existing staff to implement interventions^10-12 or added additional activity-specific programming.¹³ Such strategies may not be feasible if resources and flexibility in programming schedules are limited. More flexible strategies to increase physical activity offerings through staff training show promise for improving physical activity levels among existing after-school program participants.^12,14 Such strategies using existing resources may cost less and may have broader appeal than interventions focused on single sports. In this cluster-randomized controlled trial, we tested whether an organizational change, collaborative learning approach can be implemented in various school and community–based after-school programs.

Box Section Ref ID

This experimental study was a cluster-randomized controlled trial conducted in 20 after-school programs (10 intervention sites paired with 10 matched control sites) in Boston, Massachusetts, from the fall of 2010 through the spring of 2011. The Figure shows the flow of programs and individual participants in the study. The study investigated changes in physical activity from baseline (September 27 through November 12, 2010) to follow-up (April 25 through May 27, 2011) among children in intervention programs compared with control programs. Eligible after-school programs were identified and recruited to participate in the study through collaboration between study staff (C.M.G.) and local community program providers (eg, Boys & Girls Clubs, the YMCA, and Boston Public Schools). Programs were eligible if they served children 5 to 12 years old, enrolled at least 40 children, and ran continuously from mid-October through the end of May. Before randomization, programs were matched on key characteristics obtained from administrative records, including the after-school program provider, snack provider, physical activity facilities, and school-level racial/ethnic and socioeconomic composition. The study manager (C.M.G.) matched and enrolled programs. After baseline data collection, the programs within each matched pair were randomized to intervention or control (ie, delayed intervention) status by a masked external statistician using a computer-based random number generator. Neither program directors nor study authors were masked to randomization status, whereas data collectors were masked. After-school programs randomized to the control condition were invited to participate in the intervention the following school year (2011-2012).

The study was approved by the Harvard T. H. Chan School of Public Health Institutional Review Board and the Boston Public Schools Office of Data and Accountability. Informed consent was obtained for all study participants. Before baseline data collection, researchers (J.L.B., C.M.G., R.M.L., E.L.K., M.E.deB., and J.C.T.) visited the 20 programs at child drop-off or pickup times to distribute active consent letters available in several languages. Parents or guardians gave written permission for children’s participation, and each child provided verbal assent. The full study protocol can be found in Supplement 1.

The Out of School Nutrition and Physical Activity (OSNAP) Initiative^15,16 works with after-school programs to improve nutrition and physical activity–related practices, environments, and policies using a socioecological model¹⁷ and a community-based participatory research approach.¹⁸ The OSNAP Initiative resources and links to research evidence are available online (http://www.osnap.org). The 2 physical activity goals were (1) to include 30 minutes of moderate, fun physical activity for every child every day (including outdoor activity, if possible) and (2) to offer 20 minutes of vigorous physical activity (VPA) 3 times per week. Over a 6-month intervention period, teams from intervention sites (including program directors and staff) were invited to participate in a series of three 3-hour OSNAP Initiative learning collaborative (LC) sessions modeled after existing community partner professional development programs and prior research.^19,20 At LC sessions, teams reviewed reports of physical activity time provided and participation rates at baseline in their programs compared with the OSNAP Initiative goals. Using tools provided by study staff, after-school teams identified areas for improvement and set action plans to improve program practices, develop relevant policies, and communicate changes to parents and other partners. At LC sessions, program staff members shared progress with other teams and participated in skill development sessions. Program staff members were also offered additional physical activity skill-building sessions. These additional sessions focused on strategies to incorporate physical activity breaks and small-space activities into the program, including training on the Food & Fun After School curriculum,²¹ recess-focused training by Playworks,²² and SPARK²³ training. Between LC sessions, study staff provided intervention programs, with technical assistance and reminders via telephone, newsletter, and email. Sites were invited to apply for up to $200 to support implementation.

Data collectors visited each program for 1 week (5 weekdays) at baseline and follow-up to distribute and collect accelerometers and to observe program activities, including types and amount of physical activity provided. Data collectors recorded the start and end times of activities offered in several predetermined categories, including free active play (ie, free choice of activity) and structured active play (ie, physical activity led by program staff), and indicated how many and what grade levels of children attended each activity and whether the activity was located indoors or outdoors.

Child demographic information, including sex, race/ethnicity, grade, and age, were collected via parent report on consent forms. From the National Oceanic and Atmospheric Association, study staff obtained local weather data on hourly precipitation and temperature for each day of data collection²⁴ and the 30-year mean annual temperature.²⁵ The average daily temperature during data collection periods, percentage deviation from the average annual temperature, and presence of precipitation (yes or no) were documented for each after-school period between 2 and 6 pm to account for daily weather changes.

Baseline (September to November 2010) and follow-up (April to May 2011) data were collected on the primary outcome measure of the number of minutes of moderate to vigorous physical activity (MVPA) that individual children accumulated overall during the after-school period. Secondary outcomes included (1) VPA minutes and sedentary activity minutes accumulated overall; (2) the number of MVPA, VPA, and sedentary activity minutes accumulated in modified 10-minute bouts (periods of continuous activity intensity, allowing for 2 minutes to fall below intensity threshold)²; and (3) total accelerometer counts during the after-school period.

Students wore accelerometers (GT3X/GT1M or MTI/CSA 7164; Actigraph) on their hip on a fitted belt continuously during the after-school period for 5 school days (Monday through Friday) at baseline and at follow-up. Functionality of monitors was verified for each available accelerometer before assignment for research use. Accelerometer counts were collected in 1-minute epochs, and the number of minutes spent in MVPA were estimated with age-specific cut points for children 6 to 18 years old²⁶ used in national surveillance,² with thresholds of 4 metabolic equivalent counts for MVPA and 6 metabolic equivalent counts for VPA. Less than 100 accelerometer counts per minute was considered sedentary activity. These cut points based on 1-minute epochs demonstrate excellent accuracy in classifying activity levels among children.²⁷ Nonwear periods were defined according to established protocols as 60 consecutive minutes of zero accelerometer counts, allowing for up to 2 minutes below 100 accelerometer counts.²⁸ Implausibly high values (>20 000 counts) were replaced with imputed counts based on the mean of the last and next valid minutes (when ≤10 in succession) or were set to missing (when >10 in succession). Valid minutes were restricted to minutes during the defined after-school period for which data collectors documented students wearing accelerometers, and valid days consisted of at least 1 hour of data collected during program time. Students with 2 or more valid days at both baseline and follow-up were included in the final analysis to be consistent with criteria used in school-based evaluations²⁹ and to ensure nondifferential attrition in the sample by program. Estimates were also calculated using data from all participants, and these results were not appreciably different (eTables 1, 2, and 3 in Supplement 2).

The mean daily minutes offered to the typical child for each type of activity were calculated by summing the number of minutes recorded by data collectors in the daily log, weighted by the proportion of children present at the program who were attending the particular activity. Researchers calculated the number of days on which programs provided at least 30 minutes of physical activity time to all attendees. The mean program-level physical activity minutes offered per day were calculated for baseline and follow-up, and changes by condition were compared, accounting for clustering of days within programs.

Linear mixed-effects models were used to investigate whether students at participating OSNAP Initiative programs experienced greater increases in accelerometer-derived physical activity levels during the program period compared with those at control programs. The intervention effect was estimated by an interaction term of period (follow-up is 1, and baseline is 0) by intervention (intervention is 1, and control is 0), in addition to period and intervention status main effects. Models were adjusted at the child level for daily minutes monitored; child sex, race/ethnicity, and grade; matched pair indicators at the program level; model of accelerometer worn (to account for potential differences in output between models^30,31); and percentage deviation from the average annual temperature and the presence of precipitation at the observation day level. Program-level random intercepts were used to account for the correlation due to clustering of students within programs. A compound symmetry within-person error covariance structure was used to account for nesting of successive daily observations within students.³² The df for significance tests was calculated using the between-within method.^33,34 Adjusted means for each outcome at baseline and follow-up were estimated from models adjusting for clustering, monitored minutes, and monitor model only. Post hoc analyses using stratified mixed models examined whether intervention effects differed by sex or grade (kindergarten through 2 vs 3-6). Analyses were performed between March 2014 and August 2015 using statistical software (SAS, version 9.3; SAS Institute Inc). Sample-size calculations completed before this study were based on observed data from children in after-school programs. They suggested that, with 30 participants clustered within 10 matched intervention and control programs, assuming an intraclass correlation of children’s physical activity minutes within programs of 0.01 and a correlation of 0.6 between baseline and follow-up physical activity minutes, researchers would have 80% power to detect a mean (SD) difference of 4.6 (19.7) minutes of MVPA as a primary outcome. Standard deviations were based on data collected in 4 pilot sites.

Staff from all intervention programs attended LC sessions 1 and 2, and 9 of 10 programs participated in LC session 3. All intervention programs included the goal of increasing physical activity in their action plans. Nine of 10 intervention programs included practice, policy, or communication actions for providing 30 daily minutes of physical activity time for every child (including outdoor activity, if possible) as part of their action plans, and 6 of 10 intervention programs reported completing all specified actions. Intervention program staff attended additional physical activity skill training, including SPARK (2 programs), Food & Fun After School (4 programs), and Playworks (7 programs). On monthly reports, 5 intervention programs and 1 control program reported ever using a Food & Fun After School physical activity lesson. One intervention program used 4 Food & Fun After School activities. Five of 10 intervention programs applied for and received $200 to support implementation of their physical activity action planning goals.

Twenty programs agreed to participate and be randomized to intervention or control (ie, delayed intervention) status. Approximately half of the program participants assented to participate in both intervention (51.2% [288 of 563]) and control (53.1% [308 of 580]) programs. Of the 251 students (87.2% [251 of 288]) in intervention programs and 268 students (87.0% [268 of 308]) in control programs with valid baseline accelerometer data, longitudinal analysis of change in accelerometer-derived physical activity was conducted among 182 intervention participants (72.5% [182 of 251] retention) and 220 control participants (82.1% [220 of 268] retention). Participants were racially/ethnically diverse (Table 1). Approximately half were female (51.2% [206 of 402]), and students ranged from kindergarten through grade 6, with a mean age of 7.7 years. Children provided means of 4.2 days of physical activity data at baseline and 4.0 days at follow-up, with mean daily monitored times of 139.9 minutes at baseline and 142.5 minutes at follow-up (Table 2).

At baseline, intervention sites provided 30 or more minutes of physical activity on a mean (SD) of 2.3 (1.6) days compared with a mean (SD) of 3.2 (1.9) days provided by control sites (P = .27). Change in physical activity minutes offered to children during program time from baseline to follow-up was not significantly different between conditions (−1.2; 95% CI, −14.2 to 12.4; P = .87). Table 2 summarizes the types and locations of physical activity time provided by programs during observed days at baseline and follow-up.

Adjusting for the matched design and other variables, the intervention led to no significant change in MVPA minutes overall (P = .40) or in bouts of MVPA (P = .92) during program time (Table 3). However, participants in intervention programs increased VPA minutes per day (3.2; 95% CI, 1.8-4.7; P < .001) and VPA minutes per day in bouts (4.1; 95% CI, 2.7-5.6; P < .001) during program time relative to participants in control programs. No significant differences were observed in changes in total (P = .57) or modified bouts (P = .35) of sedentary activity time among intervention program participants compared with control participants. However, there were significant increases in total accelerometer counts per day (16 894; 95% CI, 5101-28 686; P = .005) among intervention participants compared with control participants.

Subsequent stratified analyses by groupings of kindergarten through grade 2 and grades 3 through 6 suggested a differential intervention effect on overall MVPA and VPA. Among younger participants, the intervention was associated with an increase in MVPA minutes during program time (3.9; 95% CI, 0.5-7.2; P = .02) compared with controls, while it was associated with a decrease in MVPA minutes among older participants (−6.7; 95% CI, −9.9 to −3.6; P < .001). Among younger participants, the intervention was associated with an increase in VPA (4.8; 95% CI, 2.8-6.9; P < .001) compared with controls. Among older participants, the observed increase in VPA associated with the intervention was not statistically significant (1.5; 95% CI, −0.6 to 3.5; P = .17). Intervention effects did not differ significantly by participant sex.

Children in programs participating in the OSNAP Initiative increased VPA levels by 36.0%, or an estimated 3.2 minutes per day during the program period overall, and 4.1 minutes in bouts of VPA. Overall, the programs participating in the OSNAP Initiative did not make significant changes in time allotted for physical activity compared with controls but were successful in making existing time more vigorously active for children. Evidence among youth indicates that engaging in VPA may be associated with better fitness, academic achievement, and cognition and with lower adiposity and cardiovascular risk compared with engaging in moderate physical activity.^1,35,36 While older students participating at OSNAP Initiative program sites did not increase MVPA, there was evidence that implementing the intervention was effective in increasing overall MVPA among younger program participants. Potentially, the technical assistance and tools and resources used were more applicable and appropriate to younger children. Activity patterns and preferences may change with age. Identification of program policies, practices, and communication strategies for engaging older students may be needed to achieve equitable results in these grades.

The observed relative increase in VPA per child (4.1 minutes) is similar to that in a meta-analysis⁵ of after-school program initiatives promoting physical activity and is consistent with changes seen in more recent studies of interventions adding specific programmed physical activities,³⁷ using train-the-trainer approaches for soccer-focused activity promotion,¹⁰ and building programmatic capacity through staff skill development¹² in existing program settings. The OSNAP Initiative provided technical assistance and training for existing program staff, with directed quality improvement goals that focused on improving the policies, practices, and communication strategies used in everyday program operation. The OSNAP Initiative was designed with a focus on policy and environmental changes that programs with varying existing resources could implement within existing structures to maximize sustainability. This approach was used successfully with both school-based and community-based programs. Nationally, approximately 57% of after-school participants attend school-based programs.⁹

This cluster-randomized controlled trial used a community-based approach that engaged after-school leaders and program providers in development and implementation. It also assessed changes in physical activity using objective assessment measures. Researchers used established accelerometer cut points validated in similarly aged children²⁶; however, the use of 1-minute epoch length may have missed activity of shorter duration.³⁸ Further detailed study of the context of physical activity in programs is warranted given the importance of different physical activity context–specific factors by age and sex.³⁹ Researchers cannot attribute the observed changes in physical activity to a single intervention component. Programs implemented various policy, programmatic, and communication strategies. In addition, researchers did not collect data on participants’ physical activity outside of the program period, making the effect on total daily physical activity unknown. While the participant sample was representative of children and families who consented to participate in the research with regard to race/ethnicity, it was not nationally representative of all after-school program participants. Nationally, Hispanic and black children attend after-school programs at higher rates than children of white race/ethnicity but represent a minority of all after-school program participants.⁹ Baseline nonresponse and loss to follow-up may also limit generalizability of the results.

This study tested the effectiveness of the OSNAP Initiative in increasing children’s physical activity levels in existing after-school programs. Although programs participating in the OSNAP Initiative did not allot significantly more time for physical activity, they successfully made existing time more vigorously active for children. Further research is needed to ensure an equitable effect and to evaluate sustainability. Many US children participate in after-school programs.⁹ The physical activity improvements that are possible using the OSNAP Initiative and similar training strategies focused on after-school organizational quality improvement could have a broad reach.

Accepted for Publication: September 17, 2015.

Corresponding Author: Angie L. Cradock, ScD, Department of Social and Behavioral Sciences, Harvard T. H. Chan School of Public Health, 677 Huntington Ave, Boston, MA 02215 (acradock@hsph.harvard.edu).

Published Online: December 7, 2015. doi:10.1001/jamapediatrics.2015.3406.

Author Contributions: Dr Cradock had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Cradock, Barrett, Giles, Lee, deBlois, Gortmaker.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Cradock, Barrett, Giles, Lee.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Cradock, Barrett, Gortmaker.

Obtained funding: Cradock, Gortmaker.

Administrative, technical, or material support: Cradock, Barrett, Giles, Lee, Kenney, deBlois.

Study supervision: Cradock, Giles, Lee, Gortmaker.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work is supported by Prevention Research Center cooperative agreement 1U48DP001946 with the Harvard Prevention Research Center (which includes the Nutrition and Obesity Policy Research and Evaluation Network and the Physical Activity Policy Research Network) from the National Center for Chronic Disease Prevention and Health Promotion of the Centers for Disease Control and Prevention.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: This work is solely the responsibility of the authors and does not represent official views of the Centers for Disease Control and Prevention.