A total of 3000 state-day values for each variable as defined in the text are shown, with 1 line connecting points in each state, except for dining data, which were not available for Alaska, Arkansas, Delaware, Idaho, Iowa, Maine, Mississippi, Montana, New Hampshire, North and South Dakota, Vermont, West Virginia, and Wyoming.
Each panel shows the 7-day mean daily cases per 100 000 population for states, sorted by their tertile of cumulative incidence (ie, by the area under the curve in these graphs) at the time of a policy or behavior. Tertile groupings vary across (A) school closure, (B) gathering ban, and (C) decreased time at work. Blue dashes indicate date of policy or behavioral change for states in that tertile.
Each panel shows the 7-day mean daily deaths per 100 000 population for states, sorted by their tertile of cumulative incidence (ie, by the area under the curve in these graphs) at the time of a policy or behavior. Tertile groupings vary across (A) school closure, (B) gathering ban, and (C) decreased time at work. Blue dashes indicate date of policy or behavioral change for states in that tertile.
Shading indicates 95% CIs, derived via Monte Carlo simulation with 1000 trials.
eTable 1. Date of Policy Interventions by State
eTable 2. Analysis of Collider Variables in State-Level Policies
eTable 3. Association Between Logged 7-Day Rolling Mean Daily Outcomes and School Closure, Gathering Bans, and Behavioral Change at State Level, Using Shortened Study Period (45 Days After Date of 0.5 Cases per 1000)
eTable 4. Association Between Logged 7-Day Rolling Mean Daily Outcomes and School Closure, Gathering Bans, and Behavioral Change at State Level, Using Lengthened Study Period (10 Days Before Through 60 Days After Date of 0.5 Cases per 1000)
eTable 5. Model Selection Results
eTable 6. Description of COVID-19–Related Behavioral Variables
eTable 7. Pandemic and Policy Characteristics of States Over Study Period
eTable 8. State-Level Characteristics Used as Controls
eTable 9. Association Between Logged 7-Day Rolling Mean COVID-19 Incidence and School Closure, Gathering Bans, and Behavioral Change at State Level
eTable 10. Association Between Logged 7-Day Rolling Mean COVID-19 Death Rate and School Closure, Gathering Bans, and Behavioral Change at State Level
eFigure. Model Prediction Compared With Observed COVID-19 Outcomes
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Zimmerman FJ, Anderson NW. Association of the Timing of School Closings and Behavioral Changes With the Evolution of the Coronavirus Disease 2019 Pandemic in the US. JAMA Pediatr. 2021;175(5):501–509. doi:10.1001/jamapediatrics.2020.6371
What are the independent associations of voluntary behavioral change and legal restrictions, such as state-mandated school closings, with the subsequent spread of the coronavirus disease 2019 (COVID-19) pandemic in the US?
In this cross-sectional study of US COVID-19 data, voluntary behavioral changes, such as reductions in time spent at work, had an association with reduced COVID-19 incidence and mortality that was 3 times stronger than that of school closures.
These findings suggest that less harmful ways of preventing severe acute respiratory syndrome coronavirus 2 transmission are available than mandatory school closures.
The consequences of school closures for children’s health are profound, but existing evidence on their effectiveness in limiting severe acute respiratory syndrome coronavirus 2 transmission is unsettled.
To determine the independent associations of voluntary behavioral change, school closures, and bans on large gatherings with the incidence and mortality due to coronavirus disease 2019 (COVID-19).
Design, Setting, and Participants
This population-based, interrupted-time-series analysis of lagged independent variables used publicly available observational data from US states during a 60-day period from March 8 to May 18, 2020. The behavioral measures were collected from anonymized cell phone or internet data for individuals in the US and compared with a baseline of January 3 to February 6, 2020. Estimates were also controlled for several state-level characteristics.
Days since school closure, days since a ban on gatherings of 10 or more people, and days since residents voluntarily conducted a 15% or more decline in time spent at work via Google Mobility data.
Main Outcomes and Measures
The natural log of 7-day mean COVID-19 incidence and mortality.
During the study period, the rate of restaurant dining declined from 1 year earlier by a mean (SD) of 98.3% (5.2%) during the study period. Time at work declined by a mean (SD) of 40.0% (7.9%); time at home increased by a mean (SD) of 15.4% (3.7%). In fully adjusted models, an advance of 1 day in implementing mandatory school closures was associated with a 3.5% reduction (incidence rate ratio [IRR], 0.965; 95% CI, 0.946-0.984) in incidence, whereas each day earlier that behavioral change occurred was associated with a 9.3% reduction (IRR, 0.907; 95% CI, 0.890-0.925) in incidence. For mortality, each day earlier that school closures occurred was associated with a subsequent 3.8% reduction (IRR, 0.962; 95% CI, 0.926-0.998), and each day of advance in behavioral change was associated with a 9.8% reduction (IRR, 0.902; 95% CI, 0.869-0.936). Simulations suggest that a 2-week delay in school closures alone would have been associated with an additional 23 000 (95% CI, 2000-62 000) deaths, whereas a 2-week delay in voluntary behavioral change with school closures remaining the same would have been associated with an additional 140 000 (95% CI, 65 000-294 000) deaths.
Conclusions and Relevance
In light of the harm to children of closing schools, these findings suggest that policy makers should consider better leveraging the public’s willingness to protect itself through voluntary behavioral change.
Few measures taken to combat the coronavirus disease 2019 (COVID-19) pandemic are more far-reaching and consequential than closing schools, which disrupts children’s socioemotional learning and academic performance1-3 as well as parents’ employment and education. The economic costs of school closures in the US in the spring of 2020 have been estimated to be some $2 trillion dollars4,5 and fall most heavily on those least able to withstand them.2,6,7 However, although schools are sites of influenza transmission,8 the evidence on school closures and reduced COVID-19 spread is inconclusive.
Several peer-reviewed studies9-12 find that the independent association of school closures with subsequent rates of COVID-19 is low to nonexistent. By contrast, a recent study by Auger and coauthors13 finds large associations of school closures with lower COVID-19 incidence and mortality. Using the same data and similar methods, Courtemanche and coauthors10 come to the opposite conclusion: school closures are associated with an increase in COVID-19 cases and mortality, albeit not significantly. Neither of these articles control for the contemporaneous effects of voluntary behavioral change. It may be that fear of contagion was an underlying factor for both individual behavioral change and public policy change.
This study tests the association of school closures with subsequent changes in COVID-19 incidence and deaths while controlling for contemporaneous voluntary behavioral change. The analysis adds further evidence to the policy debate by incorporating big data to analyze the timing of government policies such as school closures, closures of nonessential businesses, and stay-at-home orders relative to several measures of behavioral change in the general population. The analysis uses data-driven techniques to incorporate these co-occurring policy and behavioral changes while minimizing the risk for collider bias. It also controls for serial autocorrelation in the estimates.
For this cross-sectional study, an interrupted-time-series model was fit to state-level data on COVID-19 incidence and deaths and on several legal restrictions and measures of behavioral change. All data were deidentified. The study was deemed exempt from institutional review by the University of California, Los Angeles institutional review board. Informed consent was not required for the use of publicly available data. This analysis follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines.
Data on COVID-19 cases and deaths were taken from the New York Times.14 Population data from the 2018 American Community Survey were used to calculate rates per 1000 population.15 The primary outcome is the logged 7-day rolling mean of daily incidence and death rate, with values of 0 recoded to 0.001.
Measures for policy action take the form of the number of days since the action was implemented (eTable 1 in the Supplement). Candidate policy variables included school closure, nonessential business closures, bans on large public gatherings, stay-at-home orders, and restaurant closures.
Several proxies for behavioral change were collected, coded as the number of days since the behavioral change occurred, which was assessed as crossing a percentage of change threshold from baseline values. Time at work and time at home were assessed from Google Mobility, gathered from individual cell phone data.16,17 These variables were coded as the percentage of change relative to a pre–COVID-19 baseline of median values from January 3 to February 6, 2020. Personal hygiene behaviors were the most difficult to observe using big data, so the number of daily Google searches, by state, for the term hand sanitizer served as a proxy.18 Google search data have been previously used in epidemiologic studies, with good predictive validity for observed behavior at the state level.19,20 This term was chosen because of the consistency of public health advice (compared with mask use, for example) and because the use of hand sanitizer would require most people to obtain a product (unlike hand washing, for example). State values were normalized by dividing the number of searches by the count on the date with the most frequent searches for that term. Daily restaurant dining data were taken from OpenTable.21 Data were coded as the percentage of change in diners in 2020 relative to the same date in 2019. OpenTable data were available only for the 36 most populous states. Because daily data tend to have high variance, all variables were converted to 7-day moving means.
Consistent with previous work,13 regression analyses controlled for a set of state-level covariates:
Percentages of the population younger than 15 years and older than 65 years15;
Percentage of the population in nursing homes22;
Quartile of cumulative incidence growth rate during the first 10 days of the pandemic;
The 2018 Centers for Disease Control and Prevention Social Vulnerability Index, reweighted at the state level23;
Cumulative testing rates of less than 10, 10 to 20, and greater than 20 per 1000 population24;
Urban density, measured by the number of urban residents per square mile (<50, 50-100, 100-150, and >150)15; and
Number of days into the study period. To adjust for the fact that the pandemic hit different states on different weeks, we recentered our data by defining all time-varying data according to the number of days since the pandemic began, defined as the first day on which a state had a cumulative incidence rate of 0.5 cases per 100 000 population or higher. The study period accordingly varied by state in calendar time and was defined as the 60-day period after the beginning date.
To understand the sequencing of behavioral change and policies, each of the 4 behavioral measures was graphed against time. To help with interpretability, this graph is centered on days since school closures went into effect rather than days into the study period. Also indicated on the graphs are the dates (relative to school closures) of legal restrictions most closely related to the behavioral measure.
To limit problems associated with collider bias among the legal restrictions in the regression analysis,26 we tested the association of cumulative COVID-19 incidence at the end of the study period with indicators of legal restrictions at the state level. State-level rates increased rapidly in the early stages and then abruptly leveled off around day 30, so if this pattern did not vary according to whether a state implemented a given legal restriction, then that policy was assessed as not a plausible confounder and therefore excluded from further analyses to avoid collider bias.
Of the 5 legal restrictions identified, 3 (nonessential business closure, restaurant closure, and stay-at-home orders) were implemented in some states but not others. Because none of the restrictions with variation across states were significant at the 0.05 level in the regressions (eTable 2 in the Supplement), they were dropped from subsequent analysis. School closure and bans on large gatherings were retained as candidate explanators of the reduction in severe acute respiratory syndrome coronavirus 2 transmission.
To assess the independent associations of school closure, other policies, and behavioral variables with rates of COVID-19 incidence and fatality, 4 models were estimated:
ln(yit) = α + β1t + β2Si(t−lag) + β3Zit + εi
ln(yit) = α + β1t + β2Si(t−lag) + β3Zit + β4Gi(t−lag) + εi
ln(yit) = α + β1t + β3Zit + β5Bi(t−lag) + εi
ln(yit) = α + β1t + β2Si(t−lag) + β3Zit + β4Gi(t−lag) + β5Bi(t−lag) + εi
where y indicates the outcome (incidence or mortality), i indicates state, t indicates days since beginning of the study period, lag indicates the relevant lag period, S indicates days since school closure, Z indicates a vector of additional state-level covariates, G indicates days since the ban on gatherings of more than 10 people, and B indicates the behavioral variable control. Sensitivity analyses using alternative study periods are reported in eTables 3 and 4 in the Supplement.
Because of differences in testing capacity and baseline risk across states, we used a data-driven process with the bayesian information criteria to determine lag periods, selection of the behavior-change variable to include, and candidate thresholds for the behavioral change variable (eTable 5 in the Supplement). Based on the results across the 2 outcomes, we selected 15% decrease in time spent at work. Lag periods were not forced to be identical across the policy and behavior variables, but the highest-performing models all had nearly identical lags across the measures: 14 days for the incidence models and 21 days for the death models.
To adjust for serial autocorrelation in the dependent variable and therefore in the error terms, a second set of models were estimated using the Prais-Winsten estimator.27 All analyses were conducted in STATA, version 15 (StataCorp LLC).
There was wide variation in the behaviors during the 60-day periods in each state ranging from March 8 to May 7 to March 19 to May 18, 2020, with the rate of meals in restaurants declining by a mean (SD) of 98.3% (5.2%) during the study period and a maximum of 100% since the previous year; time at work declining by a mean (SD) of 40.0% (7.9%) and a maximum of 60% since the pre-COVID baseline; and time at home increasing by a mean (SD) of 15.4% (3.7%) and a maximum of 26% since the pre-COVID baseline. eTable 6 in the Supplement reports the descriptive statistics of the COVID-19–related behaviors during the study period.
Figure 1 shows the patterns of these changes against the timing of school closures and other legal restrictions. The experience of each state is represented separately by its own line. Behavioral changes clearly began before school closure across the 50 states. In many states, the behavioral change was about halfway to its full extent by the time schools were officially closed. Because the other policy changes, such as the closure of nonessential businesses and stay-at-home orders, occurred alongside or after school closures, the behavioral changes were well under way, and in some cases fully realized, before these other interventions took effect. Restaurant dining, for example, was reduced 50% to 100% before any restaurant closures took effect.
There was large variation in the spread of the pandemic during the first 10 days: median growth rate was just over 1300% and ranged from 180% in South Dakota to more than 5000% in New Jersey. Behavioral change began as early as 4 days before the pandemic was under way (as defined by the study period) and as late as 10 days afterward. eTable 7 in the Supplement displays characteristics of the states during the study period. Other state-level characteristics are presented in eTable 8 in the Supplement.
Figure 2 and Figure 3 show 7-day rolling mean daily incidence and mortality by state. States are grouped by tertile based on their cumulative incidence at the time of the policy or behavioral change. The dashed blue lines indicate at which day of the pandemic the respective policy or behavioral change occurred. For all 3 sets of actions, states that enacted policies or changed behaviors early in the pandemic generally had lower incidence and death throughout the study period.
The Table shows the results for the 8 regression models. The upper half of the table displays the models using logged daily incidence as the outcome, with the Prais-Winsten models as our preferred estimation method. In model 1, in which only school-closure timing and state-level controls are included, each day earlier that schools were closed was associated with an incidence rate ratio (IRR) of 0.854 (95% CI, 0.845-0.863). This finding implies that each day earlier that schools were closed was associated with a 14.6% reduction in cases. In model 2, which adds the number of days since the ban on gatherings, the association between earlier school closures and cases weakened by more than one-fourth (IRR, 0.892; 95% CI, 0.880-0.905). The IRR for gathering bans was smaller but still significant (0.949; 95% CI, 0.940-0.960). Model 3 removed both policy variables and introduced the behavior-change variable, finding that each additional day since a 15% reduction in time spent at work was associated with a 15.3% (IRR, 0.847; 95% CI, 0.838-0.856) reduction in daily COVID-19 incidence. Model 4 is the preferred specification, because it included all 3 primary regressors. Each day of delay in school closures was associated with a 3.5% reduction (IRR, 0.965; 95% CI, 0.946-0.984), whereas each day earlier that behavioral change occurred was associated with a 9.3% reduction (IRR, 0.907; 95% CI, 0.890-0.925).
The bottom half of the Table shows the results for the daily death rate. The findings were similar to those for daily incidence across the 4 Prais-Winsten models: school closures 1 day earlier were associated with a 15.3% reduction (IRR, 0.847; 95% CI, 0.829-0.865) in daily death in the short model, which fell to a 3.8% reduction (IRR, 0.962; 95% CI, 0.926-0.998) when the other primary regressors were accounted for. For the reduction in time spent at work, these estimates were a 15.9% reduction (IRR, 0.841; 95% CI, 0.824-0.860) in the short model and a 9.8% reduction (IRR, 0.902; 95% CI, 0.869-0.936) in daily deaths in the full model.
Figure 4 shows the results of simulations under various scenarios of daily incidence and cumulative cases and for deaths. The baseline scenario was the estimated value under the full model, which was close to the actual levels of incidence and death (eFigure in the Supplement). In each graph, 3 hypothetical counterfactuals were graphed:
The modeled values if there had been a 2-week delay in school closures in each state relative to the actual closure date but with behavioral change happening when it did;
The modeled values if there had been a 2-week delay in behavioral change in each state relative to the actual timing of behavioral change but with school closures happening when they did; and
The modeled values if both school closures and behavioral change had happened with a 2-week delay.
These graphs suggest that had the school closures happened with a 2-week delay, there would have been approximately 587 000 (95% CI, 213 000-1 153 000) more cases and 23 000 (95% CI, 2000-62 000) more deaths by day 60 of the pandemic than in fact occurred. Had the behavioral changes happened with a 2-week delay, there would have been some 4.3 million (95% CI, 2 542 000-6 995 000) more cases and 140 000 (95% CI, 65 000-294 000) more deaths than in fact occurred. Had both changes happened 2 weeks later than they did, there would have been 10.1 million (95% CI, 4 841 000-19 944 000) more cases and 280 000 (95% CI, 105 000-788 000) more deaths.
This analysis shows that a substantial portion of behavioral changes that were eventually embraced happened before any intervention requiring such action. The results imply that many people will act in ways that limit the spread of a pandemic disease even without being compelled to do so.
The regression analyses suggest that the school closures of mid-to-late March 2020 appear to have been associated with a reduced incidence of COVID-19 infection and associated deaths due to COVID-19. However, relative to other changes occurring in the same time frame, the estimates of association were modest. The simulations suggested that timely school closures were associated with the prevention of cases and deaths during the study period, but that the estimated number of cases and deaths averted because of voluntary changes in behavior was much larger.
The results have important implications for the interpretation of ecological studies about the effects of policy interventions, including school closures. The finding of widespread behavioral change before school closures took effect suggests that both school closures and behavioral change have a common cause: a general fear of contracting and spreading disease.
These results add clarity to the existing literature by testing the independent role of legal restrictions of school closures and bans on large gatherings when controlling for contemporaneous voluntary behavioral change. Unlike the results of Courtemanche et al,10 this analysis finds a significant and beneficial association of school closures with pandemic spread, albeit of modest magnitude. Unlike the results of Auger et al,13 this analysis finds a significant and beneficial association of bans on large gatherings with pandemic spread, albeit of modest magnitude. These differences may result from better control for serial autocorrelation and collider bias. It seems plausible that the timing of any given legal restriction is influenced by both prior rates of severe acute respiratory syndrome coronavirus 2 transmission as well as by other legal restrictions in place and, if so, each legal restriction is a collider of others.26
Voluntary behavior is connected to legal restrictions in complex ways. A previous study found evidence of asymmetric results between closures and reopening, with reopening appearing to be considered an all-clear: although there were modest independent effects of stay-at-home orders and closures of restaurants, reopening restaurants was associated with a surge in cases.28 Therefore, as policy makers plan for school reopening, it is imperative that they do so in a way that does not signal to residents that they can safely return to prepandemic behaviors. Proper communication that school reopenings are potentially hazardous but worthwhile for child well-being is one such way to achieve this aim.29,30
This study has some limitations. Data on behavior are from indirect observation using cell phone locations. Although this method has been previously used in the literature and has good validity, it is subject to measurement error, and it is not reported at geographic levels more granular than the state for privacy reasons. The publicly available data therefore miss potentially important within-state variation in behavioral change. We were unable to find any formal study of the validity of Google’s identification of work or home locations.
Local school closures happened in some areas earlier than statewide closures of schools. This analysis, like previous analyses,13 uses the statewide closures. If a substantial number of schools closed earlier in many states, it may undermine the conclusion that behavioral change happened earlier than school closures.
It is essential to be mindful of the limitations of the simulations. Because the growth in COVID-19 cases is exponential, modest changes in the early stages of the epidemic are modeled as having enormous effects over time. Therefore, extrapolating beyond the range of data is risky. Modeled results herein are presented to provide context to the empirical results, but these—and all similar exercises—should be interpreted with caution. If school closures had in fact happened 2 weeks later, the resulting surge in cases may have triggered stronger voluntary behavioral changes that would have blunted the effect of keeping schools open, leading to fewer than the estimated additional 23 000 deaths.
School closures are a blunt instrument for achieving reductions in the spread of any pandemic.31,32 They cause widespread disruption to the economy, including to the health care workforce,33,34 and they are associated with substantial harms to child development.2 Individuals in the US were changing their behavior in large numbers before they were compelled to do so by official policy, suggesting it is possible to achieve the benefits of a social distancing strategy without incurring costs to the most vulnerable members of society.
Given the uncertainty surrounding the severity and transmission dynamics of the disease, the decision to close schools in spring 2020 was reasonable. However, this analysis suggests that school closures were not the factor most strongly associated with slowing the spread of the disease. Other, less harmful, means may be found in the future to effectively limit further spread of COVID-19.
Accepted for Publication: November 18, 2020.
Published Online: February 22, 2021. doi:10.1001/jamapediatrics.2020.6371
Corresponding Author: Frederick J. Zimmerman, PhD, Center for Health Advancement, Department of Health Policy and Management, Fielding School of Public Health at UCLA, PO Box 951772, Los Angeles, CA 90095-1772 (email@example.com).
Correction: This article was corrected on March 8, 2021, to fix errors and add clarification in the Abstract, main text, and Table.
Author Contributions: Dr Zimmerman and Mr Anderson had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: All authors.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by the UCLA Clinical and Translational Science Institute grant TL1TR001883 from the National Institutes of Health/National Center for Advancing Translational Science (Mr Anderson).
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: