US State Restrictions and Excess COVID-19 Pandemic Deaths

Key Points

Question  How did state restrictions affect the number of excess COVID-19 pandemic deaths?

Findings  This cross-sectional analysis including all 50 US states plus the District of Columbia found that if all states had imposed COVID-19 restrictions similar to those used in the 10 most (least) restrictive states, excess deaths would have been an estimated 10% to 21% lower (13%-17% higher) than the 1.18 million that actually occurred during the 2-year period analyzed. Behavior changes were associated with 49% to 79% of this overall difference.

Meaning  These findings indicate that collectively, stringent COVID-19 restrictions were associated with substantial decreases in excess deaths during the pandemic.

Importance  Despite considerable prior research, it remains unclear whether and by how much state COVID-19−related restrictions affected the number of pandemic deaths in the US.

Objective  To determine how state restrictions were associated with excess COVID-19 deaths over a 2-year analysis period.

Design, Setting, and Participants  This was a cross-sectional study using state-level mortality and population data from the US Centers for Disease Control and Prevention for 2020 to 2022 compared with baseline data for 2017 to 2019. Data included the total US population, with separate estimates for younger than 45 years, 45 to 64 years, 65 to 84 years, and 85 years or older used to construct age-standardized measures. Age-standardized excess mortality rates and ratios for July 2020 to June 2022 were calculated and compared with prepandemic baseline rates. Excess death rates and ratios were then regressed on single or multiple restrictions, while controlling for excess death rates or ratios, from March 2020 to June 2020. Estimated values of the dependent variables were calculated for packages of weak vs strong state restrictions. Behavioral changes were investigated as a potential mechanism for the overall effects. Data analyses were performed from October 1, 2023, to June 13, 2024.

Exposures  Age and cause of death.

Main Outcomes  Excess deaths, age-standardized excess death rates per 100 000, and excess death ratios.

Results  Mask requirements and vaccine mandates were negatively associated with excess deaths, prohibitions on vaccine or mask mandates were positively associated with death rates, and activity limitations were mostly not associated with death rates. If all states had imposed restrictions similar to those used in the 10 most restrictive states, excess deaths would have been an estimated 10% to 21% lower than the 1.18 million that actually occurred during the 2-year analysis period; conversely, the estimates suggest counterfactual increases of 13% to 17% if all states had restrictions similar to those in the 10 least-restrictive states. The estimated strong vs weak state restriction difference was 271 000 to 447 000 deaths, with behavior changes associated with 49% to 79% of the overall disparity.

Conclusions and Relevance  This cross-sectional study indicates that stringent COVID-19 restrictions, as a group, were associated with substantial decreases in pandemic mortality, with behavior changes plausibly serving as an important explanatory mechanism. These findings do not support the views that COVID-19 restrictions were ineffective. However, not all restrictions were equally effective; some, such as school closings, likely provided minimal benefit while imposing substantial cost.

US states swiftly responded to the emergence of COVID-19 by implementing a variety of policies including initial emergency declarations, diverse forms of shutdowns and, later, mask and vaccine mandates. The initial actions were almost universal,¹ but opposition to these measures quickly developed.^2,3 This dissent spilled into the political arena where Republican governors and legislatures in multiple states limited the authority of public health officials and banned mask and vaccine mandates.^4,5

Numerous studies have evaluated the impact of policies designed to mitigate the COVID-19 pandemic.^6-8 Early analyses suggested that strong social distancing measures curtailed the initial spread of COVID-19,^9,10 with evidence of interactive effects between public and private mitigation efforts.¹¹ However, the specific modeling assumptions and design criteria of some of these studies have been questioned,^3,12,13 and these analyses generally covered only the initial stages of the pandemic.

Obtaining causal estimates is complicated because of unobserved confounding factors and possible reverse causation, particularly given that the pandemic occurred over a relatively restricted time period and its dynamics are challenging to model. These general issues are exacerbated by preexisting geographic differences in population and employment characteristics, behaviors, and policy environments that affect mortality.^5,14,15 An additional challenge is that initial COVID-19 exposure was concentrated in a small number of locations: primarily New York state, its surrounding states, and Massachusetts. These initial infection rates were probably largely idiosyncratic, reflecting factors such as having high population densities, being travel destinations, and making extensive use of public transportation.^16-18 During this early period, policy variation across states was limited¹ and the lethality of COVID-19 was especially high¹⁹ because medical professionals had little experience with and few tools available to treat severe COVID-19 infections. Consequently, studies containing data for the early pandemic period, such as those examining the overall performance of states²⁰ or groups of policy responses,²¹ faced particular challenges. Reverse causation also may have been problematic given that states with the worst initial conditions were likely to have implemented the most restrictive policies.²² Moreover, restrictions were often introduced as packages, which made it difficult to disentangle specific policy effects.

This analysis focuses on state policies aimed at limiting COVID-19−related deaths during the pandemic. The primary investigation covers the 2-year period July 2020 to June 2022. “Activity limitations” refers to mandated closures of restaurants, bars, and schools; stay-at-home orders; and restrictions on leisure time activities and public gatherings. Other limitations that were examined included general mask requirements and mask and vaccine mandates, as well as prohibitions on these measures. Strategies used to mitigate the estimation challenges described include (1) using 2 methods of accounting for state differences in age-standardized prepandemic death rates to provide likely upper and lower bounds on the restriction effects and (2) beginning the analysis approximately 4 months after the first COVID-19 deaths in the US to mitigate the largely idiosyncratic variation at the start of the pandemic. Of particular interest is the overall impact of packages of “weak” vs “strong” COVID-19 restrictions rather than individual policy effects because policy responses were generally strongly correlated. Lastly, the association of behavioral changes with estimated restriction effects and excess mortality is examined.

This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cross-sectional studies.²³ Because all data were obtained from public sources, this study did not constitute human participant research and did not require institutional review board review or exemption according to the US Department of Health and Human Services (45 CFR §46).

Death rates per 100 000 were computed using population and mortality data obtained from CDC Wonder,^24,25 age-standardized throughout based on 2020 national population shares for younger than 45 years, 45 to 64 years, 65 to 84 years, and 85 years or older. Pandemic deaths could be influenced by a variety of factors, such as differences in educational attainment, the industrial composition of employment, and the general policy environment. Although some of these could potentially be directly controlled for, others would likely not be, resulting in omitted variables biases. To address this, excess death rates, which have been widely used in the literature^26-32 and excess death ratios were constructed.

Excess death rates were computed as actual death rates divided by baseline death rates during July 2017 to June 2019, a corresponding 2-year period before the COVID-19 pandemic. These accounted for determinants of cross-state differences in levels of baseline mortality, to the extent that they persisted during the pandemic. However, because death rates rose with the emergence of COVID-19, such adjustments in levels could be insufficient because equal percentage changes would raise rates by larger absolute amounts in states with higher rather than lower prepandemic mortality rates. If so, because states with lower baseline death rates adopted more restrictive policies, using excess death rates as the dependent variable might overstate the benefits of restrictions.

An alternative was to analyze excess death ratios computed as actual deaths divided by death baseline rates. These allowed for proportional adjustments in pandemic effects but were likely to underestimate the effectiveness of restrictions when the determinants of baseline and pandemic death rates differed. For example, persons living in nursing homes, areas with high population density, or making extensive use of public transportation may have been more vulnerable to COVID-19,^16,17,33 while these factors were less closely associated with prepandemic mortality. In combination, analysis of excess death rates and ratios provided likely upper and lower bounds on the restriction effects.

Data on 7 state activity limitations were obtained from the Institute for Health Metrics and Evaluation (IHME),³⁴ including stay-at-home orders; restaurant, bar, leisure activity, primary school, and higher education closures; and restrictions on public gatherings. The percentage of days during which the limitations were in effect from July 2020 to June 2022 was calculated, and primary analysis focused on the population-weighted average of the 7 measures. Alternative sources of activity limitations were examined and shown to have similar patterns to those analyzed in this study.

The percentage of the analysis period during which all states required individuals to wear masks when outside of the home was calculated using information from Ballotpedia³⁵; for the District of Columbia, data were from the IHME.³⁴ Kaiser Family Foundation data³⁶ were used to construct variables indicating whether state government or school employees were required to be vaccinated as of February 2022, whether there were school mask requirements, and whether there were prohibitions on government vaccine or school mask mandates.

Three behaviors were examined. CDC vaccination data³⁷ were used to calculate the percentage of the population completing a primary COVID-19 vaccination series. The percentage of days individuals stated that they always wore a mask outside their home (mask use) was obtained from IHME.³⁴ Reductions in time outside the home compared with the prepandemic period (mobility reductions) came from Opportunity Insights.³⁸

Descriptive analysis was conducted to evaluate patterns of activity limitations, mask requirements, behaviors, excess death rates, and ratios from March 2020 to December 2022. This analysis demonstrated the concentration of initial COVID-19 pandemic deaths in a few areas, despite the almost universal imposition of activity limitations, motivating use of the July 2020 to June 2022 analysis period. A composite restriction score was constructed as the sum of the standardized (mean [SD], 0 [1]) activity limitations score, and standardized mask requirements plus dummy variables for state and school vaccination and school mask requirements and minus the dummy variable values for vaccination and mask prohibitions.

Age-standardized excess death rates or ratios during the 2-year period, July 2020 to June 2022 (1 observation per state), were next regressed on single or multiple restrictions, controlling for excess death rates or ratios during the preanalysis period (March 2020-June 2020) in case later outcomes were influenced by early COVID-19 exposure, although preliminary analysis indicated this was only a minor issue. Observations were weighted by 2020 state populations, and confidence intervals were based on robust standard errors.

Given the collinearity between policies, the single restriction estimates likely provided upper bounds on magnitudes of the restriction effects. Therefore, particular emphasis was placed on results for packages of weak and strong restrictions, where the former (latter) were calculated using population-weighted averages for the 10 states with the lowest (highest) overall restriction scores. Specifically, weak restrictions referred to activity limitation scores and mask requirements of 0.80 and 1.41 SDs less than the national average; no vaccine or mask mandates; and vaccine and school mask prohibition values of 0.79 and 0.75, respectively. Strong restrictions referred to activity limitation scores and mask requirements of 1.07 and 1.26 SDs, respectively, greater than the national average; universal school and state vaccination mandates and school mask requirements; and no vaccine or mask mandate prohibitions (eTable 1 in Supplement 1).

The association between behavioral changes and restriction effects was also examined by regressing (1) the 3 behaviors on restrictions and (2) excess death rates or ratios on the estimated difference in behaviors associated with weak vs strong restrictions, with these then compared with the overall restriction effect. Additional details on the methods are provided in Supplement 1.

Statistical tests were 2-tailed and P <.05 were considered statistically significant. Data analyses were performed from October 1, 2023, to June 13, 2024, using Stata, release 18.0 (StataCorp LLC).

In the US, COVID-19 deaths emerged in March 2020. After subsiding, they became more numerous and geographically distributed from October 2020 through March 2021, with additional smaller peaks near the end of 2021 and beginning of 2022 when the delta and omicron variants emerged (Figure 1A). The pattern of excess death rates was nearly identical (eFigure 1 in Supplement 1). The initial surge, from March 2020 to May 2020, was highly concentrated in 4 states (Connecticut, Massachusetts, New Jersey, and New York) containing 12% of the nation’s population but accounting for half of the COVID-19 fatalities (Figure 1B; eTable 2 in Supplement 1). During this period, there was little variation in COVID-19 restrictions. For instance, all states had declared a state of emergency by March 15, 2020,³⁹ the composite activity limitations score averaged 96.2 (of a maximum of 100) by April 7, 2020, and there were strong simultaneous behavioral responses, including large mobility reductions and increases in mask use (Figure 2A and B).

Considerable policy variation emerged in the second half of 2020, with states reducing or eliminating activity limitations and, somewhat later, mask requirements. Mobility reductions also declined rapidly during this period, as did mask use after the start of 2021. Vaccinations first became available in December 2020 and quickly became widespread, but with considerable geographic heterogeneity. Activity limitations had been essentially phased out by June 2021 and mask requirements by March 2022, at which point 23 and 11 states had instituted vaccination mandates for state and school employees, respectively, and 15 states had mandated masks in schools. Conversely, 13 states had prohibited vaccination mandates and 7 had outlawed school mask requirements (eTable 3 in Supplement 1).

With the exception of Hawaii, the states with the lowest excess death rates and ratios were in the Northeast region, while a mix of Southern and Western states had the highest rates and ratios (Figure 2C and D). At the extreme, the excess death rate in Massachusetts was less than one-fifth that of Mississippi (115 vs 590 per 100 000). The poor performance of Southern states was modestly mitigated when examining excess death ratios because of their high prepandemic death rates, but the population-weighted correlation between rates and ratios was high (0.91) and the rank ordering of states was largely unchanged. Relative performance was also similar when examining COVID-19 rather than all-cause excess death rates (eTable 4 in Supplement 1).

Figure 2E and F show monthly excess death rates and ratios for states with composite restriction scores greater than or less than the national median. Stronger restrictions were associated with lower monthly excess death rates and ratios in virtually every month from July 2020 to June 2022, with the largest gaps during the second half of 2020, when the virus first became widespread, and in September through December 2021 when the delta variant became dominant. However, strong restriction states had much higher excess death rates and ratios in the preanalysis period (March 2020 to June 2020). This early COVID-19 exposure may have lowered subsequent mortality risk; however, additional analysis revealed that these effects were minimal (eFigure 2 in Supplement 1).

In regression models controlling for a single restriction variable, mask requirements and vaccine or school mask mandates were associated with reductions in estimated excess death rates and ratios; prohibitions on mask or vaccine mandates were positively associated with them; whereas activity limitations were largely unrelated (Figure 3A and B; eTables 5 and 6 in Supplement 1). Restrictions were associated with greater use of the 3 preventive behaviors (Figure 3C and D; eTable 5 in Supplement 1). These, in turn, were negatively correlated with excess death rates and, for vaccinations, with lower excess death ratios (eTable 7 in Supplement 1).

Of primary interest were predicted effects of packages of strong vs weak restrictions, with the difference between them denoted as the “restriction premium” where negative numbers indicated lower excess death rates or ratios. Strong restrictions were associated with more favorable outcomes, including an excess death rate of 282 per 100 000 over the 2-year period that was 135 per 100 000 (32%) less than the 417 per 100 000 estimated for weak restrictions; similarly, the restriction premium was −0.048 for excess death ratios (0.188 vs 0.236), a 20% difference (Table and Figure 4). The restriction premium was larger, as a percentage of national baseline mortality rates, for excess death rates than ratios (−0.079 vs −0.048) consistent with these 2 estimates bounding the actual effects.

These results were robust to sensitivity checks including separately controlling for the 7 specific activity limitations rather than using the composite score; incorporating the overall restriction score as the single policy covariate; excluding the control for the initial pandemic period (March 2020 to June 2020) excess death rates or ratios; estimating the model over the full pandemic period (March 2020 to June 2022) with Connecticut, Massachusetts, New Jersey, and New York excluded; and using COVID-19 mortality, rather than all excess deaths, as the dependent variable (eTable 8 in Supplement 1). Larger restriction effects were also observed in the second analysis year (July 2021- June 2022) than in the first (July 2020 to June 2021).

Using the primary specifications, the estimates suggested that if all states had weak restrictions, there would have been 1.3 million to 1.4 million excess deaths from July 2020 to June 2022, that is, 271 000 to 447 000 more than estimated with universal strong restrictions, a 25% to 48% difference (Table). Compared to the 1.18 million actual excess deaths over the study period, weak restrictions in all states would have resulted in an estimated 13% to 17% increase, and strong restrictions in a 10% to 21% decrease. Behavioral changes were associated with 49% to 79% of the overall restriction premia (Table and Figure 4; eTable 9 in Supplement 1).

COVID-19 restrictions were associated with substantial reductions in excess pandemic deaths in the US. If all states had weak restrictions, as defined in the Methods section, estimated excess deaths from July 2020 to June 2022 would have been 25% to 48% higher than if all had imposed strong restrictions. Behavioral responses provided a potentially important mechanism for this, being associated with 49% to 78% of the overall difference.

This study’s methods departed from prior related research^20,21,40,41 in 2 particularly important ways. First, the analysis started in July 2020, 4 months after the first substantial COVID-19 fatalities in the US. There was little policy variation during these months, and deaths during this period were probably largely idiosyncratic—reflecting factors such as initial infections in areas that were travel destinations—and so were not strongly reflective of these policies, which also likely impacted mortality with a delay. This is important given that states with high initial mortality rates adopted or retained relatively restrictive policies and had low subsequent excess death rates. Second, while earlier studies used various methods of controlling for potential confounding factors, these efforts were almost certainly incomplete, leaving open the possibility of considerable omitted variables bias. This was partially addressed by examining excess death rates and ratios, which provided likely upper and lower bounds on the restriction effects.

These study findings do not support the views of those opposing COVID-19 restrictions who erroneously believe the restrictions did not work. To the contrary, the package of policies implemented by some states probably saved many lives. However, all restrictions were not equally successful. Evidence of protective effects was weakest for activity limitations. This may have partially concealed benefits at the start of the pandemic, before the analysis period, or reflected reverse causation given that most activity limitations were implemented before political considerations dominated, when states were likely to be especially responsive to local variations in infection rates.

Mortality reductions were not necessarily sufficient to justify imposing restrictions because they also imposed a variety of costs. Some of these, such as the “loss of liberty,” reflect normative beliefs that are difficult to precisely value. However, using value of statistical life estimates ranging from $4.7 million to $11.6 million,^42,43 the estimated lives saved from strong (vs weak) restrictions over the 2-year period were worth $1.3 trillion to $5.2 trillion—6% to 22% of 2021 gross domestic product—providing a possible benchmark against which to evaluate this loss. Supplement 1 details these calculations.

Some harms were specific and potentially more measurable. Prior research indicates that school closings hurt educational outcomes.^44-46 The absence of evidence that these also reduced pandemic deaths suggests that they may have been too aggressively pursued in some states. On the other hand, school mask mandates were probably more effective and imposed lower costs. Similarly, the social isolation experienced by nursing home residents was harmful,⁴⁷ raising questions about activity limitations that restricted personal interactions for this group.

This study had limitations. First, even with its methodological innovations, the cross-sectional design makes it difficult to identify causation. Second, with only 51 observations (1 for each state plus the District of Columbia), the data were not robust enough to include controls for other state demographic or policy characteristics, although these effects were partially accounted for, to the extent that they influenced baseline mortality. Third, although the restrictions examined were more comprehensive than in prior analyses, they were incomplete in a variety of ways. For example, they did not account for restrictions imposed at the substate level (eg, by cities and school districts). The resulting measurement error may have biased the policy effects toward zero. Conversely, they may have captured the effects of related policies that were not controlled for (eg, state or school vaccine requirements may have been correlated with other types of vaccine mandates in the state). Similarly, data were not available on the exact timing of some policies, requiring the use of dummy variables rather than more precise measures. This also precluded examination of the importance of the specific timing of policy implementation. Fourth, a limited set of behaviors was analyzed. Some of these were self-reported, potentially with error, and they may also have been correlated with omitted factors affecting policy responses in the state. Fifth, the calculation of excess deaths was inexact, although qualitatively similar results were obtained when focusing on fatalities with COVID-19 as the underlying cause. Lastly, state-level policies that were not directly related to the pandemic were not considered, except to the extent these affected baseline death rates.

This cross-sectional analysis suggests that strong COVID-19 restrictions saved lives and that the death toll was probably considerably higher than it would otherwise have been in states that resisted imposing these restrictions, banned their use, or implemented them for only relatively short periods of time. These findings may be relevant for public health approaches addressing future pandemics and provide methodological approaches that may be useful for calculating excess deaths in a variety of situations.

Accepted for Publication: May 21, 2024.

Published: July 26, 2024. doi:10.1001/jamahealthforum.2024.2006

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2024 Ruhm CJ. JAMA Health Forum.

Corresponding Author: Christopher J. Ruhm, PhD, University of Virginia, 235 McCormick Rd, PO Box 400893, Charlottesville, VA 22904-4893 (ruhm@virginia.edu).

Author Contributions: Dr Ruhm had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Ruhm.

Acquisition, analysis, or interpretation of data: Ruhm.

Drafting of the manuscript: Ruhm.

Critical review of the manuscript for important intellectual content: Ruhm.

Statistical analysis: Ruhm.

Obtained funding: Ruhm.

Administrative, technical, or material support: Ruhm.

Supervision: Ruhm.

Conflict of Interest Disclosures: Dr Ruhm reported consulting fees from opioid litigation outside the submitted work.

Data Sharing Statement: See Supplement 2.