Variation in US Hospital Mortality Rates for Patients Admitted With COVID-19 During the First 6 Months of the Pandemic

Key Points

Question  Are hospital outcomes for patients with coronavirus disease 2019 (COVID-19) improving?

Findings  In this cohort study of 38 517 adults who were admitted with COVID-19 to 955 US hospitals, rates of 30-day mortality or referral to hospice varied from 9.06% to 15.65% in the best- and worst-performing quintiles. In the early months of the pandemic, 94% of hospitals in a subset of 398 improved by at least 25%, and the strongest determinant of improvements in hospital-level outcome was a decline in community rates of infection.

Meaning  All else being equal, COVID-19 mortality in hospitals seems to be lower when the prevalence of COVID-19 in their surrounding communities is lower.

Importance  It is unknown how much the mortality of patients with coronavirus disease 2019 (COVID-19) depends on the hospital that cares for them, and whether COVID-19 hospital mortality rates are improving.

Objective  To identify variation in COVID-19 mortality rates and how those rates have changed over the first months of the pandemic.

Design, Setting, and Participants  This cohort study assessed 38 517 adults who were admitted with COVID-19 to 955 US hospitals from January 1, 2020, to June 30, 2020, and a subset of 27 801 adults (72.2%) who were admitted to 398 of these hospitals that treated at least 10 patients with COVID-19 during 2 periods (January 1 to April 30, 2020, and May 1 to June 30, 2020).

Exposures  Hospital characteristics, including size, the number of intensive care unit beds, academic and profit status, hospital setting, and regional characteristics, including COVID-19 case burden.

Main Outcomes and Measures  The primary outcome was the hospital’s risk-standardized event rate (RSER) of 30-day in-hospital mortality or referral to hospice adjusted for patient-level characteristics, including demographic data, comorbidities, community or nursing facility admission source, and time since January 1, 2020. We examined whether hospital characteristics were associated with RSERs or their change over time.

Results  The mean (SD) age among participants (18 888 men [49.0%]) was 70.2 (15.5) years. The mean (SD) hospital-level RSER for the 955 hospitals was 11.8% (2.5%). The mean RSER in the worst-performing quintile of hospitals was 15.65% compared with 9.06% in the best-performing quintile (absolute difference, 6.59 percentage points; 95% CI, 6.38%-6.80%; P < .001). Mean RSERs in all but 1 of the 398 hospitals improved; 376 (94%) improved by at least 25%. The overall mean (SD) RSER declined from 16.6% (4.0%) to 9.3% (2.1%). The absolute difference in rates of mortality or referral to hospice between the worst- and best-performing quintiles of hospitals decreased from 10.54 percentage points (95% CI, 10.03%-11.05%; P < .001) to 5.59 percentage points (95% CI, 5.33%-5.86%; P < .001). Higher county-level COVID-19 case rates were associated with worse RSERs, and case rate declines were associated with improvement in RSERs.

Conclusions and Relevance  Over the first months of the pandemic, COVID-19 mortality rates in this cohort of US hospitals declined. Hospitals did better when the prevalence of COVID-19 in their surrounding communities was lower.

One year after identifying the first cases of coronavirus disease 2019 (COVID-19) in Wuhan, China, we have little understanding of how mortality rates vary by hospital or whether mortality rates are improving. Hospital-level mortality may depend not just on patient risk factors, but also on the hospital where patients are admitted. Individual-level and hospital-level mortality rates are also likely to improve over time with increasing experience with the disease and as new treatments become available. We used data from a large national health insurer in the US to estimate the variation in hospital-level mortality among patients hospitalized with COVID-19 to examine how mortality rates changed and identify hospital-level characteristics that were associated with those mortality rates and their change.

We used deidentified administrative claims data from a large national health insurer in the US that were linked with a daily record of patients who were admitted to a hospital with a primary or secondary diagnosis of COVID-19 (eTable 3 in the Supplement) and their current status (admitted, discharged, transferred, or dead) until June 30, 2020, within the UnitedHealth Group Clinical Discovery Database. Data specific to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and COVID-19 underwent an additional check for errors in sampling and data collection described in the eAppendix in the Supplement. We obtained hospital-level characteristics from the 2018 American Hospital Association Annual Survey,¹ 2020 Medicare Impact,² and 2020 Provider of Service files.³ We obtained cumulative COVID-19 case rates for January 1 to April 30, 2020, and May 1 to June 30, 2020 in the county of each hospital in our sample from The New York Times database.⁴ This study was reviewed and deemed exempt by the institutional review board of UnitedHealth Group.

We started with all Medicare Advantage and commercial enrollees 18 years or older hospitalized with COVID-19 since January 1, 2020 (eFigure 1 in the Supplement). We then excluded patients with fewer than 6 months of insurance enrollment in 2019 (to permit comorbidity measurement using historical claims), were readmitted to or transferred to another facility within 30 days of initial admission (to prevent misattributing hospital-level outcomes among patients who were admitted to multiple hospitals), were admitted with COVID-19 after June 30, 2020 (to provide sufficient follow-up to measure 30-day event rates to July 31), and were admitted to hospitals with missing information or fewer than 10 patients (to improve the statistical reliability of our hospital-level mortality estimates). The hospitals where these patients were admitted constituted our main study sample.

To reflect an outcome that was close to a 30-day any-site mortality rate, we measured a hospital’s risk standardized event rate (RSER), which reflected a composite of either inpatient mortality or referral to hospice within 30 days of initial admission for COVID-19 based on the National Quality Forum–approved hierarchical generalized linear models while accounting for clustering of patients within hospitals.⁵ We considered the composite of mortality or hospice referral as a more complete representation of the outcome of interest. We repeated all analyses using mortality alone (eFigures 2 and 3 in the Supplement).

The hierarchical models to estimate the RSER used the methods of George et al⁶ and Silber et al^7-9 and are detailed in the eAppendix and eMethods in the Supplement. We modeled the log odds of patient events as a function of patient-level clinical and demographic variables, including age, sex, Elixhauser comorbidity indicators (eTable 3 in the Supplement),¹⁰ nursing facility admission source, and the number of days between January 1, 2020, and admission. The latter variable was chosen to account for likely improvements in patient outcomes as hospitals gained experience with COVID-19. As a sensitivity analysis, we also calculated the RSER for each hospital using the methods of Drye et al¹¹ and Normand et al.¹² The estimates from these measures were nearly identical (eFigure 9 in the Supplement). We computed each hospital’s RSER by calculating the average of the predicted risk of mortality or referral to hospice for all patients in the sample as if they had (hypothetically) been treated at this hospital.^10-13 This approach uses a fixed patient population to fairly compare hospitals.

To examine changes in hospital RSERs, we repeated these analyses in the subset of hospitals with 10 or more patients admitted for COVID-19 in the early and later part of the observation period using a period indicator variable. We again used a fixed patient population: each hospital’s RSER was calculated as the average of the predicted risk of mortality or referral to hospice for all patients in the subsample as if they had (hypothetically) been treated at this hospital during this period. In overall, early-, and later-period analyses, we ordered hospitals into quintiles based on their RSERs. We used paired Wilcoxon sign rank tests to compare differences in RSERs across periods. We visualized each hospital’s change in RSER using a Bland-Altman plot.¹³

We estimated linear regressions to measure associations between hospital-level characteristics, including the number of beds, number of intensive care unit beds, profit status, academic hospital status, hospital setting (urban/nonurban), and a hospital’s (1) RSER in the sample of 955 hospitals and (2) difference in RSER between the early and late period in the sample of 398 hospitals. The regressions included the census region of the hospital location as a fixed effect and a measure of the COVID-19 case load in the hospital’s county, which was measured as the number of cumulative cases per 10 000 residents for January 1 to April 30, 2020, in the early period and May 1 to June 30, 2020, in the late period. They also include a binary indicator to note if cases increased during the late period.

All statistical tests were 2-sided, with a significance level of P > .05. All analyses were conducted using R, version 3.6.3 (R Foundation).¹⁴ All statistical code is included in the eMethods in the Supplement.

A total of 955 hospitals were included, reflecting 38 517 inpatients with COVID-19 who were admitted between January 1, 2020, and June 30, 2020 (Table 1) from 43 states and Washington DC. Overall, 3179 patients (8.25%) died, and 1433 patients (3.7%) were referred to hospice. The mean (SD) hospital-level risk-standardized rate of 30-day inpatient mortality or referral to hospice was 11.82% (2.50%) (eFigure 4 in the Supplement). In hierarchical models, several individual-level risk factors were strongly associated with the odds of 30-day inpatient mortality or referral to hospice. Men had odds 1.29 times higher than women (95% CI, 1.20-1.38; P < .001), patients older than 85 years had odds 14.52 times higher than those aged 18 to 45 years (95% CI, 10.75-19.61; P < .001), patients transferred from a nursing facility had odds 2.43 times higher than those admitted from the community (95% CI, 2.22-2.65; P < .001), patients with metastatic cancer had odds 1.85 times higher than those without (95% CI, 1.57-2.18; P < .001), and patients admitted within the first 90 days of 2020 had odds 2.58 times higher than those admitted 150 days or more into 2020 (95% CI, 2.29-2.90; P < .001) (eFigure 2A in the Supplement). Results that used mortality as the sole outcome were similar (eFigure 2B in the Supplement).

The RSERs varied considerably across hospitals, ranging from 5.70% to 24.65% (Figure 1). The mean RSER in the worst-performing quintile of hospitals was 15.65% compared with 9.06% in the best-performing quintile of hospitals (absolute difference, 6.59 percentage points; 95% CI, 6.38%-6.80%; P < .001) (Table 2; eTable 1 in the Supplement). Variation was substantial regardless of the county-level cumulative COVID-19 case burden (eTable 2 in the Supplement). Hospital ranks based on the composite of mortality or hospice referral and on mortality alone were statistically significantly correlated (Kendall rank correlation coefficient, 0.628; P < .001) (eFigure 3 in the Supplement).

A total of 398 hospitals (41.7%) had sufficient patient volume during the early (admitted January 1, 2020, to April 30, 2020) and late (admitted May 1, 2020, to June 30, 2020) periods to compare mortality rates. The 27 801 patients (72.2%) in this subset had characteristics similar to the overall set of patients, although hospitals were slightly more likely to be larger and academic (Table 1). The overall mean (SD) RSER declined from 16.56% (3.99%) to 9.29% (2.08%) (Figure 1; eFigures 5, 6, and 7 in the Supplement). The mortality rate of all hospitals but 1 improved: 281 hospitals (70.6%) revealed a 25% to 50% reduction in event rates, and 95 hospitals (24.0%) revealed more than a 50% reduction in event rates (Figure 2). The individual positions of hospitals in the rankings changed between periods but were correlated (Kendall rank correlation, 0.4731; P < .001), suggesting that better-performing hospitals continued to be better performers (eFigure 10 in the Supplement). From the early to late period, the absolute difference in the rates of mortality or referral to hospice between the worst- and best-performing quintiles of hospitals decreased from 10.54 percentage points (95% CI, 10.03%-11.05%; P < .001) to 5.59 percentage points (95% CI, 5.33%-5.86%; P < .001) (eTable 1 in the Supplement). Even in the later period, the RSER in the worst-performing quintile of hospitals was 12.47% compared with 6.88% in the best-performing quintile of hospitals (Table 2).

We found no association between number of intensive care unit beds, academic status, profit status, or urban/nonurban setting and a hospital’s RSER (eFigure 8 in the Supplement) except that medium to large hospitals, hospitals in the Northeast, and hospitals with high county-level COVID-19 case rates had worse RSERs. These results were largely sustained in a hierarchical model, which simultaneously reflected patient and hospital attributes (eFigure 11 in the Supplement). The characteristic that was associated with the largest change in a hospital’s RSER over the 2 periods was the COVID-19 burden in the community; higher early-period community case rates were associated with improvements in RSER, and increases in community case rates were associated with worsening RSER after adjusting for other factors (Figure 3).

This study had 4 main findings. First, mortality rates for patients with COVID-19 varied significantly across US hospitals. Second, RSERs in all but 1 hospital improved over the first 6 months of the pandemic. Third, while absolute mortality differences across hospitals declined, wide differences persisted. Fourth, the characteristic most associated with RSER and its change was the community case rate of COVID-19; high county-level case rates were associated with worse RSERs and with improvements in RSERs over time. Increases in county-level case rates were associated with worsening RSERs.

This study extended past findings of variation in outcomes and improvement over time.^15,16 It also confirmed individual-level risk factors that were previously identified, including advanced age, male sex, medical comorbidities, and nursing facility sources.

A central finding of this study was that 94% of hospitals had a relative reduction in COVID-related mortality rates of more than 25% in just a few months. That rate of relative improvement is striking and encouraging, but perhaps not surprising. Early efforts at treating patients with COVID-19 were based on experience with previously known causes of severe respiratory illness. Later efforts could draw on experiences specific to SARS-CoV-2 infection. Remdesivir received a US Food and Drug Administration emergency use approval on May 1, 2020,¹⁷ the start of the study’s later period, although, to our knowledge, a survival benefit has not been shown.¹⁸ A June 16, 2020, statement from the RECOVERY investigators reported a substantial survival benefit from dexamethasone in selected patients with respiratory failure.^19,20 Considerable changes in inpatient management were tried (eg, early vs late assisted ventilation, differences in oxygen flow, prone or supine positioning, and anticoagulation). Those efforts varied in how systematically they were evaluated, but our results suggest that valuable experience was gained. Another possible reason for improvement includes greater use of masks which, theoretically, could reduce the viral inoculum and perhaps disease severity.²¹ In general, health outcomes improve with time, but this novel viral infection and our early access to a large set of patients provided an opportunity to see rapid improvement.

Despite these widespread improvements for nearly all hospitals, this study also revealed large differences in mortality or referral to hospice between the best- and worst-performing hospitals. Decades of quality measurement often reveal differences in outcomes across hospitals.^22-24 The large differences observed in this study could reflect large differences in fundamental quality, but they could also reflect different admission thresholds across hospitals. For example, although we could adjust for differences in medical comorbidities, COVID-19 presents heterogeneously, and we could not adjust for differences in the manifestations of COVID-19 itself.

This study has additional limitations. First, the calculated event rates reflect patients from a single insurer and, therefore, also a limited set of hospitals. Nevertheless, this study reflects what is to our knowledge the largest and most comprehensive sample of US patients with COVID-19 to date, covering commercially and Medicare-insured populations. Second, we are unable to measure out-of-hospital mortality. However, most COVID-19 mortality among inpatients occurs in the hospital and so should be observable in our data. We used the composite outcome of death or referral to hospice within 30 days to reflect any-site mortality more comprehensively. That composite reflects what is likely a more complete assessment of the outcome of interest than used in studies that are restricted to inpatient mortality. Nevertheless, the fundamental findings of this study were preserved in sensitivity analyses that used mortality alone as the outcome. Third, we did not measure morbidity and disability outcomes among survivors that may be meaningful. Fourth, to measure county-level disease burden, we used cumulative reported case rates that could be sensitive to varied testing availability and use. However, these values were highly correlated with death rates 1 month later, which would be less sensitive to testing availability or use (correlation coefficient, 0.881; P < .001). Fifth, in using insurance claims-based information, we were unable to examine processes of care that may also help to explain variation in outcomes.

This study also has strengths. It represents a geographically and sociodemographically diverse group of 38 517 patients and 955 hospitals, allowing confidence in the estimation of individual-level patient factors associated with mortality, and variation in hospital mortality rates and their changes.

This study revealed that outcomes for patients with COVID-19 rely not only on individual-level risk factors, but also on the hospital where care is received. This study also revealed that during the first 6 months of the COVID-19 pandemic, mortality rates in US hospitals declined sharply. Nevertheless, the characteristic that is most associated with poor or worsening hospital outcomes is high or increasing community case rates. The association between high community COVID-19 case loads and both worse RSERs and greater improvement in RSERs suggests hospitals do worse when they are burdened with cases and is consistent with imperatives to flatten the curve. As case rates of COVID-19 increase across the nation, hospital mortality outcomes may worsen.

Accepted for Publication: November 14, 2020.

Published Online: December 22, 2020. doi:10.1001/jamainternmed.2020.8193

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Asch DA et al. JAMA Internal Medicine.

Corresponding Author: David A. Asch, MD, Center for Health Care Innovation, 3400 Civic Center Blvd, PCAM South Tower 14-171, Philadelphia, PA 19104 (asch@wharton.upenn.edu).

Author Contributions: Dr Islam and Mr Buresh had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Sheils, Islam, Werner, Buresh, Chen.

Supervision: Sheils, Werner, Doshi.

Conflict of Interest Disclosures: Dr Doshi reported personal fees from AbbVie, Boehringer Ingelheim, Janssen, Kite Pharma, and Merck and grants from AbbVie, Janssen, Novartis, Merck, Pfizer, PhRMA, Regeneron, and Sanofi and outside the submitted work. No other disclosures were reported.