Mortality for Publicly Reported Conditions and Overall Hospital Mortality Rates

Importance  Federal efforts about public reporting and quality improvement programs for hospitals have focused primarily on a small number of medical conditions. Whether performance on these conditions accurately predicts the quality of broader hospital care is unknown.

Objective  To determine whether mortality rates for publicly reported medical conditions are correlated with hospitals’ overall performance.

Methods  Using national Medicare data, we compared hospital performance at 2322 US acute care hospitals on 30-day risk-adjusted mortality, aggregated across the 3 publicly reported conditions (acute myocardial infarction, congestive heart failure, and pneumonia), with performance on a composite risk-adjusted mortality rate across 9 other common medical conditions, a composite mortality rate across 10 surgical conditions, and both composites combined. We also examined the relationship between alternative surrogates of quality (hospital size and teaching status) and performance on these composite outcomes.

Results  Our sample included 6 670 859 hospitalizations for Medicare fee-for-service beneficiaries from 2008 through 2009. Hospitals in the top quartile of performance on publicly reported conditions had a 3.6% lower absolute risk-adjusted mortality rate on the combined medical-surgical composite than those in the bottom quartile (9.4% vs 13.0%; P < .001). These top performers on publicly reported conditions had 5 times greater odds of being in the top quartile on the overall combined composite risk-adjusted mortality rate (odds ratio [OR], 5.3; 95% CI, 4.3-6.5). Mortality rates for the index condition were predictive of medical (OR, 8.4; 95% CI, 6.8-10.3) and surgical (2.7; 2.2-3.3) performance when these groups were considered separately. In comparison, large size (OR, 1.9; 95% CI, 1.5-2.4) and teaching status (2.4; 1.8-3.2) showed weaker relationships with overall hospital mortality rates.

Conclusions and Relevance  Hospital performance on publicly reported conditions can potentially be used as a signal of overall hospital mortality rates.

Public reporting of hospital performance has emerged as a central strategy for improving the quality of US hospital care. The goals of public reporting include providing consumers with information so they can make choices about where to seek care, giving feedback to providers about their own performance, and providing payers with performance targets for financial incentives. Public reporting of clinical outcomes such as mortality rates has become increasingly important, in part because outcomes are easily interpretable by both consumers and policy makers and represent meaningful end results of hospital care. Starting in 2014, the Centers for Medicare & Medicaid Services, the largest payer in the nation, will begin using mortality rates as a key measure in determining hospital payments through its Value-Based Purchasing program.¹

A serious concern with current public reporting and pay-for-performance programs, however, is that they focus on only 3 conditions: acute myocardial infarction, congestive heart failure, and pneumonia.² While these conditions are common, together they account for only 13% of all hospitalizations for the elderly (and an even smaller proportion of care for the nonelderly).³ It is unclear whether high performance on these 3 conditions is an adequate measure for the overall performance of a hospital. Although recent studies have shown mortality rates for the 3 index conditions to be moderately well correlated among themselves,⁴ the degree to which these conditions reflect performance on a broader range of medical and surgical conditions is unknown.

Understanding this issue is critical for 3 reasons. First, the degree to which performance on certain marker conditions correlates with broader hospital performance can help policy makers determine whether public reporting and pay-for-performance programs should be based more broadly. Second, for consumers, this information can provide important insights into how they are likely to fare if they choose a hospital based on its performance on this set of conditions. Finally, for clinical and quality experts, these data can answer the critical question of whether there really is such a thing as a “good hospital” and provide insights into why performance seems to vary so greatly across institutions.

Therefore, in this study, we sought to answer 3 questions: first, to what extent do mortality rates for high performance on the reported conditions help consumers identify hospitals that are high performers overall or avoid hospitals that are low performers overall? Second, what is the absolute mortality gradient associated with this effect? Finally, how well do aggregate mortality rates for the 3 reported conditions compare with more traditional proxies for hospital quality, such as size (as an indicator of volume)^5,6 and major teaching status,⁷ in identifying high-quality hospitals?

We used the Medicare Provider Analysis and Review files to identify all elderly fee-for-service enrollees admitted to a nonfederal acute care hospital during 2008 or 2009. We began with 4580 acute care hospitals that treated Medicare patients in the 50 states or the District of Columbia and excluded all 1270 Critical Access Hospitals because they are not required to participate in federal public reporting and their sample sizes are generally small. We further excluded 988 acute care hospitals with fewer than 25 admissions for our medical or surgical condition of interest. Our final sample of 2322 hospitals provided 90.3% of all acute care for Medicare fee-for-service patients in the United States. We used the American Hospital Association survey⁸ from 2009 to obtain data on hospital characteristics, including size, teaching status (membership in the Council of Teaching Hospitals), ownership, geographic region, proportions of Medicare and Medicaid patients, and nursing ratios. The hospitals in our database represent 93% of all hospitals reporting to the American Hospital Association. We obtained data on the communities in which the hospitals were located using the 2009 Area Resource File.⁹

Our primary predictor was an aggregate of 30-day mortality rates for the 3 publicly reported conditions. We identified all hospitalizations with primary discharge diagnoses of acute myocardial infarction, congestive heart failure, or pneumonia (see the International Classification of Diseases, 9th Revision, Clinical Modification codes for each condition in eTables 1 and 2 in the Supplement. For every condition, each patient’s likelihood of death within 30 days of admission was adjusted for urgency of admission, age, sex, race, and 29 comorbid medical conditions using the Medicare risk adjustment model developed by the Agency for Healthcare Quality and Research.¹⁰ We then created an aggregate mortality rate for each hospital using indirect standardization, a method for creating composite outcomes that takes into account differences in the mix of conditions at each hospital as well as hospitals’ performance for each condition (see eMethods in Supplement for details).¹¹ As is customary in most studies examining hospital mortality rates, we assigned patients who were transferred to the admitting hospital.

Our outcomes of interest were composite medical and surgical 30-day mortality rates. We first built medical and surgical composite mortality rates separately and then, for our primary outcome, combined both to create a single composite score for the hospital. To select the conditions used in the medical composite rate, we took the 15 most common diagnosis-related groups among all US hospitals and excluded the 3 publicly reported conditions. Next, we excluded 3 other diagnosis-related groups because they had large clinical heterogeneity (metabolic or nutritional disorder), had low mortality rates (chest pain), or were primarily managed surgically (hip fracture). The medical composite rate was calculated across the remaining 9 conditions: stroke, arrhythmia, chronic obstructive pulmonary disease, respiratory tract infection, sepsis, urinary tract infection, gastrointestinal bleed, renal failure, and esophagitis or gastroenteritis. Because respiratory tract infections are clinically related to pneumonia, as a sensitivity analysis, we re-created the medical composite mortality rate without that condition, which had no qualitative effect on our findings. All patients included in the analysis were identified using the appropriate International Classification of Diseases, 9th Revision, Clinical Modification primary diagnosis codes (eTable 1 in Supplement) corresponding to our conditions of interest.

In building a surgical composite mortality rate, we considered 25 common general surgery and cardiothoracic procedures and selected those performed, on average, at least 10 000 times per year across the Medicare population with a median mortality of more than 2%. This produced 7 procedures: coronary artery bypass grafting, aortic valve repair, above-knee amputation, colon resection, small bowel resection, exploratory laparotomy, and pulmonary lobe resection. To this list we added 3 procedures classified by the Agency for Healthcare Quality and Research as inpatient quality indicators: abdominal aortic aneurysm repair, esophagectomy, and pancreatectomy. Patients were identified using primary International Classification of Diseases, 9th Revision, Clinical Modification procedure codes (eTable 2 in Supplement). Patients with a primary medical diagnosis and a primary procedure code of interest were allocated to the surgical group, since their care would most likely be provided primarily by a surgical service. The composite medical and surgical mortality rates were calculated in the same manner as the aggregate rate for the publicly reported conditions.

Finally, we built an overall composite, which we refer to as the overall hospital mortality rate, combining the indirect standardized medical composite and the indirect standardized surgical composite by averaging the rates. Given the large differences in volume between medical and surgical conditions, we gave equal weight to both medical and surgical composites.

We categorized hospitals into quartiles based on their aggregate mortality rate for the publicly reported conditions and designated those in the lowest mortality quartile as top performers and those in the highest mortality quartile as poor performers. We examined characteristics of the hospitals, their patient populations, and the communities in which they were based across the 4 quartiles of performance.

To assess the potential size of the benefit conferred by being a top performer on the publicly reported conditions, we calculated risk-adjusted mortality rates (overall and for medical and surgical composites separately) within each quartile of performance on the publicly reported conditions. Overall mortality rates for quartiles of performance on publicly reported conditions were compared using analysis of variance with Tukey pairwise comparisons.

We examined how well mortality for publicly reported conditions predicted performance on overall hospital mortality. Within each quartile of mortality for the publicly reported conditions, we calculated the proportion of hospitals in each quartile of overall hospital mortality. Then, to determine whether choosing a top performer on the publicly reported conditions would improve the odds of choosing a top performer in overall hospital performance, we calculated the odds of a top performer on our predictor also being in the top quartile for the overall hospital mortality rate. We also examined whether choosing a top performer on the publicly reported conditions would help avoid selecting a low performer for overall hospital mortality rate. We then used the same process to determine the effect of choosing a poor performer. We repeated these analyses on medical and surgical composites separately. Finally, we tested for a difference in the ability of the mortality rate for the index condition to predict performance on medical vs surgical outcomes by introducing an interaction term into the model.

Because we were concerned that the relationship between the aggregated mortality rates of the publicly reported conditions and the composite outcomes might be confounded by characteristics of the hospital, we built multivariable logistic regression models to adjust for these factors. The following hospital characteristics were included as covariates in the models: hospital size, teaching status, ownership, geographic region, location (urban vs rural), median county income, nurse-to-census ratio, the hospital’s Disproportionate Share Index, and the proportion of Medicare patients in the hospital.

We next examined how well 2 other commonly used markers of hospital quality—large size (a proxy for volume) and being a major teaching hospital—performed in predicting overall hospital mortality rates. We built bivariate models to calculate odds ratios (ORs) for the likelihood of large hospitals being in the top or bottom quartile of overall hospital outcomes as well as for medical and surgical outcomes separately. We repeated these analyses for major teaching hospitals.

Since public reporting programs typically report mortality rates for individual conditions, we repeated the analyses described earlier using individual reported conditions as the predictor variable and the overall hospital, medical, and surgical composites as the outcome variables. We also repeated these analyses using pairs of reported conditions as the predictor variable to determine whether a particular combination was a stronger predictor of broader overall hospital mortality rates than others. Finally, we repeated the main analyses using hospital mortality rates from the Centers for Medicare & Medicaid Services as our primary predictor.

As an additional sensitivity analysis, we assessed whether the association between the publicly reported conditions and broader hospital outcomes was consistent among relatively healthy patients as well as among those who were particularly sick. We built risk-adjusted models initially limiting our sample to just low-comorbidity patients (those with <3 comorbidities) and examined the relationship between performance on the publicly reported conditions and our outcomes of interest (overall hospital mortality, medical composite, and surgical composite). We then repeated these analyses among high-comorbidity patients (those with ≥3 comorbidities).

We considered a 2-sided P value of less than .05 to be significant. Analyses were performed using SAS version 9.2 (SAS Institute) and Stata 12.1 (StataCorp LP). This study was approved by the Office of Human Research Administration at the Harvard School of Public Health.

The top performers—hospitals in the lowest mortality quartile for the publicly reported conditions—were more often located in the Northeast and Midwest and were large, urban, teaching, and nonprofit institutions compared with the poor performers (Table 1). Top performers tended to be in counties with higher median incomes.

The patient population captured by the selected reported, medical, and surgical diagnoses represented 6 670 859 admissions, which together accounted for 43.1% of all admissions and 57.0% of deaths among the study hospitals. Patients at the top-performing hospitals were slightly older than those at the poor-performing hospitals and had a slightly lower prevalence of uncomplicated type 2 diabetes mellitus and hypertension but a higher prevalence of chronic kidney disease and cancer. Patients at top-performing hospitals were less likely to be eligible for Medicaid and more often admitted on an emergency basis (Table 2).

Overall hospital mortality was 3.6% lower at top-performing hospitals for publicly reported conditions than at hospitals in the lowest-performing quartile (9.4% vs 13.0%; P < .001). The differences in the mortality rates were similarly large for the medical and surgical composites (Figure 1).

Top-performing hospitals on the publicly reported conditions had more than a 5-fold higher odds of being in the best quartile of overall risk-adjusted hospital mortality compared with other hospitals (OR, 5.3; 95% CI, 4.3-6.5) (Table 3). Odds ratios remained statistically significant when we considered medical (OR, 8.4; 95% CI, 6.8-10.3) and surgical (2.7; 2.2-3.3) mortality separately (Table 3). Publicly reported conditions tended to be a stronger predictor of performance for medical outcomes than surgical outcomes (P < .001 for interaction). When individual reported conditions were used as the predictor, the results were similar, although the magnitude of these associations was smaller. These patterns persisted when we adjusted for hospital and community characteristics using multivariable regression models (eTable 3 in Supplement). Poor-performing hospitals had significantly increased odds of also being poor performers on overall risk-adjusted hospital mortality (OR, 4.5; 95% CI, 3.7-5.5) (eTable 4 in Supplement).

Top-performing hospitals on the publicly reported conditions had 81% lower odds of being in the worst quartile of overall mortality (OR, 0.19; 95% CI, 0.14-0.27) (Table 4). These patterns persisted for the medical and surgical composites, and after adjusting for hospital and community characteristics (eTable 5 in Supplement). Poor-performing hospitals on the publicly reported conditions had much lower odds of being in the best quartile of overall mortality (eTable 6 in Supplement).

Large hospitals (those with ≥400 beds) had somewhat higher odds of being a top performer on the overall hospital outcomes than nonlarge hospitals (OR, 1.9; 95% CI, 1.5-2.4), as did teaching hospitals compared with nonteaching hospitals (2.4; 1.8-3.2). However, both were weaker predictors of top performance than the aggregate publicly reported conditions, especially when predicting performance on the composite of medical conditions (Figure 2).

When we repeated the main analyses using mortality rates generated by the Centers for Medicare & Medicaid Services as our primary predictor, the results were qualitatively similar (eTables 7-9 in Supplement). Finally, when we examined the relationships stratified by the underlying level of comorbidity, we found nearly identical results for those with fewer than 3 comorbid conditions. However, for the small number of patients with 3 or more cormorbid conditions, the publicly reported conditions were a significant predictor of performance on the medical mortality composite but were no longer significant for the surgical or overall mortality composites (eTables 10 and 11 in Supplement).

A hospital’s 30-day mortality rates for Medicare’s 3 publicly reported conditions—acute myocardial infarction, congestive heart failure, and pneumonia—were correlated with overall hospital mortality rates, even in clinically dissimilar fields. Hospitals in the top quartile of performance on the publicly reported mortality rates had greater than 5-fold higher odds of being top performers for a combined metric across 19 common medical and surgical conditions, translating into absolute overall mortality rates that were 3.6% lower for the top performers than for the poor performers. Finally, performance on the publicly reported conditions far outperformed 2 other widely used markers of quality: size and teaching status.

It is unclear exactly why mortality rates for different conditions should be common within a hospital. One view is that common processes of care and systems characteristics may contribute to this consistency. For example, hospitals may have relatively uniform procedures for ordering and cross-checking medications, reporting critical laboratory or radiology results, or using electronic clinical decision support tools across services and departments. However, there may be greater similarity between processes and systems of care for medical conditions within a hospital than there is across medical and surgical services. This may explain the somewhat stronger effect of the publicly reported conditions on predicting medical performance vs surgical performance, particularly for high-comorbidity surgical patients, who likely need particularly specialized care. In addition, emerging data suggest that leadership plays a critical role in hospital quality, perhaps promoting a broad culture of excellence.^12,13 While the fact that the ORs were only moderately strong suggests that clinician-specific factors likely still contribute to patient outcomes, the broader findings from our work support the notion that there may actually be “good” and “bad” hospitals and that performance on a manageable set of key indicators can help identify such institutions.

The predictive ability of performance on the publicly reported conditions was considerably better than we expected and certainly better than other measures often used to identify high-quality hospitals, such as hospital size and teaching status. Summary performance on the publicly reported conditions thus offers substantial value to consumers above and beyond basic hospital characteristics. However, prior studies have shown that these types of data are used infrequently and generally only by subsets of patients.^14-16 Our results suggest that there may be substantial value in efforts to engage and empower patients to use publicly reported hospital performance to make informed choices regarding where to seek care, irrespective of the condition that brings them to the hospital.

Prior studies examining consistency in performance across clinical specialties have been mixed. Dimick and colleagues¹⁷ found that despite relatively weak correlations in mortality rates for 11 surgical procedures, a surgical composite was a good predictor of procedure-specific mortality within hospitals. On the other hand, Chassin et al,¹⁸ in their landmark 1989 study, found poor correlations in inpatient mortality across a variety of medical and surgical conditions. These findings suggested that quality may be driven by individual clinical talent or focus on specific clinical areas. We suspect that one reason for the different patterns is that care has become more technical and systematized in the past 2 decades, leading to greater consistency within hospitals.

There are limitations to our study. We estimated risk-adjusted mortality rates using administrative data, which have modest ability to account for differences in the severity of illness between hospitals. In addition, inaccuracies in administrative data may introduce misclassification; however, we suspect that misclassification would have biased us against finding a relationship.

A second concern is that unmeasured factors, such as socioeconomic status or general health of the population, might confound the relationship between publicly reported conditions and overall outcomes. This could be a significant problem if hospitals attempt to “game the system” by intentionally selecting only patients with few comorbidities. In this case, the association between outcomes for publicly reported conditions and other medical and surgical outcomes might be due to unmeasured patient factors. To address this, we used risk-adjustment models that take into account patient comorbidities and a series of multivariable models that tried to account for the socioeconomic status of the patient population. However, these approaches are imperfect, and we cannot exclude the possibility of residual confounding.

Hospital performance on 30-day mortality rates for 3 common publicly reported medical conditions—acute myocardial infarction, congestive heart failure, and pneumonia—closely predicted mortality rates across a variety of other medical and surgical conditions. This finding has important implications for national quality improvement efforts that have focused on these 3 conditions and whose utility rests on the ability of these metrics to reflect broader hospital performance. Our work also suggests that we should redouble efforts to help consumers use these metrics. Furthermore, understanding the systems and leadership characteristics common to hospitals that perform well on publicly reported conditions may help to identify components of a truly good hospital that can be used to improve mortality rates at lower-performing institutions.

Accepted for Publication: March 24, 2013.

Corresponding Author: Ashish K. Jha, MD, MPH, Department of Health Policy and Management, Harvard School of Public Health, 677 Huntington Ave, Boston, MA 02115 (ajha@hsph.harvard.edu).

Published Online: June 24, 2013. doi:10.1001/jamainternmed.2013.7049.

Author Contributions: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: McCrum, Joynt, Gawande, Jha.

Acquisition of data: McCrum, Jha.

Analysis and interpretation of data: All authors.

Drafting of the manuscript: McCrum.

Critical revision of the manuscript for important intellectual content: Joynt, Orav, Gawande, Jha.

Statistical analysis: McCrum, Orav.

Obtained funding: Jha.

Administrative, technical, and material support: Jha.

Supervision: Gawande, Jha.

Conflict of Interest Disclosures: Dr Gawande reported receiving royalties from publishers for writing and honoraria from clinical organizations and associations for lectures and teaching on approaches to improving health system outcomes, which can include increasing transparency. In the past 12 months, these publishers and organizations have included the following: Henry Holt, Picador USA, Conde Nast, HarperCollins, Pearson Publishing, Houghton Mifflin, McGraw-Hill, Harvard Business School Press, Profile Books Ltd, The New England Journal of Medicine, Penguin Books/India, Sosoh Publishing, Commonwealth Publishing, China Renmin University Press, Lua de Papel, Antoni Bosch Editor, Asa Editore, Editorial Diagonal del Grup62/Edicione, Verlagsgruppe Random House GmbH, Uitgeverij Nieuwezjds, Baronet, Znak, Humanitas, Giulio Einaudi Editore S.p.A., Sextante, Book21, Gramedia Pustaka Utama, Shinyusha Co Ltd, Domingo Publishing, Cheers Publishing, Mehta Publishing House, Misuzu Shobo, Codecs, First News-TriViet Publishing, Cengage Learning, Internazionale Magazine, Eriko Yamauchi, CBS Television, Public Broadcasting Service, TNT Network, Fresenius, National Hospice & Palliative Care Organization, PeaceHealth, Intermountain Health Care, ABT Assoc, Partners Healthcare, American Society of Anesthesiologists, American College of Surgeons, Leading Age, Carolinas Healthcare, University of Albany, American Society of Healthcare Engineers, and University of California at San Francisco.

Funding/Support: This study was supported by the Cabot Fellowship from the Center for Surgery and Public Health at Brigham and Women’s Hospital (Dr McCrum); grant 1K23HL109177-01 from the National Heart, Lung, and Blood Institute (Dr Joynt); and the Lerner Research Award from Brigham and Women’s Hospital, Division of Cardiovascular Medicine (Dr Joynt). Additional support was provided from internal department funds from the Harvard School of Public Health.

Role of the Sponsors: The funders had no role in the design, conduct, or reporting of the study.