Ross JS, Mulvey GK, Stauffer B, Patlolla V, Bernheim SM, Keenan PS, Krumholz HM. Statistical Models and Patient Predictors of Readmission for Heart FailureA Systematic Review. Arch Intern Med. 2008;168(13):1371–1386. doi:10.1001/archinte.168.13.1371
Readmission after heart failure (HF) hospitalization is an increasing focus for physicians and policy makers, but statistical models are needed to assess patient risk and to compare hospital performance. We performed a systematic review to describe models designed to compare hospital rates of readmission or to predict patients' risk of readmission, as well as to identify studies evaluating patient characteristics associated with hospital readmission, all among patients admitted for HF.
We identified relevant studies published between January 1, 1950, and November 19, 2007, by searching MEDLINE, Scopus, PsycINFO, and all 4 Ovid Evidence-Based Medicine Reviews. Eligible English-language publications reported on readmission after HF hospitalization among adult patients. We excluded experimental studies and publications without original data or quantitative outcomes.
From 941 potentially relevant articles, 117 met inclusion criteria: none contained models to compare readmission rates among hospitals, 5 (4.3%) presented models to predict patients' risk of readmission, and 112 (95.7%) examined patient characteristics associated with readmission. Studies varied in case identification, used multiple types of data sources, found few patient characteristics consistently associated with readmission, and examined differing outcomes, often either readmission alone or a combined outcome of readmission or death, measured across varying periods (from 14 days to 4 years). Two articles reported model discriminations of patient readmission risk, both of which were modest (C statistic, 0.60 for both).
Our systematic review identified no model designed to compare hospital rates of readmission, while models designed to predict patients' readmission risk used heterogeneous approaches and found substantial inconsistencies regarding which patient characteristics were predictive. Clinically, patient risk stratification is challenging. From a policy perspective, a validated risk-standardized statistical model to accurately profile hospitals using readmission rates is unavailable in the published English-language literature to date.
Readmission after hospital discharge is attracting considerable attention from the Institute of Medicine,1 the Centers for Medicare & Medicaid Services,2 and the Medicare Payment Advisory Commission3 as an indicator of the quality and efficiency of care. Readmission after heart failure (HF) hospitalization is being given particular attention because HF is one of the most common principal discharge diagnoses among Medicare beneficiaries4 and because there is wide variation among hospital HF readmission rates,3 with some studies reporting rates approaching 45% at 6 months.5 Moreover, improved hospital6,7 and postdischarge8- 12 care, including predischarge planning,13,14 home-based follow-up,15 and patient education,13,16,17 lowers HF readmission rates, suggesting that HF readmission rates might be reduced if proved interventions were more fully instituted.
There are several issues to consider in using readmission as a measure of hospital performance. Any effort to compare readmission rates across institutions must consider differences in the spectrum of patients, with particular attention to HF disease severity and comorbid disease. Numerous publications have investigated patient factors associated with readmission risk after HF hospitalization.5,18- 26 Information from these studies and others may inform efforts to compare readmission rates among hospitals after accounting for differences across hospital populations, providing some perspective on the potential performance of a statistical model developed for such use. Moreover, this information may identify patient characteristics associated with readmission that may be considered for inclusion in any developed model and assist clinicians in stratifying patients' risk for readmission. Accordingly, 3 distinct but related objectives of this systematic review are as follows: (1) to describe statistical models designed to compare hospital rates of readmission among patients admitted for HF and to determine their performance (using model operating characteristics such as C statistic or area under the receiver operating characteristic curve) and which patient characteristics were included, (2) to describe statistical models designed to predict patients' risk of readmission among patients admitted for HF and to determine their performance and which patient characteristics were included, and (3) to identify patient characteristics associated with hospital readmission among patients admitted for HF.
Relevant studies were identified by searching the following databases: (1) Ovid MEDLINE (January 1, 1950, to November 19, 2007); (2) PubMed (January 1, 1950, to November 19, 2007); (3) Scopus, an Elsevier Science Publishers abstract and citation database (January 1, 1996, to November 19, 2007); (4) Ovid PsycINFO (January 1, 1967, to November 19, 2007); and (5) all Evidence-Based Medicine Reviews on Ovid, including ACP Journal Club (January 1, 1991, to September-October 2007), Cochrane Database of Systematic Reviews, Database of Abstracts and Reviews of Effects, and Cochrane Central Register of Controlled Trials (third quarter of 2007). We searched these databases using the following strategy. First, we performed a search that included the Medical Subject Headings (MeSH) term patient readmission (exploded) and the key words readmi$ and rehosp$ (using “$” for truncation), identifying 11 240 publications using our readmission terms. Second, we performed a search that included the MeSH term risk (exploded) and the key words model$, predict$, use$, util$, and risk$, identifying 4 462 843 publications using our risk/model/prediction terms. Third, we performed a search that included the MeSH term heart failure, congestive (exploded), identifying 106 487 publications using our HF term. Fourth, we combined our patient readmission, risk/model/prediction, and HF terms. This search identified 941 articles.
We applied several inclusion and exclusion criteria that were defined a priori to these 941 articles (article selection form available as an appendix from the author). Publications eligible for inclusion reported on readmission among individual patients hospitalized for HF as a primary outcome, secondary outcome, or part of a composite outcome. We excluded abstracts, pediatric studies, non–English language studies, and publications without original data (reviews, letters, and editorials). We also excluded studies that reported results from a case series or case report and studies without quantitative outcomes. Finally, we excluded publications from experimental studies, often randomized clinical trials, that reported on the effect of an intervention at readmission (eg, discharge planning, case management programs, or pharmaceutical treatment). Because our review is focused on identifying patient characteristics that are associated with hospital readmission, publications reporting on an intervention's effect are less relevant because they rarely report on the effect of participants' (patients’) characteristics on readmission. However, we included publications that used data collected from a randomized clinical trial to examine the effect of participants' (patients’) characteristics on readmission (independent of the effect of the intervention).
Two of us (J.S.R. and B.S.) independently reviewed the titles and abstracts of retrieved publications and selected relevant articles for possible inclusion in our review. Based on this review, we excluded 795 publications that did not report on readmission among patients hospitalized at baseline for HF or that met at least 1 of our exclusion criteria.
The remaining 146 potentially eligible publications were retrieved. On detailed review of the full-text publications, we excluded 29 additional articles that met our predefined exclusion criteria. The most common reasons for exclusion included the following: patients were not hospitalized at baseline (n = 13), patients did not have HF (n = 4), there were no original data (n = 4), and readmission was not examined as an outcome (n = 4).
We developed a standardized instrument (available as an appendix from the author) to perform a detailed abstraction of the remaining 117 publications by consulting with experts in cardiology and systematic review methods, preparing an instrument for their review, piloting the instrument, and making modifications as necessary. The following variables were extracted: data source; readmission type and period; study purpose, design, and period; sample sizes (hospitals and patients); analytic strategy, including methods used to handle deaths and transfers; and candidate variables examined as predictors of hospital readmission, categorized as laboratory variables, comorbid conditions, HF severity variables, and sociodemographic variables. All extractions were confirmed by a second author, and disagreements regarding assessment and data extraction were resolved by consensus among all authors.
There were 117 publications included in our review. As per our research objectives, (1) we identified no studies that presented statistical models that were derived or designed explicitly to compare hospital rates of readmission among patients with HF, (2) we identified 5 studies (4.3%) that presented statistical models that were derived or designed explicitly to predict risk of readmission among patients with HF, and (3) we identified 112 studies (95.7%) that examined patient characteristics associated with readmission among patients with HF but did not derive a statistical model to predict patient readmission risk. In other words, none of the articles reported the development or application of a statistical model for the purpose of comparing hospitals or other health care organizations, the first objective of our systematic review.
Table 1 gives characteristics of 5 studies20,23,27- 29 that presented statistical models or risk scores derived to predict patient risk of readmission after HF hospitalization, the second objective of our systematic review. All studies examined patients within the United States, but none used national data. Two studies predicted all-cause readmission, 1 study predicted HF-specific readmission, and the remaining 2 studies predicted death or all-cause readmission; the period studied ranged from 60 days to 1 year. Three studies derived a model risk score, whereas the other 2 studies derived and validated a model risk score to predict hospital readmission.
Chin and Goldman27 used prospectively collected medical record review data from a single academic hospital in Boston, Massachusetts, of 257 patients to derive a risk score for death or all-cause readmission to any hospital within 60 days among patients with HF. Patients were identified using the admitting diagnosis in combination with clinical characteristics. The authors examined 25 candidate variables for inclusion in their risk score (Table 2), including access to care, laboratory measures, comorbid conditions, clinical characteristics, and sociodemographic characteristics. Using Cox proportional hazards regression modeling, they developed an 11-point scoring system to stratify patient risk for death or all-cause readmission within 1 year (with 0-1 indicating lowest risk and 8-11 indicating highest risk), wherein each patient is assigned 2 points for single marital status, 1 point per Charlson Comorbidity Index (to a maximum of 4), 3 points for an initial systolic blood pressure of 100 mm Hg or less, and 2 points for new ST-T wave changes on an admission electrocardiogram (no C statistic was presented).
Philbin and DiSalvo23 used registry data collected from hospitals by the New York State Department of Health as part of the Statewide Planning and Research Cooperative System of 42 731 patients in 236 hospitals to derive and validate a risk score for HF-specific readmission to any New York State hospital within 1 year among patients with HF. Patients were identified using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes. The authors examined 60 candidate variables for inclusion in their risk score, including patient dispositions, hospital characteristics, hospital course measures, sociodemographic characteristics, and comorbid conditions and the Charlson Comorbidity Index. Using multivariate logistic regression analysis, they developed a 15-point scoring system to stratify patient risk for HF readmission within 1 year (with 0-3 indicating lowest risk and 11-15 indicating highest risk), wherein each patient is assigned 4 points at baseline and 1 point is added for each of the following 11 variables: (1) black race/ethnicity; (2) primary insurance of Medicare or Medicaid; medical history of (3) ischemic heart disease, (4) valvular heart disease, (5) diabetes mellitus, (6) renal disease, (7) chronic lung disease, (8) idiopathic cardiomyopathy, or (9) cardiac surgery; (10) the use of telemetry during the index hospitalization; and (11) home health care services after hospital discharge. One point is subtracted for each of the following 4 variables: (1) treatment at a rural hospital; (2) echocardiogram or (3) cardiac catheterization performed during the index hospitalization; and (4) discharge to a skilled nursing facility (C statistic, 0.60).
Krumholz et al20 used Connecticut Medicare Provider Analysis and Review file data from the Health Care Financing Administration, supplemented with medical record review of 2176 patients in 18 hospitals, to derive and validate a statistical model to predict risk of all-cause readmission to any Connecticut hospital within 6 months among older patients admitted for HF. Patients were identified using ICD-9-CM codes. The authors examined 32 candidate variables for their statistical model, including mobility, HF disease severity, comorbid conditions, clinical characteristics, discharge medications, sociodemographic characteristics, and hospital course and laboratory measures. Using Cox proportional hazards regression modeling, their final model included the following independent predictors of readmission (no C statistic was presented): (1) hospitalization in the prior year (hazard ratio [HR], 1.25; 95% confidence interval [CI], 1.05-1.48; P =.01), (2) medical history of HF (HR, 1.23; 95% CI, 1.02-1.48; P =.03), (3) medical history of diabetes mellitus (HR, 1.17; 95% CI, 0.99-1.39; P =.07), and (4) serum creatinine level exceeding 2.5 mg/dL (to convert creatinine level to micromoles per liter, multiply by 88.4) at discharge (HR, 1.72; 95% CI, 1.35-2.18; P< .001).
Felker et al28 used data from 949 patients in 78 hospitals enrolled in the Outcomes of a Prospective Trial of Intravenous Milrinone for Exacerbations of Chronic Heart Failure Study to derive a statistical model to predict risk of death or all-cause readmission to any hospital within 60 days among patients with HF. Patients were identified by their clinical presentation on admission. The authors examined 41 candidate variables for their statistical model, including height, weight, laboratory measures, comorbid conditions, clinical characteristics, admission medications, and sociodemographic characteristics. Using multivariate logistic regression, their final model included the following independent predictors of death or readmission (C statistic, 0.69): (1) HF hospitalization in the prior year (odds ratio [OR], 1.14; 95% CI, 1.06-1.23; P < .001), (2) prior percutaneous coronary intervention (OR, 1.46; 95% CI, 1.00-2.12; P =.05), (3) systolic blood pressure (OR, 0.82; 95% CI, 0.75-0.89; P < .001), (4) serum urea nitrogen level (OR, 1.26; 95% CI, 1.14-1.41; P < .001), and (5) serum hemoglobin level (OR, 0.89; 95% CI, 0.82-0.97; P =.006).
Yamokoski et al29 used data from 373 patients in 26 hospitals enrolled in the Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheterization Effectiveness trial to derive a statistical model to predict risk of all-cause readmission to any hospital within 6 months among patients with HF. Their objective was to compare the accuracy of the model with physician and nurse estimates of the same outcome for patients. Patients were identified by their clinical presentation on admission. The authors examined 18 candidate variables for their statistical model, including clinical characteristics, discharge medications, sociodemographic characteristics, New York Heart Association class, and hospital course and laboratory measures. Using multivariate logistic regression analysis, their final model included the following independent predictors of readmission (C statistic, 0.60): (1) serum urea nitrogen level and (2) high-dose diuretics at discharge.
Table 3A and Table 3B summarizes 112 studies5,18,21,22,25,26,30- 51,133 that examined the association of specific patient characteristics with readmission among patients hospitalized for HF, the third objective of our systematic review. These studies often described a multivariate model used to adjust for the effect of patient characteristics on readmission when examining a specific patient characteristic (eg, race/ethnicity) but did not derive a statistical model to predict patient readmission risk.
These 112 studies were evenly divided between patients within (57 studies) and outside (55 studies) the United States
(Table 3A and Table 3B). One study used a case-control design, 14 studies used a cohort of patients drawn from a randomized controlled trial, 42 studies were retrospective cohort studies, and 55 studies were prospective cohort studies. Most studies (n = 62) were conducted at a single hospital. Of 57 studies that examined patients within the United States, various data sources were used: 16 studies used hospital administrative data (including data from Veteran Affairs medical centers and the Kaiser-Permanente Health System) alone or in combination with medical record review; 15 studies used medical record review, often in combination with patient self-report; 14 studies used government administrative data (ie, Medicare Provider Analysis and Review files) alone or in combination with medical record review; and 12 studies used data collected during a randomized controlled trial.
Studies from the United States used all-cause readmission (46 studies) far more often than HF-specific readmission (11 studies) as an outcome, whereas non-US studies used all-cause readmission (29 studies) and HF-specific readmission (26 studies) almost equally as often as outcomes (Table 3A and Table 3B). Follow-up periods ranged from 14 days to 4 years, with 1 year (23 studies), 6 months (22 studies), and 1 month (19 studies) being most common.
The studies also varied in methods for identifying and excluding patients hospitalized at baseline for HF. For identification, 52 studies used clinical presentations on admission, 29 studies used ICD-9-CM codes at discharge, 19 studies used admitting diagnoses, and 12 studies used diagnosis-related group codes. Forty-four studies included patients who died during the follow-up period in their main outcome analyses (ie, death or readmission), 43 studies examined death during the follow-up period as a separate outcome, 14 studies excluded these patients from analyses, and 11 studies did not discuss their analytic plan for patients who died during follow-up. Only 29 studies (25.9%) discussed their analytic plan for patients who were transferred during their baseline hospitalization; 19 studies excluded these patients from analyses, 9 studies “assigned” the hospitalization to the transferring (index) hospital, and 1 study assigned the hospitalization to the receiving hospital.
Many studies were explicitly designed to examine the association between readmission and a specific patient-level characteristic such as sex (n = 6), race/ethnicity (n = 8), socioeconomic status (n = 2), systolic ejection fraction (n = 10), duration of index hospital admission (n = 3), medical history of renal disease (n = 3) or anemia (n = 4), and serum markers, including hematocrit and levels of sodium, creatinine, C-reactive protein, serum urea nitrogen, cardiac troponin T, and B-type natriuretic peptide, while adjusting for multiple other variables. However, approximately one-third of studies reported the significance test only for the primary independent variable of interest and not for other covariates included in their analyses.
There was some consistency across studies as to which variables were included in the multivariate analyses examining the association between readmission and a specific patient-level characteristic. Among sociodemographic variables, age (81.3% of studies) and sex (71.4%) were most frequently included in analyses (Table 4), although none were consistently associated with readmission. Among comorbid conditions, diabetes mellitus (46.4%) and hypertension (41.1%) were most frequently included in analyses, although again none were consistently associated with readmission. Among HF severity variables, systolic ejection fraction (56.3%) and New York Heart Association class (34.8%) were frequently included in analyses, although neither was consistently associated with readmission. Finally, among serum markers, creatinine or serum urea nitrogen level (44.6%), sodium level (25.0%), and B-type natriuretic peptide level (21.4%) were frequently included in analyses, although only elevated B-type natriuretic peptide was consistently associated with readmission (77.3%).
Our systematic review reveals several important considerations for efforts to compare hospital-specific rates of readmission after HF hospitalization and to stratify patient risk of readmission. Among 117 studies20,23,27- 29 included in our review, we did not identify a model designed to compare readmission rates, the first objective of our systematic review. Five studies developed models to predict patient risk of readmission, the second objective of our systematic review, but none found that patient characteristics strongly predict readmission. In fact, the 117 studies examined a large number of diverse patient characteristics to determine the associated readmission risk, and no consistent predictors emerged from our review, although not all studies reported the prognostic importance of all candidate variables. In addition, there was considerable methodological heterogeneity among research examining HF readmission risk. Studies varied in analytic approach, outcome examined, follow-up period, case identification, and handling of patient deaths and transfers, demonstrating that there is little consensus on approaches to compare institutions or to stratify risk for individual patients and making the literature difficult to synthesize. Nevertheless, it is clear that the risk of readmission is high after HF hospitalization among studies with shorter and longer follow-up periods, reinforcing the importance of focusing on HF readmissions to improve quality and efficiency in health care.
The 5 studies20,23,27- 29 we identified that developed models to predict patient risk of readmission demonstrate the challenge of stratifying patient risk based on clinical and demographic characteristics. Collectively, these models considered a large number of patient characteristics, yet few were consistently associated with risk of readmission across studies. Given the differing methodological approaches used, perhaps these inconsistencies are not surprising. Among 2 statistical models that specifically examined risk of readmission (as opposed to the combined outcome of readmission or death),23,29 patient characteristics provide only modest information about readmission risk, with discriminations (as measured by C statistic) of 0.60. In contrast, statistical models used to predict mortality after HF hospitalization demonstrate better discrimination,134- 137 with C-statistic discriminations ranging from 0.67 to 0.81.
The discrimination differences between the readmission and mortality models may be a reflection of the challenges in predicting patient risk of readmission based on clinical and demographic characteristics. Each readmission model included many variables that are also found in the mortality models, which have higher predictive ability, suggesting that the lower discrimination of the readmission models is unlikely to be because a critical patient characteristic was omitted. A possible explanation is that other nonpatient factors may have a larger role in readmission risk, although this was not the focus of our systematic review. For instance, readmission risk may be more responsive to improved hospital6,7 and postdischarge8- 17 care than mortality risk, and it may depend more on health care system characteristics such as acute care hospital capacity or physician supply.138
There are some key differences between models that we identified to stratify patient risk of readmission and models that might be used to profile hospitals. First, all of the patient risk models relied on clinical information (eg, admission physical examination findings) and laboratory test results, whereas profiling models would likely need to use administrative data, particularly for national comparisons. Second, most patient risk models accounted for patient characteristics that may be inappropriate for profiling models such as length of stay, discharge disposition, inhospital events and complications, and patient income, education, and race/ethnicity.139 Accounting for such characteristics could inappropriately risk-standardize hospital performance for the differences in quality and efficiency that profiling efforts attempt to measure, including inhospital complications, excessively long or short lengths of stay, and premature discharges to skilled nursing facilities rather than to home. Third, patient risk models would be expected to include patient characteristics present on discharge (including comorbidities and complications) for optimal risk stratification, whereas profiling models should only consider patient characteristics present on admission so that hospitalization complications are not used to adjust risk. Fourth, none of the patient risk models accounted for the clustering of patients within hospitals, whereas profiling models should account for the nonindependence of readmissions to each hospital.
The Medicare Payment Advisory Commission recently recommended that Medicare should publicly report short-term readmission rates and use these rates to adjust payments for HF and other conditions, including pneumonia, acute myocardial infarction, coronary artery bypass graft surgery, and chronic obstructive pulmonary disease, emphasizing that Medicare's hospital payment system provides no explicit encouragement or reward for hospitals that reduce readmissions, although readmissions may be indicative of poor care or missed opportunities to better coordinate care.3 Despite a large literature examining HF readmissions, our review demonstrates that the evidence to support such an HF measure is insufficient. Further research focused on deriving and validating a risk-standardized statistical model that accounts for important patient characteristics associated with readmission and for clustering of patients within hospitals is critical if hospitals are to be profiled on readmission rates.
Despite growing interest among physicians and policy makers in hospital readmissions as an indicator of quality and efficiency of care,1- 3 our systematic review highlights several methodological issues that need to be resolved before hospitals are compared and profiled based on readmission rates after HF hospitalization. No such hospital-level comparative measure yet exists (to our knowledge), and the collection of evidence that we identified to inform profiling efforts found inconsistent results, varied in the inclusion of patient characteristics for risk standardization, and used heterogeneous approaches to case identification, outcomes, and follow-up periods. Readmissions after HF admission are common, and from a clinical perspective it seems difficult to stratify patient risk. From a policy perspective, a validated risk-standardized statistical model that accounts for important patient characteristics associated with readmission is necessary if hospitals are to be profiled on their readmission rates, with the caveat that patient characteristics are likely to account for only a small amount of the variation among hospitals.
Correspondence: Joseph S. Ross, MD, MHS, Departments of Geriatrics and Adult Development and Medicine, Mount Sinai School of Medicine, One Gustave L. Levy Place, Box 1070, New York, NY 10029 (email@example.com).
Accepted for Publication: January 21, 2008.
Author Contributions: Dr Ross had full access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the analysis. Study concept and design: Ross, Keenan, and Krumholz. Acquisition of data: Ross, Mulvey, Stauffer, and Patlolla. Analysis and interpretation of data: Ross, Bernheim, Keenan, and Krumholz. Drafting of the manuscript: Ross, Keenan, and Krumholz. Critical revision of the manuscript for important intellectual content: Ross, Mulvey, Stauffer, Patlolla, Bernheim, Keenan, and Krumholz. Statistical analysis: Ross. Administrative, technical, and material support: Krumholz. Study supervision: Krumholz.
Financial Disclosure: None reported.
Funding/Support: The analyses on which this publication is based were supported by contract HHSM-500-2005-CO001C from the Centers for Medicare & Medicaid Services, US Department of Health and Human Services. Dr Ross is supported by project TRP-02-149 from the Department of Veterans Affairs Health Services Research and Development Service and by the Hartford Foundation. Dr Stauffer is supported by the Department of Veteran Affairs. During the study, Dr Bernheim was supported by training grant T32AG1934 from the National Institute on Aging.
Role of the Sponsor: The Department of Veterans Affairs or the Hartford Foundation or Robert Wood Johnson Foundation had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.
Disclaimer: The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The author assumes full responsibility for the accuracy and completeness of the ideas presented. The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Previous Presentation: This study was presented at the American Heart Association's 8th Scientific Forum on Quality of Care and Outcomes Research in Cardiovascular Disease and Stroke; May 11, 2007; Washington, DC.