QI indicates quality improvement. Sorted from shorter to longer time horizon.1-3,6-13,40-76 QI indicates quality improvement.
Sorted from shorter to longer time horizon.1-3,6-13,40-76 QI indicates quality improvement.
eAppendix 1. Literature Search Methods
eAppendix 2. Classification of Readmission-related Practices
eAppendix 3. PRISMA Flow Diagram
eAppendix 4. Assessment of Bias, Reporting of QI Intervention, and Funding Sources
eAppendix 5. Standardization of Costs
eAppendix 6. Results of Regression Sensitivity Analyses that Were Limited to RCTs
Customize your JAMA Network experience by selecting one or more topics from the list below.
Nuckols TK, Keeler E, Morton S, et al. Economic Evaluation of Quality Improvement Interventions Designed to Prevent Hospital Readmission: A Systematic Review and Meta-analysis. JAMA Intern Med. 2017;177(7):975–985. doi:10.1001/jamainternmed.2017.1136
Are quality improvement interventions designed to reduce hospital readmissions associated with net savings to the health system?
In this systematic review and data analysis of economic evaluations based on data for more than 16 700 patients, hospital readmissions declined by an average of 12.1% among populations with heart failure and 6.3% among general populations, but net savings to the health system were variable. In general populations, interventions that involved engaging patients and caregivers were associated with similar effectiveness but substantially larger net savings to the health system than other interventions.
Diverse interventions can be effective at reducing readmissions, but cost savings do not consistently occur. Interventions that engage patients and family members may be associated with larger net savings.
Quality improvement (QI) interventions can reduce hospital readmission, but little is known about their economic value.
To systematically review economic evaluations of QI interventions designed to reduce readmissions.
Databases searched included PubMed, Econlit, the Centre for Reviews & Dissemination Economic Evaluations, New York Academy of Medicine's Grey Literature Report, and Worldcat (January 2004 to July 2016).
Dual reviewers selected English-language studies from high-income countries that evaluated organizational or structural changes to reduce hospital readmission, and that reported program and readmission-related costs.
Data Extraction and Synthesis
Dual reviewers extracted intervention characteristics, study design, clinical effectiveness, study quality, economic perspective, and costs. We calculated the risk difference and net costs to the health system in 2015 US dollars. Weighted least-squares regression analyses tested predictors of the risk difference and net costs.
Main Outcomes and Measures
Main outcomes measures included the risk difference in readmission rates and incremental net cost. This systematic review and data analysis is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.
Of 5205 articles, 50 unique studies were eligible, including 25 studies in populations limited to heart failure (HF) that included 5768 patients, 21 in general populations that included 10 445 patients, and 4 in unique populations. Fifteen studies lasted up to 30 days while most others lasted 6 to 24 months. Based on regression analyses, readmissions declined by an average of 12.1% among patients with HF (95% CI, 8.3%-15.9%; P < .001; based on 22 studies with complete data) and by 6.3% among general populations (95% CI, 4.0%-8.7%; P < .001; 18 studies). The mean net savings to the health system per patient was $972 among patients with HF (95% CI, −$642 to $2586; P = .23; 24 studies), and the mean net loss was $169 among general populations (95% CI, −$2610 to $2949; P = .90; 21 studies), reflecting nonsignificant differences. Among general populations, interventions that engaged patients and caregivers were associated with greater net savings ($1714 vs −$6568; P = .006).
Conclusions and Relevance
Multicomponent QI interventions can be effective at reducing readmissions relative to the status quo, but net costs vary. Interventions that engage general populations of patients and their caregivers may offer greater value to the health system, but the implications for patients and caregivers are unknown.
Being hospitalized once puts patients at risk for being hospitalized again, and readmissions are costly. Historically, about 15% to 21% of older adults were rehospitalized within 30 days, and 20% to 48% within a year.1-5 Individuals with heart failure (HF) are at even higher risk, with 26% to 50% being readmitted within 30 days and 68% to 78% within a year.4,6-13 Following the Hospital Readmission Reduction Program in 2012, 30-day readmissions have declined among older Medicare beneficiaries, including those with HF.14,15 Nonetheless, in 2013, 3.9 million 30-day readmissions occurred nationwide across all payers, including 183 000 HF readmissions, at a cost of $52.4 billion.16
To reduce readmissions, hospitals and health systems can implement quality improvement (QI) interventions. Readmissions occurring within the first week after discharge are often related to the stress of acute illness as well as heightened self-care needs, new medications, and impaired function,5,17,18 while those occurring later reflect chronic illness.5 Accordingly, some QI interventions include practices that are implemented around the time of discharge, while others are maintained longer term.19-21
Multifaceted QI interventions can be effective, but neither the cost of implementation nor the potential savings associated with declines in hospitalization have been systematically evaluated.19,22 Furthermore, it remains unclear whether effective interventions tend to produce net savings or losses to the health system.
Our objective was to systematically evaluate the clinical effectiveness and incremental net costs of QI interventions for the prevention of readmission to acute care hospitals, including within 30 days and over longer periods. We searched peer-reviewed and non–peer-reviewed literature. We then examined the types of interventions studied, clinical effectiveness, associated costs, and the quality of the economic evaluations. Finally, we examined factors associated with effectiveness and costs.
This systematic review and data analysis is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.23 The study protocol is posted on the Prospero registry (CRD42015014950).24 An 8-member technical expert panel provided input during key phases of the project.
A reference librarian (R.S.) developed search terms related to readmission, and expanded on published terms related to economic evaluation (see eAppendices 1-6 and eReferences in the Supplement).25 Databases included PubMed, Econlit, and the Centre for Reviews & Dissemination Economic Evaluations. To identify gray literature, we searched New York Academy of Medicine's Grey Literature Report and Worldcat and invited technical expert panel members to suggest non–peer-reviewed analyses. Searches were restricted to English-language documents from January 2004 to July 2016. We hand-searched citations from previous systematic reviews and other sources.19,20,26-29
Studies were eligible if they represented original investigations, applied QI interventions designed to prevent readmission, measured or modeled the cost of the interventions, and compared alternatives. Studies had to report both program costs (costs of implementation) and readmission-related costs. We excluded studies from low- to middle-income countries owing to differences in care practices and cost structures.30 All diagnoses, clinical study designs, economic evaluation approaches, and time horizons were eligible. A QI intervention was “an effort to change/improve the clinical structure, process, or outcomes of care by means of an organizational or structural change.”31(p280) Readmissions had to be the main focus (eg, studies that focused on length of stay but monitored readmissions were excluded), and patients had to be enrolled during or immediately after hospitalization. Two trained reviewers (T.K.N. and E.K.) independently examined titles, abstracts, and full-text publications to determine eligibility; discrepancies were resolved by consensus, or, when necessary, through discussion with the research team.
We obtained any additional publications related to each study, and then pairs of investigators extracted data from the articles. Discrepancies were resolved by consensus, or, when necessary, through discussion with the research team.
Experienced hospitalists and researchers determined whether each intervention involved 15 categories of readmission-related practices.19,20 To facilitate comparisons across studies, we aggregated these 15 categories into 5 groups based on the purpose of the intervention. The groups were (1) assess patient risks and needs, (2) engage patient and/or caregiver, (3) reconcile medication, (4) connect patient to usual clinicians, and (5) supplement care by usual clinicians (eTable 1 in the Supplement).
For each eligible study, investigators extracted data related to context, population, design, and reporting of the clinical evaluation, time frame(s), outcome measure(s), numbers of individuals and outcomes in intervention and comparison groups, and funding sources. Contextual variables included country, academic status, setting, and readmission rate in the control group. We stratified studies into populations that were limited to HF, general populations, and other unique populations. Clinical study designs included randomized clinical trial (RCT), non-RCT, controlled before-after analysis; uncontrolled before-after analysis, interrupted time series and repeated measures study, and modeling exercise.32 To assess potential bias, we used the Cochrane Collaboration Tool for RCTs and the Newcastle-Ottawa Scale for cohort studies.33,34 We scored selected elements from the Minimum Quality Criteria Set (items 3-7, 10-11, and 13), a tool for critically appraising the reporting of QI interventions.35 Funding sources included government, nonprofit, commercial, and none.
The clinical outcome was the risk difference, meaning the change in the readmission rate, calculated by subtracting the percentage of patients who were readmitted in the intervention group from the percentage readmitted in the control group. When studies reported clinical outcomes for more than 1 time frame, we prioritized the time frame that matched costs and the duration of the intervention. When studies reported more than 1 outcome measure, we prioritized the number of patients with all-cause readmissions, unplanned readmissions, condition-related readmissions, and then such measures reported together with deaths.
Investigators with training in economic evaluation extracted the approach (cost analyses including cost-consequences and business-case analyses vs cost-effectiveness, cost-benefit, and related analyses); perspective (hospital, health system, payer, society); time horizon; discount rate (if applicable); year and currency of costs; resources involved in implementation; and incremental program, readmission-related, and net costs.
The economic outcome was the incremental net cost to the health system per enrolled patient (“net cost”), calculated by summing incremental program and readmission-related costs and converting all costs to 2015 US dollars. To identify relevant categories of costs, we used the Quality-Cost Framework.36 Together, structure- and process-related costs comprised an intervention’s total program costs. Structure-related costs were the fixed costs associated with start-up and maintenance, such as labor costs associated with creating new protocols and training clinicians. Process-related costs were variable, recurring costs resulting from the care of individual patients, such as the cost of home visits. Outcome-related costs included health care expenditures related to outcomes, including readmission.
Reviewers assessed whether economic evaluations met basic standards by applying a modified version of the Quality of Health Economics Studies Checklist (mQHES).37,38 We divided each question into subparts for easier scoring and added 2 questions related to the adequacy of competing alternatives and credibility of the analysis overall. To calculate the total mQHES score (scale, 0-115) for each study, we determined the percentage of “yes” responses to subparts of each question, weighted each question’s raw score, and summed the weighted values.
Calculating the risk difference and net cost separately (ie, conducting a cost-consequences analysis) for each study enabled us to perform 2 sets of prespecified weighted least-squares regression analyses that assessed potential predictors of these outcomes. We chose the weighted least-squares approach (which weights studies by number of participants) over a random effects approach (which weights studies by the inverse of variance) because the 2 methods yield equivalent results, weighted least-squares models perform as well or better when there may be publication bias or small sample sizes,39 studies did not report data related to variance for cost estimates so the random effects approach was infeasible for that outcome, and using a single analytical method for both outcomes was less complex.
We stratified both the risk difference and net cost analyses by populations limited to HF and general populations, excluding studies of other unique populations. A regression model was fitted for each independent variable separately, including the readmission rate in the control group (continuous), intervention time horizon (<30 or ≥30 days), program cost per patient (continuous), and practices included in the QI intervention. We examined the number of individual practices per study, the number of groups of practices per study (both continuous), and each of the 5 groups of practices individually (yes or no for each group). In models predicting the net cost, additional continuous independent variables included the risk difference and the mQHES score. Because results may differ by study design, we performed sensitivity analyses that included only RCTs. There were too few studies for multivariate regression.
We identified 5205 articles, selecting 401 for full-text review; 48 articles met all eligibility criteria. One article reported results separately for medical and surgical patients, while another tested 2 variations on an intervention. We report these separately, yielding 50 unique studies. See eFigure in the Supplement for PRISMA Diagram and eTables 2 to 6 in the Supplement for data extracted from the studies.
Nearly all studies involved supplementing care by usual clinicians, and most involved assessing patients’ risks or needs, engaging patients or caregivers, and undertaking steps to connect patients to their usual clinicians. Medication reconciliation was not as common, particularly among populations with HF (Table 1, eTables 2-6 in the Supplement).
Twenty-five studies were limited to patients with HF and together included 5768 patients.6-13,40-56 Twenty-one studies applied to general populations, largely older adults and together included 10 445 patients.1,57-74 Four studies applied to unique populations.2,3,75,76
Twenty-nine studies were based in the United States, while the rest were in other nations. eTables 2 to 6 in the Supplement include setting and academic status.
Thirty-three studies were RCTs, 7 were non-RCTs, 5 used controlled before-after designs, 3 used uncontrolled before-after designs, and 2 were modeling exercises. All studies compared QI interventions with a status quo scenario. Among the non-RCTs, 3 compared patients who opted in vs those who opted out, and 2 did not report methods of allocation.
Thirty-nine studies reported all-cause readmissions, 2 unplanned readmissions, 6 condition-related readmissions, 2 all-cause readmissions or death, and 1 readmission that involved intensive care among patients with life-limiting illness. Three reported complete data for all-cause and condition-related readmissions (with similar results for effectiveness and costs). Most studies reported numbers of patients with readmissions; 5 studies reported only numbers of readmissions per patient.
See eTables 2 to 6 in the Supplement for risk of bias assessments, funding sources, and items from the Minimum Quality Data Set. Many studies were at risk for bias owing to the possibility of incomplete outcome data or loss to follow-up.
Forty-four of the studies were cost analyses, whereas 6 were cost-effectiveness or cost-benefit analyses. Forty-two studies considered the health system perspective, 1 considered the hospital perspective, 3 used the payer perspective, 1 used both the hospital and health system perspectives, and 3 considered the societal perspective. Overall, economic evaluation methods met basic standards, with a median mQHES score of 99.3 (interquartile range, 10.8) across the 50 studies.
Figure 1, shows program costs, readmissions-related costs, and net costs for each study in populations limited to HF, where negative costs reflect savings. Program costs were negative in 1 study, meaning that the intervention saved money even without a change in readmissions: home care services were provided more efficiently in the intervention group.56 Program costs were smaller than savings from reduced readmissions in most studies, leading to net savings. However, net financial losses occurred in 5 studies. In 1 of these studies, both readmissions and costs rose.52 In another, readmissions declined but length of stay increased in the intervention group.53 In a modeling analysis from the United Kingdom, readmissions declined but the cost per readmission was low, leading to net costs rather than savings.54 Finally, program costs were particularly high in 2 studies,6 including 1 in which readmissions did not decline.55
For general and other unique populations (Figure 2), program costs were negative in 2 studies: 1 study provided inpatient care more efficiently through an acute care for elderly patients unit,70 and the other study improved the management of percutaneous gastrostomy tubes such that the need for care at home declined.3 Net costs rose in 4 studies: both readmissions and costs climbed in 1 study,71 while costs per hospitalization rose in 3 studies, including 1 among surgical patients.72-74 See eTable 7 in the Supplement for data used to standardize costs.
Among patients with HF, the weighted mean percentage of patients who were readmitted during the study period (lasting a median of 197 days) was 50.0% in the control group and 37.9% in the intervention group, based on 22 studies with complete data. This corresponds to a risk difference of 12.1% (95% CI, 8.3%-15.9%; P < .001) (Table 2). Larger investments in program costs were associated with modestly reduced effectiveness (P = .007). Time horizon, readmission rate in the control group, groups of readmission-related practices, and the number of groups of readmission-related practices were not associated with different effectiveness. In a sensitivity analyses that was limited to 17 RCTs, overall findings were unchanged.
In general populations, the weighted mean percentage of patients who were readmitted during the study period (lasting a median of 30 days) was 22.4% in the control group and 16.1% in the intervention group, based on 18 studies with complete data and excluding the 4 studies of unique populations. This corresponds to a risk difference of 6.3% (95% CI, 4.0%-8.7%; P < .001), which is significantly smaller than the 12.1% decline in HF studies (P = .01). None of the variables examined, including time horizon and readmission-related practices, was associated with effectiveness. Results were unchanged by limiting the analysis to the 12 RCTs.
The mean net cost to the health system per patient with HF was −$972 (95% CI, −$2586 to $642; P = .23), reflecting nonsignificant net savings, based on 24 studies with complete data (Table 3). Net costs declined with increasing effectiveness (P < .001), such that an absolute 1% increase in the risk difference was associated with $342 greater savings. Time horizon, readmission rate in the control group, other readmission-related practices, and mQHES score were not associated with net costs. Findings were similar when limiting the analysis to 17 RCTs.
For general populations, the mean net cost was $169 (95% CI, −$2610 to $2949; P = .90), reflecting nonsignificant net losses. When interventions involved engaging patients and caregivers, net costs were $8282 lower than when they did not do so (−$1714 vs $6568; P = .006). Program costs, intervention effectiveness, and mQHES scores were not associated with net costs. In a sensitivity analysis limited to 12 RCTs, $100 higher program costs were associated with $261 greater savings (P = .009).
In this systematic review and meta-analysis of 50 economic evaluations that included data on more than 16 700 patients, multifaceted QI interventions reduced all-cause hospital readmissions but did not consistently yield net savings to the health system. Thirty percent of the studied interventions lasted 14 to 30 days, the time frame considered by the Medicare Hospital Readmissions Reduction Program, while the remainder generally lasted for 6 months or longer. The percentage of patients readmitted within study time frames declined by 12.1% among patients with HF (from 50.0%) and by 6.3% among general populations (from 22.4%). On average, QI interventions were associated with nonsignificant net savings of $972 among patients with HF and nonsignificant net losses of $169 in general populations. Overall, net losses occurred in 20% of studies.
Neither prior literature nor our analysis provides clear answers about which practices are most effective at reducing readmissions. In a 2014 meta-analysis19 of 42 RCTs targeting 30-day readmissions, the pooled risk ratio (RR) was 0.82 (95% CI, 0.73-0.91). This reflects an 18% relative decline, which is slightly lower than the 23% to 28% relative decline (ie, risk difference divided by baseline rate) that we observed. In the 2014 analysis, interventions that supported patient capacity for self-care were more effective, which is similar to the concept of engaging patients and caregivers. We found that engaging patients and caregivers was associated with greater net savings among general populations but not among patients with HF. In a 2014 meta-analysis77 of 47 RCTs of HF interventions that lasted 3 to 6 months, all-cause readmissions declined with multidisciplinary clinics (pooled RR, 0.70, 95% CI 0.55–0.89) and home-visiting programs (RR, 0.75; 95% CI, 0.68-0.86), while HF-related readmissions declined with structured telephone support (RR, 0.74; 95% CI, 0.61-0.90). In a 2015 Cochrane review78 of 41 RCTs, telemonitoring over 3 to 26 months (RR, 0.71; 95% CI, 0.60-0.83) and structured telephone support over 3 to 18 months (RR, 0.85; 95% CI, 0.77-0.93) reduced HF-related but not all-cause readmissions. We may have detected fewer differences in effectiveness across readmission-related practices because we focused on all-cause readmissions and because the interventions that have been subjected to economic evaluation tended to use similar types of QI practices.
Both effectiveness and costs are important to assessing value.22 Thirty-day readmissions are in decline among Medicare beneficiaries but, until now, little systematically developed information has existed on the cost of the QI interventions that may be contributing to these trends. The costs reported in our analysis reflect the health system perspective, which encompasses costs borne by hospitals, physicians, other clinicians, and public and private payers. Yet financial implications differ across stakeholders. Under fee-for-service payment systems, hospitals and inpatient physicians invest labor and other resources before discharge, and outpatient physicians, home health agencies, and skilled nursing facilities do so after. Public and private payers are the primary recipients of any savings when readmissions decline. For example, Medicare is expected to save $8.2 billion from avoided readmissions under the Hospital Readmissions Reduction Program.79 This does not include the program costs associated with QI interventions, however, because these are not borne by Medicare.
Among general populations, we found that interventions that engaged patient and caregivers offered considerably greater value because, while clinical effectiveness was the same, net savings per patient were $8282 larger (P = .006). Such interventions enhance the capacity for self-management, decreasing unnecessary reliance on the health care system. However, they also inherently increase investments of labor by patients and caregivers. Patient and caregiver opinions of these interventions, and the value of their time, were not considered by studies in this analysis.
Is implementing QI interventions to reduce readmissions is a good value? On the one hand, the interventions are generally effective and cost neutral to cost saving to the health system. Our data suggest that preventing 1 more readmission among 100 intervention enrollees with HF would, on average, yield net savings of $34 200. On the other hand, hospital readmissions only indirectly reflect patient health outcomes. Many readmissions are in the patients’ best interests and not avoidable. To fully assess value, a cost-effectiveness analysis would need to be performed from the societal perspective, considering health-related quality of life and other implications for patients and caregivers.
Regarding limitations, generalizability is a concern because QI is highly context dependent, but hospitals seem to be implementing interventions that are similar to those we evaluated.21,80 Several primary studies used nonrandomized allocation procedures or uncontrolled before-after designs, raising the possibility that other factors contributed to findings. We addressed this through sensitivity analyses in which we dropped studies with nonrandomized designs. Although most studies reported all-cause readmissions, some used other measures. Many studies were at risk for obtaining incomplete outcome data. The studies included a large number of patients and diverse settings, making it harder to compare findings across studies, but the declines in readmission rates that we observed are consistent with or slightly larger than in prior systematic reviews. In most cases, the economic evaluations met quality standards. We did not formally assess publication bias as data on the variance of cost estimates was limited. Authors may not perform economic analyses until clinical effectiveness has been demonstrated. We detected no evidence that lower-quality studies reported greater savings, as might be expected if more favorable findings were published preferentially.
Multicomponent QI interventions that are designed to reduce readmissions after hospital discharge are generally effective relative to the status quo, but net savings to health system perspective are uncertain owing to heterogeneity in results across studies. Interventions that engage general populations of patients and their family members may be of higher value to the health system than other interventions because they are associated with similar effectiveness and substantially higher net savings. However, the burden of these interventions to patients and caregivers has not been quantified.
Corresponding Author: Teryl K. Nuckols, MD, MSHS, Cedars-Sinai Medical Center, 8700 Beverly Dr, Becker 113, Los Angeles, CA 90048 (firstname.lastname@example.org).
Accepted for Publication: February 2, 2017.
Published Online: May 30, 2017. doi:10.1001/jamainternmed.2017.1136
Author Contributions: Dr Nuckols had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Nuckols, Shekelle, Keeler, Morton.
Acquisition, analysis, or interpretation of data: Nuckols, Keeler, Morton, Anderson, Doyle, Pevnick, Booth, Shanman, Arifkhanova.
Drafting of the manuscript: Nuckols, Shanman.
Critical revision of the manuscript for important intellectual content: Nuckols, Keeler, Morton, Anderson, Doyle, Pevnick, Booth, Arifkhanova, Shekelle.
Statistical analysis: Nuckols, Keeler, Morton, Anderson, Booth.
Obtained funding: Nuckols.
Administrative, technical, or material support: Nuckols, Anderson, Doyle, Pevnick, Shanman.
Study supervision: Nuckols, Morton, Shekelle.
Conflict of Interest Disclosures: Dr Nuckols has received research funding from the Collaborative Spine Research Foundation, the American Association of Neuromuscular & Electrodiagnostic Medicine, and icare New South Wales. Dr Pevnick has received research funding from the American Society of Health-System Pharmacists. Dr Shekelle has received research funding from the ECRI Institute, and royalties from UpToDate. No other disclosures are reported.
Funding/Support: The Agency for Healthcare Research and Quality (AHRQ) funded the study (R01 HS22644-01).
Role of the Funder/Sponsor: The AHRQ had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: