[Skip to Content]
[Skip to Content Landing]
Figure 1.  Periods Relevant to Analyses 1 to 3
Periods Relevant to Analyses 1 to 3

In analysis 1, changes in excess readmissions rates are inversely associated with baseline excess readmission ratios (ERRs). In hospitals with worse (or better) than mean performance during the Hospital Readmissions Reduction Program (HRRP) measurement period for fiscal year (FY) 2013, ERRs declined (or increased) during the 3-year postmeasurement period. In analysis 2, hospitals with low performance during an alternate measurement period exhibited subsequent declines in ERRs. In hospitals with worse (or better) than mean performance during an alternate measurement period, declines (or increases) in ERRs occurred during the 3-year postmeasurement period (ie, during the FY 2013 measurement period). In analysis 3, hospitals with low performance during the HRRP measurement period for FY 2013 exhibited declines in ERRs when going backward in time. In hospitals with worse (or better) than mean performance during the HRRP measurement period for FY 2013, declines (or increases) in ERRs occurred in reverse chronological order during the alternate measurement period. ACA indicates Affordable Care Act.

Figure 2.  Changes in Excess Readmission Ratio (ERR) From Fiscal Year (FY) 2013 Measurement Period to the 3-Year Postmeasurement Period
Changes in Excess Readmission Ratio (ERR) From Fiscal Year (FY) 2013 Measurement Period to the 3-Year Postmeasurement Period

Data from analysis 1 (described in Figure 1) are stratified by condition. MI indicates myocardial infarction.

Figure 3.  Changes in Excess Readmission Ratio (ERR) From an Alternate Measurement Period to 3-Year Postmeasurement Period
Changes in Excess Readmission Ratio (ERR) From an Alternate Measurement Period to 3-Year Postmeasurement Period

Postmeasurement period is in reference to the fiscal year 2013 measurement period. Data from analysis 2 (described in Figure 1) are stratified by condition. MI indicates myocardial infarction.

Table 1.  Hospital Characteristics by Target’s Summary Statistics and by Conditionsa
Hospital Characteristics by Target’s Summary Statistics and by Conditionsa
Table 2.  Changes in Excess Readmissions Ratios Explained by Regression to the Mean
Changes in Excess Readmissions Ratios Explained by Regression to the Mean
1.
Centers for Medicare & Medicaid Services. Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
2.
Centers for Medicare & Medicaid Services. Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
3.
Desai  NR, Ross  JS, Kwon  JY,  et al.  Association between hospital penalty status under the Hospital Readmission Reduction Program and readmission rates for target and nontarget conditions.  JAMA. 2016;316(24):2647-2656. doi:10.1001/jama.2016.18533PubMedGoogle ScholarCrossref
4.
Wasfy  JH, Zigler  CM, Choirat  C, Wang  Y, Dominici  F, Yeh  RW.  Readmission rates after passage of the hospital readmissions reduction program: a pre-post analysis.  Ann Intern Med. 2017;166(5):324-331. doi:10.7326/M16-0185PubMedGoogle ScholarCrossref
5.
Barnett  AG, van der Pols  JC, Dobson  AJ.  Regression to the mean: what it is and how to deal with it.  Int J Epidemiol. 2005;34(1):215-220. doi:10.1093/ije/dyh299PubMedGoogle ScholarCrossref
6.
Atkinson  G, Loenneke  JP, Fahs  CA, Abe  T, Rossow  LM.  Individual differences in the exercise-mediated blood pressure response: regression to the mean in disguise?  Clin Physiol Funct Imaging. 2015;35(6):490-491. doi:10.1111/cpf.12211PubMedGoogle ScholarCrossref
7.
Bland  JM, Altman  DG.  Regression towards the mean.  BMJ. 1994;308(6942):1499. doi:10.1136/bmj.308.6942.1499PubMedGoogle ScholarCrossref
8.
Kelly  C, Price  TD.  Correcting for regression to the mean in behavior and ecology.  Am Nat. 2005;166(6):700-707. doi:10.1086/497402PubMedGoogle ScholarCrossref
9.
Linden  A.  Assessing regression to the mean effects in health care initiatives.  BMC Med Res Methodol. 2013;13(1):119. doi:10.1186/1471-2288-13-119PubMedGoogle ScholarCrossref
10.
Yudkin  PL, Stratton  IM.  How to deal with regression to the mean in intervention studies.  Lancet. 1996;347(8996):241-243. doi:10.1016/S0140-6736(96)90410-9PubMedGoogle ScholarCrossref
11.
Bland  JM, Altman  DG.  Some examples of regression towards the mean.  BMJ. 1994;309(6957):780. doi:10.1136/bmj.309.6957.780PubMedGoogle ScholarCrossref
12.
Linden  A.  RTMCI: Stata Module to Estimate Regression to the Mean Effects With Confidence Intervals. Boston, MA: Statistical Software Components; 2013.
13.
Stigler  SM.  Regression towards the mean, historically considered.  Stat Methods Med Res. 1997;6(2):103-114. doi:10.1177/096228029700600202PubMedGoogle ScholarCrossref
14.
Press  MJ, Scanlon  DP, Ryan  AM,  et al.  Limits of readmission rates in measuring hospital quality suggest the need for added metrics.  Health Aff (Millwood). 2013;32(6):1083-1091. doi:10.1377/hlthaff.2012.0518PubMedGoogle ScholarCrossref
15.
Gupta  A. Impacts of performance pay for hospitals: the readmissions reduction program. Becker Friedman Institute for Research in Economics Working Paper No. 2017-07. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3054172. Posted October 16, 2017. Accessed May 29, 2019.
16.
Zuckerman  RB, Sheingold  SH, Orav  EJ, Ruhter  J, Epstein  AM.  Readmissions, observation, and the Hospital Readmissions Reduction Program.  N Engl J Med. 2016;374(16):1543-1551. doi:10.1056/NEJMsa1513024PubMedGoogle ScholarCrossref
17.
Ibrahim  AM, Dimick  JB, Sinha  SS, Hollingsworth  JM, Nuliyalu  U, Ryan  AM.  Association of coded severity with readmission reduction after the Hospital Readmissions Reduction Program.  JAMA Intern Med. 2018;178(2):290-292. doi:10.1001/jamainternmed.2017.6148PubMedGoogle ScholarCrossref
18.
Ody  C, Msall  L, Dafny  LS, Grabowski  DC, Cutler  DM.  Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated.  Health Aff (Millwood). 2019;38(1):36-43. doi:10.1377/hlthaff.2018.05178PubMedGoogle ScholarCrossref
19.
Dharmarajan  K, Wang  Y, Lin  Z,  et al.  Association of changing hospital readmission rates with mortality rates after hospital discharge.  JAMA. 2017;318(3):270-278. doi:10.1001/jama.2017.8444PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    EXPAND ALL
    The Principal Cause of Early Readmission
    Mark Ketterer, PhD | Henry Ford Hospital
    Diagnosis must preceed treatment. If the core behavioral cause of readmissions is not recognized, it cannot be ameliorated.

    Cognitive impairment is highly prevalent in conditions with high readmission rates, is the strongest prospective predictor of early readmission (Agarwal et al., 2016; Ketterer et al., 2017; Ketterer et al., 2014a; 2014b; Patel et al., 2015) , causes nonadherence (Cameron et al., 2010) and psychoeducational counselling for patients/families regarding reduces early readmission by 30-50% (Jasinski et al., 2018; Ketterer et al., 2016; Ketterer et al., 2019). Furthermore, cognitive impairment is unrecognized in 70-90% of cases (Agarwal
    et aal., 2016; Ketterer et al., 2014a; Raymont et al., 2004; Valcour et al., 2000).

    Persistence of low readmission rates beyond 30 days requires more study. Can families maintain the support affecting low early rates?

    Agarwal, K. S., Kazim, R., Xu, J., Borson, S., & Taffet, G. E. (2016). Unrecognized cognitive
    impairment and its effect on heart failure readmissions of elderly patients. Journal of the
    American Geriatric Society, 64, 2296–2301. PMID: 27676328.

    Cameron, J., Worrall-Carter, L., Page, K., Riegel, B., Lo, S. K., & Stewart, S. (2010). Does
    cognitive impairment predict poor self-care in patients with heart failure? Europeon Journal
    of Heart Failure, 12, 508–515. PMID: 20354031.

    Jasinski MJ, Lumley MA, Soman S, Yee J & Ketterer MW. (2018). Family consultation to reduce early hospital readmissions among patients with end stage kidney disease: a randomized, controlled trial. Clinical Journal of the American Sociaety of Nephrology 13:850-857. DOI: https://doi.org/10.2215/CJN.08450817.

    Ketterer MW, Chawa M & Paone G. (2017). Prospective correlates of early (30 day) readmissions on a cardiothoracic surgery service. Psychology, Health & Medicine 22(8):947-954. https://doi.org/10.1080.13548506.2017.1287408.

    Ketterer, M. W., Draus, C., McCord, J., Mossallam, U., & Hudson, M. (2014a). Behavioral factors and hospital admissions/readmissions in patients with CHF. Psychosomatics, 55, 45–50. PMID: 24016384.

    Ketterer MW, Ouellette D & Jennings J. (2019). Psychoeducation for chronic cognitive impairment and reduced early readmissions amongs pulmonary inpatienty. Psychology, Health & Medicine https://www. Tandfonline.com/jounalinformation?journalCode=cphm20

    Ketterer, M. W., Peltzer, J., Mossallam, U., Draus, C., Schairer, J., Rabbani, B., & McCord, J.
    (2016b). Cognitive impairment and reduced early readmissions in congestive heart failure?
    American Journal of Accountable Care, 3(16), e1–e6. https://www.ajmc.com

    Ketterer, M. W., Soman, S., & Mossallam, U. (2014b). Psychosocial risk factors and admissions/readmissions in end stage renal disease patients. Journal of Behavioral Health, 3(4), 230–233.

    Patel, A., Parikh, R., Howell, Hsieh, E., Landers, S. H., & Gorodeski, E. Z. (2015). Mini-cog performance: Novel marker of post discharge risk among patients hospitalized for heartfailure. Circulation, Heart Failure, 8, 8–16. PMID: 25477431.

    Raymont, V., Bingley, W., Buchanon, A., David, A. S., Hayward, P., Wessely, S., & Hotopf, M.
    (2004). Prevalence of mental incapacity in medical inpatients and associated risk factors:
    Cross-sectional study. Lancet, 364(9443), 1421–1427. PMID: 15488217.

    Valcour, V. G., Masaki, K. H., Curb, J. D., & Blanchette, P. L. (2000). The detection of dementia in the primary care setting. Archives of Internal Medicine, 160(19), 2964–2968. PMID:
    11041904.
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Original Investigation
    June 26, 2019

    Regression to the Mean in the Medicare Hospital Readmissions Reduction Program

    Author Affiliations
    • 1Sol Price School of Public Policy, Department of Health Policy and Management, University of Southern California, Los Angeles
    • 2Schaeffer Center for Health Policy and Economics, University of Southern California, Los Angeles
    • 3Division of General Internal Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California
    • 4Department of General Internal Medicine, University of California, Los Angeles
    • 5School of Public Health, Division of Health Policy and Management, University of Minnesota, Minneapolis
    JAMA Intern Med. 2019;179(9):1167-1173. doi:10.1001/jamainternmed.2019.1004
    Key Points

    Question  Could regression to the mean explain some of the decline in readmissions at hospitals initially classified as below-mean performers under the Medicare Hospital Readmissions Reduction Program (HRRP)?

    Findings  In this analysis of change in hospital readmission rates for specific conditions among Medicare beneficiaries, strong evidence suggests that most declines in excess readmission experienced after implementation of the HRRP at poorly performing hospitals was explained by a statistical phenomenon called regression to the mean. Regression to the mean signifies that entities farther away from the mean in one period are likely to be recorded closer to the mean in subsequent periods, simply by chance.

    Meaning  Random chance or luck rather than improvement in quality of care appears to be the primary driver of improvements in readmissions experienced at hospitals initially classified as below-mean performers under the HRRP.

    Abstract

    Importance  Excess 30-day readmissions have declined substantially in hospitals initially penalized for high readmission rates under the Medicare Hospital Readmissions Reduction Program (HRRP). Although a possible explanation is that the policy incentivized penalized hospitals to improve care processes, another is regression to the mean (RTM), a statistical phenomenon that predicts entities farther from the mean in one period are likely to fall closer to the mean in subsequent (or preceding) periods owing to random chance.

    Objective  To quantify the contribution of RTM to declining readmission rates at hospitals initially penalized under the HRRP.

    Design, Setting, and Participants  This study analyzed data from Medicare Provider and Analysis Review files to assess changes in readmissions going forward and backward in time at hospitals with high and low readmission rates during the measurement window for the first year of the HRRP (fiscal year [FY] 2013) and for a measurement window that predated the FY 2013 measurement window for the HRRP among hospitals participating in the HRRP. Hospital characteristics are based on the 2012 survey by the American Hospital Association. The analysis included fee-for-service Medicare beneficiaries 65 years or older with an index hospitalization for 1 of the 3 target conditions of heart failure, acute myocardial infarction, or pneumonia or chronic obstructive pulmonary disease and who were discharged alive from February 1, 2006, through June 30, 2014, with follow-up completed by July 30, 2014. Data were analyzed from January 23, 2018, through March 29, 2019.

    Exposures  Hospital Readmission Reduction Program penalties.

    Main Outcome and Measures  The excess readmission ratio (ERR), calculated as the ratio of a hospital’s readmissions to the readmissions that would be expected based on an average hospital with similar patients. Hospitals with ERRs of greater than 1.0 were penalized.

    Results  A total of 3258 hospitals were included in the study. For the 3 target conditions, hospitals with ERRs of greater than 1.0 during the FY 2013 measurement window exhibited decreases in ERRs in the subsequent 3 years, whereas hospitals with ERRs of no greater than 1.0 exhibited increases. For example, for patients with heart failure, mean ERRs declined from 1.086 to 1.038 (−0.048; 95% CI, −0.053 to −0.043; P < .001) at hospitals with ERRs of greater than 1.0 and increased from 0.917 to 0.957 (0.040; 95% CI, 0.036-0.044; P < .001) at hospitals with ERRs of no greater than 1.0. The same results, with ERR changes of similar magnitude, were found when the analyses were repeated using an alternate measurement window that predated the HRRP and followed up hospitals for 3 years (for patients with heart failure, mean ERRs declined from 1.089 to 1.044 [−0.045; 95% CI, −0.050 to −0.040; P < .001] at hospitals with below-mean performance and increased from 0.915 to 0.948 [0.033; 95% CI, 0.029 to 0.037; P < .001] at hospitals with above-mean performance). By comparing actual changes in ERRs with expected changes due to RTM, 74.3% to 86.5% of the improvement in ERRs for penalized hospitals was explained by RTM.

    Conclusions and Relevance  Most of the decline in readmission rates in hospitals with high rates during the measurement window for the first year of the HRRP appeared to be due to RTM. These findings seem to call into question the notion of an HRRP policy effect on readmissions.

    Introduction

    The Medicare Hospital Readmissions Reduction Program (HRRP), enacted under the Affordable Care Act in March 2010,1 imposes financial penalties on hospitals with excess 30-day readmission rates. Excess readmission rates measure whether risk-adjusted admissions for a particular condition at a hospital during a preceding reference measurement window exceed the national mean. Beginning in fiscal year (FY) 2013, Medicare penalized hospitals with excess readmissions for heart failure (HF), acute myocardial infarction (AMI), and pneumonia during a reference measurement window from July 1, 2008, through June 30, 2011.2 In particular, Medicare measured excess readmissions using the excess readmissions ratio (ERR), calculated as the ratio of a hospital’s predicted 30-day readmissions to the number that would be expected based on an average hospital with similar patients. Only hospitals with ERRs of greater than 1.0 for targeted conditions are penalized.

    Risk-adjusted readmissions for targeted conditions started to decline in 2011, shortly after the announcement of the HRRP, and the reductions were larger at penalized than nonpenalized hospitals.3,4 Some interpreted this evidence as suggesting that the HRRP’s financial penalties incentivized hospitals to improve transitions of care and implement other strategies that resulted in declines in readmissions.3,4 However, most changes in excess readmissions at penalized hospitals may be explained by regression to the mean (RTM), a statistical phenomenon that occurs when an outcome for an entity is measured repeatedly and the outcome is a random variable.5-11 With repeated measurements over time, outcomes for entities with outcomes farther away from the mean in one period are likely to be recorded closer to the mean in subsequent or preceding periods simply by chance, because more extreme values have a lower probability of occurring than do values near the mean. Therefore, RTM has the potential to produce the appearance of improvement among initially low performers and deterioration among initially high performers. Given that hospitals’ excess readmissions involve randomness, hospitals with excess readmissions before the HRRP—that is, penalized hospitals—would be more likely to exhibit subsequent reductions in excess readmissions due to chance alone. Chance or random events might play an important role in determining excess readmissions because readmissions are not only a function of care processes and treatment provided to patients but also of random biological variation in treatment efficacy or adverse effects and random events such as poor outcomes due to severe weather or falls due to hazardous living conditions.

    The objective of this analysis was to describe and quantify the role of RTM in explaining improvements in performance, relative to national means, at hospitals initially classified as below-mean performers under the HRRP. First, we tested whether hospitals with below-mean baseline performance during the initial HRRP measurement window experienced subsequent improvements in performance after announcement and implementation of the HRRP and, inversely, whether hospitals with above-mean performance experienced subsequent declines in performance after the announcement and implementation of the HRRP.

    Next, we tested 2 hypotheses to examine whether RTM rather than HRRP policy effects played a major role in explaining the above trends in excess readmissions. First, if RTM played a major role, then hospitals with worse than mean baseline performance during an earlier alternate reference measurement window that predates the implementation of the HRRP would exhibit subsequent declines in ERRs. These declines would be qualitatively similar to those observed at hospitals with low performance after the implementation of the HRRP. Second, if RTM played a major role, then hospitals with worse than mean baseline performance during the HRRP would exhibit declines in ERRs going backward in time just as they do going forward in time. Finally, for hospitals with below-mean baseline performance, we used established methods to calculate the percentage of the subsequent improvements in excess readmissions that were attributable to RTM.

    Methods
    Population and Setting

    The study was performed at the hospital level. We included fee-for-service Medicare beneficiaries 65 years or older with an index hospitalization for 1 of the 3 target conditions (HF, AMI, and pneumonia) or chronic obstructive pulmonary disease (COPD) that resulted in discharge alive from February 1, 2006, through June 30, 2014. We excluded index hospitalizations when the patient was admitted with the same condition during the prior 30 days. Study hospitals had at least 25 eligible index hospitalizations for a target condition in the excess readmission measurement period. The study was reviewed by the institutional review board (IRB) of the University of Southern California, Los Angeles, which deemed that our project meets the requirements outlined in 45 CFR 46.101(b)(4) and qualifies for exemption from IRB review. The project was also reviewed by the IRBs of all coauthors and was deemed exempt or approved with expedited review. Informed consent was not required for this study of deidentified patient data. This study met the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.

    Data Sources

    We used data from Medicare Provider and Analysis Review files from February 1, 2006, through June 30, 2014, to identify index hospitalizations and 30-day readmissions to July 30, 2014. Information on patient characteristics came from Master Beneficiary Summary files. Hospital characteristics were based on the 2012 Medicare Provider of Services files and 2012 American Hospital Association Annual Survey.

    Measures

    Following the HRRP methods, we calculated ERRs for each hospital and study period by using linear regression to estimate the predicted number of readmissions (based on patient characteristics and hospital-level random effects) and the expected number (based only on patient characteristics) and then dividing the predicted by the expected number. In these regression models, patient characteristics included age group (65-69, 70-74, 75-79, 80-84, and 85-89 years and a single group ≥90 years), sex, and condition-specific comorbidities documented during the index hospitalization. We used the ERR to measure performance because it reflected how Medicare assigned HRRP penalties and accounted for trends in mean readmission rates, whether owing to the policy or other factors.

    Based on the reference-period ERRs, we stratified eligible hospitals into 2 groups reflecting below-mean (ERR>1.0) or above-mean (ERR≤1.0) performance. These group variables were the main independent variables in the analyses. The main outcome variable was the change in the ERR from the reference period to the subsequent or preceding period.

    Statistical Analysis

    Data were analyzed from January 23, 2018, through March 29, 2019. First, we summarized the characteristics of hospitals used in the analyses. Next, we conducted 3 sets of analyses, described below, with 3 analyses per set (HF, AMI, and pneumonia). In each analysis, we performed linear regression to examine whether the ERRs for a reference period predicted changes in ERRs from the reference period to a subsequent or a preceding period.

    Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

    We examined changes in ERRs from the FY 2013 measurement window (July 1, 2008, through June 30, 2011) to the subsequent 3-year period (July 1, 2011, through June 30, 2014) after the announcement and implementation of the HRRP. We expected that for targeted conditions, hospitals with below-mean performance (ERR>1.0) would experience a decrease in ERRs, and hospitals with above-mean performance (ERR≤1.0) would experience an increase in ERRs. These effects are consistent with RTM and HRRP policy effects (Figure 1). In 1 alternate specification, we repeated this analysis by examining trends in readmissions for hospitals with ERRs of greater than 1.07 (75th percentile of the ERR distribution). Hospitals in the right tail of the ERR distribution had limited incentives to respond to the HRRP because the policy capped penalties; therefore, the hospitals would have to achieve substantial declines in readmissions to reduce the size of the penalties, and substantial declines might have been infeasible. We considered index hospitalizations for COPD in another alternate specification because the HRRP did not create incentives to lower COPD readmissions until FY 2015.

    Analysis 2: Alternate Measurement Period

    We hypothesized that with RTM, hospitals with below-mean performance during a different measurement window predating the HRRP would also exhibit subsequent improvements despite the absence of policy incentives, and performance would deteriorate at hospitals with above-mean performance. To test this hypothesis, we examined changes in ERRs from an alternate measurement period that immediately preceded the FY 2013 measurement window (February 1, 2006, through June 30, 2008; data from before 2006 were not available) to the FY 2013 measurement window.

    Analysis 3: Reverse Chronological Trends

    Regression to the mean suggests that hospitals with below-mean performance during the FY 2013 measurement window would exhibit improvement going backward in time, just as they do going forward in time. Hospitals that are above-mean performers would exhibit deterioration going backward in time. To test this hypothesis, we examined changes in ERRs in reverse-chronological order from the FY 2013 measurement window to the preceding alternate measurement period (February 1, 2006, through June 30, 2008).

    Quantifying Regression to the Mean

    For hospitals that performed below the mean during the FY 2013 measurement window, we quantified the percentage of the subsequent improvement in performance that was due to RTM.9,12 The formula, which is explained in detail in eMethods in the Supplement, captures the 3 phenomena that influence the size of the RTM effect and are easily understood intuitively. First, the expected RTM effect is greater when the outcome in the baseline period is more extreme (ie, farther from the mean of the distribution), because extreme outcomes rarely occur and consequently are very unlikely to be repeated. Second, other things being equal, the expected RTM effect is greater when the variance of the outcome in the baseline period is greater, because a greater variance means that the outcome distribution is more spread out. Consequently, any movement from the tail of the distribution at baseline to the mean in the postmeasurement period represents a greater difference in the outcome. Third, the RTM effect is smaller when the correlation in the outcome between the baseline and postmeasurement periods is higher, because a high correlation means that outcomes tend to be more persistent over time. We used methods described by Linden12 to estimate the expected RTM effect. Two-tailed P < .05 indicated significance.

    Results
    Descriptive Analyses

    Our analyses of eligible hospitals included 3116 for HF, 2196 for AMI, and 3233 for pneumonia. Medicare beneficiaries constituted about half of patients in these hospitals (Table 1). For each condition, more than 60% of hospitals were nonprofit, and more than 70% were located in metropolitan areas.

    Changes in ERRs Between Contiguous Nonoverlapping Measurement Periods
    Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

    For all 3 target conditions, hospitals with below-mean performance (ERR>1.0) in the FY 2013 measurement window exhibited improvements (decreases in ERRs) in the subsequent 3 years, whereas hospitals with above-mean (ERR≤1.0) performance exhibited deterioration (Figure 2). For HF, the mean ERRs declined from 1.086 to 1.038 (−0.048; 95% CI, −0.053 to −0.043; P < .001) at hospitals with below-mean performance and increased from 0.917 to 0.957 (0.040; 95% CI, 0.036 to 0.044; P < .001) at hospitals with above-mean performance. Similarly, for AMI, the mean ERRs declined from 1.106 to 1.043 (−0.063; 95% CI, −0.070 to −0.057; P < .001) at lower-performing hospitals and increased from 0.899 to 0.963 (0.063; 95% CI, 0.057 to 0.070; P < .001) at higher-performing hospitals. For pneumonia, the mean ERRs declined from 1.112 to 1.041 (−0.071; 95% CI, −0.077 to −0.066; P < .001) and increased from 0.898 to 0.955 (0.057; 95% CI, 0.052 to 0.062; P < .001), respectively. Results were qualitatively similar for hospitals with ERRs greater than 1.07 and for patients with COPD (see eTables 1 and 2 and eFigure 1 in the Supplement).

    Analysis 2: Alternate Measurement Period

    For all 3 target conditions, performance improved at hospitals with below-mean performance during an alternate measurement period predating the HRRP and deteriorated at hospitals with above-mean performance (Figure 3). For HF, mean ERRs declined from 1.089 to 1.044 (−0.045; 95% CI, −0.050 to −0.040; P < .001) at hospitals with below-mean performance and increased from 0.915 to 0.948 (0.033; 95% CI, 0.029 to 0.037; P < .001) at hospitals with above-mean performance. Results were similar for AMI and pneumonia.

    Analysis 3: Reverse Chronological Trends

    Results reveal an inverse association between performance during the FY 2013 measurement window and reverse chronological changes in performance from that window to the preceding period. For HF, hospitals with ERRs of greater than 1.0 saw mean ERRs decline from 1.086 to 1.051 (−0.035; 95% CI, −0.040 to −0.030; P < .001), whereas hospitals with mean ERRs equal to or below 1.0 exhibited mean increases from 0.918 to 0.951 (0.033; 95% CI, 0.029 to 0.038; P < .001). Results were similar for AMI and pneumonia (see eFigure 2 in the Supplement).

    Quantifying Regression to the Mean

    Among hospitals with below-mean baseline performance during the FY 2013 measurement window, 74.3% to 86.5% of the improvement observed from that window to the subsequent 3-year period was explained by RTM (Table 2). Similarly, 83.6% to 91.8% of the decline in performance among hospitals with above-mean baseline performance was explained by RTM.

    Discussion

    This analysis found that hospitals with below-mean performance in the measurement window for the first year of HRRP penalties experienced a decline of 4.8 to 7.1 percentage points in excess readmissions during the subsequent 3 years. However, we found strong evidence suggesting that most of this improvement was due to RTM rather than HRRP policy effects. The evidence supporting this conclusion includes the fact that we observed similar changes in excess readmissions at below-mean and above-mean performing hospitals when we defined performance during an alternative measurement period predating the HRRP and when we examined changes in performance going backward rather than forward in time. These observations held for all 3 target conditions, including HF, AMI, and pneumonia. Furthermore, readmissions for patients with COPD and for hospitals in the right tail of the ERR distribution, where there was no incentive to improve performance, show a similar trend across time. Finally, quantitative analyses indicated that at least three-fourths of the improvement observed at hospitals with below-mean performance during the FY 2013 measurement window was explained by RTM.

    Regression to the mean was first conceived by Sir Francis Dalton in 1886 to explain why tall parents have children shorter than themselves and short parents have children taller than themselves.13 Despite its long history, RTM remains an overlooked statistical phenomenon. Observers often erroneously ascribe improvements among low performers to an intervention of interest when the true cause is RTM. Consistent with our findings of RTM, Press et al14 detected decreases in readmissions from 2009 (the year before the HRRP was announced) to 2011 (the year before it was implemented) at hospitals with initially high rates and increases at hospitals with initially low rates. Our results expand on these previous findings by examining a longer time frame, including after the HRRP was implemented, using multiple strategies to document the effects of RTM and quantifying the size of that effect.

    The fact that RTM had such a large effect on trends in hospital performance has 3 important implications. First, it suggests that the HRRP explains at most a small portion of the more favorable trends in readmissions that occurred at penalized compared with nonpenalized hospitals. Hospitals that were penalized in FY 2013 experienced a financial shock due to excess readmissions, but our results suggest that this did not result in greater declines in excess readmissions. However, this does not mean that the HRRP did not lead to a decline in readmissions. Other researchers15,16 have evaluated the HRRP using analytical strategies that are not biased by RTM and found that the policy was associated with a decline in readmissions. Other investigators17,18 have suggested that the decline in risk-adjusted readmissions across all hospitals might have been inflated owing to nationwide changes in coding practices.

    The second implication of our findings is that hospital performance and the assignment of penalties under the HRRP is more strongly influenced by chance. The results suggest that hospitals that were penalized based on their ERRs in the baseline period converged to the mean in the postmeasurement period. This possibility suggests that bad luck rather than poor-quality care explains the poor performance in the baseline period. Moreover, if hospitals have limited abilities to control their performance, they may be less likely to sustain efforts to improve transition-related care and reduce readmissions during the long term.

    The third implication of our findings pertains to the ability to detect associations between trends in readmissions and mortality. Dharmarajan et al19 found hospital-level improvements in readmissions were only weakly associated with declines in mortality. Our findings suggest that chance may have played such a large role in changes in readmissions that the ability to detect a true association with mortality was limited, an example of regression dilution bias.5

    Limitations

    The results of our analysis should be viewed in light of its limitations. This study analyzes observational data on trends in readmissions at high- and low-performing hospitals after the implementation of the HRRP. Ideally, we would have preferred having data from a control group of hospitals that were unaffected by the HRRP; however, that was not possible given that the policy was implemented nationally. The study would also have been stronger if we had richer data on care processes at hospitals. With such data we may have been able to directly analyze changes in care processes at high- and low-performing hospitals and more definitively assess the extent to which RTM or changes in care processes explain trends in readmissions at these hospitals. In addition, the analysis focuses on low- and high-performing hospitals as a group. We cannot conclude whether declines in readmissions at individual hospitals were due to RTM, favorable changes in transition-related care, or other factors.

    Conclusions

    Hospitals with poor baseline performance during the first year of HRRP penalties appeared to exhibit substantial improvement relative to other hospitals nationally during the subsequent years. This analysis found that three-fourths of the relative improvement was due to RTM rather than the policy. Modifications to the HRRP that better account for the role of chance, such as excluding small hospitals with greater variability in readmissions from the HRRP or calculating mean readmissions across multiple conditions when estimating excess readmissions, might enhance its fairness and possibly also its effectiveness.

    Back to top
    Article Information

    Accepted for Publication: March 29, 2019.

    Corresponding Author: Neeraj Sood, PhD, Sol Price School of Public Policy, Department of Health Policy and Management, University of Southern California, 635 Downey Way, Verna and Peter Dauterive Hall, Ste 512, Los Angeles, CA 90089 (nsood@healthpolicy.usc.edu).

    Published Online: June 26, 2019. doi:10.1001/jamainternmed.2019.1004

    Author Contributions: Mr Joshi had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Joshi, Nuckols, Escarce, Huckfeldt, Sood.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Joshi, Nuckols, Huckfeldt, Popescu, Sood.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Joshi, Escarce, Huckfeldt, Sood.

    Obtained funding: Nuckols, Huckfeldt, Sood.

    Administrative, technical, or material support: Sood.

    Supervision: Nuckols, Escarce, Sood.

    Conflict of Interest Disclosures: Mr Joshi reported receiving grants from the Agency for Healthcare Research and Quality (AHRQ) during the conduct of the study. Dr Nuckols reported receiving grant R01 HS024284 from the AHRQ during the conduct of the study and grants from the National Institute on Drug Abuse, National Institute of Diabetes and Digestive and Kidney Diseases, Patient-Centered Outcomes Research Institute, National Institutes of Health (NIH)/National Center for Advancing Translational Sciences, National Heart, Lung and Blood Institute, AHRQ, NIH/National Institute on Aging, and American Association of Neuromuscular and Electrodiagnostic Medicine outside the submitted work. Dr Escarce reported receiving grant R01 HS024284 from the AHRQ during the conduct of the study. Dr Huckfeldt reported receiving grants from the AHRQ during the conduct of the study and funding outside of the submitted work from the Assistant Secretary of Planning and Evaluation (ASPE) in the Department of Health and Human Services, Centers for Medicare & Medicaid Services, the Florida Medical Malpractice Joint Underwriting Association Patient Safety Grant Program, and the NIH; serving as a paid consultant to the RAND Corporation on NIH-funded research and as a peer reviewer; and serving as a paid consultant to the Urban Institute on a project funded by ASPE. Dr Popescu reported receiving grants from the AHRQ during the conduct of the study. Dr. Sood reported receiving grants from the AHRQ during the conduct of the study; serving as a scientific advisor for Virta Health, a start-up aiming to reverse diabetes through medication management and lifestyle change; receiving compensation for serving as an expert for the American Medical Association advising on the competitive effects of the CVS-Aetna merger; receiving compensation for serving as an expert for a consortium of law firms representing cities and jurisdictions suing pharmaceutical firms and distributors for the opioid crisis; receiving compensation for serving as an expert for China Development Research Foundation (Government of China) and the Pharmaceutical Research and Manufacturers of America on improving policies for controlling drugs costs and improving access to prescription drugs in China; and receiving grants from NIH, National Institute for Health Care Management Foundation, and Patient-Centered Outcomes Research Institute outside the submitted work. No other disclosures were reported.

    Previous Presentation: This article was presented at the American Society of Health Economists conference; June 26, 2019; Washington, DC.

    References
    1.
    Centers for Medicare & Medicaid Services. Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
    2.
    Centers for Medicare & Medicaid Services. Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
    3.
    Desai  NR, Ross  JS, Kwon  JY,  et al.  Association between hospital penalty status under the Hospital Readmission Reduction Program and readmission rates for target and nontarget conditions.  JAMA. 2016;316(24):2647-2656. doi:10.1001/jama.2016.18533PubMedGoogle ScholarCrossref
    4.
    Wasfy  JH, Zigler  CM, Choirat  C, Wang  Y, Dominici  F, Yeh  RW.  Readmission rates after passage of the hospital readmissions reduction program: a pre-post analysis.  Ann Intern Med. 2017;166(5):324-331. doi:10.7326/M16-0185PubMedGoogle ScholarCrossref
    5.
    Barnett  AG, van der Pols  JC, Dobson  AJ.  Regression to the mean: what it is and how to deal with it.  Int J Epidemiol. 2005;34(1):215-220. doi:10.1093/ije/dyh299PubMedGoogle ScholarCrossref
    6.
    Atkinson  G, Loenneke  JP, Fahs  CA, Abe  T, Rossow  LM.  Individual differences in the exercise-mediated blood pressure response: regression to the mean in disguise?  Clin Physiol Funct Imaging. 2015;35(6):490-491. doi:10.1111/cpf.12211PubMedGoogle ScholarCrossref
    7.
    Bland  JM, Altman  DG.  Regression towards the mean.  BMJ. 1994;308(6942):1499. doi:10.1136/bmj.308.6942.1499PubMedGoogle ScholarCrossref
    8.
    Kelly  C, Price  TD.  Correcting for regression to the mean in behavior and ecology.  Am Nat. 2005;166(6):700-707. doi:10.1086/497402PubMedGoogle ScholarCrossref
    9.
    Linden  A.  Assessing regression to the mean effects in health care initiatives.  BMC Med Res Methodol. 2013;13(1):119. doi:10.1186/1471-2288-13-119PubMedGoogle ScholarCrossref
    10.
    Yudkin  PL, Stratton  IM.  How to deal with regression to the mean in intervention studies.  Lancet. 1996;347(8996):241-243. doi:10.1016/S0140-6736(96)90410-9PubMedGoogle ScholarCrossref
    11.
    Bland  JM, Altman  DG.  Some examples of regression towards the mean.  BMJ. 1994;309(6957):780. doi:10.1136/bmj.309.6957.780PubMedGoogle ScholarCrossref
    12.
    Linden  A.  RTMCI: Stata Module to Estimate Regression to the Mean Effects With Confidence Intervals. Boston, MA: Statistical Software Components; 2013.
    13.
    Stigler  SM.  Regression towards the mean, historically considered.  Stat Methods Med Res. 1997;6(2):103-114. doi:10.1177/096228029700600202PubMedGoogle ScholarCrossref
    14.
    Press  MJ, Scanlon  DP, Ryan  AM,  et al.  Limits of readmission rates in measuring hospital quality suggest the need for added metrics.  Health Aff (Millwood). 2013;32(6):1083-1091. doi:10.1377/hlthaff.2012.0518PubMedGoogle ScholarCrossref
    15.
    Gupta  A. Impacts of performance pay for hospitals: the readmissions reduction program. Becker Friedman Institute for Research in Economics Working Paper No. 2017-07. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3054172. Posted October 16, 2017. Accessed May 29, 2019.
    16.
    Zuckerman  RB, Sheingold  SH, Orav  EJ, Ruhter  J, Epstein  AM.  Readmissions, observation, and the Hospital Readmissions Reduction Program.  N Engl J Med. 2016;374(16):1543-1551. doi:10.1056/NEJMsa1513024PubMedGoogle ScholarCrossref
    17.
    Ibrahim  AM, Dimick  JB, Sinha  SS, Hollingsworth  JM, Nuliyalu  U, Ryan  AM.  Association of coded severity with readmission reduction after the Hospital Readmissions Reduction Program.  JAMA Intern Med. 2018;178(2):290-292. doi:10.1001/jamainternmed.2017.6148PubMedGoogle ScholarCrossref
    18.
    Ody  C, Msall  L, Dafny  LS, Grabowski  DC, Cutler  DM.  Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated.  Health Aff (Millwood). 2019;38(1):36-43. doi:10.1377/hlthaff.2018.05178PubMedGoogle ScholarCrossref
    19.
    Dharmarajan  K, Wang  Y, Lin  Z,  et al.  Association of changing hospital readmission rates with mortality rates after hospital discharge.  JAMA. 2017;318(3):270-278. doi:10.1001/jama.2017.8444PubMedGoogle ScholarCrossref
    ×