This graph shows the distribution of expected performance changes among hospitals after adjusting for the additional clinical and social risk factors found in the Box. Hospitals are grouped on the y axis by decile of readmission performance adjusted for age, sex, and recent comorbidities (ie, standard Centers for Medicare & Medicaid Services [CMS] adjustments). On the x axis, changes are reported in percentage points of the readmission rate. The plotted distribution reflects 5000 draws from the empirical covariances of hospitals’ unadjusted readmission rates with readmission rates predicted from the base CMS variables, as well as the additional clinical and social characteristics listed in the Box. After additional adjustments in our analysis, approximately 6.0% of hospitals originally above the median would move below the median (plotted in dark blue), while a similar proportion of hospitals below the median would be expected to move above the median (plotted in orange). See eFigure 2 in the Supplement, which plots the distribution of expected changes in the rate of 30-day readmission or mortality following additional adjustments.
These graphs display average risk-adjusted outcomes by quintile of hospitals before and after adjusting for clinical and social covariates found in the Box, in addition to base variables used by the Centers for Medicare & Medicaid Services (CMS) to adjust for risk of readmissions. The graphs display mean adjusted rates among all patients admitted to hospitals in each quintile. A and C, Hospitals are grouped into quintiles by patients’ clinical and social risk according to the additional characteristics examined. B and D, Hospitals are grouped into quintiles by the proportion of Medicare patients dually enrolled in Medicaid. The clinical and social risk of Medicare patients was assessed by constructing patient-level risk scores predicted by the average within-hospital associations between each outcome and the added clinical and social variables from the Box, controlling for base CMS variables and hospital fixed effects. Random effects models were used to estimate hospital-level averages of these risk scores or rates of dual enrollment in Medicaid, and the resulting estimates were used to group hospitals into quintiles (see eAppendix 3 in the Supplement for details).
eAppendix 1. Derivation of analytic cohort
eFigure 1. Derivation of analytic cohort
eAppendix 2. Description of covariates
eAppendix 3. Statistical analyses
eTable 1. Within-hospital association between a unit change in patient characteristics and outcomes
eTable 2. Hospital-level standard deviation and correlation of risk-adjusted readmissions/readmission or mortality, comparing base CMS adjustments vs further adjustment for patients’ clinical and social characteristics
eFigure 2. Expected changes in the risk-adjusted rate of readmission or mortality after adjusting for additional clinical and social characteristics of patients
eFigure 3. Distribution of expected changes in hospital readmission rates following adjustment for additional patient characteristics
eFigure 4. Distribution of expected changes in the composite rate of readmission or mortality following adjustment for additional patient characteristics
eTable 3. Simulated change in HRRP penalties after adjusting for additional clinical and social risk factors for readmission
eAppendix 4. Supplementary analyses
eTable 4. Differences in predicted within-hospital associations between social factors and outcomes between hospitals in the top and bottom quintiles based on Medicare patients’ social risk
eTable 5. Hospital-level standard deviation and correlation of risk-adjusted readmissions following index admissions for HRRP-targeted conditions
eFigure 5. Distribution of expected changes in readmissions following admissions for HRRP-targeted conditions
eTable 6. Hospital-level standard deviation and correlation of risk-adjusted readmissions, comparing adjustment for standard CMS variables plus dual enrollment versus adjustments for additional clinical and social characteristics
eFigure 6. Risk-adjusted readmission rates for hospitals grouped into quintiles by Medicare patients’ readmission risk or dual enrollment in Medicaid (base model includes dual enrollment status)
eTable 7. Hospital-level scores, comparing adjustment for within-hospital vs within- and between-hospital associations between patient characteristics and outcomes
Customize your JAMA Network experience by selecting one or more topics from the list below.
Roberts ET, Zaslavsky AM, Barnett ML, Landon BE, Ding L, McWilliams JM. Assessment of the Effect of Adjustment for Patient Characteristics on Hospital Readmission Rates: Implications for Pay for Performance. JAMA Intern Med. 2018;178(11):1498–1507. doi:10.1001/jamainternmed.2018.4481
To what extent are differences in hospital readmission rates explained by measurable patient characteristics not used by Medicare to adjust for risk of readmissions in pay-for-performance programs?
This observational study of Medicare claims and US Census data found that adjustment for additional clinical and social variables reduced overall hospital variation in readmission rates by 9.6%, narrowed differences in rates between hospitals serving higher vs lower proportions of high-risk Medicare patients by 54%, and reduced expected penalties by 41% among the 10% of hospitals with the largest penalty reductions.
Hospitals are penalized to some extent for serving sicker and poorer patients rather than for poorer quality of care, which highlights the need for improved risk adjustment in pay-for-performance programs.
In several pay-for-performance programs, Medicare ties payments to readmission rates but accounts only for a limited set of patient characteristics—and no measures of social risk—when assessing performance of health care providers (clinicians, practices, hospitals, or other organizations). Debate continues over whether accounting for social risk would mitigate inappropriate penalties or would establish lower standards of care for disadvantaged patients if they are served by lower-quality providers.
To assess changes in hospital performance on readmission rates after adjusting for additional clinical and social patient characteristics by using methods that distinguish the association between patient characteristics and readmission from between-hospital differences in quality.
Design, Setting, and Participants
Using Medicare claims for admissions in 2013 through 2014 and linked US Census data, we assessed several clinical and social characteristics of patients that are not currently used for risk adjustment in the Hospital Readmission Reduction Program. We compared hospital readmission rates with and without adjustment for these additional characteristics, using only the average within-hospital associations between patient characteristics and readmission as the basis for adjustment, thereby appropriately excluding hospitals’ distinct contributions to readmission from the adjustment.
Main Outcomes and Measures
All-cause readmission within 30 days of discharge.
The study sample consisted of 1 169 014 index admissions among 1 003 664 unique Medicare beneficiaries (41.5% men; mean [SD] age, 79.9 [8.3] years) in 2215 hospitals. Compared with adjustment for patient characteristics currently implemented by Medicare, adjustment for the additional characteristics reduced overall variation in hospital readmission rates by 9.6%, changed rates upward or downward by 0.37 to 0.72 percentage points for the 10% of hospitals most affected by the additional adjustments (±30.3% to ±58.9% of the hospital-level standard deviation), and would be expected to reduce penalties (in relative terms) by 52%, 46%, and 41% for hospitals with the largest 1%, 5%, and 10% of penalty reductions, respectively. The additional adjustments reduced the mean difference in readmission rates between hospitals in the top and bottom quintiles of high-risk patients by 0.53 percentage points (95% CI, 0.50-0.55; P < .001), or 54% of the difference estimated with CMS adjustments alone. Both clinical and social characteristics contributed to these reductions, and these reductions were considerably greater for conditions targeted by the Hospital Readmission Reduction Program. Adjustment for social characteristics resulted in greater changes in rates of readmission or death than in rates of readmission alone.
Conclusions and Relevance
Hospitals serving higher-risk patients may be penalized substantially because of the patients they serve rather than their quality of care. Adjusting solely for within-hospital associations may allow adjustment for additional patient characteristics to mitigate unintended consequences of pay for performance without holding hospitals to different standards because of the patients they serve.
Readmission rates are used by the Centers for Medicare & Medicaid Services (CMS) to assess quality of care and determine payment adjustments to health care providers (clinicians, practices, hospitals, or other organizations) in several Medicare pay-for-performance programs, including the Hospital Readmission Reduction Program (HRRP), the Merit-based Incentive Payment System, and the Medicare Shared Savings Program.1-3 Although the purported goal of pay for performance is to improve quality of care, there is growing concern that it may ultimately result in worse quality for vulnerable groups by penalizing providers for serving higher-risk patients, independent of the quality of care provided.4-10 Underlying this concern is the fact that CMS adjusts readmissions for few patient characteristics.11 In the HRRP, for example, CMS adjusts for patients’ age, sex, and diagnoses recorded in the 12 months of claims preceding an admission but not for other clinical factors, such as disability or patients’ full history of chronic conditions, nor for social factors such as poverty.4,12-17
Because of limited risk adjustment, and evidence that hospitals serving higher-risk patients have received larger penalties,1,18-20 some have proposed accounting for additional patient characteristics—including measures of social risk—in pay-for-performance programs.5,6,9-11,21-23 Toward this end, the 21st Century Cures Act24 has directed CMS to begin comparing readmission rates among hospitals serving similar proportions of Medicare patients dually enrolled in Medicaid when calculating HRRP penalties.25 However, this stratification approach (to begin in 2019) raises concerns that hospitals serving different patient populations will be held to different standards. If dually enrolled beneficiaries are disproportionately treated at lower-quality hospitals, the new adjustment would conceal between-hospital differences in quality that contribute to worse outcomes for socially vulnerable patients.11,26-28 Similarly, using regression instead of stratification to adjust for the dually enrolled proportion of a hospital’s patient population would inappropriately adjust for some differences in quality between hospitals in this scenario, rather than only for the association of dual enrollment with readmission—an association that would be expected in both high- and low-quality hospitals.
Risk-adjustment approaches like these are problematic because they conflate patient- and provider-level factors associated with readmission, either explicitly through stratification or implicitly by adjusting for provider-level differences that may be correlated with quality.26 Accordingly, some have argued that adjusting for social risk factors would establish a lower standard of care for providers serving socially disadvantaged groups,27 but this is not a necessary consequence of risk adjustment.3,26,29 Approaches that distinguish patient-level vulnerability from hospital-level factors can better support the goal of risk adjustment, which is to differentiate provider quality from performance differences that are due to differences in patients characteristics.3,26,29
In this study, we compared hospitals’ readmission rates before and after adjusting for several sets of patient characteristics in addition to those CMS currently uses to adjust for risk of readmissions.16 We focused on clinical and social characteristics observable in Medicare claims and linked US Census data, acknowledging in our interpretation of results that these measures incompletely describe patient risk.4,23,26,30,31 We based the adjustments on the average within-hospital associations between patient characteristics and readmissions, thereby excluding hospitals’ distinct contributions to readmissions from our adjustments.3,26,29 Using this approach, we quantified the impact of adjustment for additional patient characteristics on overall hospital variation in readmission performance, on performance scores among hospitals most affected by the adjustments, and on penalties that would be expected in a program structured like the HRRP.
We analyzed Medicare admissions from 2013 through 2014 for a random 20% sample of beneficiaries. In each year, we identified beneficiaries who had at least 1 hospital admission and were continuously enrolled in the traditional fee-for-service program during the year of admission (while alive in the case of decedents) and in the preceding year. Consistent with CMS methods for assessing readmissions, we limited our sample to patients 65 years and older (eAppendix 1 in the Supplement).16 The study was approved by the Harvard Medical School Committee on Human Studies, and informed consent was waived because the analyses used deidentified secondary data, which involved no contact with patients.
For each beneficiary in each year, we included the first admission in the year that met criteria for an index admission and also conducted a sensitivity analysis in which we randomly selected 1 admission per beneficiary annually. We defined index admissions similarly to the HRRP but included admissions for any condition (not just those targeted by the HRRP), because the larger resulting sample enabled us to more reliably estimate systematic between-hospital variation in readmission rates, which is consistent with proposals to improve HRRP accuracy by including more index conditions.32 Similar all-condition readmission rates are publicly reported by CMS and are a quality measure in the Merit-based Incentive Payment System.33 Although we included index admissions for all conditions, we analyzed 1 admission per beneficiary, rather than all available index admissions, so that the distribution of admissions per beneficiary in our sample would more closely resemble the distribution for a condition-specific index admission assessed by the HRRP. Because we sampled 1 index admission per patient annually, the readmissions rates we report are expectedly smaller than the rates reported by CMS.
We excluded from consideration as index admissions those that ended with transfer to another hospital, admissions in which the patient died in the hospital, and admissions occurring within 30 days of a hospital discharge for the same patient (ie, readmissions were not counted as index admissions). For consistency with the HRRP, which has focused largely on index admissions for unplanned reasons, we also excluded index admissions for planned procedures. To more reliably estimate between-hospital variation in readmission rates, we limited our analyses to index admissions occurring in hospitals with at least 200 admissions during 2013 through 2014 (eFigure 1 in the Supplement shows the derivation of the study sample).
For admissions meeting our inclusion criteria, we assessed whether the patient was readmitted to any acute care hospital within 30 days of the discharge date of the index admission, excluding readmissions for planned procedures.16 We also assessed a combined outcome of readmission or mortality within 30 days of discharge to capture potential effects of hospital quality on mortality, address the problems of competing risks and censoring present in analyses ignoring deaths, and explore whether social risk factors might predict death more strongly than readmission because they may reflect differences in demand for inpatient care (eg, income or supplemental insurance coverage).
Following CMS specifications for the construction of hospital-wide readmission rates, for each index admission we used Medicare enrollment and claims data to determine the base set of characteristics used by CMS for risk adjustment: patients’ age and sex, the principal diagnosis (aggregated into clinical categories),34 and the presence of 31 conditions recorded during the index admission or in the preceding 12 months (eAppendix 2 in the Supplement).16
From the Chronic Conditions Data Warehouse,35 which draws from Medicare claims since 1999 to measure beneficiaries’ cumulative chronic disease burden, we assessed the presence of 27 conditions reported prior to the index admission year. Using beneficiaries’ claims from the year prior to admission, we also constructed indicators for the condition categories included in the CMS Hierarchical Condition Categories risk score.36 We used claims from the year of admission to determine whether beneficiaries were long-term nursing home residents.37 Finally, we used enrollment data to determine whether beneficiaries were originally eligible for Medicare because of a qualifying disability and whether they had end-stage renal disease in the year of admission.
We used enrollment data from Medicare’s Master Beneficiary Summary File38 to identify beneficiaries with dual enrollment in Medicaid (a marker of low income and assets). We categorized dual enrollment by disability status to differentiate beneficiaries who became eligible for Medicaid as a result of disability and low income from those who became eligible as a result of age and low income. We also used enrollment data to assess beneficiaries’ receipt of benefits from state Medicare Savings Programs, which provide partial Medicaid benefits to low-income Medicare beneficiaries, and receipt of the Medicare Part D low-income subsidy among beneficiaries who did not automatically receive the low-income subsidy as a result of Medicaid coverage (eAppendix 2 in the Supplement).31,39
We included data from the American Community Survey40 to assess the following area-level socioeconomic variables13: educational attainment, poverty rates, household income, and the proportion of residents living alone. We assessed these characteristics among individuals 65 years and older at 2 levels of geography: US Census tracts, which contain an average of 4000 residents each and approximate beneficiaries’ immediate neighborhoods; and zip code tabulation areas, with a mean population size of approximately 9400 people (eAppendix 2 in the Supplement).31
We first fitted linear regression models predicting readmission as a function of hospital fixed effects, indicators for the month and year of admission, and different sets of patient characteristics: (1) base CMS variables alone, (2) base variables plus additional clinical variables, (3) base variable plus additional social variables, and (4) all patient variables (Box). By including hospital fixed effects in each set of models, we estimated the average within-hospital associations (ie, pooled across hospitals) between patient characteristics and readmission (eAppendix 3 in the Supplement). We used these average within-hospital associations to calculate each hospital’s predicted readmission rate based on only the observed characteristics of its patients, appropriately excluding the hospital’s distinct contribution to readmissions. For each patient-admission observation, we calculated a score by subtracting the patient’s predicted probability of readmission from an indicator of whether the patient was readmitted. The mean score for a hospital equals its adjusted readmission rate. We calculated 4 sets of scores, 1 for each set of adjusters. Results were similar when using logistic instead of linear regression for this part of the analysis (eAppendix 3 in the Supplement).
Age and sex
31 Disease indicators reported on beneficiaries’ claims in the prior yearb
Primary diagnosis of index admissionc
Disability was original reason for Medicare enrollment
End-stage renal disease
Long-term residence in a nursing homef
Dual Medicare and Medicaid enrollmenth
Recipients of a Medicare Savings Programi
Recipients of the Part D low-income subsidyj
No subsidies or prescription drug coveragek
Poverty rate, household income,l educational attainment,m and proportion of residents living alone in the beneficiary’s zip code and US Census tractn
Interactions among clinical and social variableso
Abbreviations: AHRQ, Agency for Healthcare Research and Quality; CMS,Centers for Medicare & Medicaid Services; CWW, Chronic Conditions Data Warehouse; HCC, Hierarchical Condition Categories.
a In addition to the variables listed, adjustment was made for the month of the sampled index admission (month fixed effects) and an indicator of whether the index admission was in 2014 (vs 2013).
b Indicators for the presence of 31 comorbidities reported for the patient’s index admission or on Medicare claims up to 365 days before the index admission.
c Consistent with CMS risk-adjustment methods for hospital-wide readmissions, indicators for primary admission diagnoses were included and grouped into 207 mutually exclusive categories using AHRQ Clinical Classification Software. Indicators for 78 diagnostic categories were included, and other conditions with low prevalence (<0.1%) were consolidated into a single category.
d Indicators for HCCs included in the CMS-HCC risk-adjustment model, assessed using prior-year Medicare claims. Of 70 condition indicators in the HCC risk-adjustment model, 63 were included, excluding 7 low-prevalence conditions (<0.1%).
e Data were used from the CCW, which draws from Medicare claims since 1999 to characterize beneficiaries’ accumulated burden of chronic disease, to assess the presence of 27 chronic conditions reported prior to each study year: Alzheimer disease, Alzheimer disease and related disorders or senile dementia, anemia, asthma, atrial fibrillation, benign prostatic hyperplasia, breast cancer, cataract, chronic kidney disease, chronic obstructive pulmonary disease, colorectal cancer, depression, diabetes, endometrial cancer, glaucoma, heart failure, hip or pelvic fracture, hyperlipidemia, hypertension, hypothyroidism, ischemic heart disease, lung cancer, osteoporosis, prostate cancer, acute myocardial infarction, rheumatoid arthritis, and stroke or transient ischemic attack. Indicators for each of these conditions and, to account for the potential nonadditivity of combinations of conditions, indicators for 6 or more and 9 or more conditions were included.
f Beneficiary received care in a nursing home for 2 or more months of the year.
g Beneficiaries were categorized into mutually exclusive categories of insurance coverage. Receiving prescription drug coverage through the Medicare Part D program, an employer, or another source was omitted as the reference category. See eAppendix 2 in the Supplement for details on codes used to assign patients to insurance categories.
h Beneficiaries with full Medicaid coverage (full-benefit duals). To account for differences in state Medicaid income eligibility limits for elderly and disabled persons, an indicator for dual enrollment was interacted with the state’s Medicaid income limit (in percentage points of the Federal Poverty Level).
i Includes partial Medicaid enrollees in the Qualified Medicare Beneficiary, Specified Low-Income Medicare Beneficiary, and Qualified Individual programs.
j Receives the Part D low-income subsidy without contemporaneous enrollment in Medicaid or a Medicare Savings Program.
k Beneficiary did not receive full or partial Medicaid benefits, the Part D low-income subsidy, or prescription drug coverage.
l Percent with incomes below $50 000.
m Percent with less than a high school education and the percent with a college education or higher.
n All area-level variables were assessed among residents 65 years and older. We linked US Census tract–level data to beneficiaries using the 9-digit zip code reported in Medicare enrollment data (eAppendix 2 in the Supplement).
o Two-way interactions between insurance characteristics, disability, end-stage renal disease, age, count of CCW conditions, count of HCC indicators, and long-term nursing home residence.
To model the relationships among the different scores at the hospital level, we fitted hierarchical regression models with hospital random effects to patient-admission–level scores. We used these models to estimate variances in hospital-level readmission rates adjusted for the different sets of characteristics and the covariances between the differently adjusted rates (eAppendix 3 in the Supplement). Importantly, this approach yielded estimates net of sampling error, thus eliminating the contribution of random annual fluctuations in readmission or case mix to assessments of the between-hospital variation in readmission rates. Such random fluctuations might alter annual hospital performance and penalties but without meaningful long-run consequences for policy or hospital finances. An approach that fails to account for the sampling error would exaggerate the systematic, policy-relevant effects of adjustments for patient characteristics.
Using the estimated variances and covariances for the differently adjusted rates, we simulated changes in hospital readmission rates that would be expected from adjustment for each additional set of characteristics and calculated the extent of changes among hospitals most affected by these adjustments (recognizing that some hospitals without the highest readmission rates may be among the most affected). From the relationship between publicly reported readmission rates and penalties, we then quantified the penalty changes that would be associated with the estimated changes in readmission rates (eAppendix 3 in the Supplement).
In a complementary analysis using the same approach to risk adjustment, we compared readmission rates among quintiles of hospitals categorized in 1 of 2 ways: by the hospital’s proportion of Medicare patients who were dually enrolled in Medicaid or by the hospital mean of a patient risk score calculated from all of the additional clinical and social characteristics listed in the Box (refer to eAppendix 2 in the Supplement for the accompanying description of methods). These comparisons quantified the impact of the additional adjustments on performance differences between hospitals that differed specifically in the additional characteristics we assessed or the 1 additional characteristic (dual enrollment) for which CMS plans to adjust through its stratification approach.25
We performed 3 sets of supplementary analyses. First, we assessed whether the within-hospital association between patients’ social characteristics and readmission varied systematically with the average social risk of a hospital’s patients (eAppendix 4 in the Supplement). Such a relationship would complicate the interpretation and reporting of between-hospital differences in quality, as it suggests the possibility that between-hospital differences could vary for different patient subgroups. This would not necessarily make risk adjustment inappropriate and might even provide additional insight into mechanisms that underlie differences in hospital performance for different subgroups. Second, we limited our analyses to index admissions for conditions targeted by the HRRP. Third, we included dual enrollment status in the set of base CMS adjustments, given the planned adjustment for dual enrollment.
The study sample consisted of 1 169 014 index admissions among 1 003 664 unique Medicare beneficiaries in 2215 hospitals. During the study period, the mean unadjusted readmission rate was 11.9% and ranged from 8.9% to 14.8% for hospitals in the lowest to the highest quintiles of readmission rates (Table). The unadjusted standard deviation in readmission rates across hospitals (estimated net of sampling error) was 1.8%, and after standard CMS adjustments it was 1.2%.
Patient characteristics varied substantially across hospitals (Table). In analyses adjusting for the admitting hospital, all of the patient-level indicators of greater clinical or social risk, and most of the area-level measures indicating greater social risk, were significantly associated with a greater likelihood of readmission (Table). For most characteristics we assessed, patients admitted to hospitals in the highest quintile of unadjusted readmission rates exhibited significantly greater clinical and social risk than patients admitted to hospitals with the lowest readmission rates (Table).
Compared with adjustment for the variables currently used by CMS, adjustment for the additional clinical and social characteristics reduced the variance in readmission rates across hospitals to 1.35 percentage points from 1.49 percentage points—a relative reduction of 9.6% (eTable 2 in the Supplement). As illustrated in Figure 1, after additional adjustments, readmission rates declined on average among hospitals initially above the national median (based on CMS adjustments), with the greatest reduction among hospitals starting in the highest decile, and increased on average for hospitals below the median, with the greatest increase among hospitals starting in the lowest decile.
Among the 10% of hospitals most affected by the additional clinical and social adjustments, readmission rates changed upward or downward by 0.37 to 0.72 percentage points, which was equivalent to ±30.3% to ±58.9% of the hospital-level standard deviation under standard CMS adjustments (eFigure 3 in the Supplement). The additional adjustments would be expected to reduce HRRP penalties (assessed as a percentage of hospitals’ total inpatient Medicare revenue) by 1.2 percentage points for the 1% most affected hospitals (52% of their mean penalty of 2.29%), by 0.81 percentage points for the 5% most affected hospitals (46% of their mean penalty of 1.77%), and by 0.63 percentage points for the 10% most affected hospitals (41% of their mean penalty of 1.55%) (eTable 3 in the Supplement).
The additional clinical and social characteristics both contributed to these changes in performance scores and expected penalties but overlapped considerably in the variation in hospital readmission rates they predicted (eTable 2 in the Supplement). Adjustment for the social characteristics had a substantially greater impact on the composite outcome of readmission or death than on readmission alone (eTable 2 in the Supplement).
Under current CMS methods for adjusting readmissions, the mean adjusted readmission rate was 0.98 percentage points higher (95% CI, 0.72-1.23 percentage points; P < .001) among hospitals in the highest quintile of patient readmission risk (predicted from the additional clinical and social characteristics) than among hospitals in the lowest quintile (Figure 2A). Adjusting for additional clinical characteristics narrowed this difference to 0.73 percentage points, and adjusting for both additional clinical and social characteristics narrowed it to 0.45 percentage points—a 0.53 percentage-point reduction (95% CI, 0.50-0.55 percentage points; P < .001), equivalent to 54% of the difference under base adjustments. The additional adjustments narrowed differences between hospitals in the highest and lowest quintiles of dually enrolled patients by 83% relative to the difference based on standard CMS adjustments (Figure 2B). Effects of the additional adjustments on between-quintile differences in rates of readmission or death were even greater (Figure 2C and D).
We found no evidence that within-hospital differences between patients with higher vs lower social risk were systematically greater in hospitals that served more socially high-risk patients (eTable 4 in the Supplement). The effects of adjusting for the additional social and clinical variables were greater when index admissions were limited to those for conditions targeted by the HRRP (eTable 5 and eFigure 5 in the Supplement). Specifically, these additional adjustments reduced the hospital-level variance of readmissions for HRRP-targeted conditions by 16.6% and changed readmission rates upward or downward by 0.66 to 1.14 percentage points for the most affected 10% of hospitals.
In our hospital-wide readmission measure, dual enrollment explained only part of the effects of the additional adjustments. For example, after adding dual enrollment status to the base model, adjustment for the remaining additional social and clinical characteristics reduced the variance in readmission rates between hospitals by 6.8% (eTable 6 in the Supplement), changed the most affected 10% of hospitals' readmission rates upward or downward by 0.20 to 0.42 percentage points, and narrowed differences between hospitals in the highest vs lowest quintiles of patients risk by 36% (eFigure 6 in the Supplement). Finally, our results were substantively unchanged when we sampled a random index admission per patient in each year, rather than the patient’s first index admission of the year.
We found substantial variation across hospitals in the prevalence of patients’ clinical and social risk factors for readmission that CMS does not use to adjust readmission rates. Despite the limited number and breadth of additional patient characteristics we could examine using Medicare and US Census data, adjusting for these characteristics reduced the variation in readmission rates across hospitals and had an appreciable effect on readmission rates and expected penalties for hospitals disproportionately serving clinically and socially vulnerable patients.4,41,42 These findings demonstrate that the HRRP penalizes hospitals to some extent for serving poorer and sicker patients, adding to evidence that pay-for-performance programs with limited risk adjustment could exacerbate disparities by unjustifiably transferring resources away from providers disproportionately serving higher-risk patients and by establishing incentives for providers to avoid these patients.3,4,18-20,43
Our study builds on prior studies of, and current policy for, risk adjustment in pay-for-performance programs in 2 important ways. First, we quantified the impact of adjustments on performance scores and expected penalties for the hospitals most affected by the adjustments. Prior studies of the HRRP that focused on overall changes in hospital performance rankings or penalty status may have missed salient financial consequences for an important minority of hospitals serving the highest-risk patients.43-45 In programs like the HRRP, in which penalties increase proportionally for hospitals with higher readmission rates, risk adjustment can substantially reduce the penalties incurred by hospitals with high shares of high-risk patients without major changes in rankings or the proportion of hospitals receiving a penalty.
Second, we used methods to address the commonly voiced concern that adjusting for social risk factors could lower standards for providers with more socially vulnerable patients.11,27,28 Starting in 2019, CMS plans to address the association between patients’ social risk and readmission by comparing readmission rates among hospitals serving similar proportions of Medicare patients dually enrolled in Medicaid.24,25 If dually enrolled patients are served disproportionately by low-quality hospitals, CMS’s approach would set lower standards for hospitals serving those patients, weakening incentives to achieve outcomes for socially vulnerable groups that are equal to those achieved for similar groups by other, higher-quality hospitals. Unlike CMS’s approach, our methodology could flexibly accommodate adjustment for a broader set of patient characteristics without raising concerns about introducing disparate standards of care.
Our results support adjusting for additional patient characteristics to mitigate unwarranted penalties for hospitals serving higher-risk patients and suggest that social risk adjustment may be even more important when payments are tied to health outcomes such as mortality, as opposed to utilization-based measures. However, improved risk adjustment would likely be an incomplete solution, because patient populations differ on many relevant dimensions that are difficult to measure.4,23 For example, our analysis of Medicare and US Census data could not assess the extent to which adjustment for functional limitations, cognition, household income, or social supports might further affect hospital performance and penalties.4,23 Thus, even with better risk adjustment, pay-for-performance programs may continue to impose additional costs on hospitals serving higher-risk patients that are unrelated to hospital quality, either in the form of penalties or the higher costs of avoiding penalties.46
In addition to the limited number of patient characteristics we could assess, a key limitation of our study is that we could not assess the causes of within-hospital associations between social risk factors and readmission. These might include unmeasured differences in clinical risk factors, a hospital’s inability to cover the higher costs of achieving high performance for higher-risk patients, or culturally incompetent or even discriminatory behavior by hospital staff. Because within-hospital associations did not systematically vary with the average social risk of a hospital’s patient population, we would not expect our adjustments to reward any discriminatory behavior contributing to within-hospital differences. Addressing within-hospital disparities would require additional incentives or resources supporting high performance specifically for high-risk patients and is not achieved by risk adjustment of quality measures, whose specific function is to improve assessment of between-hospital differences in quality. Particularly given the growing evidence of pay-for-performance programs’ ineffectiveness in driving quality improvement,47,48 policy makers could consider alternative improvement strategies that invest in the care of vulnerable groups to reduce within-provider disparities, rather than risk exacerbating them because of inadequate risk adjustment.
In conclusion, our study contributes to evidence of the unintended consequences of limited risk adjustment in pay-for-performance programs for providers serving clinically and socially higher-risk patients. Our results support policies to adjust readmission rates for a more comprehensive set of patient characteristics, including social risk factors, to minimize the potential for pay-for-performance programs rates to exacerbate health care disparities, as well as alternative strategies to improve quality and address disparities.
Corresponding Author: J. Michael McWilliams, MD, PhD, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115 (email@example.com).
Accepted for Publication: July 15, 2018.
Published Online: September 17, 2018. doi:10.1001/jamainternmed.2018.4481
Author Contributions: Drs Roberts, Zaslavsky, Ding, and McWilliams had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Roberts, Zaslavsky, Barnett, Landon, McWilliams.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Roberts, Zaslavsky, McWilliams.
Critical revision of the manuscript for important intellectual content: Roberts, Barnett, Landon, Ding, McWilliams.
Statistical analysis: Roberts, Zaslavsky, Ding, McWilliams.
Obtained funding: McWilliams.
Administrative, technical, or material support: Roberts, Ding.
Study supervision: Roberts, McWilliams.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by grants from the National Institutes of Health (P01 AG032952), the Laura and John Arnold Foundation, and the Marshall J Seidman Center for Studies in Health Economics and Health Care Policy at Harvard Medical School.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank Pasha Hamed, MA, Harvard Medical School, for statistical programming support. He did not receive additional compensation for his contributions.
Disclaimer: This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Laura and John Arnold Foundation.