Association Between Medicare Accountable Care Organization Implementation and Spending Among Clinically Vulnerable Beneficiaries

Key Points

Question  How have Medicare accountable care organizations (ACOs) affected spending and high-cost institutional use for all Medicare beneficiaries and specifically clinically vulnerable beneficiaries?

Findings  Using a difference-in-difference estimation, total spending decreased by $34 per beneficiary-quarter after ACO contract implementation across the overall Medicare cohort and decreased by $114 in the clinically vulnerable cohort. Hospitalization and emergency department visits decreased significantly per beneficiary-quarter in both the overall Medicare cohort and clinically vulnerable cohort.

Meaning  Medicare ACO programs are associated with modest reductions in spending and high-cost institutional use both overall and for clinically vulnerable beneficiaries.

Importance  Accountable care contracts hold physician groups financially responsible for the quality and cost of health care delivered to patients. Focusing on clinically vulnerable patients, those with serious conditions who are responsible for the greatest proportion of spending, may result in the largest effects on both patient outcomes and financial rewards for participating physician groups.

Objective  To estimate the effect of Medicare accountable care organization (ACO) contracts on spending and high-cost institutional use for all Medicare beneficiaries and for clinically vulnerable beneficiaries.

Design, Setting, and Participants  For this cohort study, 2 study populations were defined: the overall Medicare population and the clinically vulnerable subgroup of Medicare beneficiaries. The overall Medicare population was based on a random 40% sample drawn from continuously enrolled fee-for-service beneficiaries with at least 1 evaluation and management visit in a calendar year. The clinically vulnerable study population included all Medicare beneficiaries 66 years or older who had at least 3 Hierarchical Condition Categories (HCCs). Beneficiaries entered the cohort during the quarter between January 2009 to December 2011 when they first had at least 3 HCCs and remained in the cohort until death. Cohort entry was restricted to the preperiod to account for potential changes in coding practices after ACO implementation. Difference-in-difference estimations were used to compare changes in health care outcomes for Medicare beneficiaries attributed to physicians in ACOs with those attributed to non-ACO physicians from January 2009 to December 2013.

Exposures  Medicare ACOs beginning contracts in January 2012, April 2012, July 2012, and January 2013 through the Pioneer and Medicare Shared Savings Programs.

Main Outcomes and Measures  Total spending per beneficiary-quarter, spending categories, use of hospitals and emergency departments, ambulatory care sensitive admissions, and 30-day readmissions.

Results  Total spending decreased by $49 (95% CI, −$73 to −$25) per beneficiary-quarter after ACO contract implementation across the overall Medicare population (n = 15 592 600) and decreased $147 in clinically vulnerable patients (n = 8 673 823) (95% CI, −$202 to −$93). In the overall Medicare cohort, hospitalizations and emergency department visits decreased by 4.2 and 2.9 events per 1000 beneficiaries per quarter, respectively (95% CIs: −5.0 to −3.3 and −4.5 to −1.3), and hospitalizations decreased in the clinically vulnerable cohort by 4.9 and 4.1 events per 1000 beneficiaries per quarter (95% CI: −6.9 to −2.9). Changes in total spending associated with ACOs did not vary by clinical condition of beneficiaries.

Conclusions and Relevance  Medicare ACO programs are associated with modest reductions in spending and use of hospitals and emergency departments. Savings were realized through reductions in use of institutional settings in clinically vulnerable patients.

Accountable care organizations (ACOs) are one of the largest payment and delivery reforms currently under way in the United States. Over 700 ACO contracts are now in place, covering 23 million Americans.¹ Accountable care organizations are groups of health care providers collectively held responsible for the cost and quality of care for a defined patient population. Early evidence suggests that ACOs are improving quality and modestly affecting health care costs.^2-5 These gains have led to further policy changes intended to accelerate adoption of these and other alternative payment models such as the Next Generation ACO model, Accountable Health Communities model, Comprehensive ESRD Care model, and Oncology Care model.^6-9

Little is known, however, about how the ACO payment contracts implemented by Medicare may have influenced health care delivery. Accountable care organizations are hypothesized to improve care and lower costs in several ways, including better care coordination and more effective treatment of chronic illness; indeed, early evidence on quality of care improvements suggests that ACOs may be adopting this approach.^10,11 Better care, in turn, should reduce the use of high-cost institutional settings through prevention, coordination, and outpatient management. A related mechanism could be that ACOs focus more directly on the care of patients with complex medical needs. Patients who are older and have multiple chronic conditions have historically received inefficient, inappropriate, and fragmented care.^12-18 Studies of early ACO pilot programs found cost savings primarily among high-cost beneficiaries.^2,19 However, the effect of Medicare’s current ACO programs on spending and high-cost institutional use for beneficiaries—and especially for clinically vulnerable beneficiaries—remains unknown.

In this study, we focus on the question of whether the implementation of Medicare ACOs has had an effect on spending and usage for both overall and for clinically vulnerable Medicare beneficiaries, focusing in particular on the use of high-cost institutional settings including hospital stays, emergency department visits, and postacute care.

We used 5 years (January 2009-December 2013) of Medicare fee-for-service administrative claims data encompassing all Part A and B charges to compare changes in spending and usage for beneficiaries cared for by physicians in ACOs and those cared for by non-ACO physicians. Beneficiaries were annually assigned to an ACO or non-ACO physician group if the sum of charges for a subset of evaluation and management visits to the primary care physicians within the group exceeded the sum of charges for such visits to any other physician group. Following the Centers for Medicare and Medicaid Services (CMS) Medicare Shared Savings Program (MSSP) methodology to attribute patients, we used a set of outpatient codes defining evaluation and management visits to physicians, nurse practitioners, or physician assistants.²⁰ We used a difference-in-difference design to account for time-invariant differences between ACOs and non-ACOs, and differences between the preintervention and postintervention periods. Detailed methodological notes are available in eAppendix in Supplement 1. The Dartmouth institutional review board approved this study.

We defined 2 study populations, one representing the overall Medicare population and a second representing a clinically vulnerable subgroup of Medicare beneficiaries. The overall Medicare population is based on a random 40% sample we drew of continuously enrolled fee-for-service beneficiaries with at least 1 evaluation and management visit in a calendar year. The clinically vulnerable study population includes all Medicare beneficiaries age 66 or older who had at least 3 Hierarchical Condition Categories (HCCs).²¹ Patients in this cohort are required to have either 1 inpatient diagnosis or 2 outpatient diagnoses in the prior year to be assigned a condition category. Beneficiaries enter the cohort during the quarter from January 2009 to December 2011 when they first have at least 3 HCCs and remain in the cohort until death. We restricted participant entry to the period before ACO contract implementation to account for potential changes in coding practices after ACO implementation.

The health care provider groups in our study joined Medicare ACO programs in January 2012 (32 ACOs); April 2012 (27 ACOs); July 2012 (87 ACOs); and January 2013 (106 ACOs). To compare physician group performance over time and to preserve the relationship between physicians and ACOs, we assigned physicians to a preperiod organization (corresponding to an ACO) if the physicians were associated with that organization in the ACO period.

Our main outcome measures were total spending per quarter and a number of spending categories: acute care, durable medical equipment, facility, tests, long-term care, hospice, imaging, home health, procedures, evaluation and management visits, and skilled nursing facility. We also studied quarterly counts of hospitalizations, emergency department visits, and observation stays.²² Other outcome measures included ambulatory care sensitive admissions²³ and 30-day all-cause readmissions in the quarter. Quarterly measurements of outcomes allowed precise representation of an organization’s ACO status, increasing the precision of our analyses compared with yearly measurements. In 2013, Medicare excluded from research files any claims associated with substance abuse. To measure changes in outcomes over time, we applied Medicare’s approach to remove such claims from files for prior years (January 2009-December 2012).

We grouped beneficiaries by 5-year age categories to allow for nonlinear effects of age. We created indicators for disability as original reason for entitlement; sex; nursing home residence; dual enrollment in Medicare and Medicaid; black, Hispanic, and other race or ethnicity groups²⁴; hospital referral region (HRR) of residence; and 10 of the most prevalent chronic conditions, including end-stage renal disease, coronary artery disease, chronic obstructive pulmonary disease, cancer, diabetes, congestive heart failure, renal failure, polyneuropathy, cardio-respiratory failure and shock, and vascular disease. As all HCCs indicate increased risk status, we considered any beneficiary diagnosed with a condition to have that condition for the remainder of the study period. We measured the proportion in poverty in the patients’ zip code using the 2010 Census.

To estimate changes in spending and high-cost institutional use associated with ACO contracts, we compared changes in outcome variables over the ACO implementation period for the beneficiary-quarter observations attributed to ACO and non-ACO (control group) care. Our main specification was a difference-in-difference beneficiary-quarter level linear model with time and HRR indicators, with the control group made up of beneficiaries attributed to non-ACO physicians. This design controlled for fixed differences between ACO and non-ACO care, as well as biases from comparisons over time that could reflect broader trends in health care or Medicare beneficiary population health. Because individuals contribute multiple observations across the study period (up to 20 across the 5-year period), we use robust standard errors accounting for the fact that errors were likely clustered by beneficiary and by HRR, an indicator for the local health care market. Using the subscripts i, j, t, and k for individual, practice group, quarter, and HRR, respectively, the statistical models had the following linear form:

Y_itjk = ACOPost_itjβ₁ + EarlyPost_itjβ₂ + SocioDem_itβ₃ + Clinical_itβ₄ + ACO_itjβ₅ + γ_t + ω_k + ε_itjk.

The coefficient of primary interest, represented by β₁, measures the effect of a patient receiving care from an ACO after the ACO contract begins (indicated by ACOPost). We estimated a single ACO effect for all outcomes, and additionally we estimated total spending effects separately for each ACO wave by program start date and individual ACO. In the latter 2 cases, ACOPost is the interaction of a postperiod indicator with a vector of indicators for each wave of ACOs or for individual ACOs, respectively. The variable ACO can be nonzero at any time to indicate physician groups who later became ACOs, whereas ACOPost can only be nonzero after ACOs have begun Medicare contracts. Because we suspected anticipatory effects of ACO implementation, we defined EarlyPost to indicate the 2 quarters prior to ACO implementation, when the ACO had applied for and then been accepted for participation. The EarlyPost coefficient, β₂, quantifies the effect of anticipation compared with earlier preperiod quarters and removes this period of time from the evaluation of the ACOPost coefficient β₁. The SocioDem vector captures Medicaid status, disability, nursing home residency, age group, sex, race/ethnicity, and percent poverty in census tract. The Clinical vector includes chronic condition indicators and for the full cohort the beneficiary’s total number of HCCs. The variables γ_t and ω_k denote time adjustment (quarterly indicators, year indicators, and season indicators) and the HRR that the group practices in, respectively. The error term captures idiosyncratic variation in Y. All covariates with continuous or ordinal scales were centered.

We chose a linear model because (1) there were few occurrences of zeros owing to the requirement of at least 1 evaluation and management visit and the focus on clinically vulnerable patients who have intense health care usage; (2) distributions were largely symmetrical; (3) if linear models do not fit the data well, our standard errors are biased against finding a significant effect and are therefore conservative; and (4) for ease of interpretation it was desirable for the estimated coefficients to represent a dollar amount or rate. We designed our approach for the clinically vulnerable cohort and used the same models for the full cohort to maintain easy comparability.

To determine whether changes in spending and high-cost institutional use in ACOs with earlier start dates were due to a longer follow-up period, we performed a sensitivity analysis limited to the first 6 months and 1 year of post-ACO implementation for each wave. We also performed a sensitivity analysis estimating the association of ACO contracts with changes in spending for 9 clinical cohorts. Additional sensitivity analyses are described further in eAppendix in Supplement 1.

On average, spending on the clinically vulnerable cohort was 97% more than on Medicare beneficiaries overall ($24 462 annually in the clinically vulnerable attributed to ACOs vs $12 420 overall), and spending in the clinically vulnerable cohort made up 82% of overall spending in the first quarter of 2009. The clinically vulnerable cohort includes 22% of attributed Medicare beneficiaries. The top 10 conditions identified in the clinically vulnerable cohort were congestive heart failure (37%), specified heart arrhythmias (37%), chronic obstructive pulmonary disease (31%), renal failure (29%), vascular disease (26%), diabetes without complication (24%), cardiorespiratory failure and shock (17%), cancer (12%), polyneuropathy (10%), and stroke (10%). The largest component of total spending for the clinically vulnerable was acute care (36%), followed by skilled nursing facility (11%), evaluation and management visits (10%), and procedures (10%). Spending in the preperiod was greater in beneficiaries attributed to ACOs than for the control group.

Demographic and clinical characteristics of beneficiary-quarters attributed to ACO and non-ACO care were similar in the preperiod overall and in the clinically vulnerable cohort. Because patients remain in the study cohort after first observation of 3 conditions, we see that the cohort has aged in the postperiod, with associated effects such as decreases in chronic conditions and spending due to mortality in the sickest patients (Table 1).

Despite differences in the level of spending between ACOs and control populations, spending over time shows similar trends during the pre-ACO period (Figure 1). We find small significant decreases in spending associated with the Pioneer program and MSSP ACOs. Across all ACOs, average total spending decreased by $49 or 1.6% per beneficiary-quarter (95% CI, −$73 to −$25) (Table 2) and by $147 or 2.3% per beneficiary-quarter (95% CI, −$202 to −$93) in the clinically vulnerable cohort. We found a statistically significant anticipatory decrease in spending in the 6 months leading up to implementation: $51 in the full cohort (95% CI, −$76 to −$26) and −$170 in the clinically vulnerable (95% CI, −$234 to −$106). In the clinically vulnerable cohort, total spending per beneficiary-quarter decreased by $195 or 2.9% (95% CI, −$277 to −$113) for the Pioneer program; $106 (95% CI, −$204 to −$7) for April 2012 MSSPs; $215 (95% CI, −$297 to −$113) for July 2012 MSSPs; and $116 (95% CI, −$176 to $−57) for January 2013 MSSPs. A Wald test on the coefficients in the clinically vulnerable cohort showed that the total savings achieved by each ACO wave were not statistically different from each other. Estimated changes in spending ranged from savings of $155 (25th percentile) to increased spending of $20 (75th percentile) in the full cohort and savings of $305 (25th percentile) to increased spending of $78 (75th percentile) in the clinically vulnerable (Figure 2). When effects were estimated separately for each of the top condition categories, there were statistically significant decreases in total spending per beneficiary-quarter for all conditions, and decreases across conditions did not differ significantly in magnitude (Figure 3). Adjusting for ACOs that dropped out of the Pioneer program before the end of our study period did not significantly change results.

Reductions in total spending per beneficiary-quarter by ACOs in the postperiod can be explained in part by decreases in acute care spending and high-cost institutional use across all waves. Acute care spending decreased by $35 or 3.3% (95% CI, −$48 to −$22, Table 2) overall and $87 or 3.8% (95% CI, −$118 to −$56) for the clinically vulnerable cohort. Skilled nursing facility spending did not change significantly overall, and decreased by $25 or 3.7% (95% CI, −$39 to −$10) for the clinically vulnerable cohort. In the full Medicare cohort, spending for long-term care and for evaluation and management visits did not change significantly. In the clinically vulnerable cohort, spending decreased slightly for both long-term care and evaluation and management visits. Changes in spending were negligible or statistically insignificant for durable medical equipment, facility, test, hospice, imaging, home health, and procedure spending for both the full Medicare and the clinically vulnerable cohorts.

In the full Medicare cohort, ACOs were associated with a reduction in emergency department visits of 2.9 events or 1.5% per 1000 beneficiary-quarters (95% CI, −4.5 to −1.3) and a decrease in hospitalizations of 4.2 events or 4.0% (95% CI, −5.0 to −3.3). In the clinically vulnerable cohort, ACOs were not associated with a significant reduction in emergency department visits and showed a decrease in hospitalizations of 4.9 events (95% CI, −6.9 to −2.8) in the clinically vulnerable. In the full Medicare and clinically vulnerable cohorts, ACOs were associated with increases in 30-day readmissions. Changes in high-cost institutional use in both our full cohort and our clinically vulnerable cohort were negligible or statistically insignificant for ambulatory care sensitive hospitalizations, and observation stays.

To determine whether the larger reductions in spending and high-cost institutional use in ACOs with earlier start dates were due to a longer follow-up period, we limited analysis to the first year of post-ACO implementation. Decreases in total spending were similar in magnitude. This, along with the finding that the January 2013 MSSPs had small and insignificant changes in spending, suggests that reductions in total spending take time to be realized. When further limited to the first 6 months of post-ACO implementation, decreases in total spending were similar in magnitude to the first-year effects, suggesting that changes in spending are relatively constant, increasing slightly over time. Results were robust to the inclusion or exclusion of the number of hierarchical clinical conditions and indicators for specific clinical condition categories.

Medicare ACO programs are associated with modest savings on average across all beneficiaries, with savings concentrated in clinically vulnerable beneficiaries and use of institutional settings. Accountable care organizations are associated with a 3.3% reduction in hospital spending overall and a 3.7% reduction in skilled nursing spending among the clinically vulnerable, as well as significant reductions in emergency department use and hospitalizations overall. We found that the magnitude of reductions was not significantly different by the start date of ACOs. We also find an anticipatory effect of participating in ACO programs that could lower the benchmark spending figures for ACOs, making it less likely for them to achieve savings according to CMS calculations. While modest in magnitude, these findings for ACOs are consistent with other evaluations of ACO savings^2-5,19 and stand in contrast to the failure to demonstrate significant cost savings in care management programs layered on top of the traditional fee-for-service Medicare program.^25,26

Similar to previous studies of payment reform,² we observe marked differences across organizations in the effects of these payment programs, ranging from large reductions in spending to large increases in spending. However, some variation in results is to be expected, and while our models include hospital referral region fixed effects, we have not studied each of these local comparisons in depth and continue to know little about why some organizations succeed and others fail.²⁷ Differences in results may be owing to ACO formation effects: the characteristics of beneficiaries attributed to ACOs and those attributed to non-ACOs may be different within markets.²⁸ Factors contributing to success may include ACO participant characteristics such as size, homogeneity, readiness, or structural attributes (for example, whether or not the ACO includes a hospital); market conditions; choice of risk stratification tools; differences in governance models or leadership effectiveness; physician engagement strategies; electronic health records and other health information technology tools; and how providers approached disease management, care transitions, and quality improvement. Early studies of the formation of ACOs offer a sense of the diversity of participating organizations.^10,29,30

Many ACOs have instituted risk stratification and care management programs targeted at clinically vulnerable patients,^31,32 as well as programs to improve patient access to their primary care team in the hope of keeping patients out of the emergency department and the hospital when unnecessary.^33,34 These programs appear to be working to a small degree through changes in spending and use associated with the ACO program; overall, we estimate that ACOs were able to save a total of $592 million and reduce high-cost institutional use by 15 000 000 events for hospitalizations and 21 000 000 events for emergency department visits for clinically vulnerable beneficiaries during 2012 and 2013. The effects we report here are similar in magnitude to results reported from the Physician Group Practice Demonstration and the Pioneer program across all patient groups.^2,4,5

Our analyses have several limitations. First, previous research has shown that diagnoses are sensitive to regional variations that cannot be explained by differences in underlying health,^35,36 which may be a limitation of the use of HCCs as a proxy for health status. Our estimates were not sensitive to the inclusion of HCCs as a risk adjustment tool. Second, we analyze only 1 to 2 years of postimplementation data. Third, we did not measure the effect of ACOs on Part D spending, wraparound Medicaid spending, or spillovers into the Medicare Advantage program. Fourth, our data on beneficiary costs and estimated savings do not include the start-up costs invested by the ACOs to get up and running. Additionally, our data cannot study those with substance abuse, patients often included in the clinically vulnerable, due to redaction of these claims. Finally, our use of claims data has limited information on quality of care. We do know, however, that on the quality measures reported to CMS, including for example patient satisfaction with management of care transitions, readmissions, and management of chronic conditions, Medicare ACOs have shown substantial improvement over time.³⁷

Our results suggest that the ACO model has modest early benefits in terms of reduced spending and high-cost institutional use for patients with multiple clinical conditions. According to our calculations, approximately 84% of ACOs who achieved statistically significant reductions in total spending in the full cohort received shared savings from CMS in the 2012 to 2013 performance year. Given the vast differences in changes in spending across ACOs, further research on the relationship between activities within ACOs, the institutional characteristics of ACOs, and the service markets in which they operate and changes in spending and high-cost institutional use are essential to better understand what factors contribute to improved care for clinically vulnerable populations. In addition, longer-term research is needed, as it is likely that while some relatively simple changes may have shown rapid results, more structural changes will take time to produce improvements in health care outcomes.

Corresponding Author: Carrie H. Colla, PhD, The Dartmouth Institute for Health Policy & Clinical Practice, Geisel School of Medicine at Dartmouth, Level 5, WTRB, One Medical Center Dr, Lebanon, NH 03756 (Carrie.H.Colla@Dartmouth.edu).

Correction: This article was corrected online July 11, 2016, for typographical errors.

Retraction and Replacement: This article was retracted and replaced on September 11, 2017, for errors in data in the text, tables, and figures (see Supplement 2 for a copy of the replacement article with corrections highlighted and a copy of the retracted article with errors highlighted).

Published Online: June 20, 2016. doi:10.1001/jamainternmed.2016.2827

Author Contributions: Dr Colla had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Colla, Lewis, Chang, Fisher.

Acquisition, analysis, or interpretation of data: Colla, Lewis, Kao, O’Malley, Chang.

Drafting of the manuscript: Colla, Kao, Fisher.

Critical revision of the manuscript for important intellectual content: Colla, Lewis, O’Malley, Chang.

Statistical analysis: Colla, Lewis, Kao, O’Malley, Chang.

Obtained funding: Colla, Lewis.

Administrative, technical, or material support: Colla.

Study supervision: Colla, O’Malley, Chang, Fisher.

Conflict of Interest Disclosures: None reported.

Funding/Support: This research was supported by grants from the Commonwealth Fund (grant No. 20150034), the National Institute on Aging (grant No. 5R33AG044251) and the Dartmouth Clinical and Translational Science Institute (through grant No. UL1TR001086 from the National Center for Advancing Translational Sciences).

Role of the Funder/Sponsor: The funders/sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content of the article is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Additional Contributions: The authors are grateful to Stephanie R. Raymond, BA, Senior Programmer/Analyst at The Dartmouth Institute, for superb programming assistance and Ellen R. Meara, PhD, a professor at The Dartmouth Institute, for her insight and expertise. Neither Ms Raymond nor Dr Meara received additional compensation for their contributions.