Adjusted annual per-beneficiary spending means are plotted for organizations participating in the Alternative Quality Contract (AQC) (intervention group) and nonparticipating providers (control group) before and after participating organizations entered the AQC. To display spending changes that were averaged across organizations entering in 2009 or 2010, we aligned the times of entry by these 2 groups of AQC participants and adjusted for the mean spending difference between the 2 groups prior to their entry. Thus, spending differences between organizations entering in 2009 and the control group contributed to overall average differences for 2 years before AQC entry (2007-2008) and 2 years after AQC entry (2009-2010), while spending differences between organizations entering in 2010 and the control group contributed to overall average differences for 3 years before AQC entry (2007-2009) and 1 year after AQC entry (2010). The dotted line indicates the point at which participants entered the AQC. Spending means were adjusted for propensity score weights to balance the distribution of counties and beneficiary characteristics between the intervention and control groups (see eFigure in the Supplement for unadjusted means).
eAppendix. Additional Methods
eTable 1. Evaluation and Management (E&M) Service Codes Used to Assign Medicare Beneficiaries to Shared Savings Program and Pioneer Accountable Care Organizations
eTable 2. Chronic Conditions Among Medicare Beneficiaries Served by Organizations Participating in the AQC and Nonparticipating Providers
eFigure. Unadjusted Mean Annual Spending in Medicare Between AQC Participants and Nonparticipating Providers Before and After Participating Organizations Entered the AQC
Customize your JAMA Network experience by selecting one or more topics from the list below.
McWilliams JM, Landon BE, Chernew ME. Changes in Health Care Spending and Quality for Medicare Beneficiaries Associated With a Commercial ACO Contract. JAMA. 2013;310(8):829–836. doi:10.1001/jama.2013.276302
In a multipayer system, new payment incentives implemented by one insurer for an accountable care organization (ACO) may also affect spending and quality of care for another insurer’s enrollees served by the ACO. Such spillover effects reflect the extent of organizational efforts to reform care delivery and can contribute to the net impact of ACOs.
We examined whether the Blue Cross Blue Shield (BCBS) of Massachusetts’ Alternative Quality Contract (AQC), an early commercial ACO initiative associated with reduced spending and improved quality for BCBS enrollees, was also associated with changes in spending and quality for Medicare beneficiaries, who were not covered by the AQC.
Design, Setting, and Participants
Quasi-experimental comparisons from 2007-2010 of elderly fee-for-service Medicare beneficiaries in Massachusetts (1 761 325 person-years) served by 11 provider organizations entering the AQC in 2009 or 2010 (intervention group) vs beneficiaries served by other providers (control group). Using a difference-in-differences approach, we estimated changes in spending and quality for the intervention group in the first and second years of exposure to the AQC relative to concurrent changes for the control group. Regression and propensity score methods were used to adjust for differences in sociodemographic and clinical characteristics.
Main Outcomes and Measures
The primary outcome was total quarterly medical spending per beneficiary. Secondary outcomes included spending by setting and type of service, 5 process measures of quality, potentially avoidable hospitalizations, and 30-day readmissions.
Before entering the AQC, total quarterly spending per beneficiary for the intervention group was $150 (95% CI, $25-$274) higher than for the control group and increased at a similar rate. In year 2 of the intervention group’s exposure to the AQC, this difference was reduced to $51 (95% CI, −$109 to $210; P = .53), constituting a significant differential change of −$99 (95% CI, −$183 to −$16; P = .02) or a 3.4% savings relative to an expected quarterly mean of $2895. Savings in year 1 were not significant (differential change, −$34; 95% CI, −$83 to $16; P = .18). Year 2 savings derived largely from lower spending on outpatient care (differential change, −$73; 95% CI, −$97 to −$50; P < .001), particularly for beneficiaries with 5 or more conditions, and included significant differential changes in spending on procedures, imaging, and tests. Annual rates of low-density lipoprotein cholesterol testing differentially improved for beneficiaries with diabetes in the intervention group by 3.1 percentage points (95% CI, 1.4-4.8 percentage points; P < .001) and for those with cardiovascular disease by 2.5 percentage points (95% CI, 1.1-4.0 percentage points; P < .001), but performance on other quality measures did not differentially change.
Conclusions and Relevance
The AQC was associated with lower spending for Medicare beneficiaries but not with consistently improved quality. Savings among Medicare beneficiaries and previously demonstrated savings among BCBS enrollees varied similarly across settings, services, and time, suggesting that organizational responses were associated with broad changes in patient care.
In response to mounting pressures to deliver more cost-effective care, provider organizations have exhibited increasing willingness to assume financial risk for the quality and costs of the care they provide. More than 250 provider groups have contracted with Medicare as accountable care organizations (ACOs), and many similar payment arrangements have been reached with commercial insurers by these and other groups.1-4
In a multipayer system in which fee for service remains the predominant basis for reimbursement,5 new incentives to lower spending and improve quality typically do not apply to all of an organization’s patients. Efforts to reduce utilization among subsets of patients covered by risk contracts could have similar effects among other patients if such efforts extend beyond targeted case management to include broader changes that influence care for all patients. Conversely, lowering utilization among some patients could induce utilization among others by generating excess capacity, particularly for services in high demand or of uncertain clinical value.6,7 In addition, physicians seeking target incomes might respond to income effects from changes in one payer’s payments by altering the volume of services reimbursed by other payers.8
The size and direction of such spillover effects of ACO contracts are important to characterize because they indicate the extent of organizational change elicited by ACO payment incentives, contribute to the net impact of ACOs, and may influence decisions by insurers and provider groups to enter or expand ACO contracts. Because evaluations of recent initiatives by insurers to set global budgets for provider groups have been limited to the patient populations placed under budgets,9-12 the potential spillover effects of ACO contracts remain unclear.
Beginning in 2009, the implementation of the Alternative Quality Contract (AQC) by Blue Cross Blue Shield (BCBS) of Massachusetts heralded a marked but partial shift away from fee-for-service incentives for participating provider organizations.13 Under the terms of the AQC, provider organizations bear financial risk for spending in excess of a global budget, gain from reducing spending below the budget, and receive bonuses for meeting performance targets on quality measures. The incentives in the AQC are similar to those in 2-sided payment arrangements that all ACOs in Medicare programs are expected to accept by their second contract period if not earlier.14,15 The AQC was associated with lower spending and improved quality of care for BCBS commercial health maintenance organization enrollees served by participating provider groups.10,11 Savings increased in the second year of group participation and were concentrated in the outpatient care of medically complex patients, particularly imaging, procedures, and tests. We examined changes associated with the AQC in spending and quality of care for traditional fee-for-service Medicare beneficiaries.
We analyzed data from Medicare enrollment and claims files from 2007 to 2010 for all beneficiaries residing in Massachusetts. Claims data were linked via National Provider Identifiers (NPIs) to data on provider groups from the American Medical Association (AMA) Group Practice File (eAppendix in the Supplement).16,17 For physicians practicing in groups of 3 or more, the Group Practice File identifies their practice site(s) and parent organization if part of a larger group. Among primary care physicians practicing in groups of 3 or more and serving Medicare beneficiaries in Massachusetts in 2009, 94% were linked to the Group Practice File (eAppendix in the Supplement).
For each year, our study population included beneficiaries who were continuously enrolled in Parts A and B of traditional fee-for-service Medicare while alive and received at least 1 primary care service during the year (n = 1 983 921 person-years). We further restricted the study population to those who were aged 66 years or older in 2007 (n = 1 761 325 person-years) to ensure at least 1 year of eligibility for assessing the presence of chronic conditions at baseline from the Chronic Condition Warehouse (CCW).18 Thus, the study population comprised a cohort of elderly adults enrolled in Medicare at baseline but not necessarily included in all study years because of death or Medicare Advantage enrollment. We adjusted our main analyses for conditions present at baseline, rather than in each study year, to avoid potential bias from more intensive diagnostic coding by organizations participating in the AQC in response to risk-adjusted global budgets.9-11,19
We conducted quasi-experimental comparisons of beneficiaries served by the 7 provider organizations entering the AQC in 2009 and the 4 entering in 2010 (hereafter called AQC participants) with beneficiaries served by other nonparticipating providers in Massachusetts, before and after AQC payment incentives were implemented for participating organizations.13 Following the Medicare Shared Savings Program assignment rules,14 we attributed each beneficiary in each year to the provider group accounting for the most allowed charges for primary care services among all groups providing primary care to the beneficiary in that year (eAppendix and eTable 1 in the Supplement).
We used 2 complementary sources of provider group identifiers when applying the attribution rules: (1) AMA Group Practice File groupings of NPIs and (2) tax identification numbers (TINs) that indicate billing entities in Medicare claims. For each of the 11 AQC participants, we assembled all groups in the AMA Group Practice File with names matching the organization or 1 of its constituent parts into a single inclusive group with a unique identifier. We similarly grouped TINs for each AQC participant to the extent that we could identify TINs from publicly available information on nonprofit organizations20; we found TINs for at least 1 major organizational component of 7 AQC participants. We then applied the assignment rules to each beneficiary twice, using each source of provider group identifiers independently. For each year, we classified beneficiaries as members of the intervention group if they were assigned to an organization participating in the AQC via either their Group Practice File group or TIN assignment. Of beneficiary assignments to AQC participants, 91.9% were determined from Group Practice File data and 21.4% from TINs (13.3% from both). We classified other beneficiaries assigned to nonparticipating providers as the control group. Based on these assignments, the distribution of Medicare beneficiaries across AQC participants (n = 417 182 person-years) and nonparticipating providers (n = 1 344 143 person-years) was similar to that among BCBS enrollees.10,11
Our study protocol was approved by the Harvard Medical School Committee on Human Studies and Privacy Board of the Centers for Medicare and Medicaid Services.
For each beneficiary, we calculated total spending on hospital and outpatient care in each quarter from 2007 through 2010 by summing Medicare reimbursements, coinsurance amounts, and payments from other primary payers. We excluded indirect medical education and disproportionate-share hospital payments. We omitted the first quarter of 2007 from analyses because it concluded a period of transition to mandatory use of NPIs by physicians in all submitted claims.21 Because savings for BCBS enrollees in the AQC were greater for spending on outpatient care and specific types of services, including imaging, procedures, and tests, we used place-of-service codes to distinguish outpatient care from inpatient care and classified spending by Berenson-Eggers Type of Service categories examined in previous evaluations of the AQC.10,11,22 To adjust for the elimination of Medicare fees for inpatient and outpatient specialty consultations in 2010,23 we counted consultations as office or hospital visits and applied standardized fees to these broadened categories of evaluation and management services uniformly across study years (eAppendix in the Supplement).
For all beneficiaries in each year, we created an indicator of being hospitalized at least once for an ambulatory care–sensitive condition as defined by the Agency for Healthcare Research and Quality Prevention Quality Indicators.24 We focused on ambulatory care–sensitive conditions related to cardiovascular disease or diabetes, conditions targeted by quality measures in the AQC. For hospitalized beneficiaries in each year, we also created an indicator of being readmitted within 30 days of discharge at least once during the year. Results were similar when readmissions and admissions for ambulatory care–sensitive conditions were analyzed as counts.
We also constructed from claims several annual process measures of quality adapted from the Healthcare Effectiveness Data and Information Set (HEDIS): screening mammography for women aged 65 to 69 years; low-density lipoprotein cholesterol testing for beneficiaries with a history of ischemic heart disease, myocardial infarction, or stroke; and hemoglobin A1c testing, low-density lipoprotein cholesterol testing, and diabetic retinal examinations for beneficiaries with diabetes.
From Medicare enrollment files, we determined age, sex, race/ethnicity, disability on enrollment in Medicare, presence of end-stage renal disease, and Medicaid eligibility. We assessed race/ethnicity to control for differences in spending and quality of care among different racial and ethnic groups. From US Census data, we assessed poverty rates and educational attainment for the elderly population in beneficiaries’ zip code tabulation areas.25 From the CCW, we determined if beneficiaries had been diagnosed as having any of 21 conditions by January 1, 2007. Because the CCW includes diagnoses since 1999, it more completely captures the accumulated burden of these chronic diseases than risk scores derived from concurrent or recent claims only.
Using a difference-in-differences approach and linear regression, we estimated average changes in quarterly spending (or annual quality) for the intervention group in the first and second years of exposure to the AQC that were not explained by concurrent changes for the control group. We accounted for the staggered entry of organizations into the AQC by allowing different preintervention periods for organizations entering in 2009 (preintervention period = 2007-2008) and organizations entering in 2010 (preintervention period = 2007-2009) in comparisons with the control group (see eAppendix in the Supplement for model specification). First-year results were estimated by averaging differential changes in spending in 2009 for organizations entering the AQC in 2009 with differential changes in 2010 for organizations entering in 2010. Second-year results were estimated from differential changes in spending in 2010 for organizations entering in 2009.
All models included as covariates the sociodemographic and clinical characteristics described herein as well as county fixed effects. To allow nonadditive effects of multiple conditions on spending, we also included 6 indicators of having 2 or more to 7 or more conditions. In a sensitivity analysis, we adjusted for CCW conditions present at the start of each study year rather than at baseline; this analysis accounted for observed changes in chronic disease burden during the study period but could be biased by “upcoding” or by successful prevention of conditions such as myocardial infarction by AQC participants. To determine if the AQC disproportionately affected the care of medically complex Medicare beneficiaries, we stratified analyses by whether beneficiaries had 5 or more chronic conditions at baseline (23% of the study population).
In addition to regression adjustments, we used a propensity score weighting technique to balance beneficiary characteristics between the intervention and control groups.26 Specifically, we fitted a logistic regression model predicting assignment to an organization participating in the AQC as a function of all covariates, including county fixed effects to balance the geographic distribution of comparison groups. From this model, we determined the predicted probability of belonging to the opposite group for each beneficiary and weighted analyses by these probabilities.27,28
We tested 2 key assumptions of our difference-in-differences approach to isolating changes in spending and quality associated with the AQC. First, we tested whether spending trends were similar for the intervention and control groups before organizations entered the AQC and adjusted for any differences in trends. Second, we tested whether the composition of the intervention group differentially changed over time relative to the control group by comparing group differences in sociodemographic and clinical characteristics in 2007-2008 vs 2009-2010.
We also conducted a sensitivity analysis controlling for differences in organizational size between AQC participants and nonparticipating providers, using the number of attributed beneficiaries and number of affiliated physicians as measures of provider group size (eAppendix in the Supplement). By including both county effects and group size in propensity score models, we effectively limited this analysis to subgroups of beneficiaries in the intervention and control groups who shared a similar geographic distribution and were also attributed to similarly sized provider groups. We repeated this analysis excluding county effects to relax the geographical constraint.
We used Taylor series methods to adjust standard errors for clustering within provider groups and within beneficiaries over time.29 Our results were substantively similar when using generalized linear models with a log link and proportional to mean variance function for spending and logistic regression for quality indicators.30 We report 2-sided P values with a significance threshold of P < .05. All analyses were conducted with SAS version 9.2 (SAS Institute Inc).
Differential changes in sociodemographic and clinical characteristics were small for the intervention group relative to the control group, and none was statistically significant (Table 1 and eTable 2 in the Supplement). As shown in the Figure, adjusted total quarterly spending was $150 higher (95% CI, $25-$274; P = .02) for the intervention group than for the control group, and spending trends were similar before AQC incentives were implemented for participating organizations (difference in total spending trend, $2 faster per quarter for the intervention group; 95% CI, −$9 to $13; P = .69). In year 2 of the intervention group’s exposure to the AQC, the spending difference between the intervention and control groups was reduced to $51 (95% CI, −$109 to $210; P = .53), constituting a significant differential change of −$99 (95% CI, −$183 to −$16; P = .02) or a 3.4% savings relative to an expected quarterly mean of $2895 (Table 2). Savings in year 1 were not statistically significant (differential change, −$34; 95% CI, −$83 to $16; P = .18).
As summarized in Table 2, savings in year 2 were explained largely by differential changes in outpatient care (−$73; 95% CI, −$97 to −$50; P < .001) and included significant differential changes in spending on office visits (−$9; 95% CI, −$13 to −$6; P < .001), emergency department visits (−$3; 95% CI, −$6 to −$1; P = .02), minor procedures (−$12; 95% CI, −$18 to −$5; P < .001), imaging (−$13; 95% CI, −$17 to −$8; P < .001), and laboratory tests (−$4; 95% CI, −$7 to −$2; P = .001).
Estimated savings in year 2 spending on outpatient care for the intervention group were greater among beneficiaries with 5 or more conditions (−$125; 95% CI, −$169 to −$80) than among those with fewer conditions (−$61; 95% CI, −$85 to −$36; P = .002 for difference between groups). Estimated savings were slightly larger when adjusted for conditions present at the start of each study year, and the differential change in total spending in year 1 became statistically significant (−$47; 95% CI, −$91 to −$3; P = .04). Estimated savings were similar or larger when adjusted for preceding spending trends in a sensitivity analysis, except the differential change in spending on office visits was smaller and no longer significant. Results were not substantially altered by adjustment for provider organizational size, with or without adjustment for county.
Relative to the control group, annual rates of low-density lipoprotein cholesterol testing were 2.2 percentage points higher (95% CI, 0.4% -4.0%; P = .02) for beneficiaries with diabetes in the intervention group prior to entering the AQC. This difference increased further to 5.2 percentage points (95% CI, 2.5%-7.9%; P <.001) by year 2 of exposure to the AQC, constituting a significant differential improvement of 3.1 percentage points (95% CI, 1.4-4.8 percentage points; P < .001) above an expected annual rate of 77.3%. A similar differential improvement in low-density lipoprotein cholesterol testing rates occurred for beneficiaries with cardiovascular disease in the intervention group (2.5 percentage points; 95% CI, 1.1-4.0 percentage points; P < .001), but performance on other quality measures did not differentially change (Table 3).
The AQC was associated with significant reductions in spending for Medicare beneficiaries but not with consistently better quality of care. Similar to observed savings among BCBS commercial enrollees in the AQC, savings in Medicare grew in year 2 of AQC incentives, were greater for patients with more clinical conditions, derived largely from lower spending on outpatient care, and included reductions in spending on procedures, imaging, and tests.10,11 These findings suggest that global payment incentives in the AQC elicited responses from participating organizations that extended beyond targeted case management of BCBS enrollees. The AQC participants reported adopting several strategies that could have influenced patient care for which they did not bear financial risk, such as rewarding constituent physicians or groups for efficient practices, changing referral patterns, engaging in high-risk case management across multiple payers, and redesigning care processes to eliminate waste.13
A previous evaluation found a $27 reduction in total quarterly spending for BCBS enrollees in year 2 of the AQC, attributable to shifts in care to lower-priced provider groups as well as to lower utilization achieved by some participating organizations.11 This reduction was 4 times larger and entirely concentrated among medically complex enrollees who more closely resembled Medicare beneficiaries but whose spending levels were still well below average spending for beneficiaries in our study. Because of these differences in patient populations and the imprecision of our results (95% CI for total per-beneficiary savings, −$16 to −$183), we could not assess the magnitude of savings among Medicare beneficiaries relative to savings demonstrated among BCBS enrollees. Some interventions tailored specifically to BCBS enrollees would be expected to have minimal effects on care for Medicare beneficiaries (eg, case management to prevent admissions), while other targeted interventions (eg, rewarding efficiency based on care for BCBS patients) and systemic changes (eg, electronic clinical decision support to reduce inappropriate imaging) could potentially have greater effects on care for Medicare beneficiaries because of their greater disease burden. We could not attribute savings in Medicare to specific interventions among the many implemented by AQC participants.
Shifts in care away from outpatient facilities that charge higher prices, as observed among BCBS enrollees in the AQC,10,11 could have contributed to overall savings in Medicare, too, because hospital outpatient departments are paid facility fees in excess of standard Medicare reimbursements.31 Because administratively set Medicare fees otherwise vary minimally within regions, the $49 quarterly savings in spending for outpatient care not billed by hospital departments reflects changes in utilization among beneficiaries served by AQC participants.
For Medicare beneficiaries with cardiovascular disease and diabetes, we found evidence of significant improvement in 1 process measure of quality but not in 3 others included in the AQC and not in hospitalizations that might be prevented by better cardiovascular disease and diabetes control rewarded by the AQC. These weaker and inconsistent associations of the AQC with quality of care for Medicare beneficiaries suggest that quality improvement efforts by participants may have been targeted more specifically to BCBS enrollees. For example, AQC participants may have relied more heavily on analytic support from BCBS and patient-specific outreach to identify and redress quality deficits among BCBS enrollees, while broader strategies to address spending were implemented. Alternatively, a more complete assessment of quality of care for Medicare beneficiaries might reveal more consistent improvements across quality measures included in the AQC, most of which we could not assess from Medicare claims.
Our findings have several implications for payment and delivery system reforms. In general, cost-reducing spillover effects of ACO contracts with one insurer on care for other insurers’ enrollees should signal a willingness among provider organizations generating the spillovers to enter similar contracts with additional insurers; they could be rewarded for the savings and quality improvements achieved for the other insurers’ enrollees. Broad organizational responses to early ACO initiatives, like those suggested by our findings, might support a rapid transition among ACOs to global payment arrangements with multiple payers. Conversely, cost-reducing spillovers present a free-riding problem to commercial insurers engaged in ACO contracts, since competing insurers with similar provider networks could offer lower premiums without incurring the costs of managing an ACO. Additional efforts to foster multipayer participation in global payment systems, such as recent state initiatives and provisions in Pioneer Medicare ACO contracts,15,32,33 may be important.
Our study had several limitations. Several factors limited statistical power for comparing savings by the presence of prior risk-based contracts held by AQC participants with BCBS, an important predictor of savings among BCBS enrollees.10,11 The Medicare population served by AQC participants was approximately one-fifth the size of the BCBS population included in previous evaluations of the AQC, and the variance in medical spending for Medicare beneficiaries is much greater than for commercially insured patients.10,11
Differential changes in unobserved case mix could have contributed to our findings, but differences in observed patient characteristics between AQC participants and nonparticipating providers remained constant over the study period. Although our results and those of previous evaluations varied similarly across settings and services, suggesting that savings among Medicare beneficiaries were related to the AQC, participating organizations could have concurrently implemented unrelated interventions to improve care efficiency for all patients. Similarly, as legislative measures to control spending were debated in Massachusetts and Medicare ACO programs were proposed nationally, organizations may have broadened interventions related to the AQC in anticipation of future risk contracts with additional payers.34-36
Nevertheless, our study suggests that organizations in Massachusetts willing to assume greater financial risk were capable of achieving modest reductions in spending for Medicare beneficiaries without compromising quality of care. Although effects of commercial and Medicare ACO initiatives similar to the AQC may differ in other markets, these findings suggest potential for these payment models to foster systemic change in care delivery. Evaluations of ACO programs may need to consider spillover effects on other patient populations to assess their full clinical and economic benefits.
Corresponding Author: J. Michael McWilliams, MD, PhD, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115 (firstname.lastname@example.org).
Author Contributions: Dr McWilliams had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: McWilliams, Landon, Chernew.
Acquisition of data: McWilliams, Landon.
Analysis and interpretation of data: McWilliams, Landon, Chernew.
Drafting of the manuscript: McWilliams, Landon, Chernew.
Critical revision of the manuscript for important intellectual content: McWilliams, Landon, Chernew.
Statistical analysis: McWilliams, Landon, Chernew.
Obtained funding: McWilliams, Landon.
Study supervision: McWilliams, Landon.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Landon reports consultancy with Active Networks, Navigenics, Massachusetts Medical Society, United Biosource, and Research Triangle Institute. Dr Chernew reports board membership on the Medicare Payment Advisory Commission and the Congressional Budget Office. No other disclosures were reported.
Funding/Support: This study was supported by grants from the Beeson Career Development Award Program (National Institute on Aging grant K08 AG038354 and the American Federation for Aging Research), the Doris Duke Charitable Foundation (Clinical Scientist Development Award 2010053), the National Institute on Aging (grant P01 AG032952), and the Commonwealth Fund (grant 20110391).
Role of the Sponsor: The funding sources did not play a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.
Additional Contributions: We are grateful to Dana G. Safran, ScD, BCBS of Massachusetts, for comments on an earlier draft of this article and to Pasha Hamed, MA, Department of Health Care Policy, Harvard Medical School, for statistical programming support. Dr Safran was not compensated for her contributions. Mr Hamed’s contributions were supported by the same funding sources as the authors’ contributions.