Difference-in-difference (DD) and difference-in-difference-in-difference (DDD) estimates are shown for Accountable Care Organization–driven changes in prostate, breast, and colorectal cancer screening by age. aP < .001.
Difference-in-difference (DD) and difference-in-difference-in-difference (DDD) estimates are shown for Accountable Care Organization–driven changes in prostate, breast, and colorectal cancer screening by quartile of predicted survival. aP < .001. bP < .01.
eTable 1. Differential Changes in Prevalence of Breast, Colorectal, and Prostate Cancer Screening After Medicare Shared Savings Program Accountable Care Organization Attribution (Specific Screening Definition)
eTable 2. Differential Change in Appropriateness of Breast, Colorectal, and Prostate Cancer Screening After Medicare Shared Savings Program Accountable Care Organization Attribution (Specific Screening Defintion)
eFigure 1. Changes in Breast Cancer Screening After ACO Attribution Stratified by Age and Predicted Survival
eFigure 2. Changes in Colorectal Cancer Screening After ACO Attribution Stratified by Age and Predicted Survival
eFigure 3. Changes in Prostate Cancer Screening After ACO Attribution Stratified by Age and Predicted Survival
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Resnick MJ, Graves AJ, Thapa S, et al. Medicare Accountable Care Organization Enrollment and Appropriateness of Cancer Screening. JAMA Intern Med. 2018;178(5):648–654. doi:10.1001/jamainternmed.2017.8087
Does Medicare Accountable Care Organization (ACO) enrollment drive changes in appropriateness of screening for breast, colorectal, and prostate cancer?
In this population-based analysis, Medicare ACO enrollment resulted in significant improvements in appropriateness of breast and colorectal cancer screening, namely improving screening rates among those likely to benefit and withholding screening from those unlikely to benefit. Conversely, ACO enrollment was associated with significant reductions in prostate cancer screening regardless of age or predicted survival.
Widespread diffusion of alternative payment models may improve the quality of breast and colorectal cancer screening programs by targeting screening to those likely to benefit and withholding screening from those who are not.
Despite rapid diffusion of Accountable Care Organizations (ACOs), whether ACO enrollment results in observable changes in cancer screening remains unknown.
To determine whether Medicare Shared Savings Program (MSSP) ACO enrollment changes the appropriateness of screening for breast, colorectal, and prostate cancers.
Design, Setting, and Participants
For this population-based analysis of Medicare beneficiaries, we used Medicare data from 2007 through 2014 and evaluated changes in screening associated with ACO enrollment using differences-in-differences (DD) analyses. We then performed difference-in-difference-in-differences (DDD) analyses to determine whether observed changes in cancer screening associated with ACO enrollment were different across strata of appropriateness, defined using age (65-74 years vs ≥75 years) and predicted survival (top vs bottom quartile).
Main Outcomes and Measures
Rates of breast, colorectal, and prostate cancer screening measured yearly as a proportion of eligible Medicare beneficiaries undergoing relevant screening services.
Among Medicare beneficiaries, comprising 39 218 652 person-years before MSSP enrollment and 17 252 345 person-years after MSSP enrollment, breast cancer screening declined among both ACO (42.7% precontract, 38.1% postcontract) and non-ACO (37.3% precontract, 34.1% postcontract) populations. The adjusted rate of decline (DD) in the ACO population exceeded the non-ACO population by 0.79% (P < .001). This decline was most pronounced among elderly women (–2.1%), with minimal observed change among younger women (−0.26%). Baseline colorectal cancer screening rates were lower than those for breast cancer among both ACO (10.1% precontract, 10.3% postcontract) and non-ACO (9.2% precontract, 9.1% postcontract) populations. We observed an adjusted 0.24% (P = .03) increase in screening associated with ACO enrollment, most pronounced among younger Medicare beneficiaries (0.36%). For breast and colorectal cancer, we observed statistically significant differences in estimates of effect between age strata, suggesting that the ACO effect on cancer screening is mediated by age (DDD for both P < .001). Prostate cancer screening declined among ACO (35.1% precontract, 28.5% postcontract) and non-ACO (31.2% precontract, 25.7% postcontract) populations. The adjusted rate of decline in the ACO population exceeded that of the non-ACO population by 1.2%. We observed no difference in estimate of effect between age strata, suggesting that the ACO-mediated changes in prostate cancer screening are similar among younger and elderly men. Results characterizing appropriateness with predicted survival mirrored those when stratified by age.
Conclusions and Relevance
Medicare Shared Savings Program ACO enrollment is associated with more appropriate breast and colorectal screening, although the magnitude of the observed ACO effect is modest in the early ACO experience.
The Centers for Medicare and Medicaid Services (CMS) and numerous commercial payers have implemented alternative payment models to align financial incentives toward high-value health care services.1-3 Accountable Care Organizations (ACOs) were developed to promote accountability4,5 through the promise of shared savings with improvements in the cost and quality of care delivered to populations. Evaluations of early Medicare ACOs suggest modest improvements in health care spending and health care quality.6-10 Furthermore, evidence suggests that ACO enrollment may reduce use of low-value health care services.11
There remain few studies that characterize the effect of ACO enrollment on appropriateness of care delivery, namely delivering services to individuals for whom the benefit exceeds the risk (appropriate screening), and withholding services from those in whom the risk outweighs the benefit (inappropriate screening, less appropriate screening). Effective cancer screening programs must balance the anticipated benefits associated with early cancer detection against the risks of diagnosis and treatment, with age and life-expectancy thresholds commonly used for screening recommendations.12,13 As such, “overscreening” refers to screening practices in individuals outside of ranges recommended by national guidelines14 and often includes screening practices in individuals at high risk of other-cause mortality who do not stand to benefit from early cancer detection. The United States Preventive Services Task Force (USPSTF) currently recommends breast cancer screening in women age 50 through 74 years with insufficient evidence to evaluate the benefits and harms in women older than 75 years.12 The USPSTF recommends colorectal cancer screening in individuals age 50 until 75 years.13 Conversely, the most recent USPSTF statement recommends against routine prostate cancer screening.15 While screening for breast and colorectal cancer are included as preventive care quality measures in the Medicare Shared Savings Program (MSSP), the net effect of quality incentives promoting cancer screening and the financial incentives discouraging screening and downstream management of screen-detected cancers remains unknown. To this end, we sought to determine whether Medicare ACO enrollment is associated with observable changes in appropriateness of cancer screening, promoting delivery of screening services to those who stand to benefit, and withholding cancer screening from those who do not.
Using Medicare enrollment and claims from 2006 through 2013, we restricted our cohort to those older than 65 years continuously enrolled in Parts A/B. In the event of death, the beneficiary was followed through the end of the last full calendar year. This study was approved by the Vanderbilt University Medical Center institutional review board.
Our cohort was built using beneficiaries identified in the 2013 Shared Savings Beneficiary file and a 20% random sample of beneficiaries. Each beneficiary was attributed yearly from 2007 through 2013 to an ACO or non-ACO provider. To do so, the 2013 CMS ACO provider file identified all ACO-participating providers. Each beneficiary was assigned, yearly, to the primary care provider delivering the most allowed charges for qualified services (eAppendix in the Supplement) per CMS MSSP attribution rules.16 If a beneficiary did not receive any qualified service from a primary care physician, she was attributed to the specialist billing for the plurality of qualified services. The primary care physician was used as the anchor for attribution to define a theoretical population of patients treated by similar physicians prior to ACO enrollment (ACO precontract). Beneficiaries attributed to ACO providers were “ACO,” while beneficiaries attributed to non-ACO providers were “control.” Our sample was restricted to beneficiaries receiving at least 1 qualified service to minimize bias associated with default non-ACO assignment in individuals not receiving care. For the purpose of this study, ACO attribution and enrollment at the beneficiary level are considered synonymous.
Beneficiaries attributed to an ACO had one of 3 staggered contract start dates (April 1, 2012, July 1, 2012, and January 1, 2013), and ACO beneficiaries were assigned an ACO start date corresponding to their particular MSSP contract. Control beneficiaries were randomly assigned one of the three MSSP start dates to permit evaluation across staggered start dates. The start date for each ACO and control beneficiary was considered T0, and our outcome measures were identified in yearly intervals both before and after this start date.
Breast, colorectal, and prostate cancer screenings were identified using inpatient, outpatient, and carrier claims from 2006 to 2014. Each outcome was defined on a yearly basis. Breast cancer screening was identified using claims for mammography; colorectal cancer using claims for fecal occult blood testing (FOBT), sigmoidoscopy, colonoscopy, or double-contrast barium enema; and prostate cancer screening using claims for prostate-specific antigen testing. Given limitations in the use of claims to differentiate screening from diagnostic services, we developed sensitive and specific screening definitions based on available literature. We present findings using the sensitive definition of our outcomes. Results using the specific definition are consistent with our primary study findings and are available in the eAppendix in the Supplement along with detailed descriptions of algorithms to identify study outcomes.
We included age, sex, race or ethnicity, dual Medicaid eligibility, and end-stage renal disease status from the Medicare Master Beneficiary Summary Files. We assessed indicators of 27 chronic conditions from the Chronic Conditions Data Warehouse, and generated a count of conditions per beneficiary. We incorporated zip code tabulation area–level sociodemographic characteristics from US Census data including percent below the federal poverty line, percent without a high school degree, percent enrolled in Medicare, and median household income.
Appropriateness was defined using age and predicted survival. Younger beneficiaries (65-74 years) and those in the highest quartile of predicted survival were considered appropriate screening candidates. Older beneficiaries (≥75 years) and those in the lowest quartile of predicted survival were considered inappropriate screening candidates. We fit Cox proportional hazards models to estimate 5-year survival using established methods.17 The model included age, sex, 20 comorbidities, and sex by comorbidity interactions. Male-specific models (prostate cancer screening) and female-specific models (breast cancer screening), included only age and comorbidities. Further details surrounding methodology is found in the eAppendix in the Supplement. Mean (SD) predicted 5-year survival in the highest and lowest quartiles was 94.5% (0.01%) and 53.5% (0.19%), respectively.
Our precontract period was defined as 5 years before the contract start date, excluding an implementation year immediately before contract start date. We excluded the year immediately preceding the contract start date a priori owing to predicted anticipatory changes in clinical practice in preparation for financial risk. Our postcontract period was defined as 2 years after the contract start date.
For each outcome, we performed difference-in-difference (DD) analyses to assess changes in annual screening rates after ACO intervention compared to before the intervention in the ACO group relative to contemporaneous changes in the non-ACO group using linear regression. Our main DD model specification is listed in the eAppendix in the Supplement.
For prostate and breast cancer screening, our precontract period included 5 years prior to ACO implementation. For colorectal cancer screening we implemented a 2-year precontract period to capture steady baseline screening rates limited by available data and the lengthy interval between recommended screenings. A sensitivity analysis for colorectal cancer including 5 years before ACO implementation was consistent with the primary study findings. We presented DD coefficients from models including all covariates described above and with fixed effects for each hospital referral region (HRR) in each year and robust standard errors to account for clustering at the HRR level. We compared DD estimates to mean screening rates in the ACO group in the precontract period to demonstrate changes relative to baseline mean.
To evaluate changes in appropriateness of screening resulting from ACO enrollment, we used linear regression and a difference-in-difference-in-differences (DDD) approach. The DD estimates reflect the magnitude of ACO-driven changes relative to contemporaneous changes in non-ACO populations within each of the appropriateness subgroups. The DDD estimates serve to characterize the magnitude and significance of differences between subgroup-level estimates of effect. We performed separate models for each screening outcome and for each measure of appropriateness (age, predicted survival). Our DDD model specifications are listed in the eAppendix in the Supplement.
We reported DDD coefficients from unadjusted models including fixed effects for each HRR in each year and robust standard errors at the HRR level. In addition to DDD estimates, we generated predicted DD estimates for each screening measure and level of appropriateness. We compared predicted DD estimates with mean screening rates in the ACO group in the precontract period to report change relative to baseline mean. We used SAS version 9.4 (SAS Institute Inc) for statistical analysis.
Our analysis comprised a total of 56 470 997 person-years of observation, including 13 460 798 person-years of data among ACO beneficiaries and 40 010 199 person-years of data among non-ACO beneficiaries. We observed minimal differences in baseline characteristics between ACO and non-ACO beneficiaries. Furthermore, the observed differential changes in beneficiary-level and community-level baseline characteristics from pre-ACO to post-ACO enrollment were consistently small magnitude (Table 1).
Accountable Care Organization enrollment was associated with a 0.79% reduction in breast cancer screening in excess of contemporaneous reductions observed in the control population (P < .001) This absolute change corresponds to a 1.80% reduction in breast cancer screening relative to baseline screening rates (P < .001). Accountable Care Organization enrollment was associated with a 0.24% absolute increase in colorectal cancer screening when compared with contemporaneous changes in the control population, corresponding to a 2.40% increase relative to baseline screening rates (P = .03). Finally, ACO enrollment was associated with a 1.20% reduction in prostate cancer screening when compared with contemporaneous changes in the control population, corresponding to a 3.40% reduction relative to baseline screening rates. Complete data reporting differential changes in screening rates are found in Table 2 with graphical representation in the eAppendix in the Supplement.
Age was strongly associated with ACO-driven changes in breast cancer screening. We observed little change in yearly breast cancer screening rates in women younger than 75 years associated with ACO enrollment. Conversely, among women 75 years or older, we observed an absolute reduction of 2.14%, corresponding to a 6.20% reduction from baseline. We observed a DDD of 1.87% (P < .001) (Table 3; Figure 1), suggesting a statistically significant difference in the effect of ACO enrollment on breast cancer screening among younger and older women. Figures 1 and 2 display the estimates of DD effect between strata of appropriateness by age and survival with the difference between the observed estimates of effect reflecting the DDD. Similar to the effect of age, we found little change in breast cancer screening after ACO enrollment among women in the highest quartile of predicted survival and a larger magnitude reduction in breast cancer screening among women in the lowest quartile of predicted survival (Table 3; Figure 2). These estimates suggest that ACO enrollment reduces breast cancer overscreening, namely screening those greater than 75 years or at high risk for 5-year mortality.
Changes driven by ACOs in colorectal cancer screening largely were driven by increases in screening in appropriate candidates. Among beneficiaries younger than 75 years, we observed a 0.36% increase in screening in ACO beneficiaries relative to changes in the control population. Given low baseline colorectal cancer screening rates, this absolute increase corresponds to a 3.00% increase relative to baseline mean. Conversely, we observed smaller magnitude changes in screening rates among beneficiaries 75 years or older, with a 0.08% absolute increase in colorectal cancer screening, corresponding to a 1.00% relative increase compared to baseline. We observed a DDD of 0.28% (P < .001) (Table 3; Figure 1), again suggesting a statistically significant difference in the effect of ACO enrollment on colorectal cancer screening between younger and older individuals. Similar findings were observed among survival strata. These data suggest that ACO enrollment is associated with positive changes in colorectal cancer screening among appropriate candidates, with smaller magnitude effects on colorectal cancer overscreening (Table 3; Figure 2).
Enrollment in ACOs reduced prostate cancer screening across appropriateness strata. Among beneficiaries younger than 75 years, we observed a 1.31% decrease in screening in ACO beneficiaries relative to changes in the control population, corresponding to a 3.30% reduction from baseline mean. Similarly, we observed a 1.48% absolute decrease in prostate cancer screening among men 75 years or older, corresponding to a 5.00% relative reduction when compared with baseline. We observed a nonsignificant DDD of 0.17%, suggesting that the effect of ACO enrollment on prostate cancer screening is similar among younger and older men (Table 3; Figure 1). Similar findings were observed among survival strata. Taken together, these data suggest that ACO enrollment is associated with reductions in prostate cancer screening irrespective of age and predicted survival (Table 3; Figure 2).
Our study sought to characterize the effect of ACO enrollment on the dual aim of promoting screening among Medicare beneficiaries likely to benefit, and discouraging screening in individuals unlikely to benefit from early cancer detection. We found that ACO enrollment was not associated with increasing breast cancer screening among appropriate candidates but was associated with significant reduction in overscreening. Conversely, ACO enrollment was associated with improvements in colorectal cancer screening among appropriate candidates with smaller magnitude increases in screening among those unlikely to benefit from early cancer detection. Finally, ACO enrollment was associated with reduction in prostate cancer screening regardless of age or predicted survival. While the directionality of observed changes in breast and colorectal cancer screening is promising, the magnitude of effect remains modest.
Cancer care remains a key driver of Medicare spending, and accordingly, an important cost center for ACOs. The USPSTF currently recommends breast cancer screening12 and colorectal cancer screening13 in specific populations. The USPSTF recently released a prostate cancer screening draft recommendation promoting shared decision-making among men younger than 70 years and recommends against screening in men over 70 years.15 Other professional organizations advocate for shared decision-making incorporating the predicted benefits and harms,18-20 underscoring the discretionary nature of prostate cancer screening. The MSSP includes rates of breast cancer screening and colorectal cancer screening as performance measures that, in part, drive the magnitude of shared savings for which an individual ACO is eligible.
There remains room for improvement in optimizing the value of cancer screening programs. Of the $1.08 billion spent on breast cancer screening-related services, $410.6 million is spent on elderly women in whom there remains insufficient evidence to evaluate the benefits and harms of screening.21 Medicare spends $450 million annually on prostate cancer screening, $145 million of which in men 75 years and older, unlikely to benefit from early detection.22 Indeed, a significant proportion of the US population undergoes cancer screening regardless of mortality risk. Specifically, 51.6%, 34.2%, and 40.8% of patients with a risk of 5-year mortality more than 50% underwent prostate, breast, and colorectal cancer screening, respectively.23
The dual aims of the ACO model are to improve quality and reduce growth in health care costs through accountability. To this end, quality incentives drive improvements in high-value screening in appropriate populations while financial incentives drive reductions in low-value screening among inappropriate populations owing to financial liability for the complete arc of cancer diagnosis, treatment, and survivorship. McWilliams and colleagues,7 in their evaluation of early MSSP performance found no significant differential change in breast cancer screening among female ACO beneficiaries aged 65 to 69 years, with similar findings among beneficiaries attributed to Pioneer ACOs.8 Schwartz and colleagues11 found Pioneer ACO alignment to be associated with a 2.4% aggregate reduction in low-value cancer screening.
Improvement in the quality of cancer screening programs may be achieved through reducing low-value screening, increasing high-value screening, or ideally, a combination of both. Our evaluation of breast, colorectal, and prostate cancer screening found MSSP ACO enrollment to be associated with significant improvements in the quality of breast and colorectal cancer screening; however, the directionality of these improvements was inconsistent. Accountable Care Organization enrollment was associated with a reduction in low-value breast cancer screening and an increase in high-value colorectal cancer screening. Indeed, there is evidence to suggest significant breast cancer overscreening23 and colorectal cancer underscreening.24 Accordingly, observed reductions in breast cancer overscreening coupled with improvements in colorectal cancer screening may indeed reflect migration toward “optimal” population-level screening rates as a result of the combination of quality measurement and financial accountability associated with ACO implementation.
Interestingly, ACO enrollment was associated with a significant reduction in prostate cancer screening, and the observed reduction occurred in both appropriate and inappropriate populations. There is ample evidence that the prevalence of prostate-specific antigen screening continues to decline, owing to the controversy surrounding the balance of risk and benefit associated with downstream diagnosis and treatment.25,26 The observed reduction in prostate-specific antigen screening ascribed to ACO enrollment may reflect efforts to optimize health care value by reducing prostate cancer overdiagnosis and overtreatment. There remains no meaningful estimate of the “optimal” rate of population-level prostate cancer screening, and as such, whether the observed reductions in screening associated with ACO enrollment reflect movement in the right or wrong direction remains unknown. Certainly, qualitative work to characterize the implications of these findings will be essential as we seek to understand the intended and unintended consequences of payment innovations.
There are a number of limitations that merit consideration. This study evaluates performance of the MSSP, and the generalizability of our findings to commercial or Medicaid ACOs remains unknown. While ACO and non-ACO populations appeared balanced, the voluntary nature of MSSP participation raises the possibility of unmeasured confounding. Our claims-based cancer screening measures are subject to misclassification, though such misclassification is not thought to be systematic and would bias our results toward the null. The use of claims precludes evaluation of patient preference, a critical element of clinical decision-making, particularly in the context of cancer screening. Finally, our survival model has been validated in a community-dwelling population and the extent to which this model is suitable for Medicare beneficiaries in nursing homes remains unknown.
Our findings provide evidence of ACO-driven small-magnitude reductions in breast cancer overscreening, improvements in appropriate colorectal cancer screening, and a reduction in prostate cancer screening in both appropriate and inappropriate candidates. While the directionality of observed changes, particularly in breast and colorectal cancer screening is promising, the magnitude of effect is modest in the early ACO experience. Further investigation will characterize the most meaningful levers to optimize cancer screening programs in the United States.
Corresponding Author: Matthew J. Resnick, MD, MPH, MMHC, A-1302 Medical Center North, Nashville, TN 37232 (firstname.lastname@example.org).
Accepted for Publication: November 24, 2017.
Published Online: March 19, 2018. doi:10.1001/jamainternmed.2017.8087
Author Contributions: Dr Resnick had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Resnick, Graves, Buntin, Penson.
Acquisition, analysis, or interpretation of data: Resnick, Graves, Thapa, Gambrel, Tyson, Lee, Buntin.
Drafting of the manuscript: Resnick, Graves.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Resnick, Graves, Thapa, Gambrel, Lee, Buntin.
Obtained funding: Resnick.
Administrative, technical, or material support: Resnick, Graves, Tyson, Penson.
Study supervision: Resnick, Graves, Buntin, Penson.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by a Mentored Research Scholar Grant in Applied and Clinical Research (MSRG-15-103-01-CHPHS to Dr Resnick) from the American Cancer Society and by the American Urological Association/Urology Care Foundation Rising Stars in Urology Research Program.
Role of the Funder/Sponsor: The funder/sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: