To the Editor The study by Dr McWilliams and colleagues1 on spending and quality of care for Medicare beneficiaries associated with the Alternative Quality Contract (AQC) in Massachusetts contains important threats to validity that raise questions about the reliability of the results. The most important is volunteer bias.
This bias was identified as an issue with an earlier evaluation of the AQC,2 but the limitation is not mentioned in the article by McWilliams et al.1 Only 11 medical practices volunteered for participation and these practices had higher Medicare spending per beneficiary than the control practices, which raises the possibility that the practices chose to participate because they knew they could achieve savings and earn bonuses, possibly for activities that were already planned or under way. The practices that declined to participate, which became the control group, may have known that they could not achieve the required savings, further exaggerating the results. Controlling for patient differences between practices does not address the underlying selection bias.
Brown J. Spending and Quality of Care for Medicare Beneficiaries in Massachusetts. JAMA. 2013;310(24):2674–2675. doi:10.1001/jama.2013.282645