The Patient-Centered Medical Home and the Challenge of Evaluating Complex Interventions

The study by Reddy and colleagues¹ reports on an analysis of the Patient Aligned Care Team (PACT) initiative, a patient-centered medical home (PCMH) initiative from the US Department of Veterans Affairs (VA) and one of the largest PCMH initiatives in the country. Involving more than 1 million patients during 4 years (2012-2015), the study tapped into the extensive databases of the VA, including results from surveys of patients and staff specifically designed to capture elements of PCMH. Reddy et al¹ examined the association of longitudinal changes in PCMH implementation with 3 high-cost health care utilization outcomes: emergency department (ED) visits, hospitalizations for ambulatory care–sensitive conditions (ACSCs), and all-cause hospitalizations. They found no consistent association of more robust PCMH implementation with better outcomes.

While there has been considerable enthusiasm about the potential for PCMH to revolutionize primary care in the United States, PCMH is more of a collection of core principles than a set of specific interventions. Many different interventions may address the various principles of PCMH, with the result that PCMH initiatives can look very different across the organizations attempting them. Two reviews from 2017 have confirmed this variation,^2,3 and they also found limited evidence of associations with most clinical and financial outcomes, with more consistent findings involving small to moderate improvement with selected prevention and chronic disease management measures.

The complexity of the intervention being attempted in the VA PCMH initiative is evidenced by the main measure for its implementation, the PACT implementation progress index (Pi²) score. The Pi² score consists of 53 different elements, covering 8 domains, which are drawn from patient surveys, staff surveys, and measures of clinic operations.¹ This score has been used to assess many outcomes, covering all the dimensions of health care quality as well as staff satisfaction and burnout. Given that a basic principle of scientific investigation is to control as many variables as possible, ideally isolating a single independent variable to study its effect or associations, it should not be surprising that the findings of system-level studies that include multiple independent variables and multiple outcomes of PCMH interventions have included mixed results. There are many possible explanations for this outcome. Perhaps PCMH is not effective in reducing costs. However, it is also possible that out of the many elements of PACT, those that can reduce ED visits or hospitalizations were not effectively implemented or were implemented to a similar degree in the 2 periods studied.

This study is of particular interest because the same authors had previously reported impressive differences in outcomes in a cross-sectional analysis between clinics with high Pi² scores for PCMH implementation vs those with low Pi² scores.⁴ They found that the practices that received the highest Pi² scores had lower hospitalization rates for ACSC and lower ED use. In contrast, in this study that examined a 4-year implementation period, Reddy et al¹ did not find any consistent associations of longitudinal differences between PCMH implementation and rates of all-cause hospitalization, ACSC hospitalizations, and ED visits.

Why the difference between the 2 studies performed by these authors (cross-sectional⁴ vs longitudinal¹), and why has it been so difficult to show benefit from PCMH implementation? First, implementing complex system changes, such as PCMH, in health care is not easy. Possibly, high-functioning clinics that already perform well on desired outcomes may be more successful in executing changes requested by their parent organization. Cross-sectional analyses may find associations of successful implementation with high performance but these associations may not be causal. Their high performance may be associated with their high level of function rather than the implemented changes. This possibility is supported by a comparative case study by Misra-Hebert et al⁵ examining what practice setting components were associated with uptake of new team-based primary care models. Strong local leadership and stable staffing were underlying elements associated with high implementation rates. Practices with these components had high performance on multiple quality metrics, both before and after the changes. Given that much of the literature on PCMH consists of cross-sectional analyses, this study by Reddy et al¹ signals the need for caution about such reports regarding complex system interventions in which implementation is variable.

Despite the results of this study by Reddy et al¹ and the mixed results regarding PCMH initiatives in the medical literature, we continue to feel optimistic about the principles of PCMH as a pathway forward for improving outcomes in primary care. Many experts agree that investment in primary care as the foundation of the health care system is required if the goal of the system is to contain costs and improve outcomes and quality of care. There is strong support in the medical literature for the value of primary care, and many opportunities to improve primary care exist within the US health care system.⁶ International comparisons have shown that the increased investments in primary care being made in other countries, often 2-fold the 5% to 6% of health care expenditures seen in the United States, are associated with lower overall health care costs and better patient outcomes.

Thus, while the need for investment in primary care remains, perhaps it is time to move away from complex, multifaceted organizational initiatives hoping for simultaneous improvement across a broad array of outcomes. Perhaps it is time to study interventions more focused in their content, target population, and desired outcomes. A good example is the recent literature on primary care programs specifically focused on reducing readmissions.^7,8 Studies from 2014⁸ and 2019⁷ have found that several interventions, such as reliable communication with inpatient health care practitioners combined with timely primary care follow-up, were associated with reduced hospital readmissions. By slowly but steadily building and disseminating an assortment of empirically proven interventions, it may be possible to achieve more effectively the goal of high-performing primary care that addresses the triple aim of improving the individual experience of care, improving the health of populations, and reducing the per capita costs of care for populations.

Published: February 5, 2020. doi:10.1001/jamanetworkopen.2019.20827

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Phillips RS et al. JAMA Network Open.

Corresponding Author: Russell S. Phillips, MD, Center for Primary Care, Harvard Medical School, 635 Huntington Ave, Boston, MA 02115 (russell_phillips@hms.harvard.edu).

Conflict of Interest Disclosures: None reported.