Association of High-Cost Health Care Utilization With Longitudinal Changes in Patient-Centered Medical Home Implementation | Health Care Delivery Models | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Analytic Cohort Diagram
Analytic Cohort Diagram

ED indicates emergency department; VHA, Veterans Health Administration.

Figure 2.  Distribution of Clinics by Change in Patient Aligned Care Team Implementation Progress Index (Pi2) Score From 2012 to 2015
Distribution of Clinics by Change in Patient Aligned Care Team Implementation Progress Index (Pi2) Score From 2012 to 2015
Table 1.  Patient Characteristics at Baseline in 2012
Patient Characteristics at Baseline in 2012
Table 2.  Association of Change in Pi2 Score With Patient Health Care Utilization
Association of Change in Pi2 Score With Patient Health Care Utilization
1.
Edwards  ST, Bitton  A, Hong  J, Landon  BE.  Patient-centered medical home initiatives expanded in 2009-13: providers, patients, and payment incentives increased.  Health Aff (Millwood). 2014;33(10):1823-1831. doi:10.1377/hlthaff.2014.0351PubMedGoogle ScholarCrossref
2.
Dale  SB, Ghosh  A, Peikes  DN,  et al.  Two-year costs and quality in the comprehensive primary care initiative.  N Engl J Med. 2016;374(24):2345-2356. doi:10.1056/NEJMsa1414953PubMedGoogle ScholarCrossref
3.
Kelvey-Albert  M. PCMH sustainability road map for multi-site practices. Paper presented at: American Public Health Association 2019 Annual Meeting and Expo; November 2, 2019; Philadelphia, PA.
4.
Peikes  D, Zutshi  A, Genevro  JL, Parchman  ML, Meyers  DS.  Early evaluations of the medical home: building on a promising start.  Am J Manag Care. 2012;18(2):105-116.PubMedGoogle Scholar
5.
Sinaiko  AD, Landrum  MB, Meyers  DJ,  et al.  Synthesis of research on patient-centered medical homes brings systematic differences into relief.  Health Aff (Millwood). 2017;36(3):500-508. doi:10.1377/hlthaff.2016.1235PubMedGoogle ScholarCrossref
6.
Friedberg  MW.  What do you mean by medical home?  Ann Intern Med. 2016;164(6):444-445. doi:10.7326/M16-0058PubMedGoogle ScholarCrossref
7.
Jackson  GL, Williams  JW  Jr.  Does PCMH “work”?—the need to use implementation science to make sense of conflicting results.  JAMA Intern Med. 2015;175(8):1369-1370. doi:10.1001/jamainternmed.2015.2067PubMedGoogle ScholarCrossref
8.
Vest  JR, Bolin  JN, Miller  TR, Gamm  LD, Siegrist  TE, Martinez  LE.  Medical homes: “where you stand on definitions depends on where you sit.”  Med Care Res Rev. 2010;67(4):393-411. doi:10.1177/1077558710367794PubMedGoogle ScholarCrossref
9.
Nelson  KM, Helfrich  C, Sun  H,  et al.  Implementation of the patient-centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use.  JAMA Intern Med. 2014;174(8):1350-1358. doi:10.1001/jamainternmed.2014.2488PubMedGoogle ScholarCrossref
10.
Tuepker  A, Kansagara  D, Skaperdas  E,  et al.  “We’ve not gotten even close to what we want to do”: a qualitative study of early patient-centered medical home implementation.  J Gen Intern Med. 2014;29(2)(suppl 2):S614-S622. doi:10.1007/s11606-013-2690-zPubMedGoogle ScholarCrossref
11.
Taylor  HA, Greene  BR, Filerman  GL.  A conceptual model for transformational clinical leadership within primary care group practice.  J Ambul Care Manage. 2010;33(2):97-107. doi:10.1097/JAC.0b013e3181d91707PubMedGoogle ScholarCrossref
12.
Fihn  SD, Francis  J, Clancy  C,  et al.  Insights from advanced analytics at the Veterans Health Administration.  Health Aff (Millwood). 2014;33(7):1203-1211. doi:10.1377/hlthaff.2014.0054PubMedGoogle ScholarCrossref
13.
Liu  C-F, Chapko  M, Bryson  CL,  et al.  Use of outpatient care in Veterans Health Administration and Medicare among veterans receiving primary care in community-based and hospital outpatient clinics.  Health Serv Res. 2010;45(5 Pt 1):1268-1286. doi:10.1111/j.1475-6773.2010.01123.xPubMedGoogle ScholarCrossref
14.
Burgess  JF  Jr, Maciejewski  ML, Bryson  CL,  et al.  Importance of health system context for evaluating utilization patterns across systems.  Health Econ. 2011;20(2):239-251. doi:10.1002/hec.1588PubMedGoogle ScholarCrossref
15.
World Health Organization.  International Classification of Diseases, Ninth Revision (ICD-9). Geneva, Switzerland: World Health Organization; 1977.
16.
Agency for Healthcare Research and Quality.  Prevention quality indicators: technical specifications. https://www.qualityindicators.ahrq.gov/Modules/pqi_resources.aspx. Accessed October 10, 2017.
17.
Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.  J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004PubMedGoogle ScholarCrossref
18.
Hebert  PL, Batten  AS, Gunnink  E,  et al.  Reliance on Medicare providers by veterans after becoming age-eligible for Medicare is associated with the use of more outpatient services.  Health Serv Res. 2018;53(3)(suppl 3):5159-5180. doi:10.1111/1475-6773.13033PubMedGoogle ScholarCrossref
19.
Liu  CF, Manning  WG, Burgess  JF  Jr,  et al.  Reliance on Veterans Affairs outpatient care by Medicare-eligible veterans.  Med Care. 2011;49(10):911-917. doi:10.1097/MLR.0b013e31822396c5PubMedGoogle ScholarCrossref
20.
Wong  ES, Maciejewski  ML, Hebert  PL, Reddy  A, Liu  CF.  Predicting primary care use among patients in a large integrated health system: the role of patient experience measures.  Med Care. 2019;57(8):608-614. doi:10.1097/MLR.0000000000001155PubMedGoogle ScholarCrossref
21.
Reddy  A, Wong  E, Canamucio  A,  et al.  Association between continuity and team-based care and health care utilization: an observational study of Medicare-eligible veterans in VA Patient Aligned Care Team.  Health Serv Res. 2018;53(3)(suppl 3):5201-5218. doi:10.1111/1475-6773.13042PubMedGoogle ScholarCrossref
22.
David  G, Gunnarsson  C, Saynisch  PA, Chawla  R, Nigam  S.  Do patient-centered medical homes reduce emergency department visits?  Health Serv Res. 2015;50(2):418-439. doi:10.1111/1475-6773.12218PubMedGoogle ScholarCrossref
23.
Peikes  D, Chen  A, Schore  J, Brown  R.  Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials.  JAMA. 2009;301(6):603-618. doi:10.1001/jama.2009.126PubMedGoogle ScholarCrossref
24.
Huntley  AL, Thomas  R, Mann  M,  et al.  Is case management effective in reducing the risk of unplanned hospital admissions for older people: a systematic review and meta-analysis.  Fam Pract. 2013;30(3):266-275. doi:10.1093/fampra/cms081PubMedGoogle ScholarCrossref
25.
Nelson  K, Sylling  PW, Taylor  L, Rose  D, Mori  A, Fihn  SD.  Clinical quality and the patient-centered medical home.  JAMA Intern Med. 2017;177(7):1042-1044. doi:10.1001/jamainternmed.2017.0963PubMedGoogle ScholarCrossref
26.
Martsolf  GR, Kandrack  R, Baird  M, Friedberg  MW.  Estimating associations between medical home adoption, utilization, and quality: a comparison of evaluation approaches.  Med Care. 2018;56(1):25-30. doi:10.1097/MLR.0000000000000842PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Health Policy
    February 5, 2020

    Association of High-Cost Health Care Utilization With Longitudinal Changes in Patient-Centered Medical Home Implementation

    Author Affiliations
    • 1Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, Seattle, Washington
    • 2VA Puget Sound Health Care System, Seattle, Washington
    • 3Department of Medicine, School of Medicine, University of Washington, Seattle
    • 4Department of Health Services, School of Public Health, University of Washington, Seattle
    JAMA Netw Open. 2020;3(2):e1920500. doi:10.1001/jamanetworkopen.2019.20500
    Key Points español 中文 (chinese)

    Question  Is a change in patient-centered medical home implementation associated with changes in high-cost health care utilization in the US Veterans Health Administration health care system?

    Findings  In this cohort study including 1 650 976 veterans, neither improvement nor decline in patient-centered medical home implementation was consistently associated with changes in the numbers of emergency department visits, ambulatory care–sensitive condition hospitalizations, or all-cause hospitalizations.

    Meaning  This finding highlights key challenges in measuring patient-centered medical home implementation over time.

    Abstract

    Importance  In 2010, the US Veterans Health Administration (VHA) implemented one of the largest patient-centered medical home (PCMH) models in the United States, the Patient Aligned Care Team initiative. Early evaluations demonstrated promising associations with improved patient outcomes, but limited evidence exists on the longitudinal association of PCMH implementation with changes in health care utilization.

    Objective  To determine whether a change in PCMH implementation is associated with changes in emergency department (ED) visits, hospitalizations for ambulatory care–sensitive conditions (ACSCs), or all-cause hospitalizations.

    Design, Setting, and Participants  This cohort study used national patient-level data from the VHA and Centers for Medicare & Medicaid Services between October 1, 2012, and September 30, 2015. A total of 1 650 976 patients from 897 included clinics were divided into 2 cohorts: patients younger than 65 years who received primary care at VHA sites affiliated with a VHA ED and patients 65 years or older who were enrolled in both VHA and Medicare services.

    Exposures  Clinics were categorized on improvement or decline in PCMH implementation based on their Patient Aligned Care Team implementation progress index (Pi2) score.

    Main Outcomes and Measures  Change in the number of ED visits, ACSC hospitalizations, and all-cause hospitalizations among patients at each clinic site.

    Results  The study included a total of 1 650 976 patients, of whom 581 167 (35.20%) were younger than 65 years (mean [SD] age, 49.03 [10.28] years; 495 247 [85.22%] men) and 1 069 809 (64.80%) were 65 years or older (mean [SD] age, 74.64 [7.41] years; 1 050 110 [98.16%] men). Among patients younger than 65 years, there were fewer ED visits among patients seen at clinics that had improved PCMH implementation (110.8 fewer visits per 1000 patients; P < .001) and clinics that had somewhat worse implementation (69.0 fewer visits per 1000 patients; P < .001) compared with clinics that had no change in Pi2 score. There were no associations of change in Pi2 scores with all-cause hospitalizations or ACSC hospitalizations among patients younger than 65 years. In patients 65 years or older, those seen at clinics that had somewhat worse PCMH implementation experienced fewer ED visits (20.1 fewer visits per 1000 patients; P = .002) and all-cause hospitalizations (12.4 fewer hospitalizations per 1000 patients; P = .007) compared with clinics with no change in Pi2 score. There was no association between change in Pi2 score with ACSC hospitalizations among patients 65 years or older.

    Conclusions and Relevance  There were no consistent associations of change in Pi2 score with high-cost health care utilization. This finding highlights the key differences in measuring PCMH implementation longitudinally compared with cross-sectional study designs.

    Introduction

    Primary care practices across the United States are adopting the patient-centered medical home (PCMH) model to improve quality and reduce costs for patients and payers. The PCMH model is considered a major shift in primary care delivery that establishes a team-based care approach to deliver patient-centered care.1 To accomplish this goal, PCMH models often focus on activities that improve patient access and continuity, care coordination, self-management support, and population health activities.2 In the United States, more than 13 000 practices are recognized as PCMHs by the National Committee for Quality Assurance.3 Despite widespread adoption, evaluations of PCMH interventions have shown mixed results in reducing high-cost health care use, including emergency department (ED) visits and hospitalizations.4 A 2017 meta-analysis of 11 PCMH interventions5 showed no improvements in ED use, hospitalizations for ambulatory care–sensitive conditions (ACSCs), or all-cause hospitalizations. The variation in PCMH implementation results may in part be because of lack of a universal definition of PCMH, specific study designs, analytic methods, and variation in the implementation of the PCMH model.6-8

    The US Veterans Health Administration (VHA) adopted the PCMH model in 2010 under the Patient Aligned Care Team (PACT) initiative. Similar to other PCMH models, the PACT initiative represented a major restructuring of primary care services with the goal of providing team-based, patient-centered, coordinated, and comprehensive care. A 2014 study by Nelson et al9 reported wide variation in PCMH implementation in the VHA and found that clinics with more effective implementation of PCMH had lower rates of ED visits and ACSC hospitalizations, but Nelson et al9 did not find significant differences in all-cause hospitalizations. However, the PCMH is a complex intervention that requires time to fully implement changes in care delivery before it can begin translating into improved outcomes. For example, Tuepeker et al10 identified several barriers to successful implementation during the early years of PCMH implementation in the VHA, including the unavailability of necessary resources and time, inadequate staffing, variation in support for PCMH adoption across facilities, and the need for further training. Resistance to change has also been identified as a barrier to successful PCMH implementation both within and outside of the VHA.11

    Given these early barriers to PCMH implementation, it is important to examine the extent to which primary care clinics have improved over time and to assess whether these improvements are associated with patient outcomes. Few studies have examined such longitudinal changes in PCMH implementation over time in parallel with patient outcomes. This information is important to guide stakeholders in determining the long-term benefit of implementing the PCMH model. For integrated health systems, understanding the longitudinal associations between PCMH and patient outcomes can inform the value of investing resources in improving PCMH implementation. In this study, our objective was to determine whether a change in PCMH implementation, defined as whether a clinic improves or declines in its PACT implementation progress index (Pi2) score, is associated with changes in health care utilization.

    Methods

    This study was completed as part of ongoing quality improvement effort at the VHA and was not considered research activity per VHA policy. Therefore, it was not subject to institutional review board review or waiver. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    We performed a retrospective longitudinal cohort study using linked national patient-level data from the VHA and the Centers for Medicare & Medicaid Services from October 1, 2012, to September 30, 2015. The primary data source for patient demographic characteristics, clinical characteristics, and use of health services was the US Department of Veterans Affairs (VA) Corporate Data Warehouse.12 Parallel claims data from the Centers for Medicare & Medicaid Services were used to track use of services reimbursed by Medicare. Using data from the Primary Care Management Module contained within the VA Corporate Data Warehouse, we identified all patients who were enrolled in primary care in the baseline year (ie, 2012) and final year (ie, 2015). All enrolled patients in the VHA are assigned to a primary care physician recorded in the Primary Care Management Module. We further assigned these patients to clinics based on their longest primary care physician relationship during the study period.

    Previous studies have documented that veterans commonly use care both within and outside of the VHA.13,14 To more accurately capture high-cost health care utilization among veterans, we created 2 distinct cohorts (Figure 1). Our first cohort included patients who were younger than 65 years throughout the study period and received primary care at a VHA site that had an ED. Our second cohort included only patients who were 65 years or older and enrolled for both VHA and Medicare services during the baseline year. We excluded patients who were enrolled in Medicare Advantage health maintenance organizations because claims from these payers were not available. In this cohort, we used linked Medicare fee-for-service claims and VA Corporate Data Warehouse data to measure combined counts of ED visits, ACSC hospitalizations, and all-cause hospitalizations. To identify these visit types, we used a previously validated algorithm that used VHA-related stop codes, Current Procedural Terminology (CPT) codes, and Medicare revenue codes.13,14 Hospitalizations for ACSCs were based on Agency for Healthcare Research and Quality Prevention Quality Indicators and were identified through standardized protocols using International Classification of Diseases, Ninth Revision,15 and CPT codes from inpatient Medicare or VHA records.16

    Change in PCMH Implementation Using Pi2 Score

    To measure clinic-level change in PCMH implementation, we used a previously validated measure of PCMH implementation, the Pi2 score.9 The Pi2 score is a clinic-level composite that combines annual patient surveys (eg, US VA Survey of Healthcare Experiences of Patients), staff-level surveys (eg, VA PACT Personnel Survey), and administrative health care data (eg, continuity of care). Using these data, each clinic is assigned an overall score based on the 8 domains of the PCMH model (ie, access, care continuity, care coordination, comprehensiveness, self-management support, patient-centered care and communication, shared decision-making, and delegation, staffing, and team function). Development and validation of the Pi2 is described elsewhere.9 Since 2012, more than 900 clinics in the VHA have been assigned a Pi2 score annually. Scores range from 8 (ie, all domain scores are in the top quartile) to −8 (ie, all domain scores are in the bottom quartile).

    For each clinic, we calculated the change in Pi2 score between 2012 and 2015. Change in clinic-level Pi2 scores was calculated by subtracting a clinic’s Pi2 score in 2012 from the clinic’s Pi2 score in 2015. Changes in Pi2 scores ranged from −15 to 10 (Figure 2). For a clinic to have a score change of 10, they would have had scores in the bottom quartile of clinics for at least 5 domains of the Pi2 in 2012 and have scores in the top quartile of clinics for at least 5 domains of the Pi2 in 2015. Based on the original 5 categories of PCMH implementation, we created similar categories of change in PCMH implementation based on Pi2 scores. We categorized clinics into 5 categories based on their improvement or decline in PCMH implementation during the 4-year period: improved implementation (Pi2 score change, >3, which would correspond to an improvement in 3 domains of the Pi2 score), somewhat improved implementation (Pi2 score change, 2 to 3), no change in implementation (Pi2 score change, −1 to 1), somewhat worse implementation (Pi2 score change, −2 to −3), and worse implementation (Pi2 score change, <−3).

    Statistical Analysis

    A patient-level analysis was conducted for each of 3 outcomes stratified by age (≥65 vs <65 years) for a total of 6 models. Separate negative binomial models were used to estimate the number of hospitalizations and ED visits. Owing to rarity of ACSC hospitalizations, analysis for this association was modeled using logistic regression. In each model, the primary variable of interest was the categorical measure capturing change in PCMH implementation, and we adjusted for baseline patient characteristics that may have accounted for health care utilization, including age, sex, self-reported race/ethnicity (ie, non-Hispanic white, non-Hispanic black, or Hispanic or other), marital status, and income. Furthermore, we controlled for patient comorbidity using the validated comorbidity score developed by Gagne et al17 and VHA copayment status (copayment exempt vs not copayment exempt). Finally, we adjusted for baseline counts of our outcome measure in each model and adjusted for clinic-level clustering of patients by including a clinic-level random effect.

    For each model, we present the 2-sided P values from a likelihood ratio test comparing the full model with a model that did not include the change in PCMH implementation. This test demonstrates the overall association of change in PCMH implementation with the statistical model. We present the incident rate ratio (IRR) and average marginal effect for each of the 5 clinic-level change categories for each outcome. A nominal P value of .05 was used to assess all statistical hypotheses. All statistical analyses were performed using Stata statistical software version 15.0 (StataCorp).

    We conducted 2 main sensitivity analyses. Clinics may be limited in the amount they can improve or worsen in Pi2 score based on their baseline score. For example, a high-functioning PCMH clinic in 2012 can exhibit ceiling effects by which it may be limited in the how much it could improve. Therefore, we subdivided clinics into their original 5 categories based on their baseline score and modeled each outcome for the 2 cohorts based on the initial clinic score (eTable 1 in the Supplement). Our second sensitivity analysis was based on the hypothesis that a VHA clinic’s improvement would only have associations among patients who were seen primarily at the VHA. Among the cohort 65 years and older, we calculated each patient’s reliance on the VHA in the baseline year 2012 as the proportion of all outpatient visits (VHA and Medicare) received from VHA. These visits were counted based on the number of evaluation and management visits in Medicare and VHA.18,19 We then investigated the association of change in Pi2 score with high-cost health care utilization if a patient was at least 50% reliant on VHA services, a previously defined cutoff (eTable 2 in the Supplement).20

    Results

    Among 1 650 976 included patients in VHA care from 2012 to 2015, 581 167 (35.20%) were younger than 65 years (mean [SD] baseline age, 49.03 [10.28] years; 495 247 [85.22%] men) and 1 069 809 (64.80%) were 65 years or older (mean [SD] baseline age, 74.64 [7.41] years; 1 050 110 [98.16%] men) (Table 1). Compared with patients younger than 65 years, the cohort of patients 65 years and older included more white patients (320 060 patients [55.07%] vs 913 801 patients [85.42%]) and was more likely to have been treated in a rural clinic (55 032 patients [9.47%] vs 224 709 patients [21.00%]). Patients 65 years or older had a mean (SD) of 567.20 (1313.44) ED visits per 1000 patients, 269.45 (724.10) all-cause hospitalizations per 1000 patients, and 36.49 (228.74) ACSC hospitalizations per 1000 patients. Patients younger than 65 years had a mean (SD) of 869.39 (1896.51) ED visits per 1000 patients, 156.38 (607.33) all-cause hospitalizations per 1000 patients, and 7.10 (101.35) ACSC hospitalizations per 1000 patients.

    There were no consistent associations of change in a clinic’s Pi2 score with the main study outcomes. Among patients younger than 65 years, there was an association of a change in Pi2 score with ED visits. However, we found fewer ED visits among patients who were seen at clinics that had improved Pi2 scores (110.8 fewer ED visits per 1000 patients; P < .001) and clinics that had somewhat worse Pi2 scores (69.0 fewer ED visits per 1000 patients; P < .001) compared with clinics that had no change in Pi2 scores (Table 2). Furthermore, among patients younger than 65 years, we identified no associations of the change in Pi2 score with total hospitalizations or ACSC hospitalizations. Among patients 65 years or older, we found fewer ED visits among patients who were seen at clinics with worse Pi2 scores (17.9 fewer ED visits per 1000 patients; P = .03) and clinics with somewhat worse Pi2 scores (20.1 fewer ED visits per 1000 patients; P = .002). In addition, patients 65 years or older seen at clinics with somewhat worse Pi2 scores had 12.4 fewer all-cause hospitalizations per 1000 patients (P = .007) compared with patients at clinics with no change in Pi2 scores (Table 2). There were no other associations of change in Pi2 scores with ED visits or all-cause hospitalizations and no association of change in Pi2 scores with ACSC hospitalizations among patients 65 years or older.

    In our sensitivity analyses, we found that neither stratifying by baseline score nor selecting patients 65 years and older who primarily relied on VHA services had a consistent association between change in Pi2 score and our outcomes (eTable 1 and eTable 2 in the Supplement). Overall, these additional analyses did not change our quantitative or qualitative interpretation of our main results.

    Discussion

    In this cohort study, we found no consistent association of change in PCMH capabilities with changes in high-cost health care utilization, including ED visits, ACSC hospitalizations, and all-cause hospitalizations. These results were surprising, given that our previous research21 found that better PCMH implementation in care delivery elements, such as team-based care and care continuity, was significantly associated with lower rates of high-cost acute health care utilization. We originally hypothesized that improvement in a clinic’s PCMH capabilities over time would be associated with less high-cost health care utilization. This hypothesis was based on a clinic’s ability to deliver improvements in core primary care capabilities, such as access, continuity, care coordination, and team-based care, which could improve a patient’s ability to better manage chronic disease, in turn leading to fewer ED visits and hospitalizations. Indeed, a 2015 study by David et al22 found that the adoption of the PCMH model was associated with lower ED utilization among patients who were chronically ill but not among patients without chronic illness. However, many ED visits and hospitalizations may not be avoidable, and few interventions have been shown to systematically reduce ED visits or hospitalizations.23,24

    While a cross-sectional design measures the achievement of PCMH capabilities at a point in time, a longitudinal study design measures the extent to which clinics achieve improvement in PCMH capabilities and whether this improvement is associated with reductions in high-cost health care utilization in a given clinic’s patient population. This ability of clinics to transform, including experiencing improvement in PCMH capabilities over time, represents a fundamentally different construct than what has been captured in prior cross-sectional studies, to our knowledge.9,25 Our study is consistent with findings by Martsolf et al26 that suggested that longitudinal analyses could attenuate the estimated associations seen in cross-sectional designs. While these prior studies used different measures of PCMH capabilities than our study, such as National Committee for Quality Assurance–level accreditation and Medical Home Index, they also demonstrated that patient outcomes were sensitive to the methods chosen for analysis.

    These findings lead to additional questions that are critical for PCMH evaluations in and outside the VHA. For example, do certain functions or domains of the PCMH have a threshold effect? If that is the case, improvement in certain PCMH capabilities would not lead to additional improvement in outcomes. Moreover, specific domains of PCMH may be more important for improving care in specific groups of patients. For example, continuity for a patient with multiple chronic conditions may be more important than interventions to improve access to care. This heterogeneity in change in specific domains of PCMH is worth additional investigation. This information is vital as health care organizations invest and expand primary care delivery based on the principles of the PCMH.

    Limitations

    We acknowledge several limitations in our study. First, our cohort of patients included only those who were enrolled at the same clinic throughout the period. While this may limit generalizability, we chose to include patients seen at the same clinical site over time to better estimate how improvement in PCMH capabilities over time were associated with outcomes for a common set of patients. Furthermore, patients could have been seen at different clinics between 2012 and 2015, limiting the exposure of PCMH implementation on outcomes. However, we found that most patients in both cohorts were seen at the same clinic throughout this period. Second, the Pi2 score used to measure PCMH implementation is a unique and validated score to measure PCMH capabilities; however, it is difficult to measure complex elements of the PCMH. Surveys and administrative data comprising the Pi2 score may not capture all processes and aspects of the PCMH. Although the Pi2 score can differentiate clinics in a cross-sectional analysis, it may not be sufficiently sensitive to change.26 Third, PCMH implementation itself could be constrained by the resources and staff limitations within the VHA. This may limit a clinic’s ability to adopt and improve on key PCMH domains. Fourth, this is an observational study, and there may be unmeasured patient- or clinic-level factors associated with PCMH implementation and patient health care utilization that were not accounted for in our analysis.

    Conclusions

    This study found no consistent association of improvements in PCMH capabilities with high-cost health care utilization. Furthermore, we found no association of PCMH improvement with capabilities when patients primarily relied on the VHA for care and when stratifying by baseline PCMH capabilities. Our findings highlight key differences in measuring PCMH capabilities longitudinally compared with previous cross-sectional study designs and the sensitivity of estimates based on analytic method used for analyses.

    Back to top
    Article Information

    Accepted for Publication: December 9, 2019.

    Published: February 5, 2020. doi:10.1001/jamanetworkopen.2019.20500

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Reddy A et al. JAMA Network Open.

    Corresponding Author: Ashok Reddy, MD, MSc, Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S Columbian Way, MS: S-152, Seattle, WA 98108 (ashok.reddy@va.gov).

    Author Contributions: Dr Reddy had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Reddy, Gunnink, Wong, Nelson.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Reddy, Gunnink.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Reddy, Gunnink, Wong, Batten.

    Obtained funding: Reddy.

    Administrative, technical, or material support: Reddy, Nelson.

    Supervision: Reddy, Taylor, Nelson.

    Conflict of Interest Disclosures: Dr Wong reported owing stock in UnitedHealth Group and receiving personal fees from American Thoracic Society outside the submitted work. No other disclosures were reported.

    Funding/Support: This material is the result of work supported with resources and the use of facilities at the VA Puget Sound Health Care System, Seattle, Washington. This work was conducted and supported by the US Department of Veterans Affairs Office of Primary Care.

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: Dr Fihn, deputy editor of JAMA Network Open, was not involved in the editorial review of or the decision to publish this article.

    References
    1.
    Edwards  ST, Bitton  A, Hong  J, Landon  BE.  Patient-centered medical home initiatives expanded in 2009-13: providers, patients, and payment incentives increased.  Health Aff (Millwood). 2014;33(10):1823-1831. doi:10.1377/hlthaff.2014.0351PubMedGoogle ScholarCrossref
    2.
    Dale  SB, Ghosh  A, Peikes  DN,  et al.  Two-year costs and quality in the comprehensive primary care initiative.  N Engl J Med. 2016;374(24):2345-2356. doi:10.1056/NEJMsa1414953PubMedGoogle ScholarCrossref
    3.
    Kelvey-Albert  M. PCMH sustainability road map for multi-site practices. Paper presented at: American Public Health Association 2019 Annual Meeting and Expo; November 2, 2019; Philadelphia, PA.
    4.
    Peikes  D, Zutshi  A, Genevro  JL, Parchman  ML, Meyers  DS.  Early evaluations of the medical home: building on a promising start.  Am J Manag Care. 2012;18(2):105-116.PubMedGoogle Scholar
    5.
    Sinaiko  AD, Landrum  MB, Meyers  DJ,  et al.  Synthesis of research on patient-centered medical homes brings systematic differences into relief.  Health Aff (Millwood). 2017;36(3):500-508. doi:10.1377/hlthaff.2016.1235PubMedGoogle ScholarCrossref
    6.
    Friedberg  MW.  What do you mean by medical home?  Ann Intern Med. 2016;164(6):444-445. doi:10.7326/M16-0058PubMedGoogle ScholarCrossref
    7.
    Jackson  GL, Williams  JW  Jr.  Does PCMH “work”?—the need to use implementation science to make sense of conflicting results.  JAMA Intern Med. 2015;175(8):1369-1370. doi:10.1001/jamainternmed.2015.2067PubMedGoogle ScholarCrossref
    8.
    Vest  JR, Bolin  JN, Miller  TR, Gamm  LD, Siegrist  TE, Martinez  LE.  Medical homes: “where you stand on definitions depends on where you sit.”  Med Care Res Rev. 2010;67(4):393-411. doi:10.1177/1077558710367794PubMedGoogle ScholarCrossref
    9.
    Nelson  KM, Helfrich  C, Sun  H,  et al.  Implementation of the patient-centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use.  JAMA Intern Med. 2014;174(8):1350-1358. doi:10.1001/jamainternmed.2014.2488PubMedGoogle ScholarCrossref
    10.
    Tuepker  A, Kansagara  D, Skaperdas  E,  et al.  “We’ve not gotten even close to what we want to do”: a qualitative study of early patient-centered medical home implementation.  J Gen Intern Med. 2014;29(2)(suppl 2):S614-S622. doi:10.1007/s11606-013-2690-zPubMedGoogle ScholarCrossref
    11.
    Taylor  HA, Greene  BR, Filerman  GL.  A conceptual model for transformational clinical leadership within primary care group practice.  J Ambul Care Manage. 2010;33(2):97-107. doi:10.1097/JAC.0b013e3181d91707PubMedGoogle ScholarCrossref
    12.
    Fihn  SD, Francis  J, Clancy  C,  et al.  Insights from advanced analytics at the Veterans Health Administration.  Health Aff (Millwood). 2014;33(7):1203-1211. doi:10.1377/hlthaff.2014.0054PubMedGoogle ScholarCrossref
    13.
    Liu  C-F, Chapko  M, Bryson  CL,  et al.  Use of outpatient care in Veterans Health Administration and Medicare among veterans receiving primary care in community-based and hospital outpatient clinics.  Health Serv Res. 2010;45(5 Pt 1):1268-1286. doi:10.1111/j.1475-6773.2010.01123.xPubMedGoogle ScholarCrossref
    14.
    Burgess  JF  Jr, Maciejewski  ML, Bryson  CL,  et al.  Importance of health system context for evaluating utilization patterns across systems.  Health Econ. 2011;20(2):239-251. doi:10.1002/hec.1588PubMedGoogle ScholarCrossref
    15.
    World Health Organization.  International Classification of Diseases, Ninth Revision (ICD-9). Geneva, Switzerland: World Health Organization; 1977.
    16.
    Agency for Healthcare Research and Quality.  Prevention quality indicators: technical specifications. https://www.qualityindicators.ahrq.gov/Modules/pqi_resources.aspx. Accessed October 10, 2017.
    17.
    Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.  J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004PubMedGoogle ScholarCrossref
    18.
    Hebert  PL, Batten  AS, Gunnink  E,  et al.  Reliance on Medicare providers by veterans after becoming age-eligible for Medicare is associated with the use of more outpatient services.  Health Serv Res. 2018;53(3)(suppl 3):5159-5180. doi:10.1111/1475-6773.13033PubMedGoogle ScholarCrossref
    19.
    Liu  CF, Manning  WG, Burgess  JF  Jr,  et al.  Reliance on Veterans Affairs outpatient care by Medicare-eligible veterans.  Med Care. 2011;49(10):911-917. doi:10.1097/MLR.0b013e31822396c5PubMedGoogle ScholarCrossref
    20.
    Wong  ES, Maciejewski  ML, Hebert  PL, Reddy  A, Liu  CF.  Predicting primary care use among patients in a large integrated health system: the role of patient experience measures.  Med Care. 2019;57(8):608-614. doi:10.1097/MLR.0000000000001155PubMedGoogle ScholarCrossref
    21.
    Reddy  A, Wong  E, Canamucio  A,  et al.  Association between continuity and team-based care and health care utilization: an observational study of Medicare-eligible veterans in VA Patient Aligned Care Team.  Health Serv Res. 2018;53(3)(suppl 3):5201-5218. doi:10.1111/1475-6773.13042PubMedGoogle ScholarCrossref
    22.
    David  G, Gunnarsson  C, Saynisch  PA, Chawla  R, Nigam  S.  Do patient-centered medical homes reduce emergency department visits?  Health Serv Res. 2015;50(2):418-439. doi:10.1111/1475-6773.12218PubMedGoogle ScholarCrossref
    23.
    Peikes  D, Chen  A, Schore  J, Brown  R.  Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials.  JAMA. 2009;301(6):603-618. doi:10.1001/jama.2009.126PubMedGoogle ScholarCrossref
    24.
    Huntley  AL, Thomas  R, Mann  M,  et al.  Is case management effective in reducing the risk of unplanned hospital admissions for older people: a systematic review and meta-analysis.  Fam Pract. 2013;30(3):266-275. doi:10.1093/fampra/cms081PubMedGoogle ScholarCrossref
    25.
    Nelson  K, Sylling  PW, Taylor  L, Rose  D, Mori  A, Fihn  SD.  Clinical quality and the patient-centered medical home.  JAMA Intern Med. 2017;177(7):1042-1044. doi:10.1001/jamainternmed.2017.0963PubMedGoogle ScholarCrossref
    26.
    Martsolf  GR, Kandrack  R, Baird  M, Friedberg  MW.  Estimating associations between medical home adoption, utilization, and quality: a comparison of evaluation approaches.  Med Care. 2018;56(1):25-30. doi:10.1097/MLR.0000000000000842PubMedGoogle ScholarCrossref
    ×