eFigure 1. Test of Common Support for Propensity Score Model
eTable 1. Propensity Score Model Coefficients
eTable 2. Full Difference in Difference Regression Output Tables
eTable 3. Difference in Difference Regression Output for Medicaid Enrollees Only
eFigure 2. Parallel Trends
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Meyers DJ, Chien AT, Nguyen KH, Li Z, Singer SJ, Rosenthal MB. Association of Team-Based Primary Care With Health Care Utilization and Costs Among Chronically Ill Patients. JAMA Intern Med. 2019;179(1):54–61. doi:10.1001/jamainternmed.2018.5118
What is the association of a team-based primary care transformation collaborative initiative with patient health care utilization and costs?
In this study, among chronically ill patients in 18 practices who were exposed to team-based care, there was an 18% reduction in hospitalizations, a 25% reduction in emergency department visits, and a 36% reduction in ambulatory care–sensitive emergency department visits relative to 76 comparison practices. Among healthier patients, there was an increase in outpatient visits and hospitalizations.
Team-based approaches to primary care transformation may benefit patients with chronic illness by reducing the use of acute care; however, it may lead to higher use among healthier patients.
Empirical study findings to date are mixed on the association between team-based primary care initiatives and health care use and costs for Medicaid and commercially insured patients, especially those with multiple chronic conditions.
To evaluate the association of establishing team-based primary care with patient health care use and costs.
Design, Setting, and Participants
We used difference-in-differences to compare preutilization and postutilization rates between intervention and comparison practices with inverse probability weighting to balance observable differences. We fit a linear model using generalized estimating equations to adjust for clustering at 18 academically affiliated primary care practices in the Boston, Massachusetts, area between 2011 and 2015. The study included 83 953 patients accounting for 138 113 patient-years across 18 intervention practices and 238 455 patients accounting for 401 573 patient-years across 76 comparison practices. Data were analyzed between April and August 2018.
Practices participated in a 4-year learning collaborative that created and supported team-based primary care.
Main Outcomes and Measures
Outpatient visits, hospitalizations, emergency department visits, ambulatory care–sensitive hospitalizations, ambulatory care–sensitive emergency department visits, and total costs of care.
Of 322 408 participants, 176 259 (54.7%) were female; 64 030 (19.9%) were younger than 18 years and 258 378 (80.1%) were age 19 to 64 years. Intervention practices had fewer participants, with 2 or more chronic conditions (n = 51 155 [37.0%] vs n = 186 954 [46.6%]), more participants younger than 18 years (n = 337 931 [27.5%] vs n = 74 691 [18.6%]), higher Medicaid enrollment (n = 39 541 [28.6%] vs n = 81 417 [20.3%]), and similar sex distributions (75 023 women [54.4%] vs 220 097 women [54.8%]); however, after inverse probability weighting, observable patient characteristics were well balanced. Intervention practices had higher utilization in the preperiod. Patients in intervention practices experienced a 7.4% increase in annual outpatient visits relative to baseline (95% CI, 3.5%-11.3%; P < .001) after adjusting for patient age, sex, comorbidity, zip code level sociodemographic characteristics, clinician characteristics, and plan fixed effects. In a subsample of patients with 2 or more chronic conditions, there was a statistically significant 18.6% reduction in hospitalizations (95% CI, 1.5%-33.0%; P = .03), 25.2% reduction in emergency department visits (95% CI, 6.6%-44.0%; P = .007), and a 36.7% reduction in ambulatory care–sensitive emergency department visits (95% CI, 9.2%-64.0%; P = .009). Among patients with less than 2 comorbidities, there was an increase in outpatient visits (9.2%; 95% CI, 5.10%-13.10%; P < .001), hospitalizations (36.2%; 95% CI, 12.2-566.6; P = .003), and ambulatory care–sensitive hospitalizations (50.6%; 95% CI, 7.1%-329.2%; P = .02).
Conclusions and Relevance
While establishing team-based care was not associated with differences in the full patient sample, there were substantial reductions in utilization among a subset of chronically ill patients. Team-based care practice transformation in primary care settings may be a valuable tool in improving the care of sicker patients, thereby reducing avoidable use; however, it may lead to greater use among healthier patients.
There is increasing recognition that creating health care teams is critical to improving health care quality and value. Teaming refers to the dynamic activities, including coordination and collaboration, that allow individuals to work together to deliver shared goals.1 Some elements of highly functioning primary care teams and teaming have been described in isolated parts of the American health care system, yet team-based care is not the norm.2,3
Past work has found teaming can be effective at improving clinical care and outcomes.4 Introducing teams in hospital settings has reduced mortality and length of stay,4 and care delivered by geriatric teams improved elderly patients’ functional status, mental health, and independence compared with control groups.4-6 Teams based in primary initiatives have found modest gains, although findings are mixed.7-10 Gaps in the literature remain. More work is needed to understand how academic medical practices’ use of team-based care, how team-based care affects safety net practices, and which patients benefit most from these interventions.
In 2012, Harvard Medical School launched the Academic Innovations Collaborative (AIC), a multiyear, multisite care learning collaborative aimed at establishing team-based care at its affiliated primary care practices.11 We examine the association of this intervention with health care utilization and costs. We make several important contributions to the literature. First, we used an All Payer Claims Database (APCD) that provided us with all commercial and Medicaid claims for the practices in our study, granting us a more complete view of patient care than single-payer studies. Second, we used a difference-in-difference approach with inverse probability weighting to isolate the effects of the intervention on patient utilization and costs. Third, our study includes intervention practices across 6 different academic medical centers, whereas past work has typically focused on changes within just 1 medical center.
This study was approved by the Harvard T.H. Chan School of Public Health institutional review board. A waiver of patient consent was granted because the study used administrative data. The AIC initiative was inspired by the need to control increasing health care costs through more efficient care delivery and aimed to improve care through team-based management of chronic illness.11 Before the AIC, practice-level care delivery was largely decentralized because most of the included practices had not established patient care teams, and participation in multisite collaborative initiatives was limited.
The AIC intervention involved substantial reorganizing of existing clinicians into care teams, empaneling patients to primary care clinicians, and introducing a series of activities related to teamwork. Patients who sought care at the affiliated practices were assigned to a care team, and the care teams were encouraged to take ownership of all aspects of the patient’s care. Practices affiliated with Harvard Medical School could choose to opt into the intervention.
Required team activities included daily 15-minute “huddles” and implementation of population management systems (eg, systematic identification and follow-up with patients who required screenings). Team members were required to attend thrice-yearly 1.5-day learning sessions and regular interactive webinars to connect with other practices and share strategies for improving clinical quality through team-based care. To support these efforts, practices received an unrestricted $3 per member per month payment to support the transformation process during the first 2 years, and a $0.50 to $1 payment for the latter 2 years of the initiative. These funds could be used by the practice for any purpose and did not need to be directly linked to the AIC intervention.
To date, examinations of the AIC have demonstrated that transitions to team-based care involved changing practice configurations (ie, who worked with whom) and composition (ie, ratios of certain types of personnel to physicians) more than it changed the overall size of practices (ie, total number of staff within practices).12-17 They have also shown that better team dynamics (eg, team members’ ability to understand each other’s roles) were associated with greater satisfaction with clinical work among primary care clinicians and with more positive perceptions regarding patient safety among all staff. Qualitative studies have illustrated how the establishment of primary care teams can provide important scaffolding for residents when they feel stressed and inadequate during continuity clinic and with greater job satisfaction among medical assistants despite a higher workload.12-17
Our primary data source was the Massachusetts APCD from 2011 to 2015.18 The APCD contains monthly plan enrollment and 100% medical claims on all residents of Massachusetts, collected across all commercial payers and Medicaid. These files underwent substance abuse redaction19 but predate the March 2016 Supreme Court decision that allowed self-insured plans to opt out of submitting claims to APCDs.20 Our sample includes almost all medical claims data available for patients in Massachusetts younger than 65 years.
We linked National Provider Identification (NPI) numbers available in APCD claims to the Massachusetts Health Quality Partners Provider Database file21 to identify claims associated with AIC patients. Using the Massachusetts Health Quality Partners Provider Database, we identified all 18 AIC practices and 76 other academically affiliated primary care practices in the greater Boston area as a comparison group. Clinicians can practice in multiple locations, so we included any NPI affiliated with an AIC practice regardless of their other affiliations. To ensure that we captured a complete list of clinicians at each AIC practice, we used lists provided by each practice to the study team of their affiliated clinicians and looked up individual NPIs if they were not included in the Massachusetts Health Quality Partners Provider Database.
We have limited data on patient characteristics, so we included additional zip code level sociodemographic characteristics from the American Community Survey. Additionally, we included details on clinician credentials, sex, and specialty from the national NPI registry.
To attribute patients to practices, we limited our sample to enrollees with at least 6 months of enrollment within a year. We also excluded those older than 65 years and those dually eligible for Medicare and Medicaid because Medicare claims were not available for our analysis. For each year, we identified as a patient’s primary clinican the clinician NPI with which patients had the plurality of their evaluation and management visits that year. Patients could be reattributed each year if they appeared to change clinician or practice based on evaluation and management visit patterns. If a patient did not have any evaluation and management visits within a calendar year, then they were not attributed and therefore they were not included in that year for analysis.
For each patient, we collected from the APCD their total months of enrollment each year, their plan identification, their plan type (eg, health maitenance organization, preferred provider organization, or point of service) their age group (≤18 years and 19-64 years; the APCD did not include detailed age information) and their sex. We also calculated an Elixhauser Comorbidity index for each patient each year, and classified those with 2 or more chronic conditions as our chronically ill sample.22
To create an adequate comparison group, we first included all academically affiliated primary care practices in the greater Boston area. We retained practices with 3 to 100 affiliated physicians because this was the range in size for the AIC practices. We attributed patients to the 76 practices that met this criteria using the same approach as for AIC practices.
Because there were some observable differences in patient characteristics between these practices, we estimated propensity scores for assignment to an AIC practice. We used a logit model with a binary outcome of AIC assignment and covariates for sex, the patient’s age group, Medicaid status, type of plan patient was enrolled in (eg, health maintenance organization, preferred provider organization, or point of service), months of enrollment in their primary plan that year, Elixhauser comorbidity index, and zip code level educational attainment rates, race/ethnicity percentages, and poverty rates. Using the propensity score, we calculated inverse probability of treatment weights (IPTW) to balance selection on observable characteristics.23 We formally assessed covariate balance using the propensity scores to assess their performance and visually assess the assumption of common support.
Our primary outcomes of interest were patient-level utilization rates and costs following a framework used to evaluate similar interventions.24 For each patient, we calculated their annual outpatient, hospitalization, and emergency department visit rate. We also calculated their ambulatory care–sensitive emergency department utilization rate based on the New York University algorithm,25 and their ambulatory care–sensitive hospitalization rate based on the Agency for Healthcare Research and Quality algorithm.26 We were particularly interested in the ambulatory care–sensitive outcomes because these are the visits that may have been most preventable through better planned and coordinated primary care.
In addition to utilization rates, we calculated the total costs of care for each patient annually excluding drug costs. We could not include drug costs because pharmacy benefit managers were not required to report data to the APCD. All other medical costs were calculated as the total allowed amounts (ie, the copayment plus the health plan paid amount) for all of a patient’s medical services within a year, regardless of where they received those services. We fit separate cost models for Medicaid and non-Medicaid enrollees because the allowed amounts may differ between public and private payers.
To estimate the effects of the AIC intervention on outcomes, we used a standard difference-in-difference framework.27 The difference-in-difference approach allows for the calculation of the mean treatment effect of an intervention by comparing the changes in the preperiod and postperiod times between the intervention and comparison groups. For our analysis, we considered the preperiod to be 2011 and 2012, and the postperiod 2014 and 2015. We excluded data from 2013 in our analysis as that was the first year of the implementation, and for many practices, the focus was more on planning and self-assessment than adoption of new approaches to team-based care. We visually and formally checked the assumption of parallel trends in the preperiod for valid difference-in-difference inference.
To account for clustering at the practice level within the analysis, we fit a linear model using generalized estimating equations.28,29 Each model included an indicator of AIC practice, an indicator of being in the postperiod, and their interaction term that provided the difference-in-difference coefficient. We include additional covariates for sex, months of enrollment, Medicaid enrollment, a linear time trend, patient’s Elixhauser index, zip code sociodemographic characteristics, clinician credential (MD/NP/RN), specialty, and sex. We included plan fixed effects to adjust for potential differences between insurance plan benefits and weight the observations using the IPTWs.
We fit additional models stratified by at least 2 chronic conditions and less than 2 chronic conditions to test whether the intervention had differential associations among those more chronically ill. Because there was a 14% increase in patient share of Medicaid enrollees among the AIC practices relative to comparison practices, we separated the cost outcomes by payer source. We fit additional sensitivity analyses both with and without the plan fixed effects and IPTWs and among Medicaid enrollees only. All analysis was conducted in SAS, version 9.4 (SAS Institute Inc). P values less than .05 were considered statistically significant, and all P values were 2-sided.
Of all residents in the state who met the eligibility criteria, 35.4% were able to be attributed to any primary care clinician (n = 1 519 652). Our final sample included 238 455 unique patients contributing 401 573 person-years between 2011/2012 and 2014/2015 in comparison practices and 83 953 patients accounting for 138 113 patient-years in AIC practices. Descriptive characteristics of the sample are presented in Table 1, both with and without IPTW weighting. Prior to weighting, there were significant differences between the 2 practice types in all characteristics. After weighting, the differences between intervention and comparison practices were largely mitigated; however, some differences in covariates still remained, notably in percentage of Medicaid patients and months of enrollment; however, the absolute differences were small. The propensity scores met the common support assumption which is displayed visually in eFigure 1 in the Supplement. The full propensity score model is also available in eTable 1 in the Supplement.
Across all types of utilization, the AIC practices had higher rates in the preperiod than the comparison practices. In eFigure 2 in the Supplement, we present figures and empirical tests of the parallel trends assumption and find that parallel trends exist in all variables except for total cost of care and outpatient visits in the more than 2 comorbidity sample. Parallel trends were not met in any outcome in the less than 2 comorbidity subsample. In Table 2, we present the difference-in-difference models for the full sample of patients and the chronic condition strata. Results are presented per 1000 person-years. Full model output is available in eTable 2 in the Supplement.
In the full sample, there was a statistically significant 7.4% increase in outpatient visits (95% CI, 3.5%-11.3%; P < .001) for patients in intervention practices compared with those in comparisons. There were no other significant differences between AIC and non-AIC practices. In the chronically ill subsample, there was an 18.6% reduction in hospitalizations (95% CI, 2.3%-35.6%; P = .03), a 25.2% reduction in emergency department visits (95% CI, 6.4%-47.6%; P = .007), and a 36.7% reduction in ambulatory care–sensitive emergency department visits (95% CI, 17.9%-64.1%; P = .009). While the difference was not statistically significant, the total cost of care among the chronically ill subsample trended downwards.
Among patients with less than 2 comorbidities, we detected statistically significant increases in outpatient visits (9.2%; 95% CI, 5.1%-13.1%; P < .001), hospitalizations (36.2%; 95% CI, 12.2%-566.6%; P < .001) and ambulatory care–sensitive hospitalizations (50.6%; 95% CI, 7.1%-329.2%; P = .02). The increases in hospitalization were primarily driven by a larger reduction among comparison practices rather than an actual increase in intervention practices. In sensitivity analyses, we found similar trends only among Medicaid enrollees (eTable 3 in the Supplement), both in the full sample and in the 2 stratified samples.
In the chronically ill strata, we found clinically relevant reductions in hospitalizations and emergency department visits 2 years after a collaborative primary care transformation effort was initiated, and in a healthier strata, we found statistically significant increases in outpatient and hospital utilization. We are aware of 1 study that found similar results from a team-based care initiative in the Intermountain Health System.9 We believe our study makes important additional contributions because we found greater effect sizes for a lower per-member-per-month payment. The APCD data we used also allowed us a more complete view on utilization that may have taken place outside of the health systems in which our intervention practices are located.
What may have driven these associations? The increase in outpatient visits for the full sample may be a result of improved care planning and treatment in the AIC practices that could have encouraged patients to make more frequent primary care appointments. One of the earliest components of the transition to team-based care was patient empanelment, which, combined with efforts to target improvement in evidence-based care processes (eg, cancer screening and chronic illness management) may have increased planning and outreach for preventive care. These additional visits and other non–visit-based interactions that team members may initiate (eg, email or telephone consultations) offer increased opportunities to anticipate and prevent exacerbations of illness, and while they may initially cost more, there is a chance they would be beneficial for patients in the long run.
Our analysis also found statistically significant increases in outpatient use and hospitalizations among patients with less than 2 comorbidities. Hospitalization rates did decrease among intervention practices; however, they decreased at a faster rate among comparison practices, leading to the large difference in difference. However, it is difficult to interpret these results because the parallel trends assumption was not met in these models, indicating that the comparison practices may not have provided an adequate comparison for healthier patients. Nevertheless, the AIC intervention may have resulted in higher utilization. We cannot determine whether the increase in hospitalizations among this group are the result of an increase in patients getting needed care as a result of care coordination.
Our study is subject to several limitations. First, our primary source of data were administrative claims, which do not include as detailed clinical information as may be found in electronic health records. However, utilization and cost outcomes are generally well captured by insurance claims. Second, our attribution of patients to physicians and physicians to practices was imperfect. While we assigned patients according to common practice in the literature, it is possible that some patients in the final sample could have been more accurately assigned to a different practice or simply did not have enough exposure to any practice for valid assignment. Third, the APCD is limited in the number of patient characteristics available. While the IPTW weighting appears to have adequately balanced our observable covariates, there may be other differences between AIC and comparison patients that may have led to residual confounding. Fourth, the time of the study coincides with large changes to the health care system under the Affordable Care Act. While the difference-in-difference framework is robust to systemic changes as long as they do not affect the intervention and comparison groups differentially, we cannot rule out other changes that may have happened differentially at AIC practices during this time. Relatedly, we observe that AIC practices have higher utilization across most utilization types in the preperiod than the comparison practices. This may reflect other inherent differences in the types of practices included in the intervention. Fifth, our data from the Massachusetts APCD did not include claims for Medicare enrollees, limiting our analysis (and its generalizability) to a younger patient population. Finally, our results only reflect changes in utilization and costs and not the quality of patient care outcomes. More research is needed to determine the extent to which the intervention led to changes in patient health outcomes.
This study reflects changes implemented by a single network of academically affiliated practices. We observed large reductions in use for chronically ill patients and increases in utilization among healthier patients. While the same effects may not generalize to all other locations, the magnitude of the effects for chronically ill patients suggest that collaborative primary care transformation initiatives may be worthwhile to address avoidable utilization of inpatient and emergency services. However, these improvements may come at the cost of increases in some utilization that may or may not be necessary for healthier patients. Policymakers and health system leaders may do well to consider similar team-based care approaches in their health systems as part of a portfolio of efforts to bend the cost curve.
In conclusion, we found that a collaborative primary care transformation initiative conducted across 6 academic medical centers was associated with substantial reductions in emergency department and hospital utilization among a sample of chronically ill patients and increases in hospitalizations and outpatient utilization among healthier patients. Similar care transformation initiatives may be valuable for managing primary care in other settings.
Corresponding Author: David J. Meyers, MPH, Department of Health Services, Policy, and Practice, Brown University School of Public Health, 121 S Main St, Box G-S121-3, Providence, RI 02912 (firstname.lastname@example.org).
Accepted for Publication: August 7, 2018.
Published Online: November 26, 2018. doi:10.1001/jamainternmed.2018.5118
Author Contributions: Dr Meyers had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Meyers, Nguyen, Li, Singer, Rosenthal.
Drafting of the manuscript: Meyers, Chien.
Critical revision of the manuscript for important intellectual content: Nguyen, Li, Singer, Rosenthal.
Statistical analysis: Meyers, Li, Rosenthal.
Obtained funding: Chien, Singer, Rosenthal.
Administrative, technical, or material support: Meyers, Chien, Nguyen.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study is supported by funding from the Harvard Medical School Center for Primary Care and the Controlled Risk Insurance Company Risk Management Foundation of the Harvard Medical Institutions. Dr Meyers was additionally supported by an National Institute of Aging T32 training fellowship, and Mr Nguyen was additionally supported by an Agency for Healthcare Research and Quality T32 training fellowship. Additional support for data analysis was provided by Rappaport Institute for Greater Boston.
Role of the Funder/Support: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: The authors acknowledge the following individuals who designed and implemented the AIC intervention: Jenny Azzara, Asaf Bitton, Juliana DiLuca, Andrew Ellner, Kristen Goodell, Russ Phillips, Gordy Schiff, Soma Stout, Jessica Zeidman, Rick Lopez, Kim Ariyabuddhiphongs, Louise Mackisack, Holly Oh, Sam Skura, Joe Froklis, Lori Tishler, David Bor, Rob Chamberlin, Fiona McCaughan, Assaad Sayah, Kathleen Conroy, Joanne Cox, Matt Carmody, Linda Powers, Valarie Stone, Blair Fosburgh, Peter Greenspan, Eric Weil, Carol Keohane, Luke Sato, Trudy Bearden, Jonathan Sugarman, Cory Sevin, and Rebecca Steinfield.
Create a personal account or sign in to: