Effect of a Multipayer Patient-Centered Medical Home on Health Care Utilization and Quality: The Rhode Island Chronic Care Sustainability Initiative Pilot Program

Importance  The patient-centered medical home is advocated to reduce health care costs and improve the quality of care.

Objective  To evaluate the effects of the pilot program of a multipayer patient-centered medical home on health care utilization and quality.

Design  An interrupted time series design with propensity score–matched comparison practices, including multipayer claims data from 2 years before (October 1, 2006–September 30, 2008) and 2 years after (October 1, 2008–September 30, 2010) the launch of the pilot program. Uptake of the intervention was measured with audit data from the National Committee for Quality Assurance patient-centered medical home recognition process.

Setting  Five independent primary care practices and 3 private insurers in the Rhode Island Chronic Care Sustainability Initiative.

Participants  Patients in 5 pilot and 34 comparison practices.

Interventions  Financial support, care managers, and technical assistance for quality improvement and practice transformation.

Main Outcomes and Measures  Hospital admissions, emergency department visits, and 6 process measures of quality of care (3 for diabetes mellitus and 3 for colon, breast, and cervical cancer screening).

Results  The mean National Committee for Quality Assurance recognition scores of the pilot practices increased from 42 to 90 points of a possible 100 points. The pilot and comparison practices had statistically indistinguishable baseline patient characteristics and practice patterns, except for higher numbers of attributed member months per year in the pilot practices (31 130 per practice vs 14 779, P = .01) and lower rates of cervical cancer screening in the comparison practices. Although estimates of the emergency department visits and inpatient admissions of patients in the pilot practices trended toward lower utilization, the only significant difference was a lower rate of ambulatory care sensitive emergency department visits in the pilot practices. The Chronic Care Sustainability Initiative pilot program was associated with a reduction in ambulatory care–sensitive emergency department visits of approximately 0.8 per 1000 member months or approximately 11.6% compared with the baseline rate of 6.9 for emergency department visits per 1000 member months (P = .002). No significant improvements were found in any of the quality measures.

Conclusion and Relevance  After 2 years, a pilot program of a patient-centered medical home was associated with substantial improvements in medical home recognition scores and a significant reduction in ambulatory care sensitive emergency department visits. Although not achieving significance, there were downward trends in emergency department visits and inpatient admissions.

Studies^1,2 suggest that better access to primary care is associated with stronger health system performance. The patient-centered medical home is a model of care provision that is proactive and coordinated and that includes financial support and accountability for a defined population of patients. The goals are to improve quality and to reduce costs.³ Medicare, numerous state Medicaid agencies, and leading commercial insurers have launched dozens of patient-centered medical home initiatives.^4,5

Despite the promise of this reform, 2 recent systematic reviews^6,7 concluded that the effect of patient-centered medical homes was mixed. A study⁸ of Geisinger Health System’s patient-centered medical home pilot program found an 18% reduction in hospital admissions and a 36% reduction in readmissions after 2 years. Likewise, Group Health Cooperative’s pilot program⁹ had 29% fewer emergency visits and 6% fewer hospitalizations compared with other Group Health clinics, with an estimated return on investment for the medical home after 2 years of $1.50 for every dollar invested.

The findings from Geisinger and the Group Health Cooperative may not generalize to less integrated settings. An evaluation of the National Patient-Centered Medical Home Demonstration Project, which included small independent practices but not changes in payers or payments, found modest quality of care improvements after 2 years but no evidence of improvements in patient-reported outcomes.¹⁰ Two more recent studies^11,12 that examined the effect on utilization and quality of the patient-centered medical home in small practices found little to no improvement in utilization and cost and only modest quality improvements.

We report findings from the pilot study of the Rhode Island multipayer patient-centered medical home, one of the earliest efforts to transform small, independent primary care practices with financial support from multiple payers. Our evaluation focused on whether better organized, more proactive primary care reduced the use of hospital and emergency department services and improved the quality of care.

We evaluated the Rhode Island Chronic Care Sustainability Initiative (CSI). A detailed description of the intervention’s major elements is included in eAppendix 1 in the Supplement. Rhode Island’s Office of the Health Insurance Commissioner launched the initiative in 2008 with support from the 3 largest commercial insurers in the state: Blue Cross Blue Shield of Rhode Island, Neighborhood Health Plan, and United Health Care. These insurers cover approximately two-thirds of the patients seen by participating practices. Five primary care practices, including 1 federally qualified health center, with a total between them of 45 primary care providers, volunteered to participate in the pilot program.

Participating practices were required to obtain patient-centered medical home recognition from the National Committee for Quality Assurance (NCQA), track and share with each other and participating payers a set of standard clinical quality indicators, and have ongoing quality improvement activities. The NCQA recognition required that practices establish and document structural capabilities and protocols in 9 areas; the areas are listed in Table 1.¹³ Initially, the pilot practices focused their clinical quality measurement and improvement efforts on evidence-based screening and management of diabetes mellitus, coronary artery disease, and depression.

Participating health plans committed approximately $2 million in support for 2 years. The plans paid the practices approximately $3 per patient per month, recognizing the resources required both to transform the organization and focus practices’ efforts on nonbillable services, such as patient outreach and care planning outside office visits. Monthly fees were paid for patients in health maintenance organization plans that had selected or been assigned to the practice and for patients in preferred provider organization plans who received the plurality of their primary care in the practice. To receive these payments, practices needed to obtain NCQA Physician Practice Connections–Patient Centered Medical Home level I recognition by achieving a total score of 25 to 49 points of 100 points (see eAppendix 1 in the Supplement for measures and point distribution), including at least 50% achievement on 5 of 10 “must pass” elements. Practices were otherwise paid according to their preexisting contracts (primarily fee for service, with some small incentives for meeting quality measure thresholds). The payers also collectively funded 5 nurse care managers employed by each practice for the pilot program’s duration. Care managers led NCQA recognition, performed quality measurement and improvement activities, coordinated care for high-risk patients, and worked with patient registries to identify and resolve gaps in recommended care.

To evaluate the effect of the CSI pilot program 2 years after initiation, we used an interrupted time series approach with a matched comparison group. The institutional review board at the Harvard School of Public Health approved the study. The preintervention period began October 1, 2006, and ended September 30, 2008; the postintervention period began October 1, 2008, and ended September 30, 2010. The primary data source for our analysis was administrative claims data from the 3 participating health plans. To describe the practice changes through which the intervention may have affected patient care, we also collected and analyzed detailed data on the patient-centered medical home structures and processes used by the NCQA’s recognition process. The 5 practices underwent the NCQA recognition at the start of the pilot program and again in 2010, allowing for analysis of performance changes. We also interviewed practice leaders at baseline, after 1 year, and at the end of the study period to help understand the changes that occurred.

Our primary hypotheses were that the patient-centered medical home intervention would reduce rates of hospital and emergency department admissions and improve the quality of care among patients receiving care from pilot practices. Because adherence to prescription drug use may play a major role in effectively managing chronic disease, we expected that better identification and management of chronically ill patients would increase their use of prescription drugs. Hospital admissions, emergency department visits, and primary care and specialist visits and prescriptions were measured using the NCQA standard definitions that are the basis for annual health plan reporting.¹⁴ We examined hospital admissions that are sensitive to ambulatory care and emergency department visits.¹⁵ To generate a visit measure for emergency department visits that are sensitive to ambulatory care, we used a classification system developed by researchers at New York University.¹⁶ We categorized emergency department visits as nonemergent, emergent/primary care treatable, emergent/emergency department care needed but preventable/avoidable, and emergent/emergency department care needed but not preventable/avoidable. We included all but the last category of visits in our measure of visits that are sensitive to ambulatory care.

We examined changes in the quality of care on the basis of an adaptation of NCQA's Health Effectiveness Data and Information Set (HEDIS) measures, with minor modifications to eligibility requirements and look-back periods as detailed below. Initially, the CSI practices prioritized diabetes care as a focus of their patient-centered medical home activity. Thus, we examined 3 measures of diabetes care quality: the percentage of diabetic patients with (1) a glycated hemoglobin test in the last year, (2) lipid testing in the last year, and (3) an eye examination in the last year. We studied 3 preventive care measures—colon, breast, and cervical cancer screening—for patients in the appropriate age and sex groups according to HEDIS specifications, but we modified the denominator and numerator time windows to fit a single year (even though screening is not recommended annually). Appropriate colon cancer screening was defined by 1 of the following criteria: (1) fecal occult blood test during the measurement year, (2) flexible sigmoidoscopy during the measurement year or the 4 prior years, or (3) colonoscopy during the measurement year or the 9 prior years. Appropriate breast cancer screening was defined by at least 1 mammogram during the measurement year or the prior year. Appropriate cervical cancer screening was defined as 1 or more Papanicolaou tests during the measurement year or the prior 2 years. We studied the utilization of primary care and specialist office visits to assess whether the intervention altered the pattern of ambulatory care.

To describe the patient-centered medical home features adopted by the practices at baseline and in the second year of the pilot program, we obtained detailed results of their NCQA recognition audit. The NCQA data quantified their level of attainment in each of 30 elements across the 9 domains.

We used the NCQA recognition audit data to summarize mean and median scores in each domain at baseline and at the end of year 2. We did not conduct statistical testing because we studied only 5 practices. Our claims data analysis recognized that the CSI was a practice-level intervention; therefore, it was incorporated matching at the level of the practice rather than the individual patient. We attributed patients to practices providing the plurality of primary care visits to that patient; ties among physicians were resolved by assigning patients to the physician seen most recently. We excluded the few patients older than 65 years in our data because of concerns about incomplete data to the extent that Medicare paid for their care. To be included in the analysis for a given year, patients were required to have a minimum of 6 months of insurance eligibility in the year.

We identified a contemporaneous group of comparison practices for the claims data analysis. We used propensity score matching based on data from the baseline year (October 1, 2006, through September 30, 2007). In the propensity score model, we included an indicator for patient comorbidities (the Elixhauser comorbidity index, a list of 30 comorbidities that can be identified in billing data and were originally used to predict mortality¹⁷) and rates of inpatient admissions, emergency department visits, and primary care visits per 1000 member months. Matching was based on a caliper width of 0.6 SD of the logit of the propensity score with a maximum of 10 matches attributed to a pilot practice.

To validate our propensity score–matched comparison cohort, we first examined differences in practice-level patient characteristics and utilization patterns using Wilcoxon-Mann-Whitney statistics. Our interrupted time series approach relies on the assumption that control practices’ performance is on a common trend with the pilot practices; hence, any postintervention differences in pilot practices’ performance relative to that preintervention trend and concurrent control practice performance are attributable to the pilot program. A fundamental assumption is that pilot and control practices’ performance would exhibit a similar trend absent the pilot program. Therefore, in addition to testing for differences in baseline levels of performance, we further tested for differences in the quarterly trends before the pilot program in our utilization measures.

To adjust for residual confounding after matching and to address the substantial number of patients who do not use services in a typical year, we estimated the effect of the CSI practice-level measures of utilization using zero-inflated negative binomial models. These 2-part negative binomial models separately estimate the probability that a patient would have zero utilization for the entire period from a model that estimates the number of events and are used where a large share of the population has an extremely low risk of an event. The dependent variables were constructed as counts or indicators of use per quarter, and the independent variables were patient age, sex, the Elixhauser comorbidity index, an indicator for whether the patient’s primary care physician was in the pilot program, the number of months of eligibility, a linear time trend to account for secular changes in utilization that would affect both the pilot and control practices, an indicator for the postintervention period, and an interaction between the pilot and postintervention practices.

All analyses included practice fixed effects and accounted for clustering at the practice level. Marginal effects and their associated P values were calculated using the Δ method. P<.05 was considered statistically significant.

Qualitative data gathered from interviews with practice leaders were recorded and transcribed. Multiple authors (M.B.R., S.J.S., and D.E.) coded interview transcripts using a thematic approach to identify recurring concepts and specific examples related to the concepts. We used qualitative data to aid the interpretation of quantitative analyses.

All 5 practices obtained level I patient-centered medical home recognition in their first submission. At the end of the pilot program, all 5 practices had reached level III status (attaining 75-100 points). Pilot practices made notable progress in patient self-management support, electronic prescribing, and the tracking of laboratory tests and results. For advanced electronic communication, which includes the ability to message patients through secure e-mail, scores were little changed. Table 1 lists the mean and range of points for the 5 practices on the NCQA standards at baseline and year 2. Practice performance on individual domains of the NCQA recognition program is reported in eAppendix 2 in the Supplement.

Our propensity score–matching algorithm identified 34 control group practices. Pilot practices were attributed patients with a mean of 31 130 member months per practice, and comparison practices were attributed a mean of 14 779 member months per practice. Comparisons of pilot and matched control practices’ patient populations on demographics, case mix, and baseline utilization patterns revealed no significant differences other than attributed member months and rates of cervical cancer screening (Table 2). Likewise, we found no differences in the prepilot quarterly trends in utilization measures (eAppendix 3 in the Supplement presents the quarterly trends in utilization for pilot and control practices).

Table 3 presents the regression-adjusted incidence rates and odds ratios from the utilization analyses. For each independent variable, the combined effect in terms of the rate per member month and percentage change relative to the pilot baseline is also reported. The CSI pilot study was associated with a reduction in ambulatory care sensitive emergency department visits of roughly 0.8 per 1000 member months or approximately 11.6% (P = .002) compared with the baseline rate of 6.9 for such emergency department visits per 1000 member months (Table 2). Despite downward trends, however, no significant reductions were found in overall emergency department visits, inpatient admissions, or ambulatory care sensitive inpatient admissions for pilot practices compared with control practices. No differential changes were found in primary care or specialist visits or in prescription drug use. The diabetes and cancer screening quality measures revealed no significant improvements, although there were positive trends for some diabetes quality measures (Table 4).

In 2008, Rhode Island launched one of the first multipayer medical home initiatives in the United States. In contrast to prior studies of patient-centered medical homes that involved large, integrated health care provision organizations, the Rhode Island CSI involved small, independent primary care groups. The participating groups made substantial progress in meeting the NCQA Physician Practice Connections–Patient Centered Medical Home criteria during a 2-year period, with notable gains in prospective population management and in tracking and coordination of care.

After 2 years, the CSI Rhode Island pilot program was associated with a significant reduction in ambulatory care sensitive emergency department visits. Although not achieving significance, there were downward trends in 3 other utilization measures (emergency department visits, inpatient admissions, and ambulatory care sensitive inpatient admissions). For claims-based diabetes quality measures, we also found no significant changes, despite positive trends for some measures.

Compared with other patient-centered medical home evaluations, the Rhode Island CSI was relatively successful in achieving practice transformation as measured by the NCQA criteria, perhaps because practices received sustained financial and staffing support. Consistent with previous reviews,^6,7 we did not identify utilization changes that would imply overall cost savings, but practices successfully reduced emergency department visits that are sensitive to ambulatory care. Although studies of patient-centered medical homes at Geisinger Health Systems and Group Health of Puget Sound found reductions in hospitalizations, both initiatives included large numbers of patients older than 65 years. Our analysis focused on commercially insured and Medicaid managed care patients for whom inpatient health care utilization was low at baseline.

In the short term, our findings suggest that patient-centered medical home initiatives will generate only modest cost savings for payers. However, on the basis of the reported experience of the National Patient-Centered Medical Home Demonstration Project and the interviews we conducted with pilot program participants, the transformation of practices may be far from complete at 2 years. During the first year, practices focused on NCQA recognition and changes in culture and patient care management functions. In the second year, most practices focused on clinical quality improvement. Our interviews suggest that only after 2 years did the practices build the foundation for the care coordination and high-risk care management efforts that may be necessary for broader and more substantial improvements in hospital and emergency department utilization and patient outcomes. Payers and policymakers in Rhode Island appear committed to sustaining and spreading the patient-centered medical home in some form.¹⁸

Our analysis has limitations. First, the CSI intervention, like many patient-centered medical home pilot programs, was introduced in a few practices. We had limited power to detect small but potentially important changes in utilization and quality. Accordingly, the 95% CIs around most of our estimates are relatively wide. We were only able to study the initiative for 2 years. With more time and data, we might have found larger effects. Practices volunteered for the pilot program and are unlikely to be representative of all practices in Rhode Island or the United States. We had access to billing data not clinical data. The CSI practices collected clinical data as part of their quality improvement activities. Although their own reports found substantial improvement over time, they lacked comparison groups and preintervention measurements.¹⁹ Our quality measures used a 1-year time window, which is less than the NCQA requires for HEDIS reporting. Because the recommended screening frequency is not annual, the effect of a 1-year time window is to reduce apparent performance scores. However, because we were evaluating differential changes over time in the pilot group, the short time window should not have affected our inferences about the effect of the CSI. Finally, we relied on a quasiexperimental approach to identify the pilot program’s effect. Our findings may be sensitive to untestable assumptions about the comparability of the intervention and control practices.

As patient-centered medical homes continue to be studied, some payers have expanded their commitments to this model. The Center for Medicare and Medicaid Innovation has launched the Multipayer Advanced Primary Care Practice Demonstration, which allows Medicare to join existing multipayer efforts such as CSI, and the Comprehensive Primary Care Initiative, which will offer more than 2000 practices that meet patient-centered medical home and meaningful use of health information technology criteria an average care coordination fee of $20 per beneficiary per month and the opportunity to receive shared savings.²⁰ Likewise, citing early results from pilot programs in Colorado, New Hampshire, and New York, WellPoint announced plans to increase its investment in patient-centered medical home–accredited practices by 2014.²¹ Of course, the expansion of payer investments in patient-centered medical home initiatives may be driven more by the desire to move toward contracting models that assign providers comprehensive accountability for the cost and quality of care than by expectations of short-run savings.

Accepted for Publication: June 30, 2013.

Corresponding Author: Meredith B. Rosenthal, PhD, Department of Health Policy and Management, Harvard School of Public Health, 677 Huntington Ave, Boston, MA 02115 (meredith_rosenthal@harvard.edu).

Published Online: September 9, 2013. doi:10.1001/jamainternmed.2013.10063.

Author Contributions:Study concept and design: Rosenthal, Li, Schneider.

Acquisition of data: Rosenthal, Li.

Analysis and interpretation of data: All authors.

Drafting of the manuscript: Rosenthal, Friedberg, Eastman, Schneider.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Rosenthal, Eastman, Li.

Obtained funding: Rosenthal, Schneider.

Administrative, technical, or material support: Eastman.

Study supervision: Rosenthal.

Qualitative analysis: Singer.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by The Commonwealth Fund, New York, New York.

Role of the Sponsor: The Commonwealth Fund had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.