[Skip to Content]
[Skip to Content Landing]
Figure 1.
Patient Eligibility for Intensive Management Patient Aligned Care Team (ImPACT) Intervention and Evaluation
Patient Eligibility for Intensive Management Patient Aligned Care Team (ImPACT) Intervention and Evaluation

aTop 5% based on total costs or 1-year risk of hospitalization.

bMultiple exclusion criteria may apply.

CAN indicates Care Assessment Need.

Figure 2.
Mean Unadjusted Monthly Costs per Person for Intensive Management PACT (ImPACT) and Patient Aligned Care Team (PACT) Patients During Baseline and Follow-up Periods
Mean Unadjusted Monthly Costs per Person for Intensive Management PACT (ImPACT) and Patient Aligned Care Team (PACT) Patients During Baseline and Follow-up Periods

This figure presents mean unadjusted monthly costs per person (including ImPACT encounter costs) during the baseline and follow-up periods, for the ImPACT and PACT patients who were alive and present as of February 1, 2013. If patients died or left the facility, their costs were set to 0 and missing, respectively, for subsequent months. ∆ImPACT − ∆PACT indicates the change in monthly costs for ImPACT patients minus the change in monthly costs for PACT patients (ie, a negative number corresponds to a larger decline in costs over the study period among patients in ImPACT, compared with patients in PACT).

Figure 3.
Estimated Program Effects on Monthly Person-Level Costs Among Patients in Intensive Management PACT (ImPACT) vs Patient Aligned Care Team (PACT)
Estimated Program Effects on Monthly Person-Level Costs Among Patients in Intensive Management PACT (ImPACT) vs Patient Aligned Care Team (PACT)

This figure presents adjusted difference-in-difference estimates for the ImPACT program's effects on monthly person-level costs. Estimates are presented for the intention-to-treat population, and for subgroups of patients with key sociodemographic and clinical characteristics at baseline. The difference-in-differences estimate corresponds to the change in monthly costs among patients in ImPACT, minus the change in costs for patients in PACT. A negative program effect corresponds to a larger decline in costs over the study period among patients in ImPACT, compared with patients in PACT. Changes in monthly costs were estimated using linear regression, controlling for patient fixed effects. If patients died or left the facility, their costs were set to 0 and missing, respectively, for subsequent months. The gray bars and error bars represent difference-in-difference estimates and standard errors, respectively, for ImPACT's effects on monthly person-level costs.

Table 1.  
Characteristics of Intensive Management PACT (ImPACT) vs Patient-Aligned Care Team (PACT) Patients
Characteristics of Intensive Management PACT (ImPACT) vs Patient-Aligned Care Team (PACT) Patients
Table 2.  
Mean Health Care Utilization During Baseline and Follow-up Periods for Intention-to-Treat Patients in Intensive Management PACT (ImPACT) vs Patient-Aligned Care Team (PACT)a
Mean Health Care Utilization During Baseline and Follow-up Periods for Intention-to-Treat Patients in Intensive Management PACT (ImPACT) vs Patient-Aligned Care Team (PACT)a
1.
Hasselman  D.  Super-Utilizer Summit: Common Themes from Innovative Complex Care Management Programs. Hamilton, NJ: Center for Health Care Strategies, Inc; 2013.
2.
Mann  C. Medicaid and CHIP: On the Road to Reform. Alliance for Health Reform/Kaiser Family Foundation; 2011. http://www.allhealth.org/briefingmaterials/KFFAlliance_FINAL-1971.ppt. Accessed November 28, 2012.
3.
Sommers  A, Cohen  M.  Medicaid’s High Cost Enrollees: How Much Do They Drive Program Spending? Washington, DC: Kaiser Commission for Medicaid and the Uninsured; 2006.
4.
Conwell  LJ, Cohen  JW. AHRQ Statistical Brief 73: Characteristics of persons with high medical expenditures in the US civilian noninstitutionalized population, 2002. Rockville, MD: Agency for Healthcare Research and Quality; 2005. https://meps.ahrq.gov/data_files/publications/st73/stat73.pdf. Accessed November 28, 2012.
5.
Zulman  DM, Pal Chee  C, Wagner  TH,  et al.  Multimorbidity and healthcare utilisation among high-cost patients in the US Veterans Affairs Health Care System.  BMJ Open. 2015;5(4):e007771.PubMedGoogle ScholarCrossref
6.
Congressional Budget Office. High-cost Medicare beneficiaries: a CBO paper. 2005. http://www.cbo.gov/sites/default/files/cbofiles/ftpdocs/63xx/doc6332/05-03-medispending.pdf. Accessed February 26, 2014.
7.
Blumenthal  D, Chernof  B, Fulmer  T, Lumpkin  J, Selberg  J.  Caring for high-need, high-cost patients—an urgent priority.  N Engl J Med. 2016;375(10):909-911.PubMedGoogle ScholarCrossref
8.
Yee  T, Lechner  A, Carrier  E. High-Intensity Primary Care: Lessons for Physician and Patient Engagement. National Institute for Health Care Reform; 2012. http://nihcr.org/analysis/improving-care-delivery/prevention-improving-health/high-intensity-primary-care/. Accessed November 22, 2016.
9.
Peterson  D, Helfand  M, Humphrey  L, Christensen  V, Carson  S. Evidence Brief: Effectiveness of Intensive Primary Care Programs, VA-ESP Project 09-199. Washington, DC: Department of Veterans Affairs; 2013. http://www.hsrd.research.va.gov/publications/esp/primary-care.cfm. Accessed November 22, 2016.
10.
California Healthcare Foundation, California Quality Collaborative. Complex Care Management Toolkit. 2012. http://www.calquality.org/storage/documents/cqc_complexcaremanagement_toolkit_final.pdf. Accessed February 26, 2014.
11.
Bodenheimer  T. Strategies to reduce costs and improve care for high-utilizing Medicaid patients: reflections on pioneering programs. Center for Health Care Strategies, Inc; 2013. http://www.chcs.org/resource/strategies-to-reduce-costs-and-improve-care-for-high-utilizing-medicaid-patients-reflections-on-pioneering-programs/. Accessed November 20, 2016.
12.
Gawande  A. The hot spotters: Can we lower medical costs by giving the neediest patients better care? The New Yorker. January 24, 2011. http://www.newyorker.com/magazine/2011/01/24/the-hot-spotters. Accessed November 22, 2016.
13.
Hong  CS, Siegel  AL, Ferris  TG. Caring for high-need, high-cost patients: what makes for a successful care management program? The Commonwealth Fund; August 2014. http://www.commonwealthfund.org/~/media/files/publications/issue-brief/2014/aug/1764_hong_caring_for_high_need_high_cost_patients_ccm_ib.pdf. Accessed November 22, 2016.
14.
McCarthy  D, Ryan  J, Klein  S. Models of care for high-need, high-cost patients: an evidence synthesis. October 2015. http://www.commonwealthfund.org/~/media/files/publications/issue-brief/2015/oct/1843_mccarthy_models_care_high_need_high_cost_patients_ib.pdf. Accessed November 22, 2016.
15.
Green  SR, Singh  V, O’Byrne  W.  Hope for New Jersey’s city hospitals: the Camden Initiative.  Perspect Health Inf Manag. 2010;7:1d.PubMedGoogle Scholar
16.
Reuben  DB.  Physicians in supporting roles in chronic disease care: the CareMore model.  J Am Geriatr Soc. 2011;59(1):158-160.PubMedGoogle ScholarCrossref
17.
Blash  L, Chapman  S, Dower  C. The Special Care Center—a joint venture to address chronic disease. Center for the Health Professions Research Brief. 2011. http://www.iorahealth.com/wp-content/uploads/2014/07/UCSF_The_Special_Care_Center_A_Joint_Venture_to_Address_Chronic_Disease.pdf. Accessed November 28, 2012.
18.
Milstein  A, Kothari  PP. Are higher-value care models replicable? Health Affairs Blog. 2009. http://healthaffairs.org/blog/2009/10/20/are-higher-value-care-models-replicable/. Accessed November 28, 2012.
19.
Boult  C, Reider  L, Leff  B,  et al.  The effect of guided care teams on the use of health services: results from a cluster-randomized controlled trial.  Arch Intern Med. 2011;171(5):460-466.PubMedGoogle ScholarCrossref
20.
Naylor  MD, Brooten  DA, Campbell  RL, Maislin  G, McCauley  KM, Schwartz  JS.  Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial.  J Am Geriatr Soc. 2004;52(5):675-684.PubMedGoogle ScholarCrossref
21.
Counsell  SR, Callahan  CM, Tu  W, Stump  TE, Arling  GW.  Cost analysis of the Geriatric Resources for Assessment and Care of Elders care management intervention.  J Am Geriatr Soc. 2009;57(8):1420-1426.PubMedGoogle ScholarCrossref
22.
McCall  N, Cromwell  J, Urato  C. Evaluation of Medicare Care Management for High Cost Beneficiaries (CMHCB) Demonstration: Massachusetts General Hospital and Massachusetts General Physicians Organization and Massachusetts General Physicians Organization (MGH), Final Report. Centers for Medicare & Medicaid Services; September 2010. https://www.cms.gov/reports/downloads/McCall_MGH_CMHCB_Final_2010.pdf. Accessed July 18, 2016.
23.
Shumway  M, Boccellari  A, O’Brien  K, Okin  RL.  Cost-effectiveness of clinical case management for ED frequent users: results of a randomized trial.  Am J Emerg Med. 2008;26(2):155-164.PubMedGoogle ScholarCrossref
24.
Bell  J, Mancuso  D, Krupski  T,  et al.  A Randomized Controlled Trial of King County Care Partners’ Rethinking Care Intervention: Health and Social Outcomes up to Two Years Post-randomization. Seattle, WA: Center Healthcare Improvement for Addictions Mental Illness and Medically Vulnerable Populations; 2012.
25.
Sledge  WH, Brown  KE, Levine  JM,  et al.  A randomized trial of primary intensive care to reduce hospital admissions in patients with high utilization of inpatient services.  Dis Manag. 2006;9(6):328-338.PubMedGoogle ScholarCrossref
26.
National Committee for Quality Assurance. NCQA Evidence Report: Benefits of the Patient-Centered Medical Homes. June 2015. http://www.ncqa.org/Portals/0/Programs/Recognition/PCMHEvidenceReportJune2015_Web.pdf?ver=2016-02-24-143948-347. Accessed July 20, 2016.
27.
Rosland  AM, Nelson  K, Sun  H,  et al.  The patient-centered medical home in the Veterans Health Administration.  Am J Manag Care. 2013;19(7):e263-e272.PubMedGoogle Scholar
28.
Raven  MC, Doran  KM, Kostrowski  S, Gillespie  CC, Elbel  BD.  An intervention to improve care and reduce costs for high-risk patients with frequent hospital admissions: a pilot study.  BMC Health Serv Res. 2011;11:270.PubMedGoogle ScholarCrossref
29.
Dorr  DA, Wilcox  AB, Brunker  CP, Burdon  RE, Donnelly  SM.  The effect of technology-supported, multidisease care management on the mortality and hospitalization of seniors.  J Am Geriatr Soc. 2008;56(12):2195-2202.PubMedGoogle ScholarCrossref
30.
Dorr  DA, Wilcox  A, Burns  L, Brunker  CP, Narus  SP, Clayton  PD.  Implementing a multidisease chronic care model in primary care using people and technology.  Dis Manag. 2006;9(1):1-15.PubMedGoogle ScholarCrossref
31.
Brown  KE, Levine  JM, Fiellin  DA, O’Connor  P, Sledge  WH.  Primary intensive care: pilot study of a primary care-based intervention for high-utilizing patients.  Dis Manag. 2005;8(3):169-177.PubMedGoogle ScholarCrossref
32.
Lessler  DS, Krupski  A, Cristofalo  M. King County Care Partners: a community-based chronic care management system for Medicaid clients with co-occurring medical, mental, and substance abuse disorders. In: Schraeder  C, Shelton  P, eds.  Comprehensive Care Coordination for Chronically III Adults. West Sussex, UK: John Wiley & Sons, Inc; 2011:339-348.
33.
Okin  RL, Boccellari  A, Azocar  F,  et al.  The effects of clinical case management on hospital service use among ED frequent users.  Am J Emerg Med. 2000;18(5):603-608.PubMedGoogle ScholarCrossref
34.
Brown  RS, Peikes  D, Peterson  G, Schore  J, Razafindrakoto  CM.  Six features of Medicare coordinated care demonstration programs that cut hospital admissions of high-risk patients.  Health Aff (Millwood). 2012;31(6):1156-1166.PubMedGoogle ScholarCrossref
35.
Zulman  DM, Ezeji-Okoye  SC, Shaw  JG,  et al.  Partnered research in healthcare delivery redesign for high-need, high-cost patients: development and feasibility of an Intensive Management Patient-Aligned Care Team (ImPACT).  J Gen Intern Med. 2014;29(4)(suppl 4):861-869.PubMedGoogle ScholarCrossref
36.
Breland  JY, Asch  SM, Slightam  C, Wong  A, Zulman  DM.  Key ingredients for implementing intensive outpatient programs within patient-centered medical homes: a literature review and qualitative analysis.  Healthc (Amst). 2016;4(1):22-29.PubMedGoogle ScholarCrossref
37.
Wang  L, Porter  B, Maynard  C,  et al.  Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration.  Med Care. 2013;51(4):368-373.PubMedGoogle ScholarCrossref
38.
Frayne  SM, Phibbs  CS, Saechao  F,  et al.  Sociodemographics, Utilization, Costs of Care, and Health Profile. Washington, DC: Department of Veterans Affairs, Women’s Health Evaluation Initiative; 2014.
39.
Peterson  R, Gundlapalli  AV, Metraux  S,  et al.  Identifying homelessness among veterans using VA administrative data: opportunities to expand detection criteria.  PLoS One. 2015;10(7):e0132664.PubMedGoogle ScholarCrossref
40.
Marshall  GN, Hays  RD.  The Patient Satisfaction Questionnaire Short-Form (PSQ-18). Santa Monica, CA: RAND; 1994.
41.
Hibbard  JH, Mahoney  ER, Stockard  J, Tusler  M.  Development and testing of a short form of the patient activation measure.  Health Serv Res. 2005;40(6, pt 1):1918-1930.PubMedGoogle ScholarCrossref
42.
Dimick  JB, Ryan  AM.  Methods for evaluating changes in health care policy: the difference-in-differences approach.  JAMA. 2014;312(22):2401-2402.PubMedGoogle ScholarCrossref
43.
Bertrand  M, Duflo  E, Mullainathan  S.  How much should we trust differences-in-differences estimates?  Q J Econ. 2004;119(1):249-275.Google ScholarCrossref
44.
Greene  WH.  Econometric Analysis. 7th ed. San Francisco, CA: Prentice Hall; 2012.
45.
Heckman  JJ.  Randomization as an instrumental variable.  Rev Econ Stat. 1996;78(2):336-341.Google ScholarCrossref
46.
Angrist  JD, Imbens  GW, Rubin  DB.  Identification of causal effects using instrumental variables.  J Am Stat Assoc. 1996;91(434):444-455.Google ScholarCrossref
47.
Angrist  JD, Krueger  AB.  Instrumental variables and the search for identification: from supply and demand to natural experiments.  J Econ Perspect. 2001;15:69-85.Google ScholarCrossref
48.
Hong  CS, Hwang  AS, Ferris  TG. Finding a Match: How Successful Complex Care Programs Identify Patients. California Healthcare Foundation; March 2015. http://www.chcf.org/publications/2015/03/finding-match-complex-care. Accessed November 22, 2016.
49.
Hunter  G, Yoon  J, Blonigen  DM, Asch  SM, Zulman  DM.  Health care utilization patterns among high-cost VA patients with mental health conditions.  Psychiatr Serv. 2015;66(9):952-958.PubMedGoogle ScholarCrossref
Original Investigation
February 2017

Effect of an Intensive Outpatient Program to Augment Primary Care for High-Need Veterans Affairs PatientsA Randomized Clinical Trial

Author Affiliations
  • 1Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, California
  • 2Division of General Medical Disciplines, Stanford University, Stanford, California
  • 3Medical Services, VA Palo Alto Health Care System, Palo Alto, California
  • 4Health Economics Resource Center, VA Palo Alto Health Care System, Menlo Park, California
  • 5Chief of Staff Office, VA Palo Alto Health Care System, Palo Alto, California
  • 6Stanford University Human Immune Monitoring Center, Institute for Immunity Transplantation and Infection, Stanford University, Stanford, California
 

Copyright 2016 American Medical Association. All Rights Reserved.

JAMA Intern Med. 2017;177(2):166-175. doi:10.1001/jamainternmed.2016.8021
Key Points

Question  Does an intensive outpatient program for high-need patients change utilization patterns and reduce costs in an integrated setting with a patient-centered medical home?

Findings  In this randomized clinical trial of a Veterans Affairs intensive outpatient care program, the intervention was well received by a random sample of high-risk and high-cost patients but achieved reductions in hospitalization rates and costs similar to those of usual Veterans Affairs primary care.

Meaning  Implementing intensive outpatient care programs in integrated settings with well-established medical homes may not prevent hospitalizations or achieve substantial cost savings.

Abstract

Importance  Many organizations are adopting intensive outpatient care programs for high-need patients, yet little is known about their effectiveness in integrated systems with established patient-centered medical homes.

Objective  To evaluate how augmenting the Veterans Affairs (VA) medical home (Patient Aligned Care Teams [PACT]) with an Intensive Management program (ImPACT) influences high-need patients’ costs, health care utilization, and experience.

Design, Setting, and Participants  Randomized clinical trial at a single VA facility. Among 583 eligible high-need outpatients whose health care costs or hospitalization risk were in the top 5% for the facility, 150 were randomly selected for ImPACT; the remaining 433 received standard PACT care.

Interventions  The ImPACT multidisciplinary team addressed health care needs and quality of life through comprehensive patient assessments, intensive case management, care coordination, and social and recreational services.

Main Outcomes and Measures  Primary difference-in-difference analyses examined changes in health care costs and acute and extended care utilization over a 16-month baseline and 17-month follow-up period. Secondary analyses estimated the intervention’s effect on ImPACT participants (using randomization as an instrument) and for patients with key sociodemographic and clinical characteristics. ImPACT participants’ satisfaction and activation levels were assessed using responses to quality improvement surveys administered at baseline and 6 months.

Results  Of 140 patients assigned to ImPACT, 96 (69%) engaged in the program (mean [SD] age, 68.3 [14.2] years; 89 [93%] male; mean [SD] number of chronic conditions, 10 [4]; 62 [65%] had a mental health diagnosis; 21 [22%] had a history of homelessness). After accounting for program costs, adjusted person-level monthly health care expenditures decreased similarly for ImPACT and PACT patients (difference-in-difference [SE] −$101 [$623]), as did acute and extended care utilization rates. Among respondents to the ImPACT follow-up survey (n = 54 [56% response rate]), 52 (96%) reported that they would recommend the program to others, and pre-post analyses revealed modest increases in satisfaction with VA care (mean [SD] increased from 2.90 [0.72] to 3.16 [0.60]; P = .04) and communication (mean [SD] increased from 2.99 [0.74] to 3.18 [0.60]; P = .03).

Conclusions and Relevance  Intensive outpatient care for high-need patients did not reduce acute care utilization or costs compared with standard VA care, although there were positive effects on experience among patients who participated. Implementing intensive outpatient care programs in integrated settings with well-established medical homes may not prevent hospitalizations or achieve substantial cost savings.

Trial Registration  clinicaltrials.gov Identifier: NCT02932228

Introduction

Within the United States and many of its component health care systems, a small number of high-need patients—sometimes referred to as superutilizers1(p1)—account for approximately half of health care spending.2,3(p1)4-6 Many of these individuals experience multiple health issues and social stressors that complicate their health care navigation and self-management. In response to escalating health care costs, insurers, clinicians, and health care facilities are eager to identify interventions that address these patients’ health risks and reduce potentially avoidable costly service use.7

Intensive outpatient care programs have gained traction in recent years as a model for delivering comprehensive, individualized medical, mental health, and social services to high-need patients.1,8-11 Despite enthusiasm for these programs,12 evidence regarding their effects on hospitalization and costs is mixed.13,14 Some have described program savings as high as 15% to 50% in observational and retrospective evaluations,15-18 and there are examples of programs for older adults that have decreased acute care use and/or costs in randomized trials.19-22 However, rigorous evaluations of other programs have not demonstrated these effects,23-25 suggesting that contextual factors such as patient population, existing services, and resource availability may be important.

Because many early intensive outpatient programs were designed specifically for older adults,19-22 patients in fragmented safety net settings,11,13 and employed individuals,17,18 their evaluations may not be generalizable to integrated systems caring for more diverse populations. Furthermore, it is unclear whether individuals receiving comprehensive care in an effective patient-centered medical home (PCMH)26 will benefit from additional intensive services.

To advance understanding of the value of intensive outpatient care in these settings, we evaluated a program in the Veterans Affairs (VA) Health Care System, an integrated system with a well-established PCMH. The program was a quality improvement initiative offered to a random sample of patients with high health care costs and/or hospitalization risk. In this article, we describe the program’s effects on health care utilization, cost, and patient experience.

Methods
Setting and Intervention

Patients in the VA receive primary care through Patient Aligned Care Teams (PACTs), teamlets with a primary care clinician, nurse, clinical associate, and administrative associate that are supported by social work, pharmacy, and behavioral health services.27 In February 2013, the VA Palo Alto facility launched an Intensive Management PACT (ImPACT) quality improvement initiative to augment PACT primary care for high-need patients. ImPACT’s design incorporated elements from early intensive outpatient care programs15-17,23,24,28-33 and lessons from the Medicare Coordinated Care Demonstration.34 The program’s development and implementation have been described previously35,36; core elements are presented in the Box.

Box Section Ref ID
Box.

Core Elements of the Veterans Affairs (VA) Intensive Management Patient-Aligned Care Team (ImPACT) Intensive Outpatient Program

  • Multidisciplinary team (nurse practitioner, physician, social worker, and recreation therapist) that partners with and augments patients’ medical home to provide enhanced services

  • Comprehensive patient assessment (medical record review, in-person intake)

  • Identification and tracking of patients’ health-related goals, priorities, and self-care challenges

  • Assessment of patients’ physical function, cognitive impairment, social support, advance directives, medication adherence, and level of activation

  • Care management for medical and social service needs

  • Frequent contact through direct telephone line and in-person visits as needed

  • Interventions to support patients’ and caregivers’ quality of life (eg, recreation therapy)

  • Attendance at specialty care appointments to support decision making, when appropriate

  • Tracking system to monitor emergency department visits, hospitalizations, upcoming appointments

  • Weekly team discussions of high-acuity patients

  • Coordination of care with VA and non-VA clinicians, including during and after hospitalization

The Stanford University Institutional Review Board and VA Palo Alto Research and Development Committee approved this evaluation of the ImPACT program using deidentified data. Informed consent was waived due to the retrospective nature of the study. The ImPACT program was retrospectively registered on October 6, 2016 (NCT02932228) (protocol in Supplement 1).

Patient Population

Patients were eligible for ImPACT if they received care from 1 of 14 primary care clinicians and if their total VA health care costs were in the top 5% for the facility during a 9-month eligibility phase (October 1, 2011, through June 30, 2012) or if their risk for 1-year hospitalization in November 2012 was in the top 5% (using the VA’s Care Assessment Need risk-prediction algorithm).37 Patients were excluded if they were enrolled in the VA’s mental health intensive case management, home-based primary care, or palliative care programs, or if they were in an inpatient setting for more than half of the eligibility phase. Patients were also excluded if they were in the lowest cost decile or the lowest risk quartile in an effort to focus on patients who were actively high utilizers and at risk for persistent high utilization. Given limited resources, facility leadership offered ImPACT to 150 of the 583 eligible patients (using simple random sampling without replacement). This number was derived from caseloads reported by previous intensive outpatient programs.11,13 ImPACT contacted selected patients by mail followed by telephone during an enrollment period from February 1, 2013, through August 31, 2013. The remaining 433 patients received standard PACT care (Figure 1).

Data Sources and Measures

We examined health care costs and utilization among ImPACT and PACT patients during a 16-month baseline period (October 1, 2011, through January 31, 2013) and 17-month follow-up period (February 1, 2013, through June 30, 2014). We obtained sociodemographic, clinical, utilization, and cost data from VA Palo Alto’s Decision Support System office, and Care Assessment Need risk scores37 from the VA’s Regional Data Warehouse. We obtained patient insurance status, mortality data, and cost and utilization data for VA-reimbursed care outside the facility (fee-basis services) from VA Palo Alto’s Office of Business Analytics. Mortality data were updated on September 15, 2014, and validated April 16, 2015, through medical record review for patients identified as deceased and patients with VA costs of $0 for 2 consecutive months at the end of the evaluation period (n = 153).

We identified chronic condition diagnoses using International Classification of Diseases, Ninth Revision (ICD-9) codes that were documented at least once in inpatient or outpatient encounters, using an established VA algorithm.38(pp89-92) Sociodemographic and other patient characteristics included age, sex, urban/rural geographical location, race/ethnicity, non-VA health insurance, and history of homelessness (identified through the ICD-9 code 60.0, a method that could underestimate actual rates due to underdetection and undercoding39).

Our primary outcome was total cost of VA care (including inpatient, outpatient, and fee-basis care). We also examined costs of specific inpatient services (ie, acute and extended medical/surgical care and acute and extended mental health/residential care), and outpatient services (ie, emergency department [ED], primary, specialty, and mental health care). For primary analyses, we included mean monthly ImPACT encounter costs (approximately $136/patient) to capture personnel effort and workload; in sensitivity analyses we replaced encounter costs with mean monthly ImPACT personnel costs ($240/patient) to reflect program costs incurred by the facility. Utilization measures included hospitalization frequency and length of stay for each inpatient service, and number of primary, mental health, specialty care, and ED visits.

Patient experience outcomes derived from a quality improvement survey administered to ImPACT patients at baseline and 6 months after enrollment. Survey measures included the Patient Satisfaction Questionnaire (PSQ-18) (5-point subscales for general satisfaction, accessibility/convenience, and communication, modified to refer specifically to VA care)40 and the 6-item low-literacy Patient Activation Measure (PAM-6) (standardized to 100 points).41 The follow-up survey also asked about patients’ satisfaction with ImPACT services and their likelihood to recommend the program to others.

Analyses

The original evaluation design assumed that program sustainability would require demonstration of superiority under 1-tailed testing; however, for the final evaluation we used a more conservative 2-tailed test. Power from the 1-tailed and 2-tailed calculations for an analysis of covariance to detect a 20% reduction in mean costs (comparable to savings achieved by previous intensive outpatient programs over 12-18 months)16-18 was 88.6% and 81.3%, respectively, equivalent to a superiority margin of 8.5% and 6.2% reduction in mean costs, respectively.

We excluded from analyses patients who died or left the facility before the intervention start date (February 1, 2013) (8 in ImPACT, 23 in PACT) and patients who were identified in retrospective medical record review as not meeting study eligibility criteria (2 in ImPACT, 5 in PACT). This resulted in a final study population of 140 patients in ImPACT and 405 patients in PACT.

We compared characteristics of patients in ImPACT and PACT using t tests for continuous variables and χ2 tests for dichotomous and categorical variables. We examined differences in attrition (ie, departure from the health care facility) across the 2 groups using linear regression. We examined differences in mortality, including among patients who had left the facility, using Kaplan-Meier analysis. Baseline and follow-up utilization and cost data were available for all patients. In descriptive analyses, we examined changes in mean ImPACT vs PACT person-level aggregate costs and utilization over 16-month baseline and follow-up periods.

We performed intention-to-treat analyses within a difference-in-differences (D-in-D) framework to compare monthly VA health care costs and utilization among ImPACT and PACT patients, during baseline and follow-up periods.42,43 We included fixed effects to account for fixed differences across patients.44 In sensitivity analyses, we replaced person fixed effects with covariates for baseline characteristics and found similar results. For patients who died during the study period, we included monthly costs after death ($0); for patients who left the facility, we set monthly costs to missing after they were no longer receiving care from the VA facility. In secondary analyses, we estimated the effect of the intervention on patients who actively engaged in ImPACT (defined a priori as completing an intake and at least 3 additional encounters with the team) using randomization as an instrument for participation.45-47

We conducted prespecified and post hoc exploratory analyses to examine variation in program effects for key patient subgroups, with the goal of refining the program’s future patient selection criteria. Prespecified subgroups included patients with and without mental health diagnoses, patients younger than 65 years and 65 years or older, and patients who were eligible for ImPACT based on high cost vs high risk for hospitalization (non–mutually exclusive groups). Post hoc stratified analyses focused on characteristics that the clinical team perceived as potential candidates for future patient selection: the presence of heart failure, diabetes, or chronic obstructive pulmonary disease; hospitalization in the 6 months prior to program implementation; and high risk for hospitalization with a hospitalization in the 6 months prior to program implementation.

To evaluate patient experience, we used paired t tests to assess changes in mean (SD) scores for satisfaction (PSQ-18) subscales40 and activation (PAM-6)41 among ImPACT participants who completed both the baseline and follow-up surveys. We examined potential nonresponse bias by comparing sociodemographic and clinical characteristics among patients who completed the baseline survey alone, and patients who completed the baseline and follow-up surveys.

We used Stata 13.0 to perform all analyses.

Results

ImPACT and PACT patients’ baseline sociodemographic and clinical characteristics were similar (Table 1 and eTable 1 in Supplement 2). ImPACT patients’ chronic conditions included hypertension (100 [71%]), joint disorders (80 [57%]), depression (68 [49%]), coronary artery disease (51 [36%]), diabetes (47 [34%]), drug use disorder (41 [29%]), heart failure (34 [24%]), posttraumatic stress disorder (32 [23%]), and cancer (29 [21%]), all of which were present in comparable rates among PACT patients. Among the 39 chronic conditions assessed, the only condition with significantly different rates among ImPACT and PACT patients was dementia (13 [9%] vs 17 [4%], respectively; P = .02) (eTable 2 in Supplement 2).

Of the 140 ImPACT patients included in intention-to-treat analyses, 96 (69%) engaged in the program (ie, they completed an intake and had at least 3 additional encounters with the team) (Figure 1). Engaged vs nonengaged ImPACT patients were typically older (mean [SD] age, 68.3 [14.2] vs 62.1 [12.6] years; P = .01), were more likely to have non-VA health insurance (primarily Medicare) (57 [59%] vs 17 [39%]; P = .02), and were less likely to live in a rural location (4 [4%] vs 7 [16%]; P = .04), respectively. Engaged patients had lower rates of liver disease/hepatitis C (16 [17%] vs 14 [32%]; P = .04) and alcohol use disorders (16 [17%] vs 14 [32%]; P = .04) than nonengaged patients, respectively, but otherwise had similar rates of chronic conditions (eTables 1 and 2 in Supplement 2).

Patient Experience Outcomes

Among ImPACT particpants who responded to the follow-up survey (n = 54 [56% response rate among engaged patients]), 52 (96%) reported that they would recommend the program to others. A majority reported being “extremely satisfied” with the program’s medical care (28 of 40 [70%]), social work (22 of 29 [76%]), and recreation therapy and community services (25 of 32 [78%]). Among those who responded to baseline and follow-up surveys (n = 54, 56% response rate among engaged patients), there was a statistically significant increase in mean (SD) ratings for satisfaction with VA care (2.90 [0.72] to 3.16 [0.60]; P = .04) and communication (2.99 [0.74] to 3.18 [0.60]; P = .03), but not for accessibility/convenience (3.00 [0.54] to 3.10 [0.59]; P = .28). Mean (SD) activation levels were 62.7 (17.7) at baseline and 66.8 (18.2) at follow-up (50 respondents for activation questions; P = .08). Patients who completed the baseline and follow-up surveys were similar to those who completed only the baseline survey, with the exception of lower hospitalization rates in the 16 months prior to enrollment (mean [SD], 1.3 [1] vs 2.2 [2], respectively; P = .009) (eTable 1 in Supplement 2).

Clinical and Utilization Outcomes

Mortality rates were 12.1% (n = 17) and 13.6% (n = 55) for ImPACT and PACT patients, respectively (mortality rate difference = 1.4% [95% CI, −5.7% to 7.3%]; log-rank test for equality of survival functions P = .69) (eFigure 1 in Supplement 2), attrition rates due to patients leaving the facility were 8.6% (n = 12) and 5.4% (n = 22), respectively (P = .19). During the first 16 months of the intervention period, the mean (SD) number of primary care visits (including ImPACT encounters) was 21.8 (17.4) for ImPACT patients and 7.4 (7.5) for PACT patients (Table 2). ImPACT and PACT patients’ mean (SD) numbers of acute medical/surgical hospitalizations were comparable at baseline (1.3 [1.5]) and decreased at similar rates to 0.7 (1.2) for patients in ImPACT and PACT (monthly admission D-in-D [SE] = 0.01 [0.01]). Mean (SD) ED visits also decreased at similar rates, from 3.4 (3.3) to 2.1 (2.9) among ImPACT patients, and from 3.3 (3.3) to 2.1 (2.9) among PACT patients (monthly ED visit D-in-D [SE] = −0.01 [0.02]). There were no significant program effects on any other inpatient or outpatient health care utilization measures (Table 2 and eTable 3 in Supplement 2).

Cost Outcomes

After accounting for ImPACT encounter costs, mean baseline and follow-up person-level monthly costs declined at similar rates among ImPACT patients (21.0%, from $6139 to $4850) and PACT patients (20.7%, from $5821 to $4618) (Figure 2 and eFigure 2 in Supplement 2). In intention-to-treat (D-in-D) analyses, ImPACT did not have a significant effect on monthly costs (D-in-D [SE] = −$101 [$623]). The D-in-D estimate was further attenuated when ImPACT personnel monthly costs (approximately $240/patient) were substituted for ImPACT encounter monthly costs (approximately $136/patient) in sensitivity analyses. Stratified cost analyses revealed no statistically significant differences among subgroups, although nonsignificantly greater cost reductions were observed for patients younger than 65 years, patients hospitalized in the 6 months prior to enrollment, and those with certain medical conditions (Figure 3).

ImPACT was associated with a significant increase in monthly person-level primary care costs (independent of ImPACT encounter costs) (D-in-D [SE] = $30 [$14]), but there were no effects on costs associated with other outpatient or inpatient services (eTable 4 in Supplement 2). Secondary analyses accounting for active engagement in the intervention revealed similar patterns (eTable 5 in Supplement 2).

Discussion

The ImPACT program provided an opportunity to evaluate intensive outpatient care for high-need patients in an integrated system with an established PCMH. Findings from this evaluation have import for accountable care organizations and other clinical settings that are responsible for their high-risk patients’ outcomes and associated costs.7 Contrary to expectations, the program did not reduce acute care utilization or total expenditures compared with standard PCMH care, although high satisfaction rates suggest that the program may have enhanced patients’ experiences with their care.

This randomized clinical trial adds to a growing body of literature describing intensive outpatient care programs for high-need patients.11,13,14 Previous reports of cost reductions after patients enroll in these programs may stem from their observational nature.13-18 The substantial and comparable cost declines observed among ImPACT and PACT patients illustrate regression to the mean experienced by high-cost populations, and underscore the importance of rigorous evaluations of programs for these patients.

One explanation for ImPACT’s modest effects on cost is that a longer implementation and/or follow-up period is required to achieve savings. One of the few intensive outpatient programs that reduced costs in a randomized trial was a Medicare demonstration program for high-cost beneficiaries that underwent preparations and piloting for more than 2 years before the evaluation began.22 Once patients are enrolled in such programs, it takes time to build their trust, modify health behaviors (including activation), and improve chronic disease trajectories. ImPACT patients had nearly 3 times more visits to primary care compared with PACT patients, which could potentially translate to increased health maintenance and screening, and reductions in future utilization.

Our findings could also reflect a mismatch between ImPACT’s selected patients and the program’s disciplines and services. Participants in a Super-Utilizers Summit in 2013 concluded that the “holy grail” for intensive outpatient programs is to determine “which patients need which interventions in which setting by which provider.”1(p6) ImPACT patients were selected based on their cost and future hospitalization risk; however, recent hospitalization was associated with a nonsignificant greater effect in subgroup analyses (Figure 3). This and other indicators of acuity (such as clinician referral) have been used to select patients for other programs and may improve identification of individuals with modifiable risk factors.48

In addition, while ImPACT’s nurse practitioner and social worker were well equipped to manage medical and social needs, the program did not have a designated mental health clinician, and instead referred patients to the VA’s existing, comprehensive mental health services. This referral-based model may not have been adequate given the prevalence and acuity of ImPACT patients’ mental health and substance use disorders. Future programs for similar populations may benefit from including a mental health clinician and/or specialized mental health–focused interventions.49

Finally, ImPACT may have increased service use and costs in certain circumstances by uncovering unmet medical or mental health needs, and facilitating appropriate access to specialists and acute care services. Subgroup analyses, while underpowered to detect a statistically significant effect, suggest that older adults in ImPACT experienced increased costs. This could be due to ImPACT-eligible older adults shifting their care from Medicare-covered non-VA services to the VA, and warrants further investigation once Medicare claims data are available.

Limitations

Several limitations should be noted. Generalizability of findings may be limited given program implementation in a single VA facility, and the population’s high mental illness burden.35 Program effects may have been diluted because the evaluation began at program implementation, and it took 7 months to enroll all engaged patients. Furthermore, because of unanticipated delays in program initiation, some patients left the facility or died before the program began, reducing statistical power to 79% (see Methods). We attempted to address this postrandomization, preintervention dropout by conducting D-in-D analyses.

In addition, because ImPACT and PACT teams were co-located, PACT clinicians could have observed and adopted ImPACT interventions; however, they would not have had the advantage of ImPACT personnel and time, and contamination was not noted during the implementation evaluation.36 Finally, assessment of patient experience outcomes was based on a quality improvement survey of ImPACT patients (with no comparison group and a 63% response rate) and could be biased if follow-up survey completion was correlated with positive or negative experience.

Conclusions

Intensive outpatient care for randomly selected high-need patients did not reduce acute care utilization or costs when compared with standard PCMH care in the VA’s integrated system. Findings highlight the need to consider study design and context when interpreting evidence about these programs. Additional research should explore whether the value of such programs can be enhanced through modifying patient selection criteria, team structure, or patient engagement strategies. While high-need patients might benefit from intensive outpatient interventions in some settings, programs in integrated systems with well-established medical homes may not achieve substantial cost savings.

Back to top
Article Information

Corresponding Author: Donna M. Zulman, MD, MS, Division of General Medical Disciplines, Stanford University, Center for Innovation to Implementation (Ci2i), VA Palo Alto Health Care System, 795 Willow Rd, MPD-152, Menlo Park, CA 94025 (dzulman@stanford.edu).

Accepted for Publication: October 27 2016.

Published Online: December 27, 2016. doi:10.1001/jamainternmed.2016.8021

Author Contributions: Dr Zulman had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Zulman, Ezeji-Okoye, Holmes, Kahn, Asch.

Acquisition, analysis, or interpretation of data: Zulman, Pal Chee, Shaw, Holmes, Kahn.

Drafting of the manuscript: Zulman, Pal Chee, Asch.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Pal Chee, Holmes.

Obtained funding: Zulman, Ezeji-Okoye, Kahn.

Administrative, technical, or material support: Ezeji-Okoye, Kahn.

Study supervision: Ezeji-Okoye, Kahn, Asch.

Conflict of Interest Disclosures: None reported.

Funding/Support: This evaluation was supported by VA Health Services Research and Development Service (PPO 13-117) and support from the VA Office of Specialty Care Transformation (Specialty/Surgical Care Neighborhood Team Based Model Pilot Program). Dr Zulman is supported by a VA Health Services Research and Development Service Career Development Award (CDA 12-173). Dr Kahn acknowledges support for this project from Stanford Clinical and Translational Science Award (NIH UL1 TR 001083).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge Debra Hummel, DNP, ARNP-C, Kathryn Holloway, MS, CTRS, Miriam Trigueros, RACSW, Georgezetta Rogers, Sasha Smither, MSW, LCSW, Lien Nguyen, Kalkidan Asrath, Jessica Radmilovic MS, CTRS, John Chardos, MD, Ian Tong, MD, and James Hallenbeck, MD, from VA Palo Alto Health Care System, and Alan Glaseroff, MD, and Ann Lindsay, MD, from Stanford Coordinated Care, for serving as clinical advisors to the ImPACT program; Cindie Slightam, MPH, Ava Wong, MPH, and David Krauth, MPH, for assistance with manuscript preparation; Donn Garvert, MS, for assistance with data management and analyses; Danielle Cohen, MPH, and Valerie Meausoone, Stanford University, and Cindie Slightam, MPH, VA Palo Alto Health Care System, for data management and program evaluation support; and members of the VA PACT–Intensive Management National Evaluation Center (Lisa Rubenstein, MD, MPH, Evelyn Chang, MD, MSHS, and Michael Ong, MD, PhD, VA Greater Los Angeles Healthcare System, and Jean Yoon, PhD, VA Palo Alto Health Care System) for their review of study findings. VA Palo Alto data acquisition and analysis were supported by Robert Chang, Chi Pham, and the Veterans Affairs Women’s Health Evaluation Initiative; Drs Glaseroff and Lindsay, Ms Meausoone, and Mr Garvert received compensation for their contributions from the VA Office of Specialty Care Transformation (Specialty Care Neighborhood Team Based Model Pilot Program grant).

References
1.
Hasselman  D.  Super-Utilizer Summit: Common Themes from Innovative Complex Care Management Programs. Hamilton, NJ: Center for Health Care Strategies, Inc; 2013.
2.
Mann  C. Medicaid and CHIP: On the Road to Reform. Alliance for Health Reform/Kaiser Family Foundation; 2011. http://www.allhealth.org/briefingmaterials/KFFAlliance_FINAL-1971.ppt. Accessed November 28, 2012.
3.
Sommers  A, Cohen  M.  Medicaid’s High Cost Enrollees: How Much Do They Drive Program Spending? Washington, DC: Kaiser Commission for Medicaid and the Uninsured; 2006.
4.
Conwell  LJ, Cohen  JW. AHRQ Statistical Brief 73: Characteristics of persons with high medical expenditures in the US civilian noninstitutionalized population, 2002. Rockville, MD: Agency for Healthcare Research and Quality; 2005. https://meps.ahrq.gov/data_files/publications/st73/stat73.pdf. Accessed November 28, 2012.
5.
Zulman  DM, Pal Chee  C, Wagner  TH,  et al.  Multimorbidity and healthcare utilisation among high-cost patients in the US Veterans Affairs Health Care System.  BMJ Open. 2015;5(4):e007771.PubMedGoogle ScholarCrossref
6.
Congressional Budget Office. High-cost Medicare beneficiaries: a CBO paper. 2005. http://www.cbo.gov/sites/default/files/cbofiles/ftpdocs/63xx/doc6332/05-03-medispending.pdf. Accessed February 26, 2014.
7.
Blumenthal  D, Chernof  B, Fulmer  T, Lumpkin  J, Selberg  J.  Caring for high-need, high-cost patients—an urgent priority.  N Engl J Med. 2016;375(10):909-911.PubMedGoogle ScholarCrossref
8.
Yee  T, Lechner  A, Carrier  E. High-Intensity Primary Care: Lessons for Physician and Patient Engagement. National Institute for Health Care Reform; 2012. http://nihcr.org/analysis/improving-care-delivery/prevention-improving-health/high-intensity-primary-care/. Accessed November 22, 2016.
9.
Peterson  D, Helfand  M, Humphrey  L, Christensen  V, Carson  S. Evidence Brief: Effectiveness of Intensive Primary Care Programs, VA-ESP Project 09-199. Washington, DC: Department of Veterans Affairs; 2013. http://www.hsrd.research.va.gov/publications/esp/primary-care.cfm. Accessed November 22, 2016.
10.
California Healthcare Foundation, California Quality Collaborative. Complex Care Management Toolkit. 2012. http://www.calquality.org/storage/documents/cqc_complexcaremanagement_toolkit_final.pdf. Accessed February 26, 2014.
11.
Bodenheimer  T. Strategies to reduce costs and improve care for high-utilizing Medicaid patients: reflections on pioneering programs. Center for Health Care Strategies, Inc; 2013. http://www.chcs.org/resource/strategies-to-reduce-costs-and-improve-care-for-high-utilizing-medicaid-patients-reflections-on-pioneering-programs/. Accessed November 20, 2016.
12.
Gawande  A. The hot spotters: Can we lower medical costs by giving the neediest patients better care? The New Yorker. January 24, 2011. http://www.newyorker.com/magazine/2011/01/24/the-hot-spotters. Accessed November 22, 2016.
13.
Hong  CS, Siegel  AL, Ferris  TG. Caring for high-need, high-cost patients: what makes for a successful care management program? The Commonwealth Fund; August 2014. http://www.commonwealthfund.org/~/media/files/publications/issue-brief/2014/aug/1764_hong_caring_for_high_need_high_cost_patients_ccm_ib.pdf. Accessed November 22, 2016.
14.
McCarthy  D, Ryan  J, Klein  S. Models of care for high-need, high-cost patients: an evidence synthesis. October 2015. http://www.commonwealthfund.org/~/media/files/publications/issue-brief/2015/oct/1843_mccarthy_models_care_high_need_high_cost_patients_ib.pdf. Accessed November 22, 2016.
15.
Green  SR, Singh  V, O’Byrne  W.  Hope for New Jersey’s city hospitals: the Camden Initiative.  Perspect Health Inf Manag. 2010;7:1d.PubMedGoogle Scholar
16.
Reuben  DB.  Physicians in supporting roles in chronic disease care: the CareMore model.  J Am Geriatr Soc. 2011;59(1):158-160.PubMedGoogle ScholarCrossref
17.
Blash  L, Chapman  S, Dower  C. The Special Care Center—a joint venture to address chronic disease. Center for the Health Professions Research Brief. 2011. http://www.iorahealth.com/wp-content/uploads/2014/07/UCSF_The_Special_Care_Center_A_Joint_Venture_to_Address_Chronic_Disease.pdf. Accessed November 28, 2012.
18.
Milstein  A, Kothari  PP. Are higher-value care models replicable? Health Affairs Blog. 2009. http://healthaffairs.org/blog/2009/10/20/are-higher-value-care-models-replicable/. Accessed November 28, 2012.
19.
Boult  C, Reider  L, Leff  B,  et al.  The effect of guided care teams on the use of health services: results from a cluster-randomized controlled trial.  Arch Intern Med. 2011;171(5):460-466.PubMedGoogle ScholarCrossref
20.
Naylor  MD, Brooten  DA, Campbell  RL, Maislin  G, McCauley  KM, Schwartz  JS.  Transitional care of older adults hospitalized with heart failure: a randomized, controlled trial.  J Am Geriatr Soc. 2004;52(5):675-684.PubMedGoogle ScholarCrossref
21.
Counsell  SR, Callahan  CM, Tu  W, Stump  TE, Arling  GW.  Cost analysis of the Geriatric Resources for Assessment and Care of Elders care management intervention.  J Am Geriatr Soc. 2009;57(8):1420-1426.PubMedGoogle ScholarCrossref
22.
McCall  N, Cromwell  J, Urato  C. Evaluation of Medicare Care Management for High Cost Beneficiaries (CMHCB) Demonstration: Massachusetts General Hospital and Massachusetts General Physicians Organization and Massachusetts General Physicians Organization (MGH), Final Report. Centers for Medicare & Medicaid Services; September 2010. https://www.cms.gov/reports/downloads/McCall_MGH_CMHCB_Final_2010.pdf. Accessed July 18, 2016.
23.
Shumway  M, Boccellari  A, O’Brien  K, Okin  RL.  Cost-effectiveness of clinical case management for ED frequent users: results of a randomized trial.  Am J Emerg Med. 2008;26(2):155-164.PubMedGoogle ScholarCrossref
24.
Bell  J, Mancuso  D, Krupski  T,  et al.  A Randomized Controlled Trial of King County Care Partners’ Rethinking Care Intervention: Health and Social Outcomes up to Two Years Post-randomization. Seattle, WA: Center Healthcare Improvement for Addictions Mental Illness and Medically Vulnerable Populations; 2012.
25.
Sledge  WH, Brown  KE, Levine  JM,  et al.  A randomized trial of primary intensive care to reduce hospital admissions in patients with high utilization of inpatient services.  Dis Manag. 2006;9(6):328-338.PubMedGoogle ScholarCrossref
26.
National Committee for Quality Assurance. NCQA Evidence Report: Benefits of the Patient-Centered Medical Homes. June 2015. http://www.ncqa.org/Portals/0/Programs/Recognition/PCMHEvidenceReportJune2015_Web.pdf?ver=2016-02-24-143948-347. Accessed July 20, 2016.
27.
Rosland  AM, Nelson  K, Sun  H,  et al.  The patient-centered medical home in the Veterans Health Administration.  Am J Manag Care. 2013;19(7):e263-e272.PubMedGoogle Scholar
28.
Raven  MC, Doran  KM, Kostrowski  S, Gillespie  CC, Elbel  BD.  An intervention to improve care and reduce costs for high-risk patients with frequent hospital admissions: a pilot study.  BMC Health Serv Res. 2011;11:270.PubMedGoogle ScholarCrossref
29.
Dorr  DA, Wilcox  AB, Brunker  CP, Burdon  RE, Donnelly  SM.  The effect of technology-supported, multidisease care management on the mortality and hospitalization of seniors.  J Am Geriatr Soc. 2008;56(12):2195-2202.PubMedGoogle ScholarCrossref
30.
Dorr  DA, Wilcox  A, Burns  L, Brunker  CP, Narus  SP, Clayton  PD.  Implementing a multidisease chronic care model in primary care using people and technology.  Dis Manag. 2006;9(1):1-15.PubMedGoogle ScholarCrossref
31.
Brown  KE, Levine  JM, Fiellin  DA, O’Connor  P, Sledge  WH.  Primary intensive care: pilot study of a primary care-based intervention for high-utilizing patients.  Dis Manag. 2005;8(3):169-177.PubMedGoogle ScholarCrossref
32.
Lessler  DS, Krupski  A, Cristofalo  M. King County Care Partners: a community-based chronic care management system for Medicaid clients with co-occurring medical, mental, and substance abuse disorders. In: Schraeder  C, Shelton  P, eds.  Comprehensive Care Coordination for Chronically III Adults. West Sussex, UK: John Wiley & Sons, Inc; 2011:339-348.
33.
Okin  RL, Boccellari  A, Azocar  F,  et al.  The effects of clinical case management on hospital service use among ED frequent users.  Am J Emerg Med. 2000;18(5):603-608.PubMedGoogle ScholarCrossref
34.
Brown  RS, Peikes  D, Peterson  G, Schore  J, Razafindrakoto  CM.  Six features of Medicare coordinated care demonstration programs that cut hospital admissions of high-risk patients.  Health Aff (Millwood). 2012;31(6):1156-1166.PubMedGoogle ScholarCrossref
35.
Zulman  DM, Ezeji-Okoye  SC, Shaw  JG,  et al.  Partnered research in healthcare delivery redesign for high-need, high-cost patients: development and feasibility of an Intensive Management Patient-Aligned Care Team (ImPACT).  J Gen Intern Med. 2014;29(4)(suppl 4):861-869.PubMedGoogle ScholarCrossref
36.
Breland  JY, Asch  SM, Slightam  C, Wong  A, Zulman  DM.  Key ingredients for implementing intensive outpatient programs within patient-centered medical homes: a literature review and qualitative analysis.  Healthc (Amst). 2016;4(1):22-29.PubMedGoogle ScholarCrossref
37.
Wang  L, Porter  B, Maynard  C,  et al.  Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration.  Med Care. 2013;51(4):368-373.PubMedGoogle ScholarCrossref
38.
Frayne  SM, Phibbs  CS, Saechao  F,  et al.  Sociodemographics, Utilization, Costs of Care, and Health Profile. Washington, DC: Department of Veterans Affairs, Women’s Health Evaluation Initiative; 2014.
39.
Peterson  R, Gundlapalli  AV, Metraux  S,  et al.  Identifying homelessness among veterans using VA administrative data: opportunities to expand detection criteria.  PLoS One. 2015;10(7):e0132664.PubMedGoogle ScholarCrossref
40.
Marshall  GN, Hays  RD.  The Patient Satisfaction Questionnaire Short-Form (PSQ-18). Santa Monica, CA: RAND; 1994.
41.
Hibbard  JH, Mahoney  ER, Stockard  J, Tusler  M.  Development and testing of a short form of the patient activation measure.  Health Serv Res. 2005;40(6, pt 1):1918-1930.PubMedGoogle ScholarCrossref
42.
Dimick  JB, Ryan  AM.  Methods for evaluating changes in health care policy: the difference-in-differences approach.  JAMA. 2014;312(22):2401-2402.PubMedGoogle ScholarCrossref
43.
Bertrand  M, Duflo  E, Mullainathan  S.  How much should we trust differences-in-differences estimates?  Q J Econ. 2004;119(1):249-275.Google ScholarCrossref
44.
Greene  WH.  Econometric Analysis. 7th ed. San Francisco, CA: Prentice Hall; 2012.
45.
Heckman  JJ.  Randomization as an instrumental variable.  Rev Econ Stat. 1996;78(2):336-341.Google ScholarCrossref
46.
Angrist  JD, Imbens  GW, Rubin  DB.  Identification of causal effects using instrumental variables.  J Am Stat Assoc. 1996;91(434):444-455.Google ScholarCrossref
47.
Angrist  JD, Krueger  AB.  Instrumental variables and the search for identification: from supply and demand to natural experiments.  J Econ Perspect. 2001;15:69-85.Google ScholarCrossref
48.
Hong  CS, Hwang  AS, Ferris  TG. Finding a Match: How Successful Complex Care Programs Identify Patients. California Healthcare Foundation; March 2015. http://www.chcf.org/publications/2015/03/finding-match-complex-care. Accessed November 22, 2016.
49.
Hunter  G, Yoon  J, Blonigen  DM, Asch  SM, Zulman  DM.  Health care utilization patterns among high-cost VA patients with mental health conditions.  Psychiatr Serv. 2015;66(9):952-958.PubMedGoogle ScholarCrossref
×