Cost-effectiveness of Practice-Initiated Quality Improvement for Depression: Results of a Randomized Controlled Trial

Context Depression is a leading cause of disability worldwide, but treatment rates in primary care are low.

Objective To determine the cost-effectiveness from a societal perspective of 2 quality improvement (QI) interventions to improve treatment of depression in primary care and their effects on patient employment.

Design Group-level randomized controlled trial conducted June 1996 to July 1999.

Setting Forty-six primary care clinics in 6 community-based managed care organizations.

Participants One hundred eighty-one primary care clinicians and 1356 patients with positive screening results for current depression.

Interventions Matched practices were randomly assigned to provide usual care (n = 443 patients) or to 1 of 2 QI interventions offering training to practice leaders and nurses, enhanced educational and assessment resources, and either nurses for medication follow-up (QI-meds; n = 424 patients) or trained local psychotherapists (QI-therapy; n = 489). Practices could flexibly implement the interventions, which did not assign type of treatment.

Main Outcome Measures Total health care costs, costs per quality-adjusted life-year (QALY), days with depression burden, and employment over 24 months, compared between usual care and the 2 interventions.

Results Relative to usual care, average health care costs increased $419 (11%) in QI-meds (P = .35) and $485 (13%) in QI-therapy (P = .28); estimated costs per QALY gained were between $15 331 and $36 467 for QI-meds and $9478 and $21 478 for QI-therapy; and patients had 25 (P = .19) and 47 (P = .01) fewer days with depression burden and were employed 17.9 (P = .07) and 20.9 (P = .03) more days during the study period.

Conclusions Societal cost-effectiveness of practice-initiated QI efforts for depression is comparable with that of accepted medical interventions. The intervention effects on employment may be of particular interest to employers and other stakeholders.

Depression is common among primary care patients and practice guidelines are available, yet quality of care and outcomes remain poor.^1-10 Improving quality of care for depression in primary care could potentially increase well-being for patients, their families, and society at large.^11,12

Studies suggest that practice-based interventions to improve quality of treatment for depression in primary care can improve short-term clinical outcomes relative to usual care, at modest cost.^13-15 One such study of high utilizers found that a practice-based quality improvement (QI) program for depression was cost-effective relative to usual care, with longer-term improvements that included physical functioning.¹⁶ In addition, less intensive interventions, such as nurse telephone contact, also improve clinical outcomes, but those relying on clinician training alone may have limited benefits.^17,18 Most prior studies have focused on interventions implemented under research protocols in academically affiliated, organized care settings, so the extent to which results can be generalized to more naturalistic practice conditions and diverse types of practices is unclear.

Findings from Partners in Care (PIC) suggest that diverse managed primary care practices can implement such intervention programs under naturalistic practice conditions that include choice of treatment by patients and clinicians, and that such interventions improved quality of care, quality of life, clinical outcomes, and retention in employment over 1 year of follow-up.¹⁹ However, this work did not evaluate the cost-effectiveness of the practice-initiated interventions.

The current study was designed to fill those gaps. We examine societal cost-effectiveness over a 2-year period of implementation of the PIC interventions, relative to usual care, in diverse managed care practices. In addition, we examine impacts on patients' employment because of the strong interest in this outcome among purchasers.

PIC is a group-level, randomized controlled trial of practice-initiated QI programs for depression.¹⁹ PIC was fielded in 6 nonacademic managed care organizations. Forty-six of 48 primary care practices (clinics) and 181 of 183 clinicians participated. Within organizations, practices were matched into blocks of 3 clusters based on factors expected to affect outcomes (specialty mix, patient socioeconomic and demographic factors, and having on-site mental health specialists). Within blocks, practices were randomized to usual care or 1 of 2 QI programs (QI-meds or QI-therapy).

Study staff screened 27 332 consecutive patients in participating practices over a 5- to 7-month period between June 1996 and March 1997 (Figure 1). Patients were eligible if they intended to use the practice over the next 12 months and screened positive for depression, based on stem items from the World Health Organization's 12-month Composite International Diagnostic Interview (CIDI).²⁰ Patients were considered positive for depression if they reported at least 1 week of depression in the last 30 days, plus 2 or more weeks of depressed mood or loss of interest in pleasurable activities in the last year or persistent depression over the year. This indicator has a 55% positive predictive value for 12-month major depressive or dysthymic disorder by the full CIDI.⁹ Patients were ineligible if younger than 18 years, not fluent in English or Spanish, or lacking in insurance coverage for intervention therapists.

Of those completing the screener, 3918 were eligible, 2417 were available to confirm insurance eligibility, and 241 were ineligible. Of those reading the informed consent, 1356 (70%) enrolled: 443 in usual care, 424 in QI-meds, and 489 in QI-therapy practices. The study was approved by the institutional review boards of RAND and the practices.

Intervention design and implementation are described elsewhere, and all QI materials are available from RAND (http://www.rand.org/organization/health/pic.products/order.html).^19,21 Prior to implementation, practices committed to implementing the programs and the study provided a payment of up to half of the estimated practice participation costs ($35 000-$70 000). Usual care clinics received depression practice guidelines by mail. The interventions provided practices with training and resources to initiate and monitor QI programs according to local practice goals and resources. Patients and clinicians retained choice of treatment, and their use of intervention resources was optional. In effect, the study served as an external disease management firm, designing the materials, hosting initial training, and offering limited support during implementation.

For both QI-meds and QI-therapy interventions, local practice teams were trained in a 2-day workshop to provide clinician education through lectures, academic detailing, or audit and feedback, and to supervise intervention staff and conduct team oversight. Practice nurses were trained as depression specialists, following a written protocol, to assist in initial patient assessment, education, and motivation for treatment. Practice teams were given patient education pamphlets and videotapes, patient tracking forms, and clinician manuals and pocket reminder cards and were encouraged to distribute them. The materials described guideline-concordant care for depression and presented psychotherapy and antidepressant medication as equally effective.

In the QI-meds program, nurse specialists were trained to support medication adherence through monthly telephone contacts or visits for 6 or 12 months, randomized at the patient level. In the QI-therapy program, practice therapists were trained to provide individual and group cognitive behavioral therapy, following a protocol developed at the San Francisco General Hospital Depression Clinic.^22,23 This therapy was available at the primary care co-payment rate (usually $5-$10) for 6 months after enrollment. All patients could have other therapy at the usual co-payment rate. Clinical supervision was provided by local experts assisted by study experts in cognitive behavioral therapy. In all conditions, patients could have medication, therapy, or both. However, the extra QI-meds or QI-therapy resources made it easier to obtain appropriate medication or therapy, respectively.

Patients were asked to complete the screener, the telephone CIDI for depression, a detailed economic survey by telephone at baseline and 24 months of follow-up, and mail surveys at baseline and 6, 12, 18, and 24 months. Completion rates were 95% and 85% for the baseline and 24-month telephone surveys and 90%, 86%, 84%, 83%, and 85%, respectively, for the mail surveys. Subjects were eligible for follow-up unless they disenrolled from the study. Claims and encounter data from the practices were only consistently available for the first 6 months of patient follow-up, and most practices did not have pharmacy data.

Intervention Costs. These included screening, intervention materials, initial nurse specialist assessments, and 20 minutes of supervision of nurses and therapists per enrolled patient. We assigned costs to intervention activities based on data from the practices about the average cost of clinic staff. Research-specific costs were excluded. For main analyses, we assumed that follow-up visits to intervention staff were included in patient reports of outpatient visits. In sensitivity analyses, we used data from intervention logs to include such visits as intervention costs (which double counts them if they were also reported by patients directly); our results did not change substantively.

Health Care Costs. We assigned costs to patient-reported counts of emergency department visits, medical and mental health visits, and psychotropic medications used, for each follow-up. Patient report was selected due to the limitations in the available claims and encounter data. In addition, the number of outpatient visits was higher for patient surveys than claims data over the first 6 months, probably due to out-of-practice use or incomplete claims data. We excluded inpatient costs because we did not expect or observe intervention effects on them and had limited precision to analyze them.²⁴

Average costs in 1998 dollars were assigned to each component of patient-reported health care use using a national database of about 1.8 million privately insured individuals (provided by Ingenix, a benefits consulting firm in New Haven, Conn). The Ingenix data included information on provider reimbursements (ie, patient and plan payments, plus coordination of benefits), which we used as a proxy for health care costs. Specifically, we calculated the mean cost per outpatient medical visit ($46), mental health visits ($96), and emergency department visits ($450), respectively, for adults in the Ingenix data; these costs include facility charges, professional fees, and ancillary services associated with the visits, as applicable. We then multiplied the visit counts reported by PIC patients by these mean costs.

For psychotropic medications, we matched patient-reported data of medication names, daily dosages, and months of use to average costs for that combination from the Ingenix data, pooling data on generic and brand names for the same medication according to their relative proportion in the Ingenix data and summing all medications used (for reference, 20 mg of fluoxetine cost $2.20 per pill, on average).

Indirect costs of treatment include patient time costs for obtaining health care.²⁵ We assumed an average time of 30 and 45 minutes for outpatient medical and mental health visits, respectively, and added average travel and waiting times reported by patients at baseline. In addition, we assumed 3 hours for emergency department visits and 1.5 hours to fill prescriptions in a month of use. We priced patients' time using reported hourly wage at baseline and sex-specific mean wage for those not working at baseline (this may slightly overstate the value of time for nonworking patients).

Quality-Adjusted Life-Years. To measure quality-adjusted life-years (QALYs), we calculated a health utility index from the Short-Form, 12-Item Health Survey (SF-12) items collapsed into 6 health states that had been identified through cluster analysis of SF-12 physical and mental component scores.^26,27 Utility weights from this index were derived from a convenience sample of primary care patients with symptoms of depression using a standard gamble approach.²⁶ QALY weights were calculated for each 6-month follow-up time period, and we analyzed patterns over time. We call this measure "QALY-SF."

In addition, following an approach developed by Lave et al,²⁸ we developed a measure of depression-burden days and assigned utility scores from the literature to estimate QALYs.^28,29 We call this measure "QALY-DB." Specifically, for each survey from baseline through 24 months, we developed a count of positive scores on the following 3 measures: probable major depressive disorder, based on a repeat of the baseline screener¹⁹; significant depressive symptoms, based on a modified Center for Epidemiologic Studies Depression Studies (CES-D) scale^19,30; and poor mental health-related quality of life (HRQOL), based on being more than 1 SD below the population mean on the mental health subscale of the SF-12.³ We averaged the count for the beginning and end of each 6-month follow-up period and multiplied by 182 to estimate depression-burden days during the period. We summed across periods to get the 24-month total. We then used findings from the literature that a year of depression is associated with losses of 0.2 to 0.4 QALYs to convert the intervention effect on depression-burden days into the QALY-DB estimates.^9,31-34

Employment. We created a measure of days worked in each 6-month follow-up by taking the average of employment status at the start and end of each period and multiplying by 116 (the number of workdays in 6 months). We summed across periods to calculate the 24-month total. We also examined days missed from work due to illness, which patients reported for the 4 weeks preceding each follow-up survey.

All multivariate models controlled for baseline measures of patient age, sex, marital status, education, rank in the distribution of household wealth, employment status, ethnicity, medical comorbidity, depressive disorder status, the SF-12 aggregate HRQOL measures, presence of comorbid anxiety disorder, and practice randomization block.

To estimate the effects of practice-initiated QI on patients, we conducted patient-level intent-to-treat analyses, controlling for baseline patient differences that could remain after group-level randomization. We examined intervention effects on health care costs using 2-part models, due to the skewed distribution of costs. The first is the probability of positive costs, using logistic regression. The second is the log of costs given any, using ordinary least squares.³⁵ We used the smearing estimate for retransformation, applying separate factors for each intervention group to ensure consistent estimates.^36,37 We did not adjust cost models for clustering by clinic because we know of no existing software to do so for 2-part models. We expected the interventions to increase health care costs, relative to usual care; not accounting for clustering is thus conservative from a policy perspective, since not adjusting for clustering is likely to overstate the statistical significance of cost differences.

For the QALY-SF measure, we specified 3-level (repeated measurements nested within patients, and patients nested within clinics) mixed effects linear time-trend regression models, controlling for the baseline utility value in addition to the covariates listed above (except HRQOL). We calculated the area under the curve to derive values over 24 months. For days of depression burden and employment, respectively, we specified 2-level (patients nested within clinics) mixed effects linear regression models, to account for patient clustering at the practice level. For these outcomes, we examined the 24-month value directly.

Significance of comparisons across intervention groups is based on the regression coefficients. We illustrate average intervention effects relative to usual care, adjusted for patient characteristics using a direct method, ie, standardized predictions generated from each regression model. Specifically, we used the regression parameters and each individual's actual values for all covariates other than intervention status to calculate the predicted outcome assuming the patient had been assigned to usual care or to either intervention, respectively. We then calculated the mean prediction under each scenario.

We analyzed patients completing at least 1 follow-up (92% of the enrolled sample; N = 1248 total [422 in usual care, 393 in QI-meds, and 433 in QI-therapy]). The data are weighted for the probability of study enrollment and follow-up response to the characteristics of the eligible sample. We used multiple imputations for missing items at each wave.^38,39 For outcomes, we averaged predictions from 5 randomly imputed data sets and adjusted SEs for uncertainty due to imputation.^39,40

Because many tests are in the same direction as hypothesized, a formal Bonferroni correction for multiple statistical comparisons is too conservative, so we report actual P values and interpret results with multiple comparisons in mind.⁴¹

At baseline, the percentage of patients with a college education was lower for usual care (15.0%) than QI-meds (22.9%, P = .004) or QI-therapy (21.5%, P = .01). Usual care patients were more likely to have current symptoms with lifetime disorder (26.1%) compared with QI-meds (18.5%, P = .02) and QI-therapy (19.4%, P = .03) vs current disorder or symptoms but no lifetime disorder. QI-therapy patients were slightly older (by 3 years on average) (P = .02). Intervention and control patients did not differ with respect to the other baseline covariates listed above at P = .05.

Table 1 reports average per patient costs and outcomes over 24 months (including patient time costs, but not inpatient care and nonpsychotropic medications). Average total costs for usual care patients were estimated to be $3835, increasing by $419 (11%) among QI-meds participants and by $485 (13%) among QI-therapy participants. Neither intervention effect on total costs is statistically significant. Patient time costs represented 22% of the average total cost under usual care. Increases in time costs represented 3% of the incremental increase due to QI-meds and 25% of the incremental increase due to QI-therapy. The intervention effects on patient time costs were also statistically nonsignificant (details available from authors on request).

For the QALY-SF measure, the incremental increase due to QI-meds was 0.0115 QALYs over 24 months (P = .15), while the increase due to QI-therapy was 0.0226 (P = .006). Combining these point estimates with our point estimates of the incremental intervention costs yields an estimated cost per QALY of $36 467 for QI-meds and $21 478 for QI-therapy.

For the QALY-DB measure, we assumed that depression reduces the value of a life-year by 0.2 to 0.4 QALYs. ^9,28,31-34 Compared with usual care, QI-meds reduced the number of depression-burden days by 25 (P = .19), or 0.0137 to 0.0274 QALYs. QI-therapy yielded 47 fewer depression-burden days over 24 months (P = .01), or 0.0258 to 0.0515 QALYs. These point estimates yield a cost-per-QALY range of $15 331 to $30 663 for QI-meds and $9478 to $18 953 for QI-therapy.

As shown in Table 1, participants from QI-meds clinics had 17.9 more employed days relative to usual care over 24 months (P = .07), on average, and QI-therapy participants had 20.9 more employed days (P = .03). Intervention and usual care patients who were working did not differ substantively or statistically with respect to sick days at any follow-up period. For instance, intervention patients were significantly more likely to be working at the 12-month follow-up survey (65.7% in QI vs 60.8% in usual care; 95% confidence interval [CI] for difference: 0.01-0.09; P = .03). Among employed patients, however, the number of reported sick days in the previous 4 weeks was virtually identical (1.2 days in QI vs 1.1 in usual care; 95% CI for difference: − 0.5 to 0.6; P = .81). Results for other follow-up periods were similar.

We found that practice-initiated, locally implemented programs that encourage guideline-concordant care for depression can substantially reduce the individual suffering and economic consequences of depression. The point estimates for incremental costs per QALY relative to usual care were within the range of many accepted medical interventions^25,42 and substantially below the estimated value of a year of life.^25,43 Our data suggested that QI-therapy may have a better overall value in terms of cost per QALY than QI-meds, suggesting that there may be particular value to improving access to structured psychotherapy such as cognitive behavioral therapy for depressed primary care patients.

We found significant intervention effects on patients' labor supply, on the order of 1 additional month of employment over 2 years. In addition to its importance to patients, this result could suggest broader economic benefits of the intervention to families and purchasers—benefits that may not be fully captured in standard measures of QALYs, which are based on patients' HRQOL. If so, the true societal cost-effectiveness of the interventions may be more favorable than what we report.²⁵

This study has important limitations. For instance, we studied 6 practice networks, although they were chosen to be diverse. We relied on patient self-report for most measures, which would bias intent-to-treat analyses if the interventions affected patient reports. We had a relatively low enrollment rate, which we partially account for by weighting back to the eligible population. Despite a large sample size relative to most clinical trials, our cost estimates lacked precision.

Our findings suggest that practice-initiated interventions to improve quality of care for depression can substantially increase patients' and societal welfare, even when implemented locally and under flexible, naturalistic practice conditions that support patient and clinician treatment choices. These interventions increase costs for clinicians and insurers, suggesting that their widespread adoption may require increases in consumer demand or public policy initiatives that provide incentives for implementing them. But the gains observed in such an applied and real-world context suggest that improved medical care has much to offer depressed patients and their families and communities if we can create the conditions necessary to put such programs in place.