Customize your JAMA Network experience by selecting one or more topics from the list below.
Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of Disseminating Quality Improvement Programs for Depression in Managed Primary Care: A Randomized Controlled Trial. JAMA. 2000;283(2):212–220. doi:10.1001/jama.283.2.212
Author Affiliations: RAND, Health Program, Santa Monica, Calif (Drs Wells, Sherbourne, Schoenbaum, Duan, Meredith, Rubenstein, and Ms Carney); UCLA-Neuropsychiatric Institute and Department of Psychiatry and Biobehavioral Sciences (Drs Wells and Unützer) and Department of Medicine (Dr Rubenstein), School of Medicine, University of California, Los Angeles; Department of Psychiatry, Georgetown University Medical Center, Washington, DC (Dr Miranda); and Department of Medicine, VA Greater Los Angeles Healthcare System, Los Angeles, Calif (Dr Rubenstein).
Context Care of patients with depression in managed primary care settings often
fails to meet guideline standards, but the long-term impact of quality improvement
(QI) programs for depression care in such settings is unknown.
Objective To determine if QI programs in managed care practices for depressed
primary care patients improve quality of care, health outcomes, and employment.
Design Randomized controlled trial initiated from June 1996 to March 1997.
Setting Forty-six primary care clinics in 6 US managed care organizations.
Participants Of 27,332 consecutively screened patients, 1356 with current depressive
symptoms and either 12-month, lifetime, or no depressive disorder were enrolled.
Interventions Matched clinics were randomized to usual care (mailing of practice guidelines)
or to 1 of 2 QI programs that involved institutional commitment to QI, training
local experts and nurse specialists to provide clinician and patient education,
identification of a pool of potentially depressed patients, and either nurses
for medication follow-up or access to trained psychotherapists.
Main Outcome Measures Process of care (use of antidepressant medication, mental health specialty
counseling visits, medical visits for mental health problems, any medical
visits), health outcomes (probable depression and health-related quality of
life [HRQOL]), and employment at baseline and at 6- and 12-month follow-up.
Results Patients in QI (n = 913) and control (n = 443) clinics did not differ
significantly at baseline in service use, HRQOL, or employment after nonresponse
weighting. At 6 months, 50.9% of QI patients and 39.7% of controls had counseling
or used antidepressant medication at an appropriate dosage (P<.001), with a similar pattern at 12 months (59.2% vs 50.1%; P = .006). There were no differences in probability of
having any medical visit at any point (each P≥.21).
At 6 months, 47.5% of QI patients and 36.6% of controls had a medical visit
for mental health problems (P = .001), and QI patients
were more likely to see a mental health specialist at 6 months (39.8% vs 27.2%; P<.001) and at 12 months (29.1% vs 22.7%; P = .03). At 6 months, 39.9% of QI patients and 49.9% of controls still
met criteria for probable depressive disorder (P
= .001), with a similar pattern at 12 months (41.6% vs 51.2%; P = .005). Initially employed QI patients were more likely to be working
at 12 months relative to controls (P = .05).
Conclusions When these managed primary care practices implemented QI programs that
improve opportunities for depression treatment without mandating it, quality
of care, mental health outcomes, and retention of employment of depressed
patients improved over a year, while medical visits did not increase overall.
Quality of care for depression in managed primary care settings is moderate
to poor1,2 with resultant poor
outcomes.3 Because depression is expected
to be the second leading cause of disability worldwide after the turn of the
century,4 improving care for this prevalent
disorder could enhance quality of life and productivity for many primary care
Clinical trials have established the efficacy of antidepressant medications
and specific psychotherapies for depressed psychiatric and primary care patients
when care is provided by trained research personnel or under standardized
Trying to improve care through routine screening with feedback to clinicians
on patients' depression has little effect, and an early study of implementing
practice guidelines for depressed primary care patients found no outcome benefits.10 However, Katon and colleagues11,12
demonstrated that collaborative care models involving psychologists (providing
short-term psychotherapy) and psychiatrists (consulting on medications) improve
treatment rates and clinical outcomes for up to 9 months of follow-up, but
lasting practice impact may depend on the continued participation of the research
team.13 What are the impacts of quality improvement
(QI) programs for depression when disseminated to managed primary care settings
under naturalistic conditions that include reliance on usual care providers
and free choice of treatment by clinicians and patients?
In this study, we examine this question using a randomized controlled
trial of QI dissemination vs usual care. The interventions were based on literature
suggesting that provider change is more likely when the following occur: (1)
social-influence methods, such as local expert leaders, are used; (2) feedback
on performance is provided; (3) clinician workload is shifted to ancillary
staff; and (4) multiple practice-change strategies are used.14-16
The study fielded 2 similar interventions, one with enhanced resources for
supporting medication management (QI-meds) and the other with enhanced resources
for providing psychotherapy for depression (QI-therapy). Both had the same
goal of improving rates of initiation and adherence to either appropriate
antidepressant medication or psychotherapy for depression, and both were based
on the same QI methods. The study was designed to test the primary hypothesis
that dissemination of QI for depression (the combined interventions) under
naturalistic conditions will improve quality of care and health-related outcomes
over a follow-up year. We were less certain that QI would affect physical
health and employment status within 1 year and consider analyses of these
outcomes as exploratory. The study has a secondary aim of determining whether
QI interventions relying on enhanced resources for medication management or
for psychotherapy are each effective. Practice guidelines emphasize the effectiveness
of both modalities,5,6,17
but primary care practitioners are less familiar with psychotherapy and its
insurance coverage has become limited.18 We
hypothesized that each intervention is effective and provide a preliminary
analysis based on 1 year of data in this article. In addition, we test whether
the combined interventions were effective for patients with either depressive
disorder or depressive symptoms without disorder, a common condition in practice
for which there is limited evidence for treatment efficacy.11,12,15,19-24
The data are from Partners in Care, an Agency for Healthcare Research
and Quality (formerly the Agency for Health Care Policy and Research) Patient
Outcomes Research Team (PORT-II).25 The study
is a group-level randomized controlled trial.26
We recruited 6 managed care organizations, selected to be diverse in
geography and organization and to oversample Mexican Americans. The organizations
included staff and network model multispecialty group practices and rural
managed public health clinics (Table 1).
Patients had prepaid or managed fee-for-service (including Medicare and Medicaid)
coverage. Uninsured patients were included in 1 site. We approached 1 or 2
regions per organization; 1 region refused and 7 agreed. Of the 48 primary
care clinics with at least 2 clinicians, 46 participated.
We grouped clinic clusters into matched blocks of 3, based on patient
demographics, clinician specialty, and distance to mental health providers.
We created 9 blocks, 1 each for 6 regions and 3 for the seventh (stratified
by low, intermediate, or high percentage of Mexican American patients). Within
blocks, we used a random number table to assign clusters to usual care or
We asked all primary care clinicians to enroll prior to their knowledge
of intervention status; 97% (n = 181) agreed, including internists and family
practice physicians (87%) and nurse practitioners (13%).
Study staff screened consecutive patient visitors over a 5- to 7-month
period between June 1996 and March 1997. Patients were eligible if they were
depressed and intended to use the clinic as a source of care for the next
12 months. Patients were ineligible if they were younger than 18 years, had
an acute medical emergency, did not speak English or Spanish, or did not have
either insurance or a public-pay arrangement that covered care delivered by
the mental health specialists in the interventions (this criterion excluded
all fee-for-service patients in 1 site and half of them in another). In the
pilot month for the first site, patients screening positive for bipolar disorder
or alcoholism were excluded, but not for the main study. Patients learned
of their intervention status after enrollment.
The patient screener included the "stem" items for major depressive
and dysthymic disorders from the 12-month Composite International
Diagnostic Interview (CIDI)1,27
and items assessing depressed symptoms in the past month. We defined patients
as having probable disorder if they had 2 weeks or more of depressed mood
or loss of interest in pleasurable activities during the last year or persistent
depression over the year, plus having at least 1 week of depression in the
last 30 days. Based on a sample of 1485 study patients who received the full CIDI affective disorders section, the positive predictive
value of this indicator is 55%.25 By definition,
sensitivity is 100%, as the CIDI requires a positive
stem item to score a diagnosis.
The study approached 44,052 persons, but 10,120 were ineligible, mainly
because they were not patients of study providers (Figure 1). Of 27,332 patients completing the screener (85% of those
approached and eligible), 3918 were potentially eligible. Many left during
enrollment, 2417 were present to confirm insurance eligibility, and 241 had
ineligible insurance. Of those who read the informed consent, 1356 enrolled
and the remaining 21% either refused or left the clinic. We developed weights
that adjust for differential probability of enrollment and include factors
predicting enrollment as covariates in analyses.
Enrolled patients completed an interview including the affective disorders
section of the CIDI; a telephone interview on anxiety
disorder, income, wealth, and employment; and a mail survey. We obtained follow-up
surveys every 6 months for 2 years. One-year data are now available for analyses.
By 1 year, 59 patients (4.4%) had dropped out and 4 (0.3%) died. Patients
remained enrolled regardless of survey response or treatment participation.
Baseline factors associated with remaining enrolled were being younger or
residing in 4 of the sites; 1 intervention (QI-therapy) had somewhat lower
panel retention relative to usual care (P<.05).
Response rates were 95% for the telephone interview and 88% for the survey
at baseline, and 85% and 83% for the 6- and 12-month surveys. Among those
enrolled at 12 months, 12-month survey response rates were higher for patients
who were female, older, married, white, or college educated, and in 1 site.
We weighted the data for probability of enrollment and survey response and
control for predictors of retention and response in analyses.
The study was approved by the institutional review boards of all participating
institutions as well as RAND. Informed consent was obtained from all patients
The conditions were usual care, QI-meds, and QI-therapy (combined to
test our primary hypothesis). Patients and providers in all conditions had
full choice of treatment and, in interventions, over use of study resources.
For usual care, clinic medical directors were mailed the Agency for Healthcare
Research and Quality depression practice guidelines, with quick reference
guides for clinicians. The interventions were designed by the study team but
implemented by the organizations with study support. Patient screening for
probable depression was conducted by study staff. The intervention goal was
to increase the percentage of clinically depressed patients receiving any
appropriate care according to national guidelines.28
The interventions use a basic QI model but different supplemental resources
(Figure 2). The basic QI model has
4 components. The first is institutional commitment. The practices agreed
to reserve in-kind resources amounting to one half the estimated costs of
implementing the interventions and their time costs for study participation;
the other half ($30,000-$72,000, depending on practice size) was paid by the
study. In 1 site, the affiliated managed carve-out behavioral health plan
paid all practice costs. The second component is training local leaders, including
a primary care clinician, a nursing supervisor, and a mental health specialist,
to implement the interventions. The study provided a 2-day workshop that reviewed
the study's manualized model of treatment and principles of collaborative
care.29,30 Prior to patient recruitment,
these leaders distributed clinician manuals, initiated monthly lectures (80%
of primary care clinicians attended at least 1 lecture), and provided academic
detailing as needed (provided to 48% of clinicians). In addition, based on
monthly team meetings using intervention staff records, the leaders provided
audit with feedback on the clinic or individual clinician level (provided
to 60% of clinicians). The third component is study training of local staff.
Nurses were trained in a 1-day workshop to provide brief clinical assessments,
patient education, and activation based on a written manual and videotape.
The local leaders supervised nurses and therapists, and the study reviewed
progress with leaders in monthly telephone calls. Supporting materials developed
by the study were provided to the local experts. These included lecture slides,
manuals, and pocket-reference cards for clinician education; pamphlets and
videotapes for patient education; and study charts and forms for nurses and
therapists to note contacts and provide feedback to patients and clinicians.
The fourth component is patient identification. Intervention clinics were
provided with lists of their study patients, while usual care clinic patients
were told they could inform their provider, but the study did not notify their
The nurses were asked to contact intervention patients for an initial
visit (assessment, education, and activation) 2 weeks after the screening
visit. The primary care clinician was asked to consider this information in
formulating a treatment plan with the patient. Nurse visits had no co-payment.
In the QI-meds intervetion, the nurse specialists were trained to provide
follow-up assessments and support adherence through monthly contacts for 6
or 12 months (randomized at the patient level). Patients in the QI-meds intervention
had access to usual practice therapists, but not the study cognitive behavioral
In the QI-therapy intervention, the study trained local psychotherapists
to provide manualized individual and group CBT31
for 12 to 16 sessions. The organizations reduced the therapy co-payment to
the primary care visit co-payment (ie, $0-$10 instead of $20-$30). Patients
could access other therapy for the usual co-payment. Brief (4-session) CBT
was suggested as an option for patients with minor depression.
Health Outcomes. (1) We assessed depressive symptoms at baseline and follow-up using
a 23-item version of the Center for Epidemiologic Studies Depression (CES-D)
scale,32 developed by Daniel Ford (written
communication, March 1996). This version drops 6 items and adds others to
approximate Diagnostic and Statistical Manual of Mental
Disorders, Fourth Edition (DSM-IV) criteria.
Item responses were summed. We fielded the expanded and traditional versions
in the same survey and then used the Samejima graded Item Response Theory
model33 to determine that a cut point of 20
is equivalent to the standard cut point of 16 for identifying probable depression.
We used an indicator for being above or below that cut point. The power calculations
for the study were based on expecting a 10% to 15% difference by intervention
status on this measure. (2) On each follow-up survey, we repeated the screener
measure of probable disorder, referring to the prior 6 months and deleting
the dysthymia stem item. (3) Health-related quality of life (HRQOL) was measured
in the screener and follow-up by the physical and mental health summary scales
from the Short-Form 12 (SF-12).34,35
Scores were normalized to a general population mean (SD) of 50 (10), with
higher scores indicating better health. We conducted an analysis using a physical
limitations score derived from the SF-12 items, with no change in conclusions
over the summary scale.
Health Service Utilization. The surveys assessed the total medical visits, medical visits for emotional
problems, and mental health specialty visits during the prior 6 months. We
developed indicators of each type of visit and counts of visits among users.
To reduce skewness, we transformed counts (cubic root for total visits, negative
reciprocal of square root for medical visits for emotional problems, and log
transformation for specialty visits).
Quality of Care. Each survey assessed prescription medications used in the prior 30 days or for a month or more in the prior 6 months. To assess appropriate use of
antidepressant medication in the prior 6 months, we applied dosage criteria
from national guidelines to each medication used.5,6,36,37
We developed an indicator of whether the respondent had individual, group,
or family counseling by a specialist in the past 6 months. Since patients
might not receive care if they improve, we created an indicator of appropriate
care that requires that patients were free of probable disorder or had either
specialty counseling or use of appropriate antidepressant medication during
a given 6-month period.
Employment. We used items from the screener and the telephone survey to assess whether
respondents were employed at baseline. The follow-up surveys assessed current
Covariates. From the patient screener, we measured age (and when appropriate, age
squared), sex, education (less than high school, completed high school, some
college, completed college or more), and a count of having 0, 1, 2, or more
than 2 chronic medical conditions, from a total of 19. We used data from the
screener and CIDI to categorize patients as having
30-day symptoms plus either 12-month (major depressive or dysthymic) disorder,
lifetime disorder only, or no disorder (ie, symptoms only). We identified
probable lifetime disorder from a supplemental CIDI
item. Using items modeled after the Health and Retirement Study,38
we developed a household-wealth variable, summing the net value of home and
We conducted patient-level, intent-to-treat analyses. To test the primary
hypotheses for each dependent variable, we estimated multivariate regression
models with combined intervention status as the main independent variable,
with the covariates listed above plus global HRQOL at the screener. For analyses
using the baseline survey, we also included an indicator of whether that survey
was completed within 30 days of the screener. For dichotomous outcomes, we
estimated logistic regression models. For levels of visits among users, we
estimated linear models using transformed counts of visits. For continuous
HRQOL outcomes, we conducted ordinary least squares regression on untransformed
scores and analyzed end status and change scores; results were comparable,
so we present end status for ease of interpretation. To test secondary hypotheses,
we interacted the combined intervention status with disorder status (specified
as 12-month or lifetime disorder vs symptoms only) and separate indicators
for QI-meds and QI-therapy, each relative to usual care. The patients are
clustered within providers and clinics, but the intraclass correlations are
close to zero, so we report results unadjusted for cluster effects.
We present standardized predictions of the intervention effects for
each outcome. We used the regression parameters and each individual's actual
value for all covariates, other than intervention status, to generate a predicted
value for each individual, first as an intervention subject and then as a
control subject. We then averaged the intervention and control predictions.
Data are weighted for the probability of enrollment, attrition, and wave response.
We used multiple imputation for missing data at the item level. We imputed
5 data sets, averaged predictions, and adjusted SEs for uncertainty due to
Because we report many results in the same direction as hypothesized, a formal
Bonferroni42 correction for multiple statistical
comparisons is too conservative. We report actual P
values and interpret the results with multiple comparisons in mind, but apply
a less strict standard for secondary hypotheses.
The enrolled sample included 443 usual care and 913 intervention patients.
The sample was clinically and sociodemographically diverse (Table 2). About half had 12-month depressive disorder, 71% were
women, and the mean (SD) age was 44 (15) years; 57% of the sample was non-Hispanic
white, and 30% was Hispanic. Intervention patients were somewhat older and
more likely to have a college education. While controls were less likely to
have current disorder, they were more likely to have lifetime disorder; patients
did not differ by intervention status in the percentage with symptoms only
vs lifetime or current disorder (t = 0.19, P = .85).
Intervention and control patients did not differ at baseline in the
probability of having any type of visit (Table 3). Visit counts among users did not differ by intervention
status for any time period. Intervention patients were 20% more likely to
have a medical visit for emotional problems in the first 6 months, but not
in the second period. Intervention patients were 30% to 40% more likely to
have a mental health specialty visit over each follow-up period. There were
no significant differences in probability of any medical visit.
The interventions increased by 30% to 40% the rates of counseling and
of appropriate use of antidepressant medication in the first 6 months and
by 20% to 30% in the second period (Table
4). The effects on counseling and on medications in the first 6
months were largely due to new treatment among those not treated at baseline.
In the second period, the interventions increased medication use among those
initially medicated, ie, continuation therapy. The interventions increased
the probability of appropriate care in each follow-up period by about 10 percentage
points (each P<.001).
Intervention patients were less likely to have probable depression at
6- and 12-month follow-up by 7 to 10 percentage points. Intervention patients
improved more on mental HRQOL by 1 to 2 points at each time point, but not
global physical health (Table 5)
or physical limitations (not shown).
Among patients initially employed, 89.7% of intervention patients and
84.7% of control patients worked at 12 months (t
= 1.96, P = .05). Among those initially not working,
16.4% of intervention patients and 11.4% of control patients were working
at 6 months (P>.10); by 12 months, 17% to 18% of
intervention and control patients started working.
The interventions reduced the likelihood of probable disorder in the
first 6 months by 10 percentage points, relative to usual care, among those
with baseline 12-month or lifetime disorder (t =
3.02, P = .003); in the second 6 months, the likelihood
was reduced by 19 percentage points among those with symptoms only at baseline
(t = 2.60, P = .009). The
interaction of intervention by disorder status was2 borderline at 12 months
(t = 1.66, P = .096), with
a stronger QI benefit for patients with symptoms only. In the first 6 months,
QI increased appropriate care among those with 12-month or lifetime disorder
at baseline (a 17 percentage-point increase over usual care, t = 4.64, P<.001), but not among patients
with symptoms only (3 percentage-point decrease). The interaction was significant
(t = 2.26, P = .02). In
the second period, QI increased appropriate care similarly (by 9-10 percentage
points relative to usual care) among those with disorder or symptoms only.
QI-therapy and QI-meds patients, each compared with controls, were 8
to 10 percentage points less likely to have probable disorder at 6 and 12
months (lowest t = 2.15, P
= .03). At 6 months, rates of appropriate care were greater by 14 percentage
points for QI-meds patients than controls (t = 3.81, P<.001) and by 8 percentage points for QI-therapy patients
than controls (t = 2.37, P
= .002). At 12 months, QI-meds patients had higher rates of appropriate care
than did controls (t = 3.60, P<.001) or QI-therapy patients (t = 2.31, P = .02), who did not differ significantly from controls.
At each follow-up, QI-meds patients had higher rates than controls of appropriate
medication use (by 10-14 percentage points, lowest t
= 3.25, P = .001) and any specialty counseling (by
10-12 percentage points, lowest t = 3.00, P = .003). At 6 months, relative to controls, QI-therapy patients were
13 percentage points more likely to have any specialty counseling (t = 3.71, P<.001) and were 5 percentage
points more likely to use appropriate antidepressant medication (t = 1.68, P = .09). The increases in appropriate
medication use were greater for QI-meds than for QI-therapy patients at both
follow-ups (lowest t = 2.07, P = .04), and QI-meds patients were more likely than QI-therapy patients
to have any specialty counseling in the second follow-up (t = 2.07, P = .04).
We found that when diverse nonacademic managed primary care practices
implemented QI programs, quality of care, clinical outcome, mental health–related
functioning, and retention in employment of depressed patients improved over
a year. The investment required by practices to implement the interventions
was modest and typical for QI projects. The study supported identification
of patients with probable depression so practices would have to screen patients
to implement the intervention independently. The changes in process we observed
were modest, but some benefits, such as a 20% to 30% improvement in rates
of probable depression, were substantial. The improvement in HRQOL, however,
is small, but could reflect substantial societal impact when accumulated nationally.
The persistence of benefits is longer than that observed in most treatment-efficacy
studies.5,6 The potential for
policy impact is suggested by a 5 percentage-point increase in employment
retention, which may be important for employers given the high prevalence
of depressive disorders and symptoms in community samples.43
To our knowledge, no QI study has demonstrated improved employment, although
perceived interpersonal work functioning improves with efficacious treatment
for major depression.44
The interventions improved outcomes for persons with 12-month or lifetime
disorder, as well as those with symptoms only. We found early benefit for
those with disorder and delayed benefit for those with symptoms only. Prior
studies suggest modest benefits from treating minor depression.15,19-24
The interventions also increased rates of appropriate care for the disorder
group in the first 6 months and for those with disorder or symptoms only in
the second 6 months. Thus, the delayed outcome benefit for the symptoms-only
group could be due to treatment in the second 6 months as symptoms continued
or a disorder developed. While nonspecific intervention aspects could explain
this effect, we think this is an unlikely explanation for prolonged benefits.
The interventions only transiently increased the probability of having
a medical visit for mental health or emotional care without increasing total
medical visits, and with a sustained increase in use of specialists. Thus,
additional primary care visits may not be necessary to improve depression
care. Instead, a collaborative partnership, such as has been found to improve
quality and outcomes of care in other studies, may have developed.11,12 We doubt that the interventions resulted
in lasting practice change, as Lin and her colleagues found little enduring
practice effect from more intensive interventions.13
Nevertheless, we observed relatively long-term impacts on patients.
In an era of declining insurance benefits for psychotherapy, it is noteworthy
that interventions improving resources for either medication management or
psychotherapy were beneficial for primary care depressed patients across diverse
practices using internal or carve-out specialty groups. The medication-resource
intervention had more pronounced and sustained effects on our process indicators
than did the psychotherapy-resource intervention, which could lead to outcome
differences in subsequent years. While the relatively greater increase in
appropriate medication use for patients in the medication-resource intervention
compared with the therapy-resource intervention is not surprising, the greater
rate of specialty counseling in the second follow-up may be, but this could
simply reflect ongoing medication management by specialists. The type and
content of therapy may well differ for the 2 interventions due to the CBT
training that was limited to the therapy-resource intervention.
This dissemination trial is similar to a social experiment, in that
it replicates naturalistic practice conditions, including usual care providers
and full choice of treatments.45 The diversity
of organizations and patients in our study suggests broad potential applicability,
which we cannot formally address with a limited number of organizations. Whether
practices will want to use these interventions may depend on their cost-effectiveness
relative to usual care. Our study has other important limitations. There was
sample loss during enrollment, but only somewhat more so than in an observational
study.1 The direction of bias is for worse
outcomes for the intervention subjects, so we could understate the interventions'
benefits. We rely on self-report, but mostly for measures that are validated
or easy to report, like employment.
Overall, our findings suggest that provider groups and managed care
organizations may benefit patients, employers, and society through modest
efforts to improve the quality of care for depression while retaining patient
and provider treatment choice.