[Skip to Navigation]
Sign In
Figure. Flow of Patients
Figure. Flow of Patients

CBT indicates cognitive behavioral therapy.

Table 1. Baseline Patient Characteristicsa
Table 1. Baseline Patient Characteristicsa
Table 2. Anxiety Self-reported Quality of Carea
Table 2. Anxiety Self-reported Quality of Carea
Table 3. Proportion Achieving Response and Remission From Baseline BSI-12 Scorea
Table 3. Proportion Achieving Response and Remission From Baseline BSI-12 Scorea
1.
Westfall JM, Mold J, Fagnan L. Practice-based research: ”Blue Highways” on the NIH roadmap.  JAMA. 2007;297(4):403-40617244837PubMedGoogle ScholarCrossref
2.
Unützer J, Katon W, Callahan CM,  et al; IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment.  Collaborative care management of late-life depression in the primary care setting.  JAMA. 2002;288(22):2836-284512472325PubMedGoogle ScholarCrossref
3.
Wells KB, Sherbourne CD, Schoenbaum M,  et al.  Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial.  JAMA. 2000;283(2):212-22010634337PubMedGoogle ScholarCrossref
4.
Simon GE, VonKorff M, Rutter C, Wagner E. Randomized trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care.  BMJ. 2000;320(7234):550-55410688563PubMedGoogle ScholarCrossref
5.
Katon W, Robinson P, Von Korff M,  et al.  A multifaceted intervention to improve treatment of depression in primary care.  Arch Gen Psychiatry. 1996;53(10):924-9328857869PubMedGoogle ScholarCrossref
6.
Roy-Byrne PP, Craske MG, Stein MB,  et al.  A randomized effectiveness trial of cognitive-behavioral therapy and medication for primary care panic disorder.  Arch Gen Psychiatry. 2005;62(3):290-29815753242PubMedGoogle ScholarCrossref
7.
Rollman BL, Belnap BH, Mazumdar S,  et al.  A randomized trial to improve the quality of treatment for panic and generalized anxiety disorders in primary care.  Arch Gen Psychiatry. 2005;62(12):1332-134116330721PubMedGoogle ScholarCrossref
8.
Stein MB, Sherbourne CD, Craske MG,  et al.  Quality of care for primary care patients with anxiety disorders.  Am J Psychiatry. 2004;161(12):2230-223715569894PubMedGoogle ScholarCrossref
9.
Mojtabai R, Olfson M, Mechanic D. Perceived need and help-seeking in adults with mood, anxiety, or substance use disorders.  Arch Gen Psychiatry. 2002;59(1):77-8411779286PubMedGoogle ScholarCrossref
10.
Grilo CM, Money R, Barlow DH,  et al.  Pretreatment patient factors predicting attrition from a multicenter randomized controlled treatment study for panic disorder.  Compr Psychiatry. 1998;39(6):323-3329829138PubMedGoogle ScholarCrossref
11.
Hazlett-Stevens H, Craske MG, Roy-Byrne PP, Sherbourne CD, Stein MB, Bystritsky A. Predictors of willingness to consider medication and psychosocial treatment for panic disorder in primary care patients.  Gen Hosp Psychiatry. 2002;24(5):316-32112220797PubMedGoogle ScholarCrossref
12.
Nadiga DN, Hensley PL, Uhlenhuth EH. Review of the long-term effectiveness of cognitive behavioral therapy compared to medications in panic disorder.  Depress Anxiety. 2003;17(2):58-6412621593PubMedGoogle ScholarCrossref
13.
Heuzenroeder L, Donnelly M, Haby MM,  et al.  Cost-effectiveness of psychological and pharmacological interventions for generalized anxiety disorder and panic disorder.  Aust N Z J Psychiatry. 2004;38(8):602-61215298582PubMedGoogle ScholarCrossref
14.
Sullivan G, Craske MG, Sherbourne C,  et al.  Design of the Coordinated Anxiety Learning and Management (CALM) study: innovations in collaborative care for anxiety disorders.  Gen Hosp Psychiatry. 2007;29(5):379-38717888803PubMedGoogle ScholarCrossref
15.
Chilvers C, Dewey M, Fielding K,  et al; Counselling versus Antidepressants in Primary Care Study Group.  Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms.  BMJ. 2001;322(7289):772-77511282864PubMedGoogle ScholarCrossref
16.
Unützer J, Choi Y, Cook IA, Oishi S. A web-based data management system to improve care for depression in a multicenter clinical trial.  Psychiatr Serv. 2002;53(6):671-673, 67812045303PubMedGoogle ScholarCrossref
17.
Craske MG, Rose RD, Lang A,  et al.  Computer-assisted delivery of cognitive behavioral therapy for anxiety disorders in primary-care settings.  Depress Anxiety. 2009;26(3):235-24219212970PubMedGoogle ScholarCrossref
18.
Means-Christensen AJ, Sherbourne CD, Roy-Byrne PP, Craske MG, Stein MB. Using five questions to screen for five common mental disorders in primary care: diagnostic accuracy of the Anxiety and Depression Detector.  Gen Hosp Psychiatry. 2006;28(2):108-11816516060PubMedGoogle ScholarCrossref
19.
Sheehan DV, Lecrubier Y, Sheehan KH,  et al.  The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.  J Clin Psychiatry. 1998;59:(suppl 20)  22-33, quiz 34-579881538PubMedGoogle Scholar
20.
Campbell-Sills L, Norman SB, Craske MG,  et al.  Validation of a brief measure of anxiety-related severity and impairment: the Overall Anxiety Severity and Impairment Scale (OASIS).  J Affect Disord. 2009;112(1-3):92-10118486238PubMedGoogle ScholarCrossref
21.
Roy-Byrne P, Veitengruber JP, Bystritsky A,  et al.  Brief intervention for anxiety in primary care patients.  J Am Board Fam Med. 2009;22(2):175-18619264941PubMedGoogle ScholarCrossref
22.
Tsao JC, Mystkowski JL, Zucker BG, Craske MG. Impact of cognitive-behavioral therapy for panic disorder on comorbidity: a controlled investigation.  Behav Res Ther. 2005;43(7):959-97015896289PubMedGoogle ScholarCrossref
23.
Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses.  Clin Psychol Rev. 2006;26(1):17-3116199119PubMedGoogle ScholarCrossref
24.
Weissman MM, Verdeli H, Gameroff MJ,  et al.  National survey of psychotherapy training in psychiatry, psychology, and social work.  Arch Gen Psychiatry. 2006;63(8):925-93416894069PubMedGoogle ScholarCrossref
25.
Derogatis L. BSI-18: Brief Symptom Inventory 18 Administration, Scoring, and Procedures Manual. Minneapolis, MN: NCS Pearson Inc; 2001
26.
Simon GE, Katon WJ, VonKorff M,  et al.  Cost-effectiveness of a collaborative care program for primary care patients with persistent depression.  Am J Psychiatry. 2001;158(10):1638-164411578996PubMedGoogle ScholarCrossref
27.
Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure.  J Gen Intern Med. 2001;16(9):606-61311556941PubMedGoogle ScholarCrossref
28.
Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the predictions of fearfulness.  Behav Res Ther. 1986;24(1):1-83947307PubMedGoogle ScholarCrossref
29.
Sheehan DV, Harnett-Sheehan K, Raj BA. The measurement of disability.  Int Clin Psychopharmacol. 1996;11:(suppl 3)  89-958923116PubMedGoogle ScholarCrossref
30.
Centers for Disease Control and Prevention.  Measuring Healthy Days. Atlanta, GA: Centers for Disease Control and Prevention; 2000
31.
Ware JE Jr, Kosinski M, Bowker D, Gandek B. How to Score Version 2 of the SF-12 Health Survey (With a Supplement Documenting Version 1). Lincoln, RI: Quality Metric Inc; 2002
32.
Steketee G, Perry JC, Goisman RM,  et al.  The psychosocial treatments interview for anxiety disorders: a method for assessing psychotherapeutic procedures in anxiety disorders.  J Psychother Pract Res. 1997;6(3):194-2109185065PubMedGoogle Scholar
33.
Littell RC, Milliken GA, Stroup WW, Wolfinger RD. SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996
34.
Little R, Rubin D. Statistical Analysis With Missing Data. New York, NY: John Wiley & Sons; 1987
35.
Brick JM, Kalton G. Handling missing data in survey research.  Stat Methods Med Res. 1996;5(3):215-2388931194PubMedGoogle ScholarCrossref
36.
Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance.  Biometrika. 1988;75(4):800-802Google ScholarCrossref
37.
Nuovo J, Melnikow J, Chang D. Reporting number needed to treat and absolute risk reduction in randomized controlled trials.  JAMA. 2002;287(21):2813-281412038920PubMedGoogle ScholarCrossref
38.
Moore A. What is a number needed to treat? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/NNT.pdf. Accessed December 1, 2009
39.
Feeny NC, Zoellner LA, Mavissakalian MR, Roy-Byrne PP. What would you choose? sertraline or prolonged exposure in community and PTSD treatment seeking women.  Depress Anxiety. 2009;26(8):724-73119544313PubMedGoogle ScholarCrossref
40.
Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication.  Arch Gen Psychiatry. 2005;62(6):617-62715939839PubMedGoogle ScholarCrossref
Original Contribution
May 19, 2010

Delivery of Evidence-Based Treatment for Multiple Anxiety Disorders in Primary Care: A Randomized Controlled Trial

Author Affiliations

Author Affiliations: Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine and Harborview Center for Healthcare Improvement for Addictions, Mental Illness, and Medically Vulnerable Populations (CHAMMP), Seattle, Washington (Drs Roy-Byrne and Welch); Departments of Psychology (Drs Craske and Rose) and Psychiatry and Biobehavioral Sciences (Dr Bystritsky), David Geffen School of Medicine, University of California, Los Angeles; Department of Psychiatry (Drs Sullivan and Edlund) and VA South Central Mental Illness Research, Education, and Clinical Center (Dr Sullivan), University of Arkansas for Medical Sciences, Little Rock; Departments of Psychiatry (Drs Lang, Chavira, Campbell-Sills, and Stein) and Family and Preventive Medicine (Dr Stein), University of California, San Diego; VA San Diego Health Care System Center of Excellence for Stress and Mental Health, San Diego, California (Dr Lang); and RAND Corporation, Santa Monica, California (Drs Sullivan, Golinelli, and Sherbourne).

JAMA. 2010;303(19):1921-1928. doi:10.1001/jama.2010.608
Abstract

Context Improving the quality of mental health care requires moving clinical interventions from controlled research settings into real-world practice settings. Although such advances have been made for depression, little work has been performed for anxiety disorders.

Objective To determine whether a flexible treatment-delivery model for multiple primary care anxiety disorders (panic, generalized anxiety, social anxiety, and posttraumatic stress disorders) would be better than usual care (UC).

Design, Setting, and Patients A randomized controlled effectiveness trial of Coordinated Anxiety Learning and Management (CALM) compared with UC in 17 primary care clinics in 4 US cities. Between June 2006 and April 2008, 1004 patients with anxiety disorders (with or without major depression), aged 18 to 75 years, English- or Spanish-speaking, were enrolled and subsequently received treatment for 3 to 12 months. Blinded follow-up assessments at 6, 12, and 18 months after baseline were completed in October 2009.

Intervention CALM allowed choice of cognitive behavioral therapy (CBT), medication, or both; included real-time Web-based outcomes monitoring to optimize treatment decisions; and a computer-assisted program to optimize delivery of CBT by nonexpert care managers who also assisted primary care clinicians in promoting adherence and optimizing medications.

Main Outcome Measures Twelve-item Brief Symptom Inventory (BSI-12) anxiety and somatic symptoms score. Secondary outcomes included proportion of responders (≥50% reduction from pretreatment BSI-12 score) and remitters (total BSI-12 score <6).

Results A significantly greater improvement for CALM vs UC in global anxiety symptoms was found (BSI-12 group mean differences of −2.49 [95% confidence interval {CI}, −3.59 to −1.40], −2.63 [95% CI, −3.73 to −1.54], and −1.63 [95% CI, −2.73 to −0.53] at 6, 12, and 18 months, respectively). At 12 months, response and remission rates (CALM vs UC) were 63.66% (95% CI, 58.95%-68.37%) vs 44.68% (95% CI, 39.76%-49.59%), and 51.49% (95% CI, 46.60%-56.38%) vs 33.28% (95% CI, 28.62%-37.93%), with a number needed to treat of 5.27 (95% CI, 4.18-7.13) for response and 5.50 (95% CI, 4.32-7.55) for remission.

Conclusion For patients with anxiety disorders treated in primary care clinics, CALM compared with UC resulted in greater improvement in anxiety symptoms, depression symptoms, functional disability, and quality of care during 18 months of follow-up.

Trial Registration clinicaltrials.gov Identifier: NCT00347269

Improving the quality of mental health care requires continued efforts to move evidence-based treatments of proven efficacy into real-world practice settings with wide variability in patient characteristics and clinician skill.1 The effectiveness of one approach, collaborative care, is well established for primary care depression,2-5 but has been infrequently studied for anxiety disorders,6,7 despite their common occurrence in primary care.8 The multiplicity of anxiety disorders and the fact that anxious patients are less likely to seek9 and harder to engage10 in treatment may be contributing factors. Furthermore, whereas effective treatment for both anxiety and depressive disorders relies in part on pharmacotherapy, psychosocial treatments such as cognitive behavioral therapy (CBT) are important for patients who are anxious. Not only do these patients strongly prefer psychological treatment over medications,10,11 but also CBT may have advantages over pharmacotherapy in terms of maintaining clinical improvements over time.12,13

In response to primary care clinician preferences for interventions that have the capacity to address a range of common mental disorders rather than just one, we designed a flexible treatment delivery model, Coordinated Anxiety Learning and Management (CALM),14 and compared its effectiveness to care as usual (usual care [UC]). The CALM model addresses the 4 most common anxiety disorders—panic disorder (PD), generalized anxiety disorder (GAD), social anxiety disorder (SAD), and posttraumatic stress disorder (PTSD)—even when they co-occur with depression. It also optimizes treatment engagement by allowing choice of treatment modality15 (pharmacotherapy, CBT, or both) and provision of additional treatment when needed.2 A Web-based outcomes system is used to facilitate measurement-based care16 and a computer-assisted program helps guide nonexpert care managers in delivering evidence-based CBT.17 In this way, CALM seeks to accommodate the complexity of real-world clinical settings, while maximizing fidelity to the evidence-base in the context of a broad range of patients, clinicians, practice settings, and payers.

We hypothesized that the intervention would be better than UC in reducing psychic and somatic symptoms of anxiety and in improving global measures of functioning, health-related quality of life, and quality of care delivered. We expected small to moderate effect sizes similar to those found in previous collaborative care studies for depression.

Methods
Design, Setting, and Patients

Between June 2006 and April 2008, 1004 primary care patients with PD, GAD, SAD, PTSD, or all 4 were enrolled in the CALM study. A total of 17 clinics in Little Rock, Arkansas, Los Angeles County, San Diego, California, and Seattle, Washington, serving more than 35 000 patients with more than 780 000 annual visits, were purposively selected based on a number of considerations, including clinician interest, space availability, size and diversity of the patient population, and insurance mix. Primary care professionals (120 internists and 28 family physicians) referred all potential patients, facilitated by an optional 5 question anxiety screener.18 To determine eligibility, referred patients met with a specially trained clinician, the Anxiety Clinical Specialist (ACS). The 14 ACS personnel (11 women and 3 men) included 6 social workers, 5 registered nurses, 2 master-level psychologists, and 1 doctoral-level psychologist. Eight ACS personnel had some mental health experience, 4 were familiar with but had no formal training with CBT, and 7 had some psychopharmacology experience.

Eligible patients were patients at participating clinics, aged 18 to 75 years, who met Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) criteria for 1 or more of PD, GAD, SAD, or PTSD (based on the Mini International Neuropsychiatric Interview19 administered by the ACS personnel after formal training and diagnostic reliability testing), and scored at least 8 (moderate anxiety symptoms on a scale ranging from 0-20) on the Overall Anxiety Severity and Impairment Scale (OASIS), validated as clinically significant in a separate analysis.20 Co-occurring major depression was permitted. Persons unlikely to benefit from CALM (ie, unstable medical conditions, marked cognitive impairment, active suicidal intent or plan, psychosis, bipolar I disorder, and substance abuse of dependence except for alcohol and marijuana abuse) were excluded. Patients already receiving ongoing CBT or medication from a psychiatrist (n = 7) were excluded, as were persons who could not speak English or Spanish (n = 2). All patients gave written informed consent for the study, which was approved by each institution's institutional review board.

After a baseline interview, patients were randomized to intervention or UC, using an automated computer program at RAND Corporation (Santa Monica, California), where all posteligibility assessments were conducted by telephone. Randomization was stratified by clinic and presence of comorbid major depression using a permuted block design. Block size was masked to all clinical site study members. The Figure describes patient flow from eligibility screening through consent and randomization.

Intervention

The CALM model used a Web-based monitoring system,16 which was modeled on the Improving Mood-Promoting Access to Collaborative Treatment (IMPACT) intervention,2 with newly developed anxiety content and a computer-assisted CBT program.17 The ACS personnel received 6 half days of didactics, which focused on mastering the CBT program, plus motivational interviewing (modified for anxiety concerns) to enhance engagement, outreach strategies for ethnic-racial and impoverished minorities, and a medication algorithm for anxiety.21 CBT training also included role-playing and required successful completion of 2 training patients over several months.

Patients in the intervention group initially received their preferred treatment, which was either medication, CBT, or both, during 10 to 12 weeks. Because the effects of CBT delivered for one disorder are known to generalize to comorbid disorders,22 patients with multiple anxiety disorders were asked to choose the most disabling or distressing disorder to focus on with the expectation that their comorbid disorders would also improve. The CBT program, a repackaging based on already validated CBT treatments,23 included 5 generic modules (education, self-monitoring, hierarchy development, breathing training, and relapse prevention) and 3 modules (cognitive restructuring and exposure to internal and external stimuli) tailored to the 4 specific anxiety disorders. CBT was administered by the ACS (typically in 6 to 8 weekly sessions), while medication was prescribed. A local study psychiatrist provided single-session medication management training to clinicians using a simple algorithm, as needed consultation by telephone or e-mail, and very rarely a face-to-face assessment for patients who were complex or treatment refractory. The algorithm emphasized first-line use of selective serotonin reuptake inhibitor or serotonin norepinephrine reuptake inhibitor antidepressants, dose optimization, adverse effect monitoring, followed by second and third step combinations of 2 antidepressants or an antidepressant and benzodiazepine for patients who were refractory.21 For medication management, the ACS provided adherence monitoring, counseling to avoid alcohol and optimize sleep hygiene and behavioral activity, and relayed medication suggestions from the supervising psychiatrist to the primary care physician.

The ACS tracked patient outcomes on a Web-based system by entering scores for the OASIS and a 3-item version of the Patient Health Questionnaire 9, and examining graphical progress over time. The goal was either clinical remission, defined as an OASIS of less than 5 (mild), sufficient improvement such that the patient did not want further treatment, or improvement with residual symptoms or other emergent problems requiring a nonprotocol psychotherapy (ie, dialectical behavior therapy, family or dynamic psychotherapy). Patients who were symptomatic and thought to benefit from additional treatment with CBT or medication could receive more of the same modality (stepping up) or the alternative modality (stepping over) for up to 3 more steps of treatment. After treatment completion, patients were entered into continued care and received monthly follow-up telephone calls to reinforce CBT skills, medication adherence, or both. The ACS personnel interacted regularly with primary care physicians in person and by telephone. Primary care physicians remained the clinician of record and prescribed all medications. All ACS personnel received weekly supervision from a psychiatrist and psychologist.

Usual Care

Patients in the UC group continued to be treated by their physician in the usual manner with no intervention (ie, with medication, counseling [7 of 17 clinics had limited in-clinic mental health resources, usually a single clinician with limited familiarity with evidence-based psychotherapy24], or referral to a mental health specialist). After the eligibility diagnostic interview, the only contact patients in the UC group had with study personnel was for assessment by telephone.

Assessments

The assessment battery was administered at baseline and 6, 12, and 18 months via centralized telephone survey by the RAND Survey Research Group who were blinded to treatment assignment. Blinding was maintained by interviewers not asking patients about their intervention status and filling out an incident report if a patient spontaneously mentioned their intervention status. The last participant was assessed in October 2009. Because prior studies indicated outcome differences by ethnic-racial groupings,3 racial/ethnicity data were obtained by participant self-report using standard classification.

The primary outcome measure was a generic measure of 2 key components of all anxiety disorders, psychic and somatic anxiety (the 12-item Brief Symptom Inventory [BSI-12] subscales for anxiety and somatization25). Response was defined as at least a 50% reduction on the BSI-12, or meeting the definition of remission; and remission was defined as a face-valid per-item score of less than 0.5 (between none and mild, total BSI-12 score <6), consistent with previous analyses using the BSI for depression outcomes.26 Secondary outcome measures included Patient Health Questionnaire 8 depression,27 anxiety sensitivity,28 and functional status (Sheehan Disability,29 the Centers for Disease Control and Prevention Healthy Days Measure of restricted activity days,30 and 12-item Short Form version 231). Quality of care was measured by patient self-report of psychotropic medication type, dose, and adherence, and number and consistency of CBT elements occurring in reported psychotherapy sessions.32 For patients in the intervention group, more detailed information on the number and type (CBT vs medication/care management) of sessions was extracted from the Web-based management system.

Statistical Analysis

During the proposal phase, we had assumed an attrition rate of 28% at month 18. Therefore, we had anticipated needing a sample size of 1040 to detect effect sizes of 0.3 SDs with at least 80% power. Although the enrolled sample size (n = 1004) was marginally smaller than projected, participant attrition was lower (20% at month 18), yielding a larger than anticipated sample size at the follow-up time points.

We compared demographics and baseline anxiety and depression disorders rates by intervention group using t tests and χ2 tests for continuous and categorical variables, respectively. To estimate the intervention effect over time, we jointly modeled the outcomes at the 4 assessment times (baseline and 6, 12, and 18 months) by time, intervention, time × intervention, and site. Time was treated as a categorical variable. To avoid restrictive assumptions, the covariance of the outcomes at the 4 assessment times was left unstructured. We fitted the proposed model using a restricted maximum likelihood approach, which produces valid estimates under the missing-at-random assumption.33 This approach correctly handles the additional uncertainty arising from missing data and uses all available data to obtain unbiased estimates for model parameters.34 This is an efficient way for conducting an intention-to-treat analysis, because it includes all the patients with a baseline assessment. For cross-sectional analyses, such as those assessing the percentage of responders at the 3 follow-up times, we used attrition weights to correctly account for those patients that missed 1 or more follow-up assessments.35

Statistical software used for all analyses was SAS version 9 (SAS Institute Inc, Cary, North Carolina). All P values were 2-tailed and adjusted using Hochberg's correction method36 to account for multiple comparisons.

Results
Sample Selection, Attrition, and Description

The Figure depicts study patient flow and reasons for noneligibility. A total of 1062 of 1620 patients (66%) who were referred were eligible for the study, of which 1036 (98%) consented to participate and 1004 (97%) were randomized. More than 80% of patients were assessed at each evaluation window (6, 12, and 18 months), and study retention was high and similar in both study groups. Nonresponse was related to younger age, less education, and higher BSI-12 and OASIS scores at 6 months; younger age, higher BSI-12, Sheehan Disability, and OASIS scores, higher rate of panic, and higher rate among Hispanics at 12 months; and younger age, higher BSI-12 and OASIS scores, lower preference for current health state, higher rate of panic, and higher rate among Hispanics at 18 months. Table 1 shows that the sample was 71% women, ethnically diverse (44% nonwhite), and broad in age range. It was a fairly ill group with more than half having at least 2 chronic medical conditions and at least 2 anxiety disorders, and two-thirds with comorbid major depression. The intervention and UC groups were comparable on all baseline characteristics.

Intervention Participation

After the baseline assessment, 482 of 503 patients (95%) randomized to the intervention group had at least 1 intervention contact. During the course of the year, patients had mean (SD) of 7.0 (4.1) (median, 8) CBT visits (35/3386 [1%] by telephone and 11/3386 [<1%] focused on depression) and 2.24 (3.57) (median, 1) medication/care management visits (462/1078 [43%] by telephone). Of the 482 patients, 166 (34%) had only CBT visits, 43 (9%) had only medication/care management visits, and 273 (57%) had some of both. Visits for 218 patients (45%) were confined to the first 3 months and 424 patients (88%) had all visits by 6 months. A small proportion of patients (69/482 [14%]) also had an in-person visit with the study psychiatrist.

Quality of Care

Table 2 depicts self-reported quality of care received at baseline and 6, 12, and 18 months for the 2 groups, using progressively more stringent definitions of care quality. At both 6-month (54.8%; 95% confidence interval [CI], 51.0%-58.7%; vs 9.98%; 95% CI, 6.08%-13.88%) and 12-month (21.6%; 95% CI, 18.2%-25.1%; vs 9.31%; 95% CI, 5.83%-12.79%) assessments, significantly more patients in the intervention group received psychotherapy with at least 3 of 6 CBT elements (eg, exposure, relaxation, cognitive restructuring, homework) usually or always delivered. At 6 months only, significantly more patients in the intervention group either took medication of appropriate21 type, dose, and duration (≥2 months) or had an appropriate change in medication (dose increase or medication switch/addition) if they were already receiving medication (25.4%; 95% CI, 21.3%-29.4%; vs 17.1%; 95% CI, 13.5%-20.7%). Rates of overall psychotropic use did not differ between the 2 groups over time.

Main Outcome Measures

eTable 1 examines trajectories of adjusted means over time for the primary BSI-12 outcome, and for all secondary outcomes. The BSI-12 scores were significantly lower for patients in the intervention group at 6 months (mean difference, −2.49 points; 95% CI, −3.59 to −1.40 points; P < .001), 12 months (mean difference, −2.63 points; 95% CI, −3.73 to −1.54; P < .001), and 18 months (mean difference, −1.63 points; 95% CI, −2.73 to −0.53; P = .05), with effect sizes of −0.30 (95% CI, −0.43 to −0.17), −0.31 (95% CI, −0.44 to −0.18), and −0.18 (95% CI, −0.30 to −0.06). Outcome measures for patients in the intervention group were significantly better for all other measures, except physical health and satisfaction with medical care. Effect sizes were small to medium depending on the measure and were greatest at 12 months. There were no significant differences in intervention effect over time × site, and all 4 disorders showed significant effects on the main BSI-12 outcome (eTable 2).

Table 3 shows that a significantly (P < .001) higher proportion of patients in the intervention group responded and remitted, respectively. Response (including remission) rates at 6, 12, and 18 months were 57.46% (95% CI, 52.84%-62.08%), 63.66% (95% CI, 58.95%-68.37%), and 64.64% (95% CI, 59.95%-69.32%) vs 36.80% (95% CI, 32.21%-41.39%), 44.68% (95% CI, 39.76%-49.59%), and 51.47% (95% CI, 46.49%-56.45%) for intervention vs UC, respectively; and remission rates were 43.40% (95% CI, 38.78%-48.03%), 51.49% (95% CI, 46.60%-56.38%), and 51.06% (95% CI, 46.16%-55.96%) vs 27.49% (95% CI, 23.25%-31.72%), 33.28% (95% CI, 28.62%-37.93%), and 36.77% (95% CI, 31.99%-41.55%) for intervention vs UC, respectively. The number needed to treat (defined as 1/difference between intervention and control response or remission) at 12 months was 5.27 (95% CI, 4.18-7.13) for response and 5.50 (95% CI, 4.32-7.55) for remission.

Comment

These findings document the feasibility, acceptability (or satisfaction), and clinical effectiveness of a care delivery model designed to treat patients with any of 4 common anxiety disorders across 17 primary care clinics varying in patient characteristics, payer types, and organization. The model used both real-time outcomes monitoring and a computer-guided, modular CBT program, which ensured a high degree of fidelity in CBT application. Although different anxiety disorders (with or without depression) were targeted with this single intervention, effect sizes were similar to those obtained in previous anxiety effectiveness studies that had focused solely on PD, GAD, or both.6,7 The number needed to treat was well within the range for treatments in medicine that are generally considered to be efficacious,37,38 and beneficial effects of the intervention persisted for at least 1 year after clinical visits had ceased, suggesting a long-term effect.

Our study had a number of limitations. It was designed to test delivery of a blended package of treatments known to be evidence-based and we cannot determine which components of the blended intervention (eg, preference, CBT, medication, Web-outcomes monitoring) accounted for the results. Patients, a third of whom had failed at least 1 course of pharmacotherapy, were relatively well-educated and were referred to the study, all of which may have enhanced CBT engagement and response. We relied on self-report rather than review of medical records to assess amount and quality of treatment and used a relatively lean assessment battery intended to cover more domains while minimizing participant burden. The participating clinics had a higher than usual amount of in-house mental health resources in UC, although this may have led to an underestimate of the benefit of CALM vs UC models.

The positive outcomes as a whole may have been mediated by higher rates of quality CBT at 6 and 12 months and higher quality medication treatment at 6 months. This improved quality of care was facilitated by real-time outcomes monitoring, which allowed for adjustment of type and amount of delivered treatment, and a computer-guided modular CBT program, which ensured high fidelity when delivered by nonexperts, although the relative contribution of each to improved outcomes cannot be determined. The high rate of selection of CBT treatment by patients confirms previous findings11,39 that anxious patients prefer psychosocial treatment approaches. Also, the persistence of anxiety despite pharmacologic treatment in more than half the sample at baseline may have further reinforced this preference. Because the intervention devoted most of its training resources to the CBT program, it is possible that the medication management component could be further improved with more focus on this modality.

The flexibility of treatment (eg, variation in number and type of sessions, and in criteria for continuing further treatment, use of both telephone and in-person contact), the targeting of multiple disorders, and the clinical effectiveness across a range of patients and clinics suggest that the CALM treatment delivery model should be broadly applicable in primary care. However, implementation of this model will require reimbursement mechanisms for care management that are not currently available. In this vein, forthcoming analyses about the cost of CALM will be needed to help payers decide whether to support its uptake in clinical settings. Furthermore, the in-house model used by CALM would be less feasible for small or rurally located practices, which might require a more centrally located care manager and perhaps Internet or telephone delivery to serve multiple small or remote practices. Nonetheless, the success of the model tested here demonstrates that addressing multiple common mental disorders in the context of one delivery model is feasible and effective and could serve as a template for the development of unified approaches to management of the multiple psychiatric comorbidities that are the rule rather than the exception in both the general population40 and in clinical practice.

Back to top
Article Information

Corresponding Author: Peter Roy-Byrne, MD, Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Harborview Medical Center, 325 Ninth Ave, PO Box 359911, Seattle, WA 98104 (roybyrne@u.washington.edu).

Author Contributions: Dr Sherbourne had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Bystritsky, Welch, Chavira, Golinelli, Campbell-Sills, Sherbourne, Stein.

Acquisition of data: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Lang, Bystritsky, Welch, Chavira, Golinelli, Campbell-Sills, Sherbourne, Stein.

Analysis and interpretation of data: Roy-Byrne, Craske, Sullivan, Bystritsky, Golinelli, Sherbourne, Stein.

Drafting of the manuscript: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Lang, Bystritsky, Welch, Chavira, Golinelli, Campbell-Sills, Sherbourne, Stein.

Critical revision of the manuscript for important intellectual content: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Lang, Bystritsky, Welch, Chavira, Golinelli, Campbell-Sills, Sherbourne, Stein.

Statistical analysis: Golinelli, Sherbourne.

Obtained funding: Roy-Byrne, Craske, Sullivan, Sherbourne, Stein.

Administrative, technical, or material support: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Lang, Bystritsky, Chavira, Golinelli, Campbell-Sills, Sherbourne, Stein.

Study supervision: Roy-Byrne, Craske, Sullivan, Rose, Edlund, Bystritsky, Welch, Chavira, Campbell-Sills, Sherbourne, Stein.

Financial Disclosures: Dr Roy-Byrne reported receiving research grant support from the National Institutes of Health; having served as a paid member of advisory boards for Jazz Pharmaceuticals and Solvay Pharmaceuticals (1 meeting for each); having received honoraria for CME-sponsored speaking from the American Psychiatric Association, Anxiety Disorders Association of America, CME LLC, CMP Media, Current Medical Directions, Imedex, Massachusetts General Hospital Academy, and PRIMEDIA Healthcare; and serving as editor in chief for Journal Watch Psychiatry (Massachusetts Medical Society) and Depression and Anxiety (Wiley-Liss Inc). Dr Roy-Byrne reported also serving as an expert witness on multiple legal cases related to anxiety; none involving pharmaceutical companies or specific psychopharmacology issues. Dr Bystritsky reported serving as a paid consultant for Jazz Pharmaceuticals. Dr Stein reported receiving or having received research support from the US Department of Defense, Eli Lilly, GlaxoSmithKline, Hoffmann-La Roche, National Institutes of Health, and the US Veterans Affairs Research Program; and is currently or has been a paid consultant for AstraZeneca, Avera Pharmaceuticals, BrainCells Inc, Bristol-Myers Squibb, Comprehensive NeuroScience; Eli Lilly, Forest Laboratories, GlaxoSmithKline, Hoffmann-La Roche, Jazz Pharmaceuticals, Johnson & Johnson, Mindsite, Pfizer, Sepracor, and Transcept Pharmaceuticals. No other authors reported any financial disclosures.

Funding/Support: This work was supported by grants U01 MH057858 and K24 MH065324 (Dr Roy-Byrne), U01 MH058915 (Dr Craske), U01 MH070022 (Dr Sullivan), U01 MH070018 (Dr Sherbourne), and U01MH057835 and K24 MH64122 (Dr Stein) from the National Institute of Mental Health.

Role of the Sponsor: The CALM study's oversight was managed by the National Institute of Mental Health data and safety monitoring board, which has a rotating panel of members. The National Institute of Mental Health had no other involvement with the design and conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.

Scientific Advisory Board: Frank Verlain deGruy III, MD, MSFM (Department of Family Medicine, University of Colorado Denver School of Medicine, Aurora); Wayne Katon, MD, and Jürgen Unützer, MD, MPH, MA (Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle); Lisa V. Rubenstein, MD, MSPH (Department of Medicine, David Geffen School of Medicine and VA Greater Los Angeles Center of Excellence for the Study of Healthcare Provider Behavior, University of California, Los Angeles; and RAND Corporation, Santa Monica, California); Kenneth Wells, MD, MPH (Jane and Terry Semel Institute for Neuroscience and Human Behavior and Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles; Department of Health Services, UCLA School of Public Health; and RAND Corporation, Santa Monica, California). These board members received compensation for their consultation.

Ethnic Advisory Board: Peter J. Guarnaccia, PhD (Department of Human Ecology and the Institute for Health, Health Care Policy and Aging Research, Rutgers University, New Brunswick, New Jersey); Maga Jackson-Triche, MD, MSHS (VA Northern California Health Care System and University of California Davis School of Medicine, Sacramento); Jeanne Miranda, PhD (Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles); David T. Takeuchi, PhD (School of Social Work and Department of Sociology, University of Washington, Seattle). These board members received compensation for their consultation.

Anxiety Clinical Specialists: Skye M. Adams, LMSW, and Sandy Sanders, MSW, LCSW (University of Arkansas for Medical Sciences, Little Rock); Michelle Behrooznia, MA, Shadha Hami Cissell, MSW, and Michele S. Smith, PhD (University of California, San Diego); Cindy Chumley (High Desert Medical Group, Lancaster, California); Laura Constantinides, RN (Southern California Permanente Medical Group, San Diego County); Margie Fort, MSW, LICSW, Alice S. Friedman, MSN, ARNP, Kelly H. Koo, MS, Molly Roston, MSW, and Jodi Rubinstein, LICSW (University of Washington, Seattle); James W. Miller, MA, CRC (Desert Medical Group, Palm Springs, California); Angelica Ruiz, CCRC (Desert Oasis Healthcare, Palm Desert, California). These individuals were compensated for their work on the project. Supervising Psychiatrists: C. Winston Brown, MD, Mohit Chopra, MD, and Dan-Vy Mui, MD (University of Arkansas for Medical Sciences, Little Rock); James W. Gaudet, MD (Southern California Permanente Medical Group, San Diego County); Stuart Levine, MD, MHA (University of California, Los Angeles, and HealthCare Partners, Torrance, California); R. Christopher Searles, MD (University of California, San Diego); Jason P. Veitengruber, MD (University of Washington at Harborview Medical Center, Seattle). These individuals were compensated for their work on the project. Primary Care Clinic Champions: Lee C. Abel, MD (Little Rock Diagnostic Clinic, Little Rock, Arkansas); Basil Abramowitz, MD, and Stacey Coleman, DO (Sharp Rees Stealy, San Diego County, California); Lisa D. Chew, MD, MPH, and Robert Crittendon, MD (Harborview Medical Center, Seattle, Washington); Matthew G. Deneke, MD (University of Arkansas for Medical Sciences, Little Rock); Anne M. Eacker, MD (University of Washington Medical Center Roosevelt, Seattle); Erwin Guzman, MD (Southern California Permanente Medical Group, San Diego County); Ralph Joseph, MD (St. Vincent's Family Clinic, Little Rock, Arkansas); Gene A. Kallenberg, MD (University of California, San Diego); Richard Kovar, MD, FAAFP, and Carrie Rubenstein, MD (Country Doctor Community Health Centers, Seattle, Washington); T. Putnam “Putter” Scott, MD (Neighborcare Health, Seattle, Washington); Ivan Womboldt, CRCC (Desert Medical Group, Palm Springs, California). These individuals were compensated for work on this project either directly or via administrative fee payments to their clinics. RAND Survey Research Group: Barbara Levitan, BA (RAND Corporation, Santa Monica, California. Ms. Levitan was compensated for work on this project. Translators: Velma Barrios, MA (University of California, Los Angeles); Avelina Martinez, MAT, ATA-Certified Translator from English into Spanish (Austin, Texas). These individuals were compensated for their work on this project. Proficiency Raters: Laura B. Allen, PhD, and Ancy E. Cherian, PhD (University of California, Los Angeles). These individuals were compensated for their work on this project. Analysis: Bernadette Benjamin, MS (RAND Corporation, Santa Monica, California); Jutta M. Joesch, PhD, and Imara I. West, MPH (University of Washington, Seattle). These individuals were compensated for their work on the project. Programming of Web-based Outcomes System: Youlim Choi, MS (University of Washington, Seattle). Mr Choi was compensated for his work on this project. Programming of Computer CBT Program: Vivid Concept LLC (http://www.vividconcept.com). Study Coordinators: Kristin Bumgardner, BS (University of Washington, Seattle and central coordination of overall study); Daniel Dickson, BA, Daniel Glenn, BA, and Michael E. J. Reding, MA (University of California, Los Angeles); Christina Reaves, MPH (University of Arkansas for Medical Sciences, Little Rock). These individuals were compensated for their work on the project. National Institute of Mental Health: Matthew V. Rudorfer, MD (Adult Treatment and Preventive Interventions Research Branch, Division of Services and Intervention Research, National Institute of Mental Health, National Institutes of Health, Bethesda, MD) and Elizabeth Zachariah, MS (Clinical Trials Operations and Biostatistics Unit, Division of Services and Intervention Research, National Institute of Mental Health, National Institutes of Health, Bethesda, Maryland).

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health.

Previous Presentation: This study was presented in part at the Annual Meeting of the Anxiety Disorders Association of America; March 5, 2010; Baltimore, Maryland.

References
1.
Westfall JM, Mold J, Fagnan L. Practice-based research: ”Blue Highways” on the NIH roadmap.  JAMA. 2007;297(4):403-40617244837PubMedGoogle ScholarCrossref
2.
Unützer J, Katon W, Callahan CM,  et al; IMPACT Investigators. Improving Mood-Promoting Access to Collaborative Treatment.  Collaborative care management of late-life depression in the primary care setting.  JAMA. 2002;288(22):2836-284512472325PubMedGoogle ScholarCrossref
3.
Wells KB, Sherbourne CD, Schoenbaum M,  et al.  Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial.  JAMA. 2000;283(2):212-22010634337PubMedGoogle ScholarCrossref
4.
Simon GE, VonKorff M, Rutter C, Wagner E. Randomized trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care.  BMJ. 2000;320(7234):550-55410688563PubMedGoogle ScholarCrossref
5.
Katon W, Robinson P, Von Korff M,  et al.  A multifaceted intervention to improve treatment of depression in primary care.  Arch Gen Psychiatry. 1996;53(10):924-9328857869PubMedGoogle ScholarCrossref
6.
Roy-Byrne PP, Craske MG, Stein MB,  et al.  A randomized effectiveness trial of cognitive-behavioral therapy and medication for primary care panic disorder.  Arch Gen Psychiatry. 2005;62(3):290-29815753242PubMedGoogle ScholarCrossref
7.
Rollman BL, Belnap BH, Mazumdar S,  et al.  A randomized trial to improve the quality of treatment for panic and generalized anxiety disorders in primary care.  Arch Gen Psychiatry. 2005;62(12):1332-134116330721PubMedGoogle ScholarCrossref
8.
Stein MB, Sherbourne CD, Craske MG,  et al.  Quality of care for primary care patients with anxiety disorders.  Am J Psychiatry. 2004;161(12):2230-223715569894PubMedGoogle ScholarCrossref
9.
Mojtabai R, Olfson M, Mechanic D. Perceived need and help-seeking in adults with mood, anxiety, or substance use disorders.  Arch Gen Psychiatry. 2002;59(1):77-8411779286PubMedGoogle ScholarCrossref
10.
Grilo CM, Money R, Barlow DH,  et al.  Pretreatment patient factors predicting attrition from a multicenter randomized controlled treatment study for panic disorder.  Compr Psychiatry. 1998;39(6):323-3329829138PubMedGoogle ScholarCrossref
11.
Hazlett-Stevens H, Craske MG, Roy-Byrne PP, Sherbourne CD, Stein MB, Bystritsky A. Predictors of willingness to consider medication and psychosocial treatment for panic disorder in primary care patients.  Gen Hosp Psychiatry. 2002;24(5):316-32112220797PubMedGoogle ScholarCrossref
12.
Nadiga DN, Hensley PL, Uhlenhuth EH. Review of the long-term effectiveness of cognitive behavioral therapy compared to medications in panic disorder.  Depress Anxiety. 2003;17(2):58-6412621593PubMedGoogle ScholarCrossref
13.
Heuzenroeder L, Donnelly M, Haby MM,  et al.  Cost-effectiveness of psychological and pharmacological interventions for generalized anxiety disorder and panic disorder.  Aust N Z J Psychiatry. 2004;38(8):602-61215298582PubMedGoogle ScholarCrossref
14.
Sullivan G, Craske MG, Sherbourne C,  et al.  Design of the Coordinated Anxiety Learning and Management (CALM) study: innovations in collaborative care for anxiety disorders.  Gen Hosp Psychiatry. 2007;29(5):379-38717888803PubMedGoogle ScholarCrossref
15.
Chilvers C, Dewey M, Fielding K,  et al; Counselling versus Antidepressants in Primary Care Study Group.  Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms.  BMJ. 2001;322(7289):772-77511282864PubMedGoogle ScholarCrossref
16.
Unützer J, Choi Y, Cook IA, Oishi S. A web-based data management system to improve care for depression in a multicenter clinical trial.  Psychiatr Serv. 2002;53(6):671-673, 67812045303PubMedGoogle ScholarCrossref
17.
Craske MG, Rose RD, Lang A,  et al.  Computer-assisted delivery of cognitive behavioral therapy for anxiety disorders in primary-care settings.  Depress Anxiety. 2009;26(3):235-24219212970PubMedGoogle ScholarCrossref
18.
Means-Christensen AJ, Sherbourne CD, Roy-Byrne PP, Craske MG, Stein MB. Using five questions to screen for five common mental disorders in primary care: diagnostic accuracy of the Anxiety and Depression Detector.  Gen Hosp Psychiatry. 2006;28(2):108-11816516060PubMedGoogle ScholarCrossref
19.
Sheehan DV, Lecrubier Y, Sheehan KH,  et al.  The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10.  J Clin Psychiatry. 1998;59:(suppl 20)  22-33, quiz 34-579881538PubMedGoogle Scholar
20.
Campbell-Sills L, Norman SB, Craske MG,  et al.  Validation of a brief measure of anxiety-related severity and impairment: the Overall Anxiety Severity and Impairment Scale (OASIS).  J Affect Disord. 2009;112(1-3):92-10118486238PubMedGoogle ScholarCrossref
21.
Roy-Byrne P, Veitengruber JP, Bystritsky A,  et al.  Brief intervention for anxiety in primary care patients.  J Am Board Fam Med. 2009;22(2):175-18619264941PubMedGoogle ScholarCrossref
22.
Tsao JC, Mystkowski JL, Zucker BG, Craske MG. Impact of cognitive-behavioral therapy for panic disorder on comorbidity: a controlled investigation.  Behav Res Ther. 2005;43(7):959-97015896289PubMedGoogle ScholarCrossref
23.
Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses.  Clin Psychol Rev. 2006;26(1):17-3116199119PubMedGoogle ScholarCrossref
24.
Weissman MM, Verdeli H, Gameroff MJ,  et al.  National survey of psychotherapy training in psychiatry, psychology, and social work.  Arch Gen Psychiatry. 2006;63(8):925-93416894069PubMedGoogle ScholarCrossref
25.
Derogatis L. BSI-18: Brief Symptom Inventory 18 Administration, Scoring, and Procedures Manual. Minneapolis, MN: NCS Pearson Inc; 2001
26.
Simon GE, Katon WJ, VonKorff M,  et al.  Cost-effectiveness of a collaborative care program for primary care patients with persistent depression.  Am J Psychiatry. 2001;158(10):1638-164411578996PubMedGoogle ScholarCrossref
27.
Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure.  J Gen Intern Med. 2001;16(9):606-61311556941PubMedGoogle ScholarCrossref
28.
Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the predictions of fearfulness.  Behav Res Ther. 1986;24(1):1-83947307PubMedGoogle ScholarCrossref
29.
Sheehan DV, Harnett-Sheehan K, Raj BA. The measurement of disability.  Int Clin Psychopharmacol. 1996;11:(suppl 3)  89-958923116PubMedGoogle ScholarCrossref
30.
Centers for Disease Control and Prevention.  Measuring Healthy Days. Atlanta, GA: Centers for Disease Control and Prevention; 2000
31.
Ware JE Jr, Kosinski M, Bowker D, Gandek B. How to Score Version 2 of the SF-12 Health Survey (With a Supplement Documenting Version 1). Lincoln, RI: Quality Metric Inc; 2002
32.
Steketee G, Perry JC, Goisman RM,  et al.  The psychosocial treatments interview for anxiety disorders: a method for assessing psychotherapeutic procedures in anxiety disorders.  J Psychother Pract Res. 1997;6(3):194-2109185065PubMedGoogle Scholar
33.
Littell RC, Milliken GA, Stroup WW, Wolfinger RD. SAS System for Mixed Models. Cary, NC: SAS Institute Inc; 1996
34.
Little R, Rubin D. Statistical Analysis With Missing Data. New York, NY: John Wiley & Sons; 1987
35.
Brick JM, Kalton G. Handling missing data in survey research.  Stat Methods Med Res. 1996;5(3):215-2388931194PubMedGoogle ScholarCrossref
36.
Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance.  Biometrika. 1988;75(4):800-802Google ScholarCrossref
37.
Nuovo J, Melnikow J, Chang D. Reporting number needed to treat and absolute risk reduction in randomized controlled trials.  JAMA. 2002;287(21):2813-281412038920PubMedGoogle ScholarCrossref
38.
Moore A. What is a number needed to treat? http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/NNT.pdf. Accessed December 1, 2009
39.
Feeny NC, Zoellner LA, Mavissakalian MR, Roy-Byrne PP. What would you choose? sertraline or prolonged exposure in community and PTSD treatment seeking women.  Depress Anxiety. 2009;26(8):724-73119544313PubMedGoogle ScholarCrossref
40.
Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication.  Arch Gen Psychiatry. 2005;62(6):617-62715939839PubMedGoogle ScholarCrossref
×