Combined Pharmacotherapy and Cognitive Behavioral Therapy for Adults With Alcohol or Substance Use Disorders: A Systematic Review and Meta-analysis | Clinical Pharmacy and Pharmacology | JAMA Network Open | JAMA Network
[Skip to Navigation]
Figure.  Flow of Study Inclusion (PRISMA Diagram)
Flow of Study Inclusion (PRISMA Diagram)

CBT indicates cognitive behavioral therapy.

aIncludes studies diverted to other meta-analytic reports on CBT without pharmacotherapy, technology-delivered CBT, and CBT with dual-disorder populations.

Table 1.  CBT Combined With Pharmacotherapy Efficacy at Posttreatment by Type of Contrast Condition
CBT Combined With Pharmacotherapy Efficacy at Posttreatment by Type of Contrast Condition
Table 2.  CBT Combined With Pharmacotherapy Efficacy at Follow-up by Type of Contrast Condition
CBT Combined With Pharmacotherapy Efficacy at Follow-up by Type of Contrast Condition
Table 3.  Subgroup Analyses of Heterogeneous Pooled Effect Sizes
Subgroup Analyses of Heterogeneous Pooled Effect Sizes
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    June 19, 2020

    Combined Pharmacotherapy and Cognitive Behavioral Therapy for Adults With Alcohol or Substance Use Disorders: A Systematic Review and Meta-analysis

    Author Affiliations
    • 1Department of Psychology, University of California, Los Angeles
    • 2Yale School of Medicine, New Haven, Connecticut
    • 3Center for Alcohol and Addiction Studies, Brown University, Providence, Rhode Island
    JAMA Netw Open. 2020;3(6):e208279. doi:10.1001/jamanetworkopen.2020.8279
    Key Points español 中文 (chinese)

    Question  Is cognitive behavioral therapy associated with improved outcomes for alcohol and other substance use disorders in the context of pharmacotherapy for addiction?

    Findings  This systemic review and meta-analysis including 30 studies found that combined cognitive behavioral therapy and pharmacotherapy was associated with increased benefit compared with usual care and pharmacotherapy. Cognitive behavioral therapy did not perform better than another evidence-based modality in this context or as an add-on to combined usual care and pharmacotherapy.

    Meaning  These findings suggest that best practices in addiction treatment should include pharmacotherapy plus cognitive behavioral therapy or another evidence-based therapy, rather than usual clinical management or nonspecific counseling services.


    Importance  Substance use disorders (SUDs) represent a pressing public health concern. Combined behavioral and pharmacological interventions are considered best practices for addiction. Cognitive behavioral therapy (CBT) is a first-line intervention, yet the superiority of CBT compared with other behavioral treatments when combined with pharmacotherapy remains unclear. An understanding of the effects of combined CBT and pharmacotherapy will inform best-practice guidelines for treatment of SUD.

    Objective  To conduct a meta-analysis of the published literature on combined CBT and pharmacotherapy for adult alcohol use disorder (AUD) or other SUDs.

    Data Sources  PubMed, Cochrane Register, MEDLINE, PsychINFO, and Embase databases from January 1, 1990, through July 31, 2019, were searched. Keywords were specified in 3 categories: treatment type, outcome type, and study design. Collected data were analyzed through September 30, 2019.

    Study Selection  Two independent raters reviewed abstracts and full-text articles. English language articles describing randomized clinical trials examining CBT in combination with pharmacotherapy for AUD and SUD were included.

    Data Extraction and Synthesis  Inverse-variance weighted, random-effects estimates of effect size were pooled into 3 clinically informative subgroups: (1) CBT plus pharmacotherapy compared with usual care plus pharmacotherapy, (2) CBT plus pharmacotherapy compared with another specific therapy plus pharmacotherapy, and (3) CBT added to usual care and pharmacotherapy compared with usual care and pharmacotherapy alone. Sensitivity analyses included assessment of study quality, pooled effect size heterogeneity, publication bias, and primary substance moderator effects.

    Main Outcomes and Measures  Substance use frequency and quantity outcomes after treatment and during follow-up were examined.

    Results  The sample included 62 effect sizes from 30 unique randomized clinical trials that examined CBT in combination with some form of pharmacotherapy for AUD and SUD. The primary substances targeted in the clinical trial sample were alcohol (15 [50%]), followed by cocaine (7 [23%]) and opioids (6 [20%]). The mean (SD) age of the patient sample was 39 (6) years, with a mean (SD) of 28% (12%) female participants per study. The following pharmacotherapies were used: naltrexone hydrochloride and/or acamprosate calcium (26 of 62 effect sizes [42%]), methadone hydrochloride or combined buprenorphine hydrochloride and naltrexone (11 of 62 [18%]), disulfiram (5 of 62 [8%]), and another pharmacotherapy or mixture of pharmacotherapies (20 of 62 [32%]). Random-effects pooled estimates showed a benefit associated with combined CBT and pharmacotherapy over usual care (g range, 0.18-0.28; k = 9). However, CBT did not perform better than another specific therapy, and evidence for the addition of CBT as an add-on to combined usual care and pharmacotherapy was mixed. Moderator analysis showed variability in effect direction and magnitude by primary drug target.

    Conclusions and Relevance  The present study supports the efficacy of combined CBT and pharmacotherapy compared with usual care and pharmacotherapy. Cognitive behavioral therapy did not perform better than another evidence-based modality (eg, motivational enhancement therapy, contingency management) in this context or as an add-on to combined usual care and pharmacotherapy. These findings suggest that best practices in addiction treatment should include pharmacotherapy plus CBT or another evidence-based therapy, rather than usual clinical management or nonspecific counseling services.


    Substance use disorders (SUDs) represent a pressing public health concern, which calls for clinicians and scientists to identify and implement best practices in treatment.1,2 To that end, the combination of pharmacological and behavioral interventions has long been considered the criterion standard in addiction care,3-5 although differences between best practices for alcohol use disorder (AUD) and SUD have been noted.6 For SUDs without a US Food and Drug Administration (FDA)–approved pharmacotherapy, such as cocaine, methamphetamine, and cannabis, behavioral treatments are the principal approach.7 Cognitive behavioral therapy (CBT) is a first-line behavioral approach for treating AUD and other SUDs (AUD/SUD).8 Cognitive behavioral therapy is a time-limited, multisession intervention that targets cognitive, affective, and environmental risks for substance use and provides training in behavioral self-control skills to help an individual achieve and maintain abstinence or harm reduction.

    Despite the importance of combined pharmacological and behavioral interventions for AUD/SUD, few meta-analyses on this intervention approach have been performed. Typically, meta-analytic reviews in the AUD/SUD literature have been conducted on specific pharmacotherapies,9 groups of pharmacotherapies,10-12 or specific behavioral interventions, such as CBT. As a result, the evidence-informed guideline will relate only to the selection of a single, stand-alone therapy, whether pharmacological or behavioral, and not their combination. For example, in a review of 122 clinical trials of AUD pharmacotherapies delivered in outpatient settings,10 the authors could not conclude about the efficacy of pharmacotherapies when combined with a behavioral cointervention.

    The meta-analytic evidence on CBT supports efficacy at short- and long-term follow-ups.13 In an early review (1999) of 26 studies by Irvin et al,14 the authors found CBT to be generally effective across a range of conditions, but effect sizes were roughly 5 times higher when CBT was combined with pharmacotherapy than when delivered as a stand-alone intervention. This subgroup analysis was based on 4 studies and should therefore be interpreted with caution.14 In 2009, Magill and Ray15 followed up this work with a meta-analysis of 53 CBT clinical trials, reporting a similar overall effect size and a larger effect when CBT was combined with pharmacotherapy than when delivered alone, but the difference in effect-size magnitude between groups was smaller than observed in the previous review including 13 studies.

    The goal of this meta-analysis is to provide an up-to-date and comprehensive review of CBT in conjunction with pharmacotherapy for AUD/SUD. This meta-analysis provides effect-size estimates across 3 distinct subgroups that may be informative to best-practice guidelines or individual clinician decision-making: (1) CBT plus pharmacotherapy compared with usual care (eg, clinical management, nonspecific drug counseling) plus pharmacotherapy, (2) CBT plus pharmacotherapy compared with another specific therapy (eg, motivational enhancement therapy, contingency management) plus pharmacotherapy, and (3) CBT added to usual care and pharmacotherapy compared with usual care and pharmacotherapy alone. Sensitivity analyses included tests of heterogeneity, study influence, and publication bias. Given the robust literature on CBT for addiction,15,16 the critical role of pharmacotherapy in addiction treatment,1,2,17-19 and the notion that combined treatments may be most effective,3,17,18 this meta-analytic review seeks to inform clinical practice and best-practice guidelines for addiction.

    Literature Search Strategy

    A literature search was conducted by a trained research assistant (J.W.) through July 31, 2019, as part of a broader meta-analysis project on CBT for substance use. First, we performed an all-fields search by treatment (cognitive behavioral therapy OR relapse prevention OR coping skills training), outcome (alcohol OR cocaine OR methamphetamine OR stimulant OR opiate OR heroin OR opioid OR marijuana OR cannabis OR illicit drug OR substances OR dual disorder OR polysubstance OR dual diagnosis), and study terms (efficacy OR randomized controlled trial OR randomized clinical trial) in the PubMed database. Then, we searched the Cochrane Register, Embase, and EBSCO databases (ie, MEDLINE and PsychINFO). Abstract screening was performed by 2 raters in abstrkr.20 A bibliographic search of CBT reviews was also performed to identify any candidate studies not identified by the original search methods14,16,21-24 This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.

    The Figure summarizes study inclusion for the present report on combined CBT and pharmacological interventions for adult AUD/SUD (PRISMA diagram). The final meta-analytic sample consisted of 30 studies and 62 effect sizes. No studies that fit the search criteria have been published since 2016. The protocol for this meta-analysis was not registered but was scientifically reviewed at the National Institutes of Health.

    Primary Study Inclusion

    Studies were English-language, peer-reviewed articles published from January 1, 1990, through July 31, 2019. Studies were primary outcome reports of randomized clinical trials. Given the importance of experimental contrast type in estimating effect-size magnitude in clinical trials,25,26 we used this design factor as a primary subgroup variable. Studies were included if they targeted adult populations (aged ≥18 years) meeting criteria for AUD or other drug use disorder (ie, Diagnostic and Statistical Manual of Mental Disorders, Third Edition Revised through Fifth Edition) or problematic use.27 Treatment must have been identified as either cognitive behavioral or relapse prevention. Concomitant treatment with pharmacotherapy for AUD/SUD was required for inclusion. Studies of CBT delivered in either individual or group format were included.

    Primary Study Characteristic Variables

    Several study characteristic variables were examined in this meta-analysis. Study-level descriptors were mean (SD) age, percentage of female participants, percentage of white participants, percentage of black participants, percentage of Latinx participants, primary drug outcome (ie, alcohol, cocaine/stimulants, opioids, other), substance use severity (ie, dependence, abuse, or heavy use), treatment length (ie, number of planned sessions), treatment delivery (ie, individual or group format), study context (ie, community sample, specialty substance use or mental health clinic, medical setting, college setting, criminal justice setting, or other setting), publication country (ie, United States or other), and study-level risk of bias.28 Effect-size subgroup variables were (1) type of experimental contrast (ie, CBT plus pharmacotherapy compared with usual care plus pharmacotherapy, CBT plus pharmacotherapy compared with another specific therapy plus pharmacotherapy, CBT added to usual care and pharmacotherapy compared with usual care and pharmacotherapy alone), (2) type of substance use outcome type (ie, frequency and quantity), and (3) outcome point (posttreatment or follow-up). Data extraction guidelines were detailed in a study codebook available on request. Data were extracted in 2 independent passes conducted by trained raters (L.A.R., L.R.M., and J.W.). Final data entry where disagreement was observed required a consensus review by another author (M.M.).

    Primary Study Outcome Variables

    The standardized mean difference was used to measure efficacy outcomes in this meta-analysis. The Hedges g statistic includes a correction for a slight upward bias in the estimated population effect.29 Before pooling, effect sizes were weighted by the inverse of the estimate variance to allow larger studies more influence on the overall effect size.30 Primary studies typically provided data on more than 1 outcome; therefore, data for effect-size estimation were selected based on a decisional hierarchy in the following order: (1) biological assay measures, (2) measures of frequency or quantity in the form of means (SDs), (3) sample proportions, and (4) other outcomes (eg, diagnostic measures). Most studies reported posttreatment outcomes only (18 [60%]), and if multiple months of follow-up data were reported, the latest time point was selected. Effect sizes were reverse scored as needed (eg, number of days with alcohol use) such that a positive effect size indicated a positive treatment outcome. Finally, when univariate outcome data were not reported, test statistics were transformed using available formulae.31 Effect size magnitude was interpreted using the follow benchmarks: 0.2 indicates small; 0.5, medium; and 0.8, large.32

    Statistical Analysis

    Data were analyzed through September 30, 2019. Comprehensive Meta-analysis, version 3.0,33 was used for all analyses. Effect sizes for alcohol and other drug use were pooled using a random-effects model where there was an assumed distribution for the population effect size with systematic and random sources of variability.34 The significance of the Q test determined whether statistically significant between-study heterogeneity was present, and the I2 statistic provided a percentage of heterogeneity estimate, regardless of statistical significance. When I2 estimates exceeded 50%,33 primary drug outcome was tested as an effect-size moderator. We conducted other sensitivity analyses, including trimmed estimates with influential studies (ie, a study that, if removed, would change the substantive interpretation of the pooled effect size) removed and tests for publication bias.

    Sample-Level Descriptive Data

    The sample included 30 unique AUD/SUD randomized clinical trials3,7,35-62 that examined CBT in combination with some form of pharmacotherapy. The study publication range was 1992 to 2016. The median sample size was 82 participants, with a range of 3035 to 917.3 The primary substance targeted in these clinical trials was alcohol (15 [50%]), followed by cocaine (7 [23%]) and opioids (6 [20%]). The sample mean (SD) participant age was 39 (6) years, with a mean (SD) of 28% (12%) female participants. Although reporting of race and ethnicity were inconsistent, the mean (SD) percentages were as follows: 66% (26%) white (21 of 30 studies), 35% (28%) black (15 of 30 studies), and 9% (7%) Latinx (15 of 30 studies). Diagnostically, study inclusion primarily (ie, 95%) targeted individuals with abuse or dependence as defined by Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) criteria.63 The CBT portion of these combined interventions was 73% individual and 26% group delivered, and 1 study36 used a mixture of individual and group sessions. The mean number of planned sessions was 16 (range, 4-48), and recruitment contexts were primarily specialty substance use or mental health clinics (20 [68%]), medical settings (5 [16%]), and community advertising (5 [16%]). The following pharmacotherapies were examined in this review: naltrexone hydrochloride and/or acamprosate sodium (26 of 62 effect sizes [42%]), methadone hydrochloride or combined buprenorphine hydrochloride and naltrexone (Suboxone) (11 of 62 [18%]), disulfiram (5 of 62 [8%]), and another pharmacotherapy or a mixture of pharmacotherapies (20 of 62 [32%]) (see Table 1 and Table 2 for details). Study-level risk-of-bias assessment showed 18 studies (60%) were low risk.28 When studies were designated as unclear or high risk, this was typically owing to (1) the presence of baseline differences between conditions, (2) no report of blinding of personnel and participants, and (3) no report of blinding of outcome assessment. Finally, most studies (21 [70%]) were published in the United States. Of studies conducted outside the United States, the following countries were represented: Germany, Sweden, the Netherlands, Switzerland, Finland, China, and Australia.

    Tables 1 and 2 describe each study with respect to key design characteristics and effect sizes and are separated by posttreatment and follow-up outcomes, respectively. For outcomes of interest, biological assay/frequency measures and quantity measures are considered primary, and when available both are reported in Tables 1 and 2. Finally, the sample was distributed in roughly equal thirds with respect to the types of conditions to which combined CBT and pharmacotherapy was compared; narrative results and Tables 1 and 2 are organized by these clinically informative subgroups, and given the distinctiveness of these comparator conditions, no overall pooled effect size is reported. For pictorial plot information, see eFigures 1 through 10 in the Supplement.

    CBT Plus Pharmacotherapy Compared With Usual Care Plus Pharmacotherapy

    When CBT plus pharmacotherapy was compared with usual care plus pharmacotherapy, the effect for CBT on posttreatment frequency outcomes was small, homogeneous, and statistically significant (g = 0.18 [95% CI, 0.01-0.35]; P = .04; τ2 = 0.00, Q > 0.05, I2 = 0%),37,43-49 but only 1 study37 provided follow-up effect-size data (g = 0.24 [95% CI, −0.15 to 0.62]). For quantity outcomes,37,43,46,47,49 effects were small to moderate, homogenous, and significant (g = 0.28 [95% CI, 0.03-0.54]; P = .03; τ2 = 0.03; Q > 0.05; I2 = 31%), and only 2 studies37,38 provided follow-up effect-size data (g = 0.50 [95% CI, 0.01-0.89] and g = 2.00 [95% CI, 1.40-2.60], respectively). Among the studies in this subgroup, no evidence was found for influential studies or publication bias.

    CBT Plus Pharmacotherapy Compared With Another Specific Therapy Plus Pharmacotherapy

    Studies that compared CBT with another specific therapy suggested no unique benefit of adding CBT to pharmacotherapy compared with other evidence-based modalities. For posttreatment frequency outcomes, effects were homogeneous and nonsignificant (g = 0.05 [95% CI, −0.13 to 0.23]; P = .58; τ2 = 0.03; Q > 0.05; I2 = 35%).7,45,50-53,55-57 A similar pattern of frequency results was found at follow-up (g = −0.02 [95% CI, −0.29 to 0.26]; P = .89; τ2 = 0.03; Q > 0.05; I2 = 11%).39,52,53,57,60 Quantity posttreatment outcomes were also nonsignificant but statistically heterogeneous (g = 0.09 [95% CI, −0.42 to 0.60]; P = .74; τ2 = 0.23; Q < 0.05; I2 = 84%).41,50,51,54 At follow-up, only Longabaugh et al39 reported quantity outcomes (g = 0.34 [95% CI, −0.29 to 0.96]). Sensitivity analyses showed no influential studies or evidence of publication bias.

    CBT as an Add-on to Usual Care and Pharmacotherapy Compared With Usual Care and Pharmacotherapy Alone

    The final subgroup of studies examined CBT as an add-on to combined usual care and pharmacotherapy and suggested no clear added benefit of CBT in this context. Frequency results were heterogeneous and nonsignificant after treatment (g = 0.06 [95% CI, −0.22 to 0.34]; P = .67; τ2 = 0.13; Q < 0.05; I2 = 76%)3,35,36,42,53,54,57-59 and at follow-up (g = 0.17 [95% CI, −0.05 to 0.38]; P = .13; τ2 = 0.04; Q > 0.05; I2 = 51%).3,42,53,57,59,60,62 Only 3 studies (all AUD trials) provided quantity effect-size data, and these studies showed very different measures of effect (Anton et al3: g = 0.08 [95% CI, −0.14 to 0.31]; Epstein et al65: g = −0.14 [95% CI, −0.54 to 0.25]; Morgenstern et al40: g = 2.90 [95% CI, 2.32-3.48]). At follow-up, the effect size for quantity outcomes in the COMBINE Study3 was g = 0.04 (95% CI, −0.19 to 0.26) for collapsed pharmacotherapy, medication management, and combined behavioral intervention conditions vs pharmacotherapy and medication management only conditions. Sensitivity analyses showed no evidence of publication bias but showed 2 influential studies35,40 (eFigure 3 in the Supplement).

    Primary Drug Target as a Moderator of Between-Study Heterogeneity

    In this review of CBT combined with pharmacotherapy for AUD/SUD frequency and quantity outcomes after treatment and at follow-up, most effect estimates showed little to no statistical heterogeneity. This outcome suggests that the variation between studies within a given pooled effect size was random rather than systematic.33 The exception was CBT combined with pharmacotherapy in contrast to another specific therapy combined with pharmacotherapy in relation to posttreatment quantity outcomes (Q < 0.05; I2 = 84%)41,50,51,54 and CBT as an addition to combined usual care and pharmacotherapy in relation to posttreatment (Q < 0.05; I2 = 76%)3,35,36,42,53,54,57-59 and follow-up (Q > 0.05; I2 = 51%)3,42,53,57,59,60,62 frequency outcomes. Given that the sample of studies targeted different substances, each primary drug was examined as a subgroup moderator, and residual I2 values were reported. Results are presented in Table 3. These analyses show that pooled effect-size direction and/or magnitude varied by primary drug outcome in this review. However, residual heterogeneity was present in 3 of the 8 subgroups, suggesting that this a priori moderator was an informative variable in this meta-analysis but did not explain all the systematic variance between studies. As a result, random-effects estimates are a particularly appropriate metric for this review.


    To our knowledge, this is the first targeted meta-analysis of CBT in combination with pharmacotherapy for adults with AUD and other SUDs to summarize the data in a manner relevant to clinical practice guidelines. Furthermore, we provided pooled effect size estimates by consumption outcome type and outcome point. Most of these subgroup estimates showed acceptable homogeneity, which suggests that the selected variables were informative effect-size modifiers for the sample of clinical trials reviewed. In the present study, a small and statistically significant effect size was observed across outcome type and time for CBT combined with pharmacotherapy when compared with usual care combined with pharmacotherapy. For context in interpreting this effect, meta-analyses among this patient population generally show effect sizes that are in the small-to-moderate range,41,66,67 and this includes effect sizes for pharmacological interventions.11,68,69 This contrast suggests that prescribing clinicians should favor CBT over usual care to improve clinical outcomes for addiction, in the context of pharmacotherapy.

    The second subgroup contrast was combined CBT and pharmacotherapy compared with another specific therapy combined with pharmacotherapy. Here, results suggested no unique benefit of adding CBT to pharmacotherapy compared with other evidence-based behavioral modalities. Such modalities may include contingency management, motivation enhancement therapy, 12-step facilitation, and interpersonal therapy, all of which have received some level of empirical support for addiction,70,71 including meta-analytic support at various follow-up periods.23 The lack of superiority of CBT over other evidence-based behavioral treatment for addiction is consistent with our recent findings,13 and this meta-analysis extends this result to combined pharmacotherapy and behavioral treatments. Although there may be evidence of some advantage for contingency management,72 removal of contingency management trials in this review did not change our substantive conclusions. This suggests that CBT is not superior to other evidence-based behavioral treatments for addiction, yet when combined with the aforementioned superiority to usual care, we suggest that clinicians favor an evidence-based behavioral therapy, CBT or otherwise, in conjunction with pharmacological treatments.

    The third contrast in this meta-analysis assessed CBT as an add-on to usual care and pharmacotherapy compared with usual care and pharmacotherapy alone. In interpreting these findings, several explanations come to mind. First, there was substantial heterogeneity in the effect sizes obtained in these studies, suggesting unique study-specific factors could further explain outcome variability. This speculation is supported by the 2 influential studies observed in this subgroup.35,40 Moreover, moderator analyses by primary drug target showed variability in effect-size direction and magnitude with effects for cocaine and stimulant studies showing a range from moderate and negative35,41 to large and positive57,58 effects. This variability may be due, in part, to the lack of FDA-approved pharmacotherapy for cocaine/stimulant use disorder.73 In other words, FDA approval in this case was potentially confounded with the primary drug target. Second, some studies reported participants’ poor adherence to the CBT protocol, which may directly affect outcome.42,65 Third, the COMBINE Study is a large trial, which reported no benefit of the combined behavioral intervention over medication management. Close inspection of the medication management protocol for this study suggests it was a systematic, intensive, and rather robust intervention74,75 not readily comparable to standard clinical care. Together, these findings speak to the difficulty of measuring the benefit of an add-on component in complex clinical settings where multiple interventions are simultaneously administered.


    This study has some limitations to consider. First, our primary goal was to derive valid, random-effects estimates characterized by effect modifiers. In other words, the goal was to avoid combining apples and oranges,76 which is a common criticism against meta-analysis. Although we consider our subgroup approach a strength, some effect size estimates were composed of a small number of primary studies, and this may result in underpowered analyses. Second, there may be some concern about fidelity and other sources of variability in what constituted this sample of CBT interventions, given that a range of manual implementations were reviewed. Unfortunately, fidelity to behavioral intervention protocol is often poorly reported in the clinical trial literature77 and could not be consistently measured for the present study. Similarly, this meta-analysis did not account for medication compliance or dose as a potential source of variability. The exclusion of 6 non-English articles is another potential limitation of this study. Finally, study results should be considered in the context of what constitutes an optimal outcome in clinical research with adult AUD/SUD. We selected consumption measures and favored biological assay variables in SUD trials, but equally important may be improvements in overall functioning,78 in which behavioral therapies, such as CBT, may be more likely to demonstrate effects.18,19 Therefore, the degree to which the outcomes presented in this review reflect an ideal end point remains an ongoing discussion for the field.


    Current clinical and research practices suggest that the rationale for combining behavioral therapy and pharmacotherapy is to provide support and skills while the patient is waiting for the medication effects to become apparent, to enhance treatment adherence, to improve treatment and study retention, and to address symptoms and problems that the medication will not address (eg, skills building).18,19,74 In an effort to inform best practices in addiction treatment, the following take-home points are drawn from the results of this meta-analysis. First, our results suggest that prescribing clinicians should favor CBT over usual clinical management to ensure optimal clinical outcomes for addiction, in the context of pharmacotherapy. This conclusion is based in our comparison of CBT plus pharmacotherapy vs usual care plus pharmacotherapy. Second, CBT is not superior to other evidence-based behavioral treatments for addiction, yet in the context of its superiority to usual care, our findings suggest that clinicians should favor an evidence-based behavioral therapy, CBT or otherwise, in conjunction with pharmacological treatment. Third, the add-on benefit of CBT compared with pharmacotherapy and usual care was not clearly supported and suggests that benefit of CBT as an adjunct requires further investigation.

    Back to top
    Article Information

    Accepted for Publication: April 11, 2020.

    Published: June 19, 2020. doi:10.1001/jamanetworkopen.2020.8279

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Ray LA et al. JAMA Network Open.

    Corresponding Author: Lara A. Ray, PhD, UCLA, 1285 Franz Hall, PO Box 159563, Los Angeles, CA 90095 (

    Author Contributions: Drs Ray and Magill had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Ray, Meredith, Magill.

    Acquisition, analysis, or interpretation of data: All authors.

    Drafting of the manuscript: Ray, Magill.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Ray, Meredith, Walthers, Magill.

    Obtained funding: Ray, Magill.

    Administrative, technical, or material support: Ray, Meredith, Walthers, Magill.

    Supervision: Ray, Magill.

    Conflict of Interest Disclosures: Dr Ray reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism (NIAAA) during the conduct of the study. Dr Carroll reported receiving grants from the National Institute on Drug Abuse and the NIAAA during the conduct of the study and having a patent to CBT4CBT with royalties paid. Dr Magill reported receiving grants from the NIAAA during the conduct of the study. No other disclosures were reported.

    Funding/Support: This study was supported by grants AA026006 and AA025704 from the NIAA.

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Volkow  ND, Jones  EB, Einstein  EB, Wargo  EM.  Prevention and treatment of opioid misuse and addiction: a review.   JAMA Psychiatry. 2019;76(2):208-216. doi:10.1001/jamapsychiatry.2018.3126 PubMedGoogle ScholarCrossref
    Kranzler  HR, Soyka  M.  Diagnosis and pharmacotherapy of alcohol use disorder: a review.   JAMA. 2018;320(8):815-824. doi:10.1001/jama.2018.11406 PubMedGoogle ScholarCrossref
    Anton  RF, O’Malley  SS, Ciraulo  DA,  et al; COMBINE Study Research Group.  Combined pharmacotherapies and behavioral interventions for alcohol dependence: the COMBINE study: a randomized controlled trial.   JAMA. 2006;295(17):2003-2017. doi:10.1001/jama.295.17.2003 PubMedGoogle ScholarCrossref
    Carroll  KM, Weiss  RD.  The role of behavioral interventions in buprenorphine maintenance treatment: a review.   Am J Psychiatry. 2017;174(8):738-747. doi:10.1176/appi.ajp.2016.16070792 PubMedGoogle ScholarCrossref
    National Institute on Drug Abuse.  Principles of Drug Addiction Treatment: A Research Based Guide. National Institute on Drug Abuse; 2007.
    Bradley  KA, Kivlahan  DR.  Bringing patient-centered care to patients with alcohol use disorders.   JAMA. 2014;311(18):1861-1862. doi:10.1001/jama.2014.3629 PubMedGoogle ScholarCrossref
    Carroll  KM, Fenton  LR, Ball  SA,  et al.  Efficacy of disulfiram and cognitive behavior therapy in cocaine-dependent outpatients: a randomized placebo-controlled trial.   Arch Gen Psychiatry. 2004;61(3):264-272. doi:10.1001/archpsyc.61.3.264 PubMedGoogle ScholarCrossref
    Substance Abuse and Mental Health Services Administration. National survey on drug use and health 2014. Published 2014. Accessed December 2019.
    Srisurapanont  M, Jarusuraisin  N.  Naltrexone for the treatment of alcoholism: a meta-analysis of randomized controlled trials.   Int J Neuropsychopharmacol. 2005;8(2):267-280. doi:10.1017/S1461145704004997 PubMedGoogle ScholarCrossref
    Jonas  DE, Amick  HR, Feltner  C,  et al.  Pharmacotherapy for adults with alcohol use disorders in outpatient settings: a systematic review and meta-analysis.   JAMA. 2014;311(18):1889-1900. doi:10.1001/jama.2014.3628 PubMedGoogle ScholarCrossref
    Maisel  NC, Blodgett  JC, Wilbourne  PL, Humphreys  K, Finney  JW.  Meta-analysis of naltrexone and acamprosate for treating alcohol use disorders: when are these medications most helpful?   Addiction. 2013;108(2):275-293. doi:10.1111/j.1360-0443.2012.04054.x PubMedGoogle ScholarCrossref
    Barnett  PG, Rodgers  JH, Bloch  DA.  A meta-analysis comparing buprenorphine to methadone for treatment of opiate dependence.   Addiction. 2001;96(5):683-690. doi:10.1046/j.1360-0443.2001.9656834.x PubMedGoogle ScholarCrossref
    Magill  M, Ray  L, Kiluk  B,  et al.  A meta-analysis of cognitive-behavioral therapy for alcohol or other drug use disorders: treatment efficacy by contrast condition.   J Consult Clin Psychol. 2019;87(12):1093-1105. doi:10.1037/ccp0000447 PubMedGoogle ScholarCrossref
    Irvin  JE, Bowers  CA, Dunn  ME, Wang  MC.  Efficacy of relapse prevention: a meta-analytic review.   J Consult Clin Psychol. 1999;67(4):563-570. doi:10.1037/0022-006X.67.4.563 PubMedGoogle ScholarCrossref
    Magill  M, Ray  LA.  Cognitive-behavioral treatment with adult alcohol and illicit drug users: a meta-analysis of randomized controlled trials.   J Stud Alcohol Drugs. 2009;70(4):516-527. doi:10.15288/jsad.2009.70.516 PubMedGoogle ScholarCrossref
    Longabaugh  R, Morgenstern  J.  Cognitive-behavioral coping-skills therapy for alcohol dependence: current status and future directions.   Alcohol Res Health. 1999;23(2):78-85.PubMedGoogle Scholar
    Kampman  K, Jarvis  M.  American Society of Addiction Medicine (ASAM) national practice guideline for the use of medications in the treatment of addiction involving opioid use.   J Addict Med. 2015;9(5):358-367. doi:10.1097/ADM.0000000000000166 PubMedGoogle ScholarCrossref
    Carroll  KM.  Integrating psychotherapy and pharmacotherapy to improve drug abuse outcomes.   Addict Behav. 1997;22(2):233-245. doi:10.1016/S0306-4603(96)00038-X PubMedGoogle ScholarCrossref
    Carroll  KM.  Manual-guided psychosocial treatment: a new virtual requirement for pharmacotherapy trials?   Arch Gen Psychiatry. 1997;54(10):923-928. doi:10.1001/archpsyc.1997.01830220041007 PubMedGoogle ScholarCrossref
    Wallace  BC, Small  K, Brodley  CE, Lau  J, Trikalinos  TA. Deploying an interactive machine learning system in an evidence-based practice center: abstrackr. In: Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium; January 2012. Association for Computing Machinery; 2012:819-824.
    Carroll  KM, Kiluk  BD.  Cognitive behavioral interventions for alcohol and drug use disorders: through the stage model and back again.   Psychol Addict Behav. 2017;31(8):847-861. doi:10.1037/adb0000311 PubMedGoogle ScholarCrossref
    Miller  WR, Wilbourne  P, Hetema  J. What works? a summary of alcohol treatment outcome research. In: Hester RK, Miller WR, eds.  Handbook of Alcoholism Treatment Approaches: Effective Alternatives. Allyn & Bacon; 2003.
    Sayegh  CS, Huey  SJ, Zara  EJ, Jhaveri  K.  Follow-up treatment effects of contingency management and motivational interviewing on substance use: a meta-analysis.   Psychol Addict Behav. 2017;31(4):403-414. doi:10.1037/adb0000277 PubMedGoogle ScholarCrossref
    Chan  B, Freeman  M, Kondo  K,  et al.  Pharmacotherapy for methamphetamine/amphetamine use disorder: a systematic review and meta-analysis.   Addiction. 2019;114(12):2122-2136. doi:10.1111/add.14755 PubMedGoogle ScholarCrossref
    Imel  ZE, Wampold  BE, Miller  SD, Fleming  RR.  Distinctions without a difference: direct comparisons of psychotherapies for alcohol use disorders.   Psychol Addict Behav. 2008;22(4):533-543. doi:10.1037/a0013171 PubMedGoogle Scholar
    Wampold  BE, Imel  ZE.  The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work. Routledge; 2015. doi:10.4324/9780203582015
    Saunders  JB, Aasland  OG, Babor  TF, de la Fuente  JR, Grant  M.  Development of the Alcohol Use Disorders Identification Test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption—II.   Addiction. 1993;88(6):791-804. doi:10.1111/j.1360-0443.1993.tb02093.x PubMedGoogle Scholar
    Higgins  JP, Altman  DG, Gøtzsche  PC,  et al; Cochrane Bias Methods Group; Cochrane Statistical Methods Group.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.   BMJ. 2011;343:d5928. doi:10.1136/bmj.d5928 PubMedGoogle Scholar
    Cooper  H, Hedges  LV, Valentine  JC, eds.  The Handbook of Research Synthesis and Meta-analysis. Russell Sage Foundation; 2019. doi:10.7758/9781610448864
    Hedges  LV, Olkin  I.  Statistical Methods for Meta-analysis. Academic Press; 2014.
    Lipsey  MW, Wilson  DB.  Practical Meta-analysis. Sage Publications Inc; 2001.
    Cohen  J.  A power primer.   Psychol Bull. 1992;112(1):155-159. doi:10.1037/0033-2909.112.1.155 PubMedGoogle Scholar
    Borenstein  M, Hedges  L, Higgins  J, Rothstein  H. Regression in meta-analysis. Comprehensive Meta-analysis Manual. Posted June 2015. Accessed January 2020.
    Hedges  LV, Vevea  JL.  Fixed-and random-effects models in meta-analysis.   Psychol Methods. 1998;3(4):486-504. doi:10.1037/1082-989X.3.4.486 Google Scholar
    Dürsteler-MacFarland  KM, Farronato  NS, Strasser  J,  et al.  A randomized, controlled, pilot trial of methylphenidate and cognitive-behavioral group therapy for cocaine dependence in heroin prescription.   J Clin Psychopharmacol. 2013;33(1):104-108. doi:10.1097/JCP.0b013e31827bfff4 PubMedGoogle Scholar
    Pan  S, Jiang  H, Du  J,  et al.  Efficacy of cognitive behavioral therapy on opiate use and retention in methadone maintenance treatment in China: a randomised trial.   PLoS One. 2015;10(6):e0127598. doi:10.1371/journal.pone.0127598 PubMedGoogle Scholar
    Wetzel  H, Szegedi  A, Scheurich  A,  et al; NeVeR Study Group.  Combination treatment with nefazodone and cognitive-behavioral therapy for relapse prevention in alcohol-dependent men: a randomized controlled study.   J Clin Psychiatry. 2004;65(10):1406-1413. doi:10.4088/JCP.v65n1017 PubMedGoogle Scholar
    Heinälä  P, Alho  H, Kiianmaa  K, Lönnqvist  J, Kuoppasalmi  K, Sinclair  JD.  Targeted use of naltrexone without prior detoxification in the treatment of alcohol dependence: a factorial double-blind, placebo-controlled trial.   J Clin Psychopharmacol. 2001;21(3):287-292. doi:10.1097/00004714-200106000-00006 PubMedGoogle Scholar
    Longabaugh  R, Wirtz  PW, Gulliver  SB, Davidson  D.  Extended naltrexone and broad spectrum treatment or motivational enhancement therapy.   Psychopharmacology (Berl). 2009;206(3):367-376. doi:10.1007/s00213-009-1615-3 PubMedGoogle Scholar
    Morgenstern  J, Kuerbis  AN, Chen  AC, Kahler  CW, Bux  DA  Jr, Kranzler  HR.  A randomized clinical trial of naltrexone and behavioral therapy for problem drinking men who have sex with men.   J Consult Clin Psychol. 2012;80(5):863-875. doi:10.1037/a0028615 PubMedGoogle Scholar
    Bertholet  N, Daeppen  JB, Wietlisbach  V, Fleming  M, Burnand  B.  Reduction of alcohol consumption by brief alcohol intervention in primary care: systematic review and meta-analysis.   Arch Intern Med. 2005;165(9):986-995. doi:10.1001/archinte.165.9.986 PubMedGoogle Scholar
    Scherbaum  N, Kluwig  J, Specka  M,  et al.  Group psychotherapy for opiate addicts in methadone maintenance treatment: a controlled trial.   Eur Addict Res. 2005;11(4):163-171. doi:10.1159/000086397 PubMedGoogle Scholar
    Balldin  J, Berglund  M, Borg  S,  et al.  A 6-month controlled naltrexone study: combined effect with cognitive behavioral therapy in outpatient treatment of alcohol dependence.   Alcohol Clin Exp Res. 2003;27(7):1142-1149. doi:10.1097/01.ALC.0000075548.83053.A9PubMedGoogle Scholar
    Carroll  KM, Rounsaville  BJ, Gordon  LT,  et al.  Psychotherapy and pharmacotherapy for ambulatory cocaine abusers.   Arch Gen Psychiatry. 1994;51(3):177-187. doi:10.1001/archpsyc.1994.03950030013002PubMedGoogle Scholar
    Carroll  KM, Nich  C, Ball  SA, McCance  E, Rounsavile  BJ.  Treatment of cocaine and alcohol dependence with psychotherapy and disulfiram.   Addiction. 1998;93(5):713-727. doi:10.1046/j.1360-0443.1998.9357137.xPubMedGoogle Scholar
    O’Malley  SS, Jaffe  AJ, Chang  G, Schottenfeld  RS, Meyer  RE, Rounsaville  B.  Naltrexone and coping skills therapy for alcohol dependence. A controlled study.   Arch Gen Psychiatry. 1992;49(11):881-887. doi:10.1001/archpsyc.1992.01820110045007PubMedGoogle Scholar
    O’Malley  SS, Rounsaville  BJ, Farren  C,  et al.  Initial and maintenance naltrexone treatment for alcohol dependence using primary care vs specialty care: a nested sequence of 3 randomized trials.   Arch Intern Med. 2003;163(14):1695-1704. doi:10.1001/archinte.163.14.1695PubMedGoogle Scholar
    Schmitz  JM, Stotts  AL, Rhoades  HM, Grabowski  J.  Naltrexone and relapse prevention treatment for cocaine-dependent patients.   Addict Behav. 2001;26(2):167-180. doi:10.1016/S0306-4603(00)00098-8PubMedGoogle Scholar
    Schmitz  JM, Stotts  AL, Sayre  SL, DeLaune  KA, Grabowski  J.  Treatment of cocaine-alcohol dependence with naltrexone and relapse prevention therapy.   Am J Addict. 2004;13(4):333-341. doi:10.1080/10550490490480982PubMedGoogle Scholar
    Anton  RF, Moak  DH, Latham  P,  et al.  Naltrexone combined with either cognitive behavioral or motivational enhancement therapy for alcohol dependence.   J Clin Psychopharmacol. 2005;25(4):349-357. doi:10.1097/ Scholar
    Davidson  D, Gulliver  SB, Longabaugh  R, Wirtz  PW, Swift  R.  Building better cognitive-behavioral therapy: is broad-spectrum treatment more effective than motivational-enhancement therapy for alcohol-dependent patients treated with naltrexone?   J Stud Alcohol Drugs. 2007;68(2):238-247. doi:10.15288/jsad.2007.68.238PubMedGoogle Scholar
    De Wildt  WA, Schippers  GM, Van Den Brink  W, Potgieter  AS, Deckers  F, Bets  D.  Does psychosocial treatment enhance the efficacy of acamprosate in patients with alcohol problems?   Alcohol Alcohol. 2002;37(4):375-382. doi:10.1093/alcalc/37.4.375PubMedGoogle Scholar
    Ling  W, Hillhouse  M, Ang  A, Jenkins  J, Fahey  J.  Comparison of behavioral treatment conditions in buprenorphine maintenance.   Addiction. 2013;108(10):1788-1798. doi:10.1111/add.12266PubMedGoogle Scholar
    Oslin  DW, Lynch  KG, Pettinati  HM,  et al.  A placebo-controlled randomized clinical trial of naltrexone in the context of different levels of psychosocial intervention.   Alcohol Clin Exp Res. 2008;32(7):1299-1308. doi:10.1111/j.1530-0277.2008.00698.xPubMedGoogle Scholar
    Otto  MW, Hearon  BA, McHugh  RK,  et al.  A randomized, controlled trial of the efficacy of an interoceptive exposure-based CBT for treatment-refractory outpatients with opioid dependence.   J Psychoactive Drugs. 2014;46(5):402-411. doi:10.1080/02791072.2014.960110PubMedGoogle Scholar
    Pettinati  HM, Kampman  KM, Lynch  KG,  et al.  Gender differences with high-dose naltrexone in patients with co-occurring cocaine and alcohol dependence.   J Subst Abuse Treat. 2008;34(4):378-390. doi:10.1016/j.jsat.2007.05.011PubMedGoogle Scholar
    Rawson  RA, Huber  A, McCann  M,  et al.  A comparison of contingency management and cognitive-behavioral approaches during methadone maintenance treatment for cocaine dependence.   Arch Gen Psychiatry. 2002;59(9):817-824. doi:10.1001/archpsyc.59.9.817PubMedGoogle Scholar
    Schmitz  JM, Mooney  ME, Moeller  FG, Stotts  AL, Green  C, Grabowski  J.  Levodopa pharmacotherapy for cocaine dependence: choosing the optimal behavioral therapy platform.   Drug Alcohol Depend. 2008;94(1-3):142-150. doi:10.1016/j.drugalcdep.2007.11.004PubMedGoogle Scholar
    Tucker  T, Ritter  A, Maher  C, Jackson  H.  A randomized control trial of group counseling in a naltrexone treatment program.   J Subst Abuse Treat. 2004;27(4):277-288. doi:10.1016/j.jsat.2004.08.003PubMedGoogle Scholar
    Saunders  EC, McGovern  MP, Lambert-Harris  C, Meier  A, McLeman  B, Xie  H.  The impact of addiction medications on treatment outcomes for persons with co-occurring PTSD and opioid use disorders.   Am J Addict. 2015;24(8):722-731. doi:10.1111/ajad.12292PubMedGoogle Scholar
    Berner  MM, Wahl  S, Brueck  R,  et al; PREDICT study group.  The place of additional individual psychotherapy in the treatment of alcoholism: a randomized controlled study in nonresponders to anticraving medication-results of the PREDICT study.   Alcohol Clin Exp Res. 2014;38(4):1118-1125. doi:10.1111/acer.12317PubMedGoogle Scholar
    Fiellin  DA, Barry  DT, Sullivan  LE,  et al.  A randomized trial of cognitive behavioral therapy in primary care-based buprenorphine.   Am J Med. 2013;126(1):74.e11-74.e17. doi:10.1016/j.amjmed.2012.07.005PubMedGoogle Scholar
    American Psychiatric Association.  Diagnostic and Statistical Manual of Mental Disorders. 5th ed. American Psychiatric Association; 2013.
    Starosta  AN, Leeman  RF, Volpicelli  JR.  The BRENDA model: integrating psychosocial treatment and pharmacotherapy for the treatment of alcohol use disorders.   J Psychiatr Pract. 2006;12(2):80-89. doi:10.1097/00131746-200603000-00003PubMedGoogle Scholar
    Epstein  DH, Hawkins  WE, Covi  L, Umbricht  A, Preston  KL.  Cognitive-behavioral therapy plus contingency management for cocaine use: findings during treatment and across 12-month follow-up.   Psychol Addict Behav. 2003;17(1):73-82. doi:10.1037/0893-164X.17.1.73 PubMedGoogle Scholar
    Lundahl  BW, Kunz  C, Brownell  C, Tollefson  D, Burke  BL.  A meta-analysis of motivational interviewing: twenty-five years of empirical studies.   Res Soc Work Pract. 2010;20(2):137-160. doi:10.1177/1049731509347850 Google Scholar
    Prendergast  ML, Podus  D, Chang  E, Urada  D.  The effectiveness of drug abuse treatment: a meta-analysis of comparison group studies.   Drug Alcohol Depend. 2002;67(1):53-72. doi:10.1016/S0376-8716(02)00014-5 PubMedGoogle Scholar
    Fullerton  CA, Kim  M, Thomas  CP,  et al.  Medication-assisted treatment with methadone: assessing the evidence.   Psychiatr Serv. 2014;65(2):146-157. doi:10.1176/ PubMedGoogle Scholar
    Streeton  C, Whelan  G.  Naltrexone, a relapse prevention maintenance treatment of alcohol dependence: a meta-analysis of randomized controlled trials.   Alcohol Alcohol. 2001;36(6):544-552. doi:10.1093/alcalc/36.6.544 PubMedGoogle Scholar
    Carroll  KM, Onken  LS.  Behavioral therapies for drug abuse.   Am J Psychiatry. 2005;162(8):1452-1460. doi:10.1176/appi.ajp.162.8.1452 PubMedGoogle Scholar
    Ray  LAPD, Bujarski  S, Grodin  E,  et al.  State-of-the-art behavioral and pharmacological treatments for alcohol use disorder.   Am J Drug Alcohol Abuse. 2019;45(2):124-140. doi:10.1080/00952990.2018.1528265 PubMedGoogle Scholar
    Dutra  L, Stathopoulou  G, Basden  SL, Leyro  TM, Powers  MB, Otto  MW.  A meta-analytic review of psychosocial interventions for substance use disorders.   Am J Psychiatry. 2008;165(2):179-187. doi:10.1176/appi.ajp.2007.06111851 PubMedGoogle Scholar
    Haile  CN, Kosten  TR.  Pharmacotherapy for stimulant-related disorders.   Curr Psychiatry Rep. 2013;15(11):415-415. doi:10.1007/s11920-013-0415-y PubMedGoogle Scholar
    COMBINE Study Research Group.  Testing combined pharmacotherapies and behavioral interventions in alcohol dependence: rationale and methods.   Alcohol Clin Exp Res. 2003;27(7):1107-1122. doi:10.1111/j.1530-0277.2003.tb02873.x PubMedGoogle Scholar
    Pettinati  HM, Volpicelli  JR, Pierce  JD  Jr, O’Brien  CP.  Improving naltrexone response: an intervention for medical practitioners to enhance medication compliance in alcohol dependent patients.   J Addict Dis. 2000;19(1):71-83. doi:10.1300/J069v19n01_06 PubMedGoogle Scholar
    Wilson  DB.  Meta-analyses in alcohol and other drug abuse treatment research.   Addiction. 2000;95(suppl 3):S419-S438. doi:10.1046/j.1360-0443.95.11s3.9.x PubMedGoogle Scholar
    Bellg  AJ, Borrelli  B, Resnick  B,  et al; Treatment Fidelity Workgroup of the NIH Behavior Change Consortium.  Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium.   Health Psychol. 2004;23(5):443-451. doi:10.1037/0278-6133.23.5.443 PubMedGoogle Scholar
    Kiluk  BD, Fitzmaurice  GM, Strain  EC, Weiss  RD.  What defines a clinically meaningful outcome in the treatment of substance use disorders: reductions in direct consequences of drug use or improvement in overall functioning?   Addiction. 2019;114(1):9-15. doi:10.1111/add.14289 PubMedGoogle Scholar