[Skip to Navigation]
Sign In
Figure 1.  Network Plot
Network Plot
Figure 2.  Aggregated Meta-analytic Effect Sizes for Efficacy at Posttreatment
Aggregated Meta-analytic Effect Sizes for Efficacy at Posttreatment

The number in each cell shows the relative treatment effect size between the column-defining treatment and the row-defining treatment. The outcome is depression symptom severity in Patient Health Questionnaire–9 (PHQ-9), and results are presented as mean difference (MD) (95% CIs). Estimates in light blue are derived from aggregated data network meta-analysis, where MD less than 0 favors the column-defining treatment of each cell. Estimates in light brown are derived from the pairwise meta-analyses, where MD less than 0 favors the row-defining treatment of each cell. iCBT indicates internet-based cognitive behavioral therapy; TAU, treatment as usual; WL, waiting list.

Figure 3.  Aggregated Meta-analytic Effect Sizes for Efficacy Over the Long Term
Aggregated Meta-analytic Effect Sizes for Efficacy Over the Long Term

Interpretation of this Figure as per Figure 2. iCBT indicates internet-based cognitive behavioral therapy; TAU, treatment as usual; WL, waiting list.

Table 1.  Study Characteristics
Study Characteristics
Table 2.  Case Examples of Individual Patient Response to Guided vs Unguided iCBT vs TAU
Case Examples of Individual Patient Response to Guided vs Unguided iCBT vs TAU
1.
McLaughlin  KA.  The public health impact of major depression: a call for interdisciplinary prevention efforts.   Prev Sci. 2011;12(4):361-371. doi:10.1007/s11121-011-0231-8PubMedGoogle ScholarCrossref
2.
Ustün  TB, Ayuso-Mateos  JL, Chatterji  S, Mathers  C, Murray  CJL.  Global burden of depressive disorders in the year 2000.   Br J Psychiatry. 2004;184(5):386-392. doi:10.1192/bjp.184.5.386PubMedGoogle ScholarCrossref
3.
Wittchen  H-U, Jacobi  F, Rehm  J,  et al.  The size and burden of mental disorders and other disorders of the brain in Europe 2010.   Eur Neuropsychopharmacol. 2011;21(9):655-679. doi:10.1016/j.euroneuro.2011.07.018PubMedGoogle ScholarCrossref
4.
Chisholm  D, Sweeny  K, Sheehan  P,  et al.  Scaling-up treatment of depression and anxiety: a global return on investment analysis.   Lancet Psychiatry. 2016;3(5):415-424. doi:10.1016/S2215-0366(16)30024-4PubMedGoogle ScholarCrossref
5.
Cuijpers  P, Noma  H, Karyotaki  E, Vinkers  CH, Cipriani  A, Furukawa  TA.  A network meta-analysis of the effects of psychotherapies, pharmacotherapies and their combination in the treatment of adult depression.   World Psychiatry. 2020;19(1):92-107. doi:10.1002/wps.20701PubMedGoogle ScholarCrossref
6.
Patel  V, Saxena  S, Lund  C,  et al.  The Lancet Commission on global mental health and sustainable development.   Lancet. 2018;392(10157):1553-1598. doi:10.1016/S0140-6736(18)31612-XPubMedGoogle ScholarCrossref
7.
Holmes  EA, O’Connor  RC, Perry  VH,  et al.  Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science.   Lancet Psychiatry. 2020;7(6):547-560. doi:10.1016/S2215-0366(20)30168-1PubMedGoogle ScholarCrossref
8.
Wind  TR, Rijkeboer  M, Andersson  G, Riper  H.  The COVID-19 pandemic: the ‘black swan’ for mental health care and a turning point for e-health.   Internet Interv. 2020;20:100317. doi:10.1016/j.invent.2020.100317PubMedGoogle Scholar
9.
Andersson  G, Titov  N, Dear  BF, Rozental  A, Carlbring  P.  Internet-delivered psychological treatments: from innovation to implementation.   World Psychiatry. 2019;18(1):20-28. doi:10.1002/wps.20610PubMedGoogle ScholarCrossref
10.
Karyotaki  E, Riper  H, Twisk  J,  et al.  Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data.   JAMA Psychiatry. 2017;74(4):351-359. doi:10.1001/jamapsychiatry.2017.0044PubMedGoogle ScholarCrossref
11.
Fairburn  CG, Patel  V.  The impact of digital technology on psychological treatments and their dissemination.   Behav Res Ther. 2017;88:19-25. doi:10.1016/j.brat.2016.08.012PubMedGoogle ScholarCrossref
12.
Cuijpers  P, Noma  H, Karyotaki  E, Cipriani  A, Furukawa  TA.  Effectiveness and acceptability of cognitive behavior therapy delivery formats in adults with depression: a network meta-analysis.   JAMA Psychiatry. 2019;76(7):700-707. doi:10.1001/jamapsychiatry.2019.0268PubMedGoogle ScholarCrossref
13.
Debray  TP, Schuit  E, Efthimiou  O,  et al; GetReal Workpackage.  An overview of methods for network meta-analysis using individual participant data: when do benefits arise?   Stat Methods Med Res. 2018;27(5):1351-1364. doi:10.1177/0962280216660741PubMedGoogle ScholarCrossref
14.
Efthimiou  O, Debray  TP, van Valkenhoef  G,  et al; GetReal Methods Review Group.  GetReal in network meta-analysis: a review of the methodology.   Res Synth Methods. 2016;7(3):236-263. doi:10.1002/jrsm.1195PubMedGoogle ScholarCrossref
15.
Riley  RD, Lambert  PC, Abo-Zaid  G.  Meta-analysis of individual participant data: rationale, conduct, and reporting.   BMJ. 2010;340:c221. doi:10.1136/bmj.c221PubMedGoogle ScholarCrossref
16.
Karyotaki  E, Furukawa  TA, Efthimiou  O, Riper  H, Cuijpers  P.  Guided or self-guided internet-based cognitive-behavioural therapy (iCBT) for depression? Study protocol of an individual participant data network meta-analysis.   BMJ Open. 2019;9(6):e026820. doi:10.1136/bmjopen-2018-026820PubMedGoogle Scholar
17.
Kleiboer  A, Smit  J, Bosmans  J,  et al.  European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries.   Trials. 2016;17(1):387. doi:10.1186/s13063-016-1511-1PubMedGoogle ScholarCrossref
18.
Cuijpers  P, Schuurmans  J.  Self-help interventions for anxiety disorders: an overview.   Curr Psychiatry Rep. 2007;9(4):284-290. doi:10.1007/s11920-007-0034-6PubMedGoogle ScholarCrossref
19.
Cuijpers  P, Karyotaki  E, Ciharova  M.  A Meta-analytic Database of Randomised Trials on Psychotherapies for Depression. Open Science Foundation. 2019.
20.
Kessler  RC, van Loo  HM, Wardenaar  KJ,  et al.  Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder.   Epidemiol Psychiatr Sci. 2017;26(1):22-36. doi:10.1017/S2045796016000020PubMedGoogle ScholarCrossref
21.
Bockting  CL, Hollon  SD, Jarrett  RB, Kuyken  W, Dobson  K.  A lifetime approach to major depressive disorder: the contributions of psychological interventions in preventing relapse and recurrence.   Clin Psychol Rev. 2015;41:16-26. doi:10.1016/j.cpr.2015.02.003PubMedGoogle ScholarCrossref
22.
Higgins  J, Green  S. Assessing risk of bias in included studies.  Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 2011.
23.
Kroenke  K, Spitzer  RL, Williams  JB.  The PHQ-9: validity of a brief depression severity measure.   J Gen Intern Med. 2001;16(9):606-613. doi:10.1046/j.1525-1497.2001.016009606.xPubMedGoogle ScholarCrossref
24.
Wahl  I, Löwe  B, Bjorner  JB,  et al.  Standardization of depression measurement: a common metric was developed for 11 self-report depression measures.   J Clin Epidemiol. 2014;67(1):73-86. doi:10.1016/j.jclinepi.2013.04.019PubMedGoogle ScholarCrossref
25.
König  J, Krahn  U, Binder  H.  Visualizing the flow of evidence in network meta-analysis and characterizing mixed treatment comparisons.   Stat Med. 2013;32(30):5414-5429. doi:10.1002/sim.6001PubMedGoogle ScholarCrossref
26.
White  IR, Barrett  JK, Jackson  D, Higgins  JP.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression.   Res Synth Methods. 2012;3(2):111-125. doi:10.1002/jrsm.1045PubMedGoogle ScholarCrossref
27.
Quartagno  M, Grund  S, Carpenter  J.  jomo: a flexible package for two-level joint modelling multiple imputation.   R Journal. 2019. doi:10.32614/RJ-2019-028Google Scholar
28.
Rücker  G, Schwarzer  G, Krahn  U. netmeta: Network meta-analysis using Frequentist methods. R package version 0.9-8, 2018.
29.
Rubin  DB.  Multiple Imputation for Nonresponse in Surveys. Vol 81: John Wiley & Sons; 2004.
30.
Plummer  M. rjags: Bayesian graphical models using MCMC. R package version 4-8. 2018.
31.
Egger  M, Davey Smith  G, Schneider  M, Minder  C.  Bias in meta-analysis detected by a simple, graphical test.   BMJ. 1997;315(7109):629-634. doi:10.1136/bmj.315.7109.629PubMedGoogle ScholarCrossref
32.
Rhodes  KM, Turner  RM, Higgins  JP.  Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data.   J Clin Epidemiol. 2015;68(1):52-60. doi:10.1016/j.jclinepi.2014.08.012PubMedGoogle ScholarCrossref
33.
Andersson  G, Bergström  J, Holländare  F, Carlbring  P, Kaldo  V, Ekselius  L.  Internet-based self-help for depression: randomised controlled trial.   Br J Psychiatry. 2005;187(5):456-461. doi:10.1192/bjp.187.5.456PubMedGoogle ScholarCrossref
34.
Beevers  CG, Pearson  R, Hoffman  JS, Foulser  AA, Shumake  J, Meyer  B.  Effectiveness of an internet intervention (Deprexis) for depression in a united states adult sample: a parallel-group pragmatic randomized controlled trial.   J Consult Clin Psychol. 2017;85(4):367-380. doi:10.1037/ccp0000171PubMedGoogle ScholarCrossref
35.
Berger  T, Hämmerli  K, Gubser  N, Andersson  G, Caspar  F.  Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help.   Cogn Behav Ther. 2011;40(4):251-266. doi:10.1080/16506073.2011.616531PubMedGoogle ScholarCrossref
36.
Choi  I, Zou  J, Titov  N,  et al.  Culturally attuned Internet treatment for depression amongst Chinese Australians: a randomised controlled trial.   J Affect Disord. 2012;136(3):459-468. doi:10.1016/j.jad.2011.11.003PubMedGoogle ScholarCrossref
37.
Christensen  H, Griffiths  KM, Jorm  AF.  Delivering interventions for depression by using the internet: randomised controlled trial.   BMJ. 2004;328(7434):265. doi:10.1136/bmj.37945.566632.EEPubMedGoogle ScholarCrossref
38.
de Graaf  LE, Gerhards  SA, Arntz  A,  et al.  Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.   Br J Psychiatry. 2009;195(1):73-80. doi:10.1192/bjp.bp.108.054429PubMedGoogle ScholarCrossref
39.
Farrer  L, Christensen  H, Griffiths  KM, Mackinnon  A.  Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial.   PLoS One. 2011;6(11):e28099. doi:10.1371/journal.pone.0028099PubMedGoogle Scholar
40.
Forand  NR, Barnett  JG, Strunk  DR, Hindiyeh  MU, Feinberg  JE, Keefe  JR.  Efficacy of guided iCBT for depression and mediation of change by cognitive skill acquisition.   Behav Ther. 2018;49(2):295-307. doi:10.1016/j.beth.2017.04.004PubMedGoogle ScholarCrossref
41.
Forsell  E, Bendix  M, Holländare  F,  et al.  Internet delivered cognitive behavior therapy for antenatal depression: a randomised controlled trial.   J Affect Disord. 2017;221:56-64. doi:10.1016/j.jad.2017.06.013PubMedGoogle ScholarCrossref
42.
Geraedts  AS, Kleiboer  AM, Wiezer  NM, van Mechelen  W, Cuijpers  P.  Short-term effects of a web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial.   J Med Internet Res. 2014;16(5):e121. doi:10.2196/jmir.3185PubMedGoogle Scholar
43.
Gilbody  S, Littlewood  E, Hewitt  C,  et al; REEACT Team.  Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial.   BMJ. 2015;351:h5627. doi:10.1136/bmj.h5627PubMedGoogle ScholarCrossref
44.
Gilbody  S, Brabyn  S, Lovell  K,  et al; REEACT collaborative.  Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial.   Br J Psychiatry. 2017;210(5):362-367. doi:10.1192/bjp.bp.116.192435PubMedGoogle ScholarCrossref
45.
Hallgren  M, Helgadóttir  B, Herring  MP,  et al.  Exercise and internet-based cognitive-behavioural therapy for depression: multicentre randomised controlled trial with 12-month follow-up.   Br J Psychiatry. 2016;209(5):414-420. doi:10.1192/bjp.bp.115.177576PubMedGoogle ScholarCrossref
46.
Johansson  R, Sjöberg  E, Sjögren  M,  et al.  Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial.   PLoS One. 2012;7(5):e36905. doi:10.1371/journal.pone.0036905PubMedGoogle Scholar
47.
Kessler  D, Lewis  G, Kaur  S,  et al.  Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial.   Lancet. 2009;374(9690):628-634. doi:10.1016/S0140-6736(09)61257-5PubMedGoogle ScholarCrossref
48.
Kivi  M, Eriksson  MC, Hange  D,  et al.  Internet-based therapy for mild to moderate depression in Swedish primary care: short term results from the PRIM-NET randomized controlled trial.   Cogn Behav Ther. 2014;43(4):289-298. doi:10.1080/16506073.2014.921834PubMedGoogle ScholarCrossref
49.
Klein  JP, Berger  T, Schröder  J,  et al.  Effects of a psychological internet intervention in the treatment of mild to moderate depressive symptoms: results of the EVIDENT study, a randomized controlled trial.   Psychother Psychosom. 2016;85(4):218-228. doi:10.1159/000445355PubMedGoogle ScholarCrossref
50.
Lintvedt  OK, Griffiths  KM, Sørensen  K,  et al.  Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial.   Clin Psychol Psychother. 2013;20(1):10-27. doi:10.1002/cpp.770PubMedGoogle ScholarCrossref
51.
Meyer  B, Berger  T, Caspar  F, Beevers  CG, Andersson  G, Weiss  M.  Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.   J Med Internet Res. 2009;11(2):e15. doi:10.2196/jmir.1151PubMedGoogle Scholar
52.
Meyer  B, Bierbrodt  J, Schröder  J,  et al  Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial.   Internet Interventions. 2015;2(1):48-59. doi:10.1016/j.invent.2014.12.003Google ScholarCrossref
53.
Milgrom  J, Danaher  BG, Gemmill  AW,  et al.  Internet cognitive behavioral therapy for women with postnatal depression: a randomized controlled trial of MumMoodBooster.   J Med Internet Res. 2016;18(3):e54. doi:10.2196/jmir.4993PubMedGoogle Scholar
54.
Mira  A, Bretón-López  J, García-Palacios  A, Quero  S, Baños  RM, Botella  C.  An Internet-based program for depressive symptoms using human and automated support: a randomized controlled trial.   Neuropsychiatr Dis Treat. 2017;13:987-1006. doi:10.2147/NDT.S130994PubMedGoogle ScholarCrossref
55.
Mohr  DC, Duffecy  J, Ho  J,  et al.  A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression.   PLoS One. 2013;8(8):e70086. doi:10.1371/journal.pone.0070086PubMedGoogle Scholar
56.
Montero-Marín  J, Araya  R, Pérez-Yus  MC,  et al.  An internet-based intervention for depression in primary Care in Spain: a randomized controlled trial.   J Med Internet Res. 2016;18(8):e231. doi:10.2196/jmir.5695PubMedGoogle Scholar
57.
Moritz  S, Schilling  L, Hauschildt  M, Schröder  J, Treszl  A.  A randomized controlled trial of internet-based therapy in depression.   Behav Res Ther. 2012;50(7-8):513-521. doi:10.1016/j.brat.2012.04.006PubMedGoogle ScholarCrossref
58.
Perini  S, Titov  N, Andrews  G.  Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial.   Aust N Z J Psychiatry. 2009;43(6):571-578. doi:10.1080/00048670902873722PubMedGoogle ScholarCrossref
59.
Phillips  R, Schneider  J, Molosankwe  I,  et al.  Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention.   Psychol Med. 2014;44(4):741-752. doi:10.1017/S0033291713001323PubMedGoogle ScholarCrossref
60.
Pugh  NE, Hadjistavropoulos  HD, Dirkse  D.  A randomised controlled trial of therapist-assisted, internet-delivered cognitive behavior therapy for women with maternal depression.   PLoS One. 2016;11(3):e0149186. doi:10.1371/journal.pone.0149186PubMedGoogle Scholar
61.
Richards  D, Timulak  L, O’Brien  E,  et al.  A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression.   Behav Res Ther. 2015;75:20-31. doi:10.1016/j.brat.2015.10.005PubMedGoogle ScholarCrossref
62.
Rosso  IM, Killgore  WD, Olson  EA,  et al.  Internet-based cognitive behavior therapy for major depressive disorder: a randomized controlled trial.   Depress Anxiety. 2017;34(3):236-245. doi:10.1002/da.22590PubMedGoogle ScholarCrossref
63.
Ruwaard  J, Schrieken  B, Schrijver  M,  et al.  Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up.   Cogn Behav Ther. 2009;38(4):206-221. doi:10.1080/16506070802408086PubMedGoogle ScholarCrossref
64.
Sheeber  LB, Seeley  JR, Feil  EG,  et al.  Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression.   J Consult Clin Psychol. 2012;80(5):739-749. doi:10.1037/a0028820PubMedGoogle ScholarCrossref
65.
Smith  J, Newby  JM, Burston  N,  et al.  Help from home for depression: a randomised controlled trial comparing internet-delivered cognitive behaviour therapy with bibliotherapy for depression.   Internet Interv. 2017;9:25-37. doi:10.1016/j.invent.2017.05.001PubMedGoogle ScholarCrossref
66.
Spek  V, Nyklícek  I, Smits  N,  et al.  Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial.   Psychol Med. 2007;37(12):1797-1806. doi:10.1017/S0033291707000542PubMedGoogle ScholarCrossref
67.
Vernmark  K, Lenndin  J, Bjärehed  J,  et al.  Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression.   Behav Res Ther. 2010;48(5):368-376. doi:10.1016/j.brat.2010.01.005PubMedGoogle ScholarCrossref
68.
Warmerdam  L, van Straten  A, Twisk  J, Riper  H, Cuijpers  P.  Internet-based treatment for adults with depressive symptoms: randomized controlled trial.   J Med Internet Res. 2008;10(4):e44. doi:10.2196/jmir.1094PubMedGoogle Scholar
69.
Yeung  A, Wang  F, Feng  F,  et al.  Outcomes of an online computerized cognitive behavioral treatment program for treating chinese patients with depression: a pilot study.   Asian J Psychiatr. 2018;38:102-107. doi:10.1016/j.ajp.2017.11.007PubMedGoogle ScholarCrossref
70.
Zagorscak  P, Heinrich  M, Sommer  D, Wagner  B, Knaevelsrud  C.  Benefits of individualized feedback in internet-based interventions for depression: a randomized controlled trial.   Psychother Psychosom. 2018;87(1):32-45. doi:10.1159/000481515PubMedGoogle ScholarCrossref
71.
Williams  AD, O’Moore  K, Blackwell  SE, Smith  J, Holmes  EA, Andrews  G.  Positive imagery cognitive bias modification (CBM) and internet-based cognitive behavioral therapy (iCBT): a randomized controlled trial.   J Affect Disord. 2015;178:131-141. doi:10.1016/j.jad.2015.02.026PubMedGoogle ScholarCrossref
72.
Titov  N, Andrews  G, Davies  M, McIntyre  K, Robinson  E, Solley  K.  Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance.   PLoS One. 2010;5(6):e10939. doi:10.1371/journal.pone.0010939PubMedGoogle Scholar
73.
Titov  N, Dear  BF, Ali  S,  et al.  Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial.   Behav Ther. 2015;46(2):193-205. doi:10.1016/j.beth.2014.09.008PubMedGoogle ScholarCrossref
74.
Löbner  M, Pabst  A, Stein  J,  et al.  Computerized cognitive behavior therapy for patients with mild to moderately severe depression in primary care: a pragmatic cluster randomized controlled trial (@ktiv).   J Affect Disord. 2018;238:317-326. doi:10.1016/j.jad.2018.06.008PubMedGoogle ScholarCrossref
75.
Karyotaki  E, Ebert  DD, Donkin  L,  et al.  Do guided internet-based interventions result in clinically relevant changes for patients with depression? an individual participant data meta-analysis.   Clin Psychol Rev. 2018;63:80-92. doi:10.1016/j.cpr.2018.06.007PubMedGoogle ScholarCrossref
76.
Kroenke  K, Strine  TW, Spitzer  RL, Williams  JB, Berry  JT, Mokdad  AH.  The PHQ-8 as a measure of current depression in the general population.   J Affect Disord. 2009;114(1-3):163-173. doi:10.1016/j.jad.2008.06.026PubMedGoogle ScholarCrossref
77.
Khaled  SM.  Prevalence and potential determinants of subthreshold and major depression in the general population of Qatar.   J Affect Disord. 2019;252:382-393. doi:10.1016/j.jad.2019.04.056PubMedGoogle ScholarCrossref
78.
Forsell  E, Jernelöv  S, Blom  K,  et al.  Proof of concept for an adaptive treatment strategy to prevent failures in internet-delivered CBT: a single-blind randomized clinical trial with insomnia patients.   Am J Psychiatry. 2019;176(4):315-323. doi:10.1176/appi.ajp.2018.18060699PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    January 20, 2021

    Internet-Based Cognitive Behavioral Therapy for Depression: A Systematic Review and Individual Patient Data Network Meta-analysis

    Author Affiliations
    • 1Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
    • 2Department of Clinical Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
    • 3Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts
    • 4Department of Psychiatry, University of Oxford, Oxford, England
    • 5Amsterdam Public Health Research Institute, Amsterdam, the Netherlands
    • 6Department of Clinical Psychology and Psychotherapy, University of Wuppertal, Wuppertal, Germany
    • 7Department of Health Promotion and Human Behavior, Department of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
    JAMA Psychiatry. 2021;78(4):361-371. doi:10.1001/jamapsychiatry.2020.4364
    Key Points

    Question  What are the patient-specific relative outcomes of guided vs unguided internet-based cognitive behavioral therapy (iCBT) for depression over the short and the long term?

    Findings  In this systematic review and meta-analysis of 39 studies comprising 9751 participants, individuals with mild/subthreshold depression was associated with little or no benefit from therapeutic guidance, while guided iCBT was associated with more effectiveness in individuals with moderate and severe depression. Both iCBT modalities outperformed the TAU regardless of depression severity.

    Meaning  Although guided iCBT was associated with greater improvement compared with unguided iCBT on average, many people with depression may still benefit from the iCBT without therapeutic guidance, and optimizing treatment assignment would considerably expand treatment coverage worldwide.

    Abstract

    Importance  Personalized treatment choices would increase the effectiveness of internet-based cognitive behavioral therapy (iCBT) for depression to the extent that patients differ in interventions that better suit them.

    Objective  To provide personalized estimates of short-term and long-term relative efficacy of guided and unguided iCBT for depression using patient-level information.

    Data Sources  We searched PubMed, Embase, PsycInfo, and Cochrane Library to identify randomized clinical trials (RCTs) published up to January 1, 2019.

    Study Selection  Eligible RCTs were those comparing guided or unguided iCBT against each other or against any control intervention in individuals with depression. Available individual patient data (IPD) was collected from all eligible studies. Depression symptom severity was assessed after treatment, 6 months, and 12 months after randomization.

    Data Extraction and Synthesis  We conducted a systematic review and IPD network meta-analysis and estimated relative treatment effect sizes across different patient characteristics through IPD network meta-regression.

    Main Outcomes and Measures  Patient Health Questionnaire–9 (PHQ-9) scores.

    Results  Of 42 eligible RCTs, 39 studies comprising 9751 participants with depression contributed IPD to the IPD network meta-analysis, of which 8107 IPD were synthesized. Overall, both guided and unguided iCBT were associated with more effectiveness as measured by PHQ-9 scores than control treatments over the short term and the long term. Guided iCBT was associated with more effectiveness than unguided iCBT (mean difference [MD] in posttreatment PHQ-9 scores, −0.8; 95% CI, −1.4 to −0.2), but we found no evidence of a difference at 6 or 12 months following randomization. Baseline depression was found to be the most important modifier of the relative association for efficacy of guided vs unguided iCBT. Differences between unguided and guided iCBT in people with baseline symptoms of subthreshold depression (PHQ-9 scores 5-9) were small, while guided iCBT was associated with overall better outcomes in patients with baseline PHQ-9 greater than 9.

    Conclusions and Relevance  In this network meta-analysis with IPD, guided iCBT was associated with more effectiveness than unguided iCBT for individuals with depression, benefits were more substantial in individuals with moderate to severe depression. Unguided iCBT was associated with similar effectiveness among individuals with symptoms of mild/subthreshold depression. Personalized treatment selection is entirely possible and necessary to ensure the best allocation of treatment resources for depression.

    Introduction

    Depression is a major public health issue, taking an enormous toll on individuals, public health care systems, and society as a whole.1-3 Broadly accessible treatment is required to reduce this burden.4 Both psychotherapy and pharmacotherapy can treat depression effectively.5 Nevertheless, psychotherapy is unavailable to most of the world’s population owing to costs, availability of trained clinicians, and stigma.6 Further, the current coronavirus disease 2019 (COVID-19) pandemic has displaced and dislocated mental health services, while social and community containment measures, associated distress, loss, and potential financial difficulties are likely to be long lasting and impactful.7,8

    Over the past 20 years, the mental health care available for depression has undergone a major technological revolution. Psychological interventions, such as cognitive behavioral therapy (CBT), are increasingly delivered over the internet (iCBT).9 These interventions can be delivered either with or without therapeutic support, usually termed guided and unguided iCBT. Unguided iCBT is more scalable and affordable,10,11 but previous studies have shown that guidance generally results in better outcomes.12 These studies have mainly reported group average effects of iCBT, providing little insight into patient attributes that may differentiate outcomes. It may be that some patients are helped as much by unguided as guided iCBT. If so, knowledge of attributes that predict such individual differences could be valuable in guiding optimized resource allocation. Doing this is challenging because extensive examination of prognostic moderator variable requires thousands of patients to be compared in order to achieve sufficient statistical power.

    Individual patient data network meta-analysis (IPD-NMA) is an evidence synthesis method that can be used to estimate the relative efficacy of multiple competing interventions by pooling individual patient data across multiple studies.13,14 Because this approach uses patient-level data, interactions between baseline individual characteristics and treatment type can be examined with more power than in individual trials.15 We performed a systematic review and IPD-NMA to investigate the relative efficacy of guided vs unguided iCBT for depression and the influence of patient characteristics on their relative efficacy.

    Methods

    The methods are described in detail in our study protocol (for discrepancies between the study protocol and this IPD-NMA, see the eAppendix in the Supplement).16

    Eligibility Criteria

    Eligible studies included (1) randomized clinical trials (RCTs); (2) comparing either guided and unguided iCBT against each other, or against any type of control condition (treatment as usual, waiting list); (3) in adults with depressive symptoms, as established by specified cutoffs on self-report scales or diagnostic interviews. Studies were excluded if the intervention (1) did not include cognitive restructuring as one of the main components; (2) was delivered only through smartphones; (3) was blended with face-to-face treatment17; and (5) targeted primarily a physical illness. No language restrictions were applied.

    Unguided iCBT was defined as CBT delivered via the internet where automated and technical support was permitted, but not support related to the therapeutic content.18 Guided iCBT was defined as CBT delivered via the internet that involved therapeutic support, either synchronous or asynchronous, delivered by a professional or a paraprofessional (nonspecialists in mental health care but trained to deliver iCBT).

    Study Identification and Selection Process

    We used our established database of RCTs examining psychological treatments for adult depression. This database is based on ongoing systematic searches of PubMed, Embase, PsycInfo, and the Cochrane Library, and has been described in detail elsewhere.19 The search algorithm for PubMed is available in the eAppendix in the Supplement. We also searched reference lists from previous meta-analyses and asked primary authors whether they were aware of other eligible studies.

    Data Collection and Data Items

    The authors provided deidentified data for each patient, where available: baseline, 6-month, and 12-month postrandomization scores of depressive symptoms; age; sex; educational level (primary, secondary, tertiary education); relationship status (in relationship yes/no); employment status (employed, unemployed, student, other); and treatment adherence (number of completed sessions/total number of sessions). Variables were chosen based on previous literature20,21 and availability across included trials. We also extracted study-level information (ie, recruitment method). After obtaining all eligible data sets, 2 independent authors merged all eligible data sets (E.K. and C.M.) and checked the data for accuracy against the published reports of the articles.

    Risk of Bias Assessment

    Two independent authors (E.K. and F.Mg.B.) assessed the risk of bias in the included studies using 4 items of the Cochrane Risk of bias tool: (4) random sequence generation, (2) allocation concealment, (3) selective outcome reporting, and (4) other possible sources of bias (ie, baseline differences between the groups).22 We did not evaluate blinding of participants, personnel, and assessors because our primary outcome is based on self-report measures, and blinding is rarely possible in psychotherapy research. We considered a trial at high risk of attrition bias if it had overall more than 50% study dropout and/or more than 30% imbalance in missing outcomes between groups.16

    Data Analysis

    This NMA focused on the differential effects of the examined interventions on depression symptom severity on the Patient Health Questionnaire–9 (PHQ-9)23 at posttreatment. The PHQ-9 was the most commonly used scale across the eligible studies (available for 4703 participants across 15 studies). Other depression scales were converted into PHQ-9 scores using established conversion algorithms.24 When no conversion algorithms existed, the study was excluded. Outcomes were assessed at posttreatment, 6 months, and 12 months following randomization. To assess transitivity in the network,14 we checked the distribution of possible effect size modifiers in the studies grouped by comparison. We assessed heterogeneity by estimating prediction intervals for all pairwise meta-analyses, and via the estimated values of τ for aggregate data NMAs (AD-NMA). We checked inconsistency in the networks using a local approach (back-calculation)25 as well as a global test (design-by-treatment).26 To retain patients with missing outcomes in analyses, we created 20 multiply imputed data sets using the jomo package in R (The R Foundation), taking into account the stratification of patients in studies.27 In each multiply imputed data set we performed PMAs after grouping studies comparing the same 2 interventions, as well as AD-NMA using the netmeta package in R.28 We assumed random treatment effects, allowing for a common heterogeneity parameter (τ) for all comparisons in the network. This parameter corresponds to the standard deviation of the random effects of across trials (assumed normal). We synthesized results from all data sets using the Rubin rules.29

    As a sensitivity analysis, we performed a complete case analysis, ie, only including patients with information on their final outcome at postintervention and follow-up assessments. In addition, we ran a series of subgroup NMAs to test possible differences in the examined studies: (1) commercial vs nonprofit iCBT programs; (2) guidance provided by paraprofessionals/lay therapists vs BA/MSc/PhD student in clinical psychology vs licensed psychologists and/or psychotherapists; (3) studies conducted in the United States vs other; and (4) studies that originally used PHQ-9 vs other. To facilitate clinical interpretation of our findings, we calculated response rates (≥50% reduction of the baseline symptoms) for the comparison guided vs unguided iCBT. To further explore the association of baseline severity with response rates, we ran a subgroup analysis using baseline PHQ-9 scores: less than 10 (mild depressive symptoms); 10 to 15 (moderate depression); 15 to 19 (moderately severe depression); and more than 19 (severe depression).

    Next, we performed a separate bayesian IPD network meta-regression in each multiply imputed data set. To avoid possible issues with overfitting and aiming at better generalizability of results, we used bayesian least absolute shrinkage and selection operator to model treatment-covariate interactions. Bayesian analyses were performed using rjags in R.30

    To assess small study effects (publication bias) that might compromise the validity of our results, we created contour-enhanced funnel plots and performed the Egger test31 to check for asymmetry after grouping active treatments. To explore whether there were systematic differences between available and unavailable studies that did not provide IPD, we synthesized the latter in AD-NMA, and compared results with the former. More details about the statistical methods are provided in the eAppendix in the Supplement. Finally, we used the shiny package in R to develop a web application to showcase all results from our IPD network meta-regression model. To evaluate the certainty of evidence, we used the GRADE methodology (eAppendix in the Supplement).32

    Results
    Study Selection and IPD Obtained

    The PRISMA flow diagram shows the study selection process (eAppendix in the Supplement). Up to January 2019, we screened 2552 full texts and identified 42 eligible RCTs, 39 of which provided patient-level data on 9751 individuals.33-71 Three studies (7%) did not contribute their data owing to university regulations72,73 or administrative burden.74

    Study Characteristics

    Table 1 presents the study characteristics. Twenty-four of 39 included studies recruited participants in the community, 11 through clinical or mixed sources, and 4 used other recruitment sources (ie, workplace). Twenty-one studies compared the effects of guided iCBT with control, and 13 studies compared unguided iCBT with control. Control groups included treatment as usual (n = 15) and waiting list (n = 22). Five studies compared guided and unguided iCBT directly with each other. Twelve studies used a commercial iCBT program, while in 27 RCTs the iCBT program was developed in house/nonprofit. The interventions comprised 5 to 18 online sessions (mean [SD], 8.0 [2.8]) delivered more than 5 to 14 weeks (mean [SD], 9 [2.5] weeks). In guided iCBT groups, guidance was provided by paraprofessionals/lay therapists (n = 6), BA/MSc/PhD student in clinical psychology (n = 14), and licensed psychologists and/or psychotherapists (n = 5). Figure 1 shows the network graph. The studies were conducted across 12 countries (across Europe, North America, and China).

    Risk of Bias Assessment

    Overall, risk of bias was low across the included studies. All but 1 study had an acceptable sequence generation and allocation concealment. One trial was at high risk of selection bias because the study recruiter drew colored balls from a bag to randomize.62 We had access to the full databases of the included studies; thus, we could use all available depression measures regardless of whether they have been included in the published reports of the trials. Therefore, all trials were at low risk of selective reporting. Moreover, the included trials were free from other sources of bias except for 1 study that reported baseline imbalances.36 Following our protocol,16 we did not evaluate performance and assessment bias. However, we acknowledge that performance bias can occur and accordingly, we have considered this in our GRADE assessment (eAppendix in the Supplement). Finally, we retained all randomized individuals in our analysis, and thus our findings are at relatively low risk of attrition bias.

    IPD Synthesis

    Of the 9751 participants in the 39 studies, 1071 (10.9%) did not have usable information on our primary outcome measure (ie, there was no established algorithm to convert the depression measure into PHQ-9 scores34,45) and were excluded from further analyses. We also excluded 312 participants because their baseline depression scores were below the threshold of mild depressive symptoms (PHQ-9 score < 5). Finally, 1 study had 50% dropout in the intervention and 0% in the control.61 Following the protocol, we excluded this study from all subsequent analyses (eAppendix in the Supplement). Thus, we report the outcomes of 8107 patients across 36 studies. The PHQ-9 mean (SD) scores at baseline were 13.7 (4.3) for guided iCBT, 14.2 (4.9) for unguided iCBT, 15.2 (5.3) for treatment as usual (TAU), and 13.2 (4.6) for waiting list and at posttreatment, 7.6 (5.0), 9.2 (5.9), 9.8 (5.5), and 12.0 (6.4) for guided iCBT, unguided iCBT, TAU, and waiting list, respectively. Overall, assessment of transitivity did not indicate systematic differences across comparisons.

    Aggregated Data Network Meta-analyses

    All pairwise meta-analyses are reported in the eAppendix in the Supplement. There was evidence of considerable heterogeneity in most comparisons. The outcomes of AD-NMAs at posttreatment assessment (Figure 2) indicated that guided iCBT was more effective than unguided iCBT (mean difference [MD] in PHQ-9 score, −0.8; 95% CI, −1.4 to −0.2), TAU (MD, −1.7; 95% CI, −2.3 to −1.1), and waiting list (MD, −3.3; 95% CI, −3.9 to −2.6). Unguided iCBT reduced symptoms compared with TAU (MD, −0.9; 95% CI, −1.5 to −0.3) and waiting list (MD, −2.5; 95% CI, −3.2 to −1.8). The heterogeneity parameter was τ = 0.6. Main results are also presented as standardized mean difference (SMD) in the eAppendix in the Supplement. Similar outcomes were observed using a complete case analysis and when including only recent trials (published after 2012 and 2013; eAppendix in the Supplement). Moreover, the 95% CI of the estimates largely overlapped in the rest of the examined subgroups, suggesting that there was no strong evidence of subgroup differences (eAppendix in the Supplement). The average study dropout rate was 25% for guided iCBT, 29% for unguided iCBT, 19% for waiting list, and 22% for TAU. Among the 25 studies reporting on treatment adherence, the average adherence was 76% for guided iCBT and 54% for unguided iCBT.

    Eight studies reported 6-month postrandomization data. Results of AD-NMA showed no significant difference between guided and unguided iCBT at 6 months (Figure 3). Both guided and unguided iCBT reduced depressive symptoms compared with TAU at 6-month postrandomization (MD for guided iCBT vs TAU, −1.1; 95% CI, −1.7 to −0.5). Similar outcomes were observed across 8 studies reporting on 12-month postrandomization outcomes (MD for guided iCBT vs TAU, −0.5; 95% CI, −1.1 to 0.1). In all analyses, we found no evidence of network inconsistency, but we found weak evidence of publication bias.

    Response Rates

    Overall, 48% of participants receiving guided iCBT responded, while 37% responded in unguided iCBT. When splitting participants into severity groups, we found that 46% of those with moderate depressive symptoms at the baseline (n = 3164) responded in the guided iCBT group compared with 39% in the unguided iCBT group (difference in response rate: 7%). However, 55% of those with moderately severe symptoms (n = 1762) at the baseline responded in the guided iCBT group compared with 40% in unguided iCBT (difference in response rate: 13%). Results of response rates are provided in the eAppendix in the Supplement.

    IPD Network Meta-analyses

    We performed an IPD network meta-regression using baseline depression severity, sex, age, relationship, and employment status as covariates that were reported in most studies. Results indicated that baseline severity was the most important prognostic factor. Higher depression at baseline was associated with higher symptoms at all posttreatment assessments. Not being employed was also associated with poorer outcomes, while sex was not associated (eAppendix in the Supplement). We found strong evidence that baseline severity was associated with effect sizes for guided and unguided iCBT, such that the higher the baseline severity, the larger the benefit of therapeutic guidance. For a PHQ-9 score of 5 to 9 (mild/subthreshold depression), there was either no or a small difference in postintervention outcome between guided and unguided iCBT. However, guided iCBT resulted in better outcomes than unguided iCBT for moderate depression (PHQ-9 score, 10-14), with increasing advantage estimated for moderately severe (PHQ-9 score, 15-19) and severe depression (PHQ-9 score > 19). Both iCBT modalities were superior to TAU and waiting list regardless of baseline severity. Common τ was 0.9. Because of the large number of possible combinations of patient characteristics, we provide the estimates of guided compared with unguided iCBT at posttreatment for 4 random case examples in Table 2. The full range of estimated relative treatment effect sizes for any combination of patient covariates, at posttreatment, 6 months following randomization, and 12 months following randomization can be explored using an interactive online application: https://cinema.ispm.unibe.ch/shinies/iCBT/. There was no evidence of a systematic difference between available and unavailable studies72-74 (eAppendix in the Supplement).

    Discussion

    We assessed data from 36 RCTs including 8107 participants with symptoms of depression from 12 countries. Both guided and unguided iCBT were associated with greater reduction in depressive symptoms than TAU and waiting list at posttreatment, at 6 months following randomization, and 12 months following randomization. Overall, guided iCBT was more effective than unguided iCBT at posttreatment, but differences diminished over the long term. Because both unguided and guided iCBT were associated with better outcomes than control conditions over the long term, unguided iCBT has considerable potential for improving long-term results of interventions with constrained economic and workforce resources. However, baseline severity was a substantial modifier of the differential benefit of guided over unguided iCBT, suggesting that even the short-term incremental benefit of guided vs unguided iCBT is limited to patients with baseline PHQ-9 scores of more than 9.

    The finding that guided iCBT is associated with more effectiveness than unguided is consistent with previous literature examining their average effects.12 The methods of IPD-NMA allowed us to identify subgroups of patients for whom such average effects might not apply. For instance, posttreatment effects of guided and unguided iCBT do not differ among male patients with mild depressive symptoms who were employed and in a relationship. The modifying role of baseline severity is in line with previous research showing that individuals with more severe initial depression are more likely to respond to guided internet-based interventions.75

    The finding that unguided iCBT was associated with more effectiveness than TAU in both the short and longer term contrasts with the findings of our previous conventional NMA, which showed no evidence of difference between unguided iCBT and TAU at posttreatment.12 However, in this IPD-NMA, we could include 2 of the largest RCTs examining the effects of unguided iCBT49,70 (>2000 participants), which were not included in our previous work.12 Also, our analyses were performed using all randomized participants, which is not always possible in conventional NMAs. Therefore, this IPD-NMA provides stronger evidence and improves the precision of previous findings.

    We were also able to identify long-term differential effect sizes in subgroups of patients (see the online application: https://cinema.ispm.unibe.ch/shinies/iCBT/). Conclusions regarding longer-term outcomes should be interpreted cautiously owing to the small number of studies (n = 8), although these studies had large sample sizes and our analyses had adequate power (n >3700 at both follow-ups).

    Strengths and Limitations

    Among the strengths of this study was its high power to detect effect-size modification by synthesizing IPD from direct and indirect comparisons. Moreover, we examined differential roles of guided and unguided iCBT in both the short and the long term. We were also able to include most eligible RCTs (93%) with 8107 participants, making this, to our knowledge, the largest study on individual patient differences in response to iCBT for depression to date. Finally, the risk of bias in the included trials was overall low, and we did not find strong evidence for small-study effect sizes, publication bias, or network inconsistency, suggesting that our analyses were relatively free from critical biases.

    Some limitations should be considered when interpreting our findings. First, we were not able to examine all factors previous research has indicated as influencing depression prognosis (ie, duration of symptoms, number of previous episodes, or comorbidities). In an effort to retain as many observations as possible, we focused on commonly reported variables across the included trials. Second, the included trials were mostly conducted in Western countries, potentially limiting the generalizability to other settings. Third, although the estimated difference between guided and unguided iCBT is small in some individuals with mild symptoms (ie, if baseline PHQ-9 score was 7), the confidence intervals of the pooled estimates are wide, suggesting that we cannot yet exclude the possibility of a clinically significant benefit of guided over unguided iCBT. Finally, only 9 studies recruited participants mainly from clinical settings. However, these were some of the largest studies included in the present IPD-NMA (n = 4269 participants). Therefore, in this sample there was a good representation of patients referred from clinical services. Furthermore, people seeking treatment in the community represent the population that is likely to access iCBT services in the real world.

    Conclusions

    These findings open new avenues for treatment decision-making. Subthreshold depression (PHQ-9 score = 5-9) is prevalent in approximately 15% to 20% of the general population.23,76,77 Given that individuals with mild depressive symptoms may benefit comparably from guided and unguided iCBT, the latter could be disseminated to a large number of people experiencing mild depressive symptoms at a favorable cost, with therapeutic guidance being prioritized for patients with moderate and severe symptoms. Further, a plethora of online self-help programs are available in the community. Individuals who seek self-treatment on the internet are making an implicit “no guidance” choice. Our work indicates that this may not be the best choice for everyone, and that individuals signing up for fully automated programs should be advised that they might benefit from therapeutic support working through the program.

    To further inform personalized treatment selection, future studies should systematically examine a range of possible effect size modifiers, such as number of previous depressive episodes, symptom duration, concurrent use of medications, and comorbidities. Such trials should examine the actual clinical utility of these predictors, for instance, by using adaptive treatment strategies.78 Future efforts should also focus on challenges of scaling up iCBT, including improving adherence, especially for unguided programs. Furthermore, only a few studies include disadvantaged individuals who may experience difficulties in using the internet owing to poverty, locality, or education. Moreover, future trials should investigate whether outcomes differ by ethnic or racial minority status and how to enrich our knowledge on how to approach different groups in the population. Finally, before disseminating and implementing iCBT widely, it is important to further examine its effectiveness and acceptability in treating major depression in primary and secondary mental health care settings. Further research is warranted on actual dissemination and implementation of iCBT.

    In summary, personalized treatment selection is possible and very much needed, as one size does not fit all. To assist clinicians and patients in choosing the right iCBT modality, we have developed an interactive application available at https://cinema.ispm.unibe.ch/shinies/iCBT/. Shared clinical decision-making should involve the patients’ values and preferences, history, and any previous or concurrent treatments so as to provide the best and most suitable intervention while maximizing human resources available.

    Back to top
    Article Information

    Corresponding Author: Eirini Karyotaki, PhD, Department of Global Health and Social Medicine, Harvard Medical School, 641 Huntington Ave, Boston, MA 02115 (eirini_karyotaki@hms.harvard.edu).

    Accepted for Publication: November 23, 2020.

    Published Online: January 20, 2021. doi:10.1001/jamapsychiatry.2020.4364

    Correction: This article was corrected on February 17, 2021, to fix a typo in the Abstract.

    The Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration Authors: Heleen Riper, PhD; Vikram Patel, MBBS, PhD; Adriana Mira, PhD; Alan W. Gemmil, PhD; Albert S. Yeung, ScD, MD; Alfred Lange, PhD; Alishia D. Williams, PhD; Andrew Mackinnon, PhD; Anna Geraedts, PhD; Annemieke van Straten, PhD; Björn Meyer, PhD; Cecilia Björkelund, PhD; Christine Knaevelsrud, PhD; Christopher G. Beevers, PhD; Cristina Botella, PhD; Daniel R. Strunk, PhD; David C. Mohr, PhD; David D. Ebert, PhD; David Kessler, PhD; Derek Richards, PhD; Elizabeth Littlewood, PhD; Erik Forsell, PhD; Fan Feng, PhD; Fang Wang, PhD; Gerhard Andersson, PhD; Heather Hadjistavropoulos, PhD; Heleen Christensen, PhD; Iony D. Ezawa, MSc; Isabella Choi, PhD; Isabelle M. Rosso, PhD; Jan Philipp Klein, MD, PhD; Jason Shumake, PhD; Javier Garcia-Campayo, MD, PhD; Jeannette Milgrom, PhD; Jessica Smith, MSc; Jesus Montero-Marin, PhD; Jill M. Newby, PhD; Juana Bretón-López, PhD; Justine Schneider, PhD; Kristofer Vernmark, PhD; Lara Bücker, PhD; Lisa B. Sheeber, PhD; Lisanne Warmerdam, PhD; Louise Farrer, PhD; Manuel Heinrich, MSc; Marcus J. H. Huibers, PhD; Marie Kivi, PhD; Martin Kraepelien, PhD; Nicholas R. Forand, PhD; Nicky Pugh, PhD; Nils Lindefors, PhD; Ove Lintvedt, PhD; Pavle Zagorscak, MSc; Per Carlbring, PhD; Rachel Phillips, MSc; Robert Johansson, PhD; Ronald C. Kessler, PhD; Sally Brabyn, MSc; Sarah Perini, MSc; Scott L. Rauch, MD, PhD; Simon Gilbody, PhD; Steffen Moritz, PhD; Thomas Berger, PhD; Victor Pop, PhD; Viktor Kaldo, PhD; Viola Spek, PhD; Yvonne Forsell, PhD.

    Affiliations of The Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration Authors: Department of Clinical Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands (Riper, van Straten, Huibers); Department of Global Health and Social Medicine, Harvard Medical School, Boston, Massachusetts (Patel); Amsterdam Public Health Research Institute, Amsterdam, the Netherlands (Riper, van Straten, Huibers); Department of Research and Innovation, GGZ inGeest, Specialized Mental Health Care, Amsterdam, the Netherlands (Riper); PROMOSAM Excellence in Research Program, MINECO, Valencia, Spain (Mira); Department of Personality, Evaluation and Psychological Treatment, Valencia University, Valencia, Spain (Mira); Parent-Infant Research Institute, Department of Clinical and Health Psychology, Heidelberg Repatriation Hospital, Austin Health, Ivanhoe, Victoria, Australia (Gemmil, Milgrom); Depression Clinical and Research Program, Massachusetts General Hospital, Harvard University, Boston, Massachusetts (Yeung); Department of Clinical Psychology, University of Amsterdam, Amsterdam, the Netherlands (Lange); Department of Psychology, University of New South Wales, Sydney, New South Wales, Australia (Williams, Newby); Prince of Wales Hospital, Black Dog Institute, University of New South Wales, Sydney, New South Wales, Australia (Mackinnon, Christensen, Newby); Center for Mental Health, University of Melbourne, Melbourne, Victoria, Australia (Mackinnon); HumanTotalCare, Utrecht, the Netherlands (Geraedts); Research Department, Gaia AG, Hamburg, Germany (Meyer); Department of Psychology, City, University of London, London, England (Meyer); Primary Health Care, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg School of Public Health and Community Medicine, Gothenburg, Sweden (Björkelund); Department for Clinical Psychological Intervention, Freie Universität Berlin, Berlin, Germany (Knaevelsrud, Heinrich, Zagorscak); Department of Psychology and Institute for Mental Health Research, The University of Texas at Austin (Beevers, Shumake); Department of Basic Psychology, Clinic and Psychobiology, Jaume I University, Castellon, Spain (Botella, Bretón-López); CIBER Fisiopatología Obesidad y Nutrición (CIBERObn), Instituto Salud Carlos III, Madrid, Spain (Botella, Bretón-López); Department of Psychology, The Ohio State University, Columbus, Ohio (Strunk, Ezawa); Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, Illinois (Mohr); Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Germany (Ebert); Centre for Academic Primary Care, Bristol Medical School (Population Health Sciences), University of Bristol, Bristol, England (D. Kessler); National Institute for Health Research Bristol Biomedical Research Centre, University Hospitals Bristol NHS Foundation Trust and University of Bristol, Bristol, England (D. Kessler); E-mental Health Research Group, Trinity College Dublin School of Psychology, Dublin, Ireland (Richards); Clinical Research & Innovation, SilverCloud Health, Dublin, Ireland (Richards, Brabyn); Department of Health Sciences, University of York, York, England (Littlewood, Gilbody); Centre for Psychiatry Research, Department of Clinical Neuroscience, Karolinska Institutet & Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden (E. Forsell, Andersson, Kraepelien, Lindefors, Kaldo); Benson Henry Institute for Mind Body Medicine, Massachusetts General Hospital, Harvard University, Boston, Massachusetts (Feng); Department of Psychology and Sleep Medicine, Guang’anmen Hospital, China Academy of Chinese Medical Sciences, Beijing, China (Wang); Department of Behavioral Sciences and Learning, Linköping University, Linköping, Sweden (Andersson, Vernmark); Department of Psychology, University of Regina, Regina, Saskatchewan, Canada (Hadjistavropoulos, Pugh); Central Clinical School, Brain and Mind Centre, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia (Choi); McLean Hospital, Belmont, Massachusetts (Rosso, Rauch); Harvard Medical School, Boston, Massachusetts (Rosso); Department of Psychiatry and Psychotherapy, Luebeck University, Luebeck, Germany (Klein); Aragon Institute for Health Research (IIS Aragón), Miguel Servet University Hospital, Zaragoza, Spain (Garcia-Campayo); Primary Care Prevention and Health Promotion Research Network, RedIAPP, Madrid, Spain (Garcia-Campayo); Imperial Clinical Trials Unit, Imperial College London, London, England (Smith); Warneford Hospital, Department of Psychiatry, University of Oxford, Oxford, England (Montero-Marin); Institute of Mental Health, University of Nottingham, Nottingham, England (Schneider); Psykologpartners, Linkoping, Sweden (Vernmark); Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany (Bücker, Moritz); Oregon Research Institute, Eugene, Oregon (Sheeber); National Health Care Institute, Diemen, the Netherlands (Warmerdam); Centre for Mental Health Research, The Australian National University, Canberra, Australia (Farrer); Department of Psychology and AgeCap, University of Gothenburg, Gothenburg, Sweden (Kivi); Department of Psychiatry and Behavioral Health, The Ohio State University Wexner Medical Center, Columbus, Ohio (Forand); Department of Psychiatry, The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York (Forand); Norwegian Center for E-health research, Tromsø, Norway (Lintvedt); Department of Psychology, Stockholm University, Stockholm, Sweden (Carlbring, Johansson); Faculty of Medicine, Imperial College London School of Public Health, London, England (Phillips); Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts (R. C. Kessler); The Clinical Psychology Centre, Sydney, New South Wales, Australia (Perini); Hull York Medical School, University of York, York, England (Gilbody); Department of Clinical Psychology and Psychotherapy, University of Bern, Bern, Switzerland (Berger); Department of Clinical and Medical Health Psychology, Tilburg University, Tilburg, the Netherlands (Pop, Spek); Department of Psychology, Faculty of Health and Life Sciences, Linnaeus University, Växjö, Sweden (Kaldo); Section of Epidemiology and Public Health Intervention Research, Department of Global Public Health, Karolinska Institutet, Stockholm Health Care Services, Region Stockholm, Sweden (Y. Forsell).

    Author Contributions: Drs Karyotaki and Efthimiou had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Furukawa and Cuijpers share last authorship.

    Concept and design: Karyotaki, Efthimiou, Riper, Patel, Lange, van Straten, Botella, Mohr, Garcia-Campayo, Newby, Breton, Singh, Lintvedt, Gilbody, Y. Forsell, Cuijpers.

    Acquisition, analysis, or interpretation of data: Karyotaki, Efthimiou, Miguel, Maas genannt Bermpohl, Mira, Gemmill, Yeung, Williams, Mackinnon, Geraedts, Meyer, Bjorkelund, Knaevelsrud, Beevers, Botella, Strunk, Ebert, D. Kessler, Richards, Littlewood, E. Forsell, Feng, Wang, Andersson, Hadjistavropoulos, Christensen, Ezawa, Choi, Rosso, Klein, Shumake, Garcia-Campayo, Milgrom, Smith, Montero-Marin, Schneider, Vernmark, Buecker, Sheeber, Warmerdam, Farrer, Heinrich, Huibers, Kivi, Kraepelien, Forand, Lindefors, Zagorscak, Carlbring, Phillips, Johansson, R. Kessler, Brabyn, Perini, Rauch, Gilbody, Moritz, Berger, Kaldo, Pop, Spek, Furukawa, Cuijpers.

    Drafting of the manuscript: Karyotaki, Efthimiou, Yeung, Newby, Lintvedt, Moritz.

    Critical revision of the manuscript for important intellectual content: Karyotaki, Efthimiou, Miguel, Maas genannt Bermpohl, Riper, Patel, Mira, Gemmill, Lange, Williams, Mackinnon, Geraedts, van Straten, Meyer, Bjorkelund, Knaevelsrud, Beevers, Botella, Strunk, Mohr, Ebert, D. Kessler, Richards, Littlewood, E. Forsell, Feng, Wang, Andersson, Hadjistavropoulos, Christensen, Ezawa, Choi, Rosso, Klein, Shumake, Garcia-Campayo, Milgrom, Smith, Montero-Marin, Newby, Breton, Schneider, Vernmark, Buecker, Sheeber, Warmerdam, Farrer, Heinrich, Huibers, Kivi, Kraepelien, Forand, Singh, Lindefors, Zagorscak, Carlbring, Phillips, Johansson, R. Kessler, Brabyn, Perini, Rauch, Gilbody, Berger, Kaldo, Pop, Spek, Y. Forsell, Furukawa, Cuijpers.

    Statistical analysis: Efthimiou, Mackinnon, Huibers, Lintvedt, Gilbody.

    Obtained funding: Karyotaki, Botella, Rosso, Montero-Marin, Schneider, Sheeber.

    Administrative, technical, or material support: Miguel, Mira, Geraedts, Bjorkelund, Knaevelsrud, Botella, Strunk, Mohr, Ebert, Richards, Littlewood, E. Forsell, Andersson, Hadjistavropoulos, Christensen, Klein, Shumake, Smith, Newby, Buecker, Heinrich, Kivi, Forand, Singh, Zagorscak, Brabyn, Moritz, Spek, Y. Forsell, Cuijpers.

    Supervision: Riper, Patel, Botella, Wang, Rosso, Garcia-Campayo, Carlbring, R. Kessler, Gilbody, Cuijpers.

    Conflict of Interest Disclosures: Dr Björkelund reports grants from Swedish Social Insurance Agency Sweden during the conduct of the study. Dr Knaevelsrud reported grants from Techniker Krankenkasse (Public Health Care Company) during the conduct of the study and personal fees from Oberbergklinik and Servier outside the submitted work. Dr Beevers reported grants from the National Institutes of Health and personal fees from Association for Psychological Science outside the submitted work. Dr Mohr reported personal fees from Otsuka Pharmaceuticals, Apple Inc, Pear Therapeutics, One Mind Foundation, and other support from Adaptive Health Equity Interest outside the submitted work. Dr Ebert reported other support from GET.ON Insitute (HelloBetter Shareholder of digital therapeutic company); personal fees from Minddistrict Consultancy, Lantern Consultancy Fees, Sanofi Consultancy, and Novartis Consultancy Fees outside the submitted work; and holding IP rights to several digital therapeutics for the prevention and treatment of mental health disorders. Dr Richards reported other support from SilverCloud Health Salary outside the submitted work. Dr Hadjistavropoulos reported grants from Canadian Institutes of Health Research, Saskatchewan Health Research Foundation, and Rx&D Research Foundation during the conduct of the study. Dr Christensen reported potentially receiving royalties to MoodGYM if it is successful commercially and is Director of the Black Dog Institute, which creates e-health interventions. Dr Klein reported grants from German Ministry of Health II A 5-2512 FSB 052 during the conduct of the study; grants from Servier; personal fees from Springer, Beltz, Elsevier, and Hogrefe outside the submitted work; and payments for workshops on psychotherapy for chronic depression and on psychiatric emergencies. Dr Shumake reported grants from National Institute for Mental Health during the conduct of the study. Dr García-Campayo reports grants from Instituto de salud Carlos III during the conduct of the study. Dr Heinrich reported grants from Techniker Krankasse (Public Health Care Company) during the conduct of the study. Dr Zagorscak reported grants from Techniker Krankenkasse (Public Health Care Company) during the conduct of the study. Dr R. Kessler reported personal fees from Datastat Inc and personal fees from Sage Pharmaceuticals, and Takeda during the conduct of the study. Dr Rauch reported grants from USA MRAA during the conduct of the study. Dr Furukawa reported grants from Mitsubishi-Tanabe and personal fees from Mitsubishi-Tanabe, MSD, and Shionogi outside the submitted work; in addition, Dr Furukawa had a patent for 2018-177688 pending for smartphone CBT apps and a patent for copyrights licensed for Kokoro-app smartphone CBT app. No other disclosures were reported.

    Funding/Support: Dr Karyotaki was supported by the Netherlands Organization for Health Research and Development (project 019.182SG.001). Dr Efthimiou was supported by project grant 180083 from the Swiss National Science Foundation.

    Role of the Funder/Sponsor: The Netherlands Organization for Health Research and Development and the Swiss National Science Foundation had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. The decision to submit the article for publication was a condition of the funding and was made before any results were available.

    Additional Contributions: We dedicate this research to the memory of Dr Jeroen Ruwaard, formerly of the GGZ in Geest Specialized Mental Health Care in Amsterdam, who contributed individual patient data from an original trial to this IPD-NMA but sadly passed away during this project. Therefore, we express our sincere appreciation to Jeroen’s contribution to the field of internet-based interventions.

    References
    1.
    McLaughlin  KA.  The public health impact of major depression: a call for interdisciplinary prevention efforts.   Prev Sci. 2011;12(4):361-371. doi:10.1007/s11121-011-0231-8PubMedGoogle ScholarCrossref
    2.
    Ustün  TB, Ayuso-Mateos  JL, Chatterji  S, Mathers  C, Murray  CJL.  Global burden of depressive disorders in the year 2000.   Br J Psychiatry. 2004;184(5):386-392. doi:10.1192/bjp.184.5.386PubMedGoogle ScholarCrossref
    3.
    Wittchen  H-U, Jacobi  F, Rehm  J,  et al.  The size and burden of mental disorders and other disorders of the brain in Europe 2010.   Eur Neuropsychopharmacol. 2011;21(9):655-679. doi:10.1016/j.euroneuro.2011.07.018PubMedGoogle ScholarCrossref
    4.
    Chisholm  D, Sweeny  K, Sheehan  P,  et al.  Scaling-up treatment of depression and anxiety: a global return on investment analysis.   Lancet Psychiatry. 2016;3(5):415-424. doi:10.1016/S2215-0366(16)30024-4PubMedGoogle ScholarCrossref
    5.
    Cuijpers  P, Noma  H, Karyotaki  E, Vinkers  CH, Cipriani  A, Furukawa  TA.  A network meta-analysis of the effects of psychotherapies, pharmacotherapies and their combination in the treatment of adult depression.   World Psychiatry. 2020;19(1):92-107. doi:10.1002/wps.20701PubMedGoogle ScholarCrossref
    6.
    Patel  V, Saxena  S, Lund  C,  et al.  The Lancet Commission on global mental health and sustainable development.   Lancet. 2018;392(10157):1553-1598. doi:10.1016/S0140-6736(18)31612-XPubMedGoogle ScholarCrossref
    7.
    Holmes  EA, O’Connor  RC, Perry  VH,  et al.  Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science.   Lancet Psychiatry. 2020;7(6):547-560. doi:10.1016/S2215-0366(20)30168-1PubMedGoogle ScholarCrossref
    8.
    Wind  TR, Rijkeboer  M, Andersson  G, Riper  H.  The COVID-19 pandemic: the ‘black swan’ for mental health care and a turning point for e-health.   Internet Interv. 2020;20:100317. doi:10.1016/j.invent.2020.100317PubMedGoogle Scholar
    9.
    Andersson  G, Titov  N, Dear  BF, Rozental  A, Carlbring  P.  Internet-delivered psychological treatments: from innovation to implementation.   World Psychiatry. 2019;18(1):20-28. doi:10.1002/wps.20610PubMedGoogle ScholarCrossref
    10.
    Karyotaki  E, Riper  H, Twisk  J,  et al.  Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data.   JAMA Psychiatry. 2017;74(4):351-359. doi:10.1001/jamapsychiatry.2017.0044PubMedGoogle ScholarCrossref
    11.
    Fairburn  CG, Patel  V.  The impact of digital technology on psychological treatments and their dissemination.   Behav Res Ther. 2017;88:19-25. doi:10.1016/j.brat.2016.08.012PubMedGoogle ScholarCrossref
    12.
    Cuijpers  P, Noma  H, Karyotaki  E, Cipriani  A, Furukawa  TA.  Effectiveness and acceptability of cognitive behavior therapy delivery formats in adults with depression: a network meta-analysis.   JAMA Psychiatry. 2019;76(7):700-707. doi:10.1001/jamapsychiatry.2019.0268PubMedGoogle ScholarCrossref
    13.
    Debray  TP, Schuit  E, Efthimiou  O,  et al; GetReal Workpackage.  An overview of methods for network meta-analysis using individual participant data: when do benefits arise?   Stat Methods Med Res. 2018;27(5):1351-1364. doi:10.1177/0962280216660741PubMedGoogle ScholarCrossref
    14.
    Efthimiou  O, Debray  TP, van Valkenhoef  G,  et al; GetReal Methods Review Group.  GetReal in network meta-analysis: a review of the methodology.   Res Synth Methods. 2016;7(3):236-263. doi:10.1002/jrsm.1195PubMedGoogle ScholarCrossref
    15.
    Riley  RD, Lambert  PC, Abo-Zaid  G.  Meta-analysis of individual participant data: rationale, conduct, and reporting.   BMJ. 2010;340:c221. doi:10.1136/bmj.c221PubMedGoogle ScholarCrossref
    16.
    Karyotaki  E, Furukawa  TA, Efthimiou  O, Riper  H, Cuijpers  P.  Guided or self-guided internet-based cognitive-behavioural therapy (iCBT) for depression? Study protocol of an individual participant data network meta-analysis.   BMJ Open. 2019;9(6):e026820. doi:10.1136/bmjopen-2018-026820PubMedGoogle Scholar
    17.
    Kleiboer  A, Smit  J, Bosmans  J,  et al.  European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): study protocol for a randomized controlled, non-inferiority trial in eight European countries.   Trials. 2016;17(1):387. doi:10.1186/s13063-016-1511-1PubMedGoogle ScholarCrossref
    18.
    Cuijpers  P, Schuurmans  J.  Self-help interventions for anxiety disorders: an overview.   Curr Psychiatry Rep. 2007;9(4):284-290. doi:10.1007/s11920-007-0034-6PubMedGoogle ScholarCrossref
    19.
    Cuijpers  P, Karyotaki  E, Ciharova  M.  A Meta-analytic Database of Randomised Trials on Psychotherapies for Depression. Open Science Foundation. 2019.
    20.
    Kessler  RC, van Loo  HM, Wardenaar  KJ,  et al.  Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder.   Epidemiol Psychiatr Sci. 2017;26(1):22-36. doi:10.1017/S2045796016000020PubMedGoogle ScholarCrossref
    21.
    Bockting  CL, Hollon  SD, Jarrett  RB, Kuyken  W, Dobson  K.  A lifetime approach to major depressive disorder: the contributions of psychological interventions in preventing relapse and recurrence.   Clin Psychol Rev. 2015;41:16-26. doi:10.1016/j.cpr.2015.02.003PubMedGoogle ScholarCrossref
    22.
    Higgins  J, Green  S. Assessing risk of bias in included studies.  Cochrane Handbook for Systematic Reviews of Interventions Version 5.1. 2011.
    23.
    Kroenke  K, Spitzer  RL, Williams  JB.  The PHQ-9: validity of a brief depression severity measure.   J Gen Intern Med. 2001;16(9):606-613. doi:10.1046/j.1525-1497.2001.016009606.xPubMedGoogle ScholarCrossref
    24.
    Wahl  I, Löwe  B, Bjorner  JB,  et al.  Standardization of depression measurement: a common metric was developed for 11 self-report depression measures.   J Clin Epidemiol. 2014;67(1):73-86. doi:10.1016/j.jclinepi.2013.04.019PubMedGoogle ScholarCrossref
    25.
    König  J, Krahn  U, Binder  H.  Visualizing the flow of evidence in network meta-analysis and characterizing mixed treatment comparisons.   Stat Med. 2013;32(30):5414-5429. doi:10.1002/sim.6001PubMedGoogle ScholarCrossref
    26.
    White  IR, Barrett  JK, Jackson  D, Higgins  JP.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression.   Res Synth Methods. 2012;3(2):111-125. doi:10.1002/jrsm.1045PubMedGoogle ScholarCrossref
    27.
    Quartagno  M, Grund  S, Carpenter  J.  jomo: a flexible package for two-level joint modelling multiple imputation.   R Journal. 2019. doi:10.32614/RJ-2019-028Google Scholar
    28.
    Rücker  G, Schwarzer  G, Krahn  U. netmeta: Network meta-analysis using Frequentist methods. R package version 0.9-8, 2018.
    29.
    Rubin  DB.  Multiple Imputation for Nonresponse in Surveys. Vol 81: John Wiley & Sons; 2004.
    30.
    Plummer  M. rjags: Bayesian graphical models using MCMC. R package version 4-8. 2018.
    31.
    Egger  M, Davey Smith  G, Schneider  M, Minder  C.  Bias in meta-analysis detected by a simple, graphical test.   BMJ. 1997;315(7109):629-634. doi:10.1136/bmj.315.7109.629PubMedGoogle ScholarCrossref
    32.
    Rhodes  KM, Turner  RM, Higgins  JP.  Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data.   J Clin Epidemiol. 2015;68(1):52-60. doi:10.1016/j.jclinepi.2014.08.012PubMedGoogle ScholarCrossref
    33.
    Andersson  G, Bergström  J, Holländare  F, Carlbring  P, Kaldo  V, Ekselius  L.  Internet-based self-help for depression: randomised controlled trial.   Br J Psychiatry. 2005;187(5):456-461. doi:10.1192/bjp.187.5.456PubMedGoogle ScholarCrossref
    34.
    Beevers  CG, Pearson  R, Hoffman  JS, Foulser  AA, Shumake  J, Meyer  B.  Effectiveness of an internet intervention (Deprexis) for depression in a united states adult sample: a parallel-group pragmatic randomized controlled trial.   J Consult Clin Psychol. 2017;85(4):367-380. doi:10.1037/ccp0000171PubMedGoogle ScholarCrossref
    35.
    Berger  T, Hämmerli  K, Gubser  N, Andersson  G, Caspar  F.  Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help.   Cogn Behav Ther. 2011;40(4):251-266. doi:10.1080/16506073.2011.616531PubMedGoogle ScholarCrossref
    36.
    Choi  I, Zou  J, Titov  N,  et al.  Culturally attuned Internet treatment for depression amongst Chinese Australians: a randomised controlled trial.   J Affect Disord. 2012;136(3):459-468. doi:10.1016/j.jad.2011.11.003PubMedGoogle ScholarCrossref
    37.
    Christensen  H, Griffiths  KM, Jorm  AF.  Delivering interventions for depression by using the internet: randomised controlled trial.   BMJ. 2004;328(7434):265. doi:10.1136/bmj.37945.566632.EEPubMedGoogle ScholarCrossref
    38.
    de Graaf  LE, Gerhards  SA, Arntz  A,  et al.  Clinical effectiveness of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial.   Br J Psychiatry. 2009;195(1):73-80. doi:10.1192/bjp.bp.108.054429PubMedGoogle ScholarCrossref
    39.
    Farrer  L, Christensen  H, Griffiths  KM, Mackinnon  A.  Internet-based CBT for depression with and without telephone tracking in a national helpline: randomised controlled trial.   PLoS One. 2011;6(11):e28099. doi:10.1371/journal.pone.0028099PubMedGoogle Scholar
    40.
    Forand  NR, Barnett  JG, Strunk  DR, Hindiyeh  MU, Feinberg  JE, Keefe  JR.  Efficacy of guided iCBT for depression and mediation of change by cognitive skill acquisition.   Behav Ther. 2018;49(2):295-307. doi:10.1016/j.beth.2017.04.004PubMedGoogle ScholarCrossref
    41.
    Forsell  E, Bendix  M, Holländare  F,  et al.  Internet delivered cognitive behavior therapy for antenatal depression: a randomised controlled trial.   J Affect Disord. 2017;221:56-64. doi:10.1016/j.jad.2017.06.013PubMedGoogle ScholarCrossref
    42.
    Geraedts  AS, Kleiboer  AM, Wiezer  NM, van Mechelen  W, Cuijpers  P.  Short-term effects of a web-based guided self-help intervention for employees with depressive symptoms: randomized controlled trial.   J Med Internet Res. 2014;16(5):e121. doi:10.2196/jmir.3185PubMedGoogle Scholar
    43.
    Gilbody  S, Littlewood  E, Hewitt  C,  et al; REEACT Team.  Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial.   BMJ. 2015;351:h5627. doi:10.1136/bmj.h5627PubMedGoogle ScholarCrossref
    44.
    Gilbody  S, Brabyn  S, Lovell  K,  et al; REEACT collaborative.  Telephone-supported computerised cognitive-behavioural therapy: REEACT-2 large-scale pragmatic randomised controlled trial.   Br J Psychiatry. 2017;210(5):362-367. doi:10.1192/bjp.bp.116.192435PubMedGoogle ScholarCrossref
    45.
    Hallgren  M, Helgadóttir  B, Herring  MP,  et al.  Exercise and internet-based cognitive-behavioural therapy for depression: multicentre randomised controlled trial with 12-month follow-up.   Br J Psychiatry. 2016;209(5):414-420. doi:10.1192/bjp.bp.115.177576PubMedGoogle ScholarCrossref
    46.
    Johansson  R, Sjöberg  E, Sjögren  M,  et al.  Tailored vs. standardized internet-based cognitive behavior therapy for depression and comorbid symptoms: a randomized controlled trial.   PLoS One. 2012;7(5):e36905. doi:10.1371/journal.pone.0036905PubMedGoogle Scholar
    47.
    Kessler  D, Lewis  G, Kaur  S,  et al.  Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial.   Lancet. 2009;374(9690):628-634. doi:10.1016/S0140-6736(09)61257-5PubMedGoogle ScholarCrossref
    48.
    Kivi  M, Eriksson  MC, Hange  D,  et al.  Internet-based therapy for mild to moderate depression in Swedish primary care: short term results from the PRIM-NET randomized controlled trial.   Cogn Behav Ther. 2014;43(4):289-298. doi:10.1080/16506073.2014.921834PubMedGoogle ScholarCrossref
    49.
    Klein  JP, Berger  T, Schröder  J,  et al.  Effects of a psychological internet intervention in the treatment of mild to moderate depressive symptoms: results of the EVIDENT study, a randomized controlled trial.   Psychother Psychosom. 2016;85(4):218-228. doi:10.1159/000445355PubMedGoogle ScholarCrossref
    50.
    Lintvedt  OK, Griffiths  KM, Sørensen  K,  et al.  Evaluating the effectiveness and efficacy of unguided internet-based self-help intervention for the prevention of depression: a randomized controlled trial.   Clin Psychol Psychother. 2013;20(1):10-27. doi:10.1002/cpp.770PubMedGoogle ScholarCrossref
    51.
    Meyer  B, Berger  T, Caspar  F, Beevers  CG, Andersson  G, Weiss  M.  Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial.   J Med Internet Res. 2009;11(2):e15. doi:10.2196/jmir.1151PubMedGoogle Scholar
    52.
    Meyer  B, Bierbrodt  J, Schröder  J,  et al  Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial.   Internet Interventions. 2015;2(1):48-59. doi:10.1016/j.invent.2014.12.003Google ScholarCrossref
    53.
    Milgrom  J, Danaher  BG, Gemmill  AW,  et al.  Internet cognitive behavioral therapy for women with postnatal depression: a randomized controlled trial of MumMoodBooster.   J Med Internet Res. 2016;18(3):e54. doi:10.2196/jmir.4993PubMedGoogle Scholar
    54.
    Mira  A, Bretón-López  J, García-Palacios  A, Quero  S, Baños  RM, Botella  C.  An Internet-based program for depressive symptoms using human and automated support: a randomized controlled trial.   Neuropsychiatr Dis Treat. 2017;13:987-1006. doi:10.2147/NDT.S130994PubMedGoogle ScholarCrossref
    55.
    Mohr  DC, Duffecy  J, Ho  J,  et al.  A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression.   PLoS One. 2013;8(8):e70086. doi:10.1371/journal.pone.0070086PubMedGoogle Scholar
    56.
    Montero-Marín  J, Araya  R, Pérez-Yus  MC,  et al.  An internet-based intervention for depression in primary Care in Spain: a randomized controlled trial.   J Med Internet Res. 2016;18(8):e231. doi:10.2196/jmir.5695PubMedGoogle Scholar
    57.
    Moritz  S, Schilling  L, Hauschildt  M, Schröder  J, Treszl  A.  A randomized controlled trial of internet-based therapy in depression.   Behav Res Ther. 2012;50(7-8):513-521. doi:10.1016/j.brat.2012.04.006PubMedGoogle ScholarCrossref
    58.
    Perini  S, Titov  N, Andrews  G.  Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial.   Aust N Z J Psychiatry. 2009;43(6):571-578. doi:10.1080/00048670902873722PubMedGoogle ScholarCrossref
    59.
    Phillips  R, Schneider  J, Molosankwe  I,  et al.  Randomized controlled trial of computerized cognitive behavioural therapy for depressive symptoms: effectiveness and costs of a workplace intervention.   Psychol Med. 2014;44(4):741-752. doi:10.1017/S0033291713001323PubMedGoogle ScholarCrossref
    60.
    Pugh  NE, Hadjistavropoulos  HD, Dirkse  D.  A randomised controlled trial of therapist-assisted, internet-delivered cognitive behavior therapy for women with maternal depression.   PLoS One. 2016;11(3):e0149186. doi:10.1371/journal.pone.0149186PubMedGoogle Scholar
    61.
    Richards  D, Timulak  L, O’Brien  E,  et al.  A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression.   Behav Res Ther. 2015;75:20-31. doi:10.1016/j.brat.2015.10.005PubMedGoogle ScholarCrossref
    62.
    Rosso  IM, Killgore  WD, Olson  EA,  et al.  Internet-based cognitive behavior therapy for major depressive disorder: a randomized controlled trial.   Depress Anxiety. 2017;34(3):236-245. doi:10.1002/da.22590PubMedGoogle ScholarCrossref
    63.
    Ruwaard  J, Schrieken  B, Schrijver  M,  et al.  Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up.   Cogn Behav Ther. 2009;38(4):206-221. doi:10.1080/16506070802408086PubMedGoogle ScholarCrossref
    64.
    Sheeber  LB, Seeley  JR, Feil  EG,  et al.  Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression.   J Consult Clin Psychol. 2012;80(5):739-749. doi:10.1037/a0028820PubMedGoogle ScholarCrossref
    65.
    Smith  J, Newby  JM, Burston  N,  et al.  Help from home for depression: a randomised controlled trial comparing internet-delivered cognitive behaviour therapy with bibliotherapy for depression.   Internet Interv. 2017;9:25-37. doi:10.1016/j.invent.2017.05.001PubMedGoogle ScholarCrossref
    66.
    Spek  V, Nyklícek  I, Smits  N,  et al.  Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial.   Psychol Med. 2007;37(12):1797-1806. doi:10.1017/S0033291707000542PubMedGoogle ScholarCrossref
    67.
    Vernmark  K, Lenndin  J, Bjärehed  J,  et al.  Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression.   Behav Res Ther. 2010;48(5):368-376. doi:10.1016/j.brat.2010.01.005PubMedGoogle ScholarCrossref
    68.
    Warmerdam  L, van Straten  A, Twisk  J, Riper  H, Cuijpers  P.  Internet-based treatment for adults with depressive symptoms: randomized controlled trial.   J Med Internet Res. 2008;10(4):e44. doi:10.2196/jmir.1094PubMedGoogle Scholar
    69.
    Yeung  A, Wang  F, Feng  F,  et al.  Outcomes of an online computerized cognitive behavioral treatment program for treating chinese patients with depression: a pilot study.   Asian J Psychiatr. 2018;38:102-107. doi:10.1016/j.ajp.2017.11.007PubMedGoogle ScholarCrossref
    70.
    Zagorscak  P, Heinrich  M, Sommer  D, Wagner  B, Knaevelsrud  C.  Benefits of individualized feedback in internet-based interventions for depression: a randomized controlled trial.   Psychother Psychosom. 2018;87(1):32-45. doi:10.1159/000481515PubMedGoogle ScholarCrossref
    71.
    Williams  AD, O’Moore  K, Blackwell  SE, Smith  J, Holmes  EA, Andrews  G.  Positive imagery cognitive bias modification (CBM) and internet-based cognitive behavioral therapy (iCBT): a randomized controlled trial.   J Affect Disord. 2015;178:131-141. doi:10.1016/j.jad.2015.02.026PubMedGoogle ScholarCrossref
    72.
    Titov  N, Andrews  G, Davies  M, McIntyre  K, Robinson  E, Solley  K.  Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance.   PLoS One. 2010;5(6):e10939. doi:10.1371/journal.pone.0010939PubMedGoogle Scholar
    73.
    Titov  N, Dear  BF, Ali  S,  et al.  Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial.   Behav Ther. 2015;46(2):193-205. doi:10.1016/j.beth.2014.09.008PubMedGoogle ScholarCrossref
    74.
    Löbner  M, Pabst  A, Stein  J,  et al.  Computerized cognitive behavior therapy for patients with mild to moderately severe depression in primary care: a pragmatic cluster randomized controlled trial (@ktiv).   J Affect Disord. 2018;238:317-326. doi:10.1016/j.jad.2018.06.008PubMedGoogle ScholarCrossref
    75.
    Karyotaki  E, Ebert  DD, Donkin  L,  et al.  Do guided internet-based interventions result in clinically relevant changes for patients with depression? an individual participant data meta-analysis.   Clin Psychol Rev. 2018;63:80-92. doi:10.1016/j.cpr.2018.06.007PubMedGoogle ScholarCrossref
    76.
    Kroenke  K, Strine  TW, Spitzer  RL, Williams  JB, Berry  JT, Mokdad  AH.  The PHQ-8 as a measure of current depression in the general population.   J Affect Disord. 2009;114(1-3):163-173. doi:10.1016/j.jad.2008.06.026PubMedGoogle ScholarCrossref
    77.
    Khaled  SM.  Prevalence and potential determinants of subthreshold and major depression in the general population of Qatar.   J Affect Disord. 2019;252:382-393. doi:10.1016/j.jad.2019.04.056PubMedGoogle ScholarCrossref
    78.
    Forsell  E, Jernelöv  S, Blom  K,  et al.  Proof of concept for an adaptive treatment strategy to prevent failures in internet-delivered CBT: a single-blind randomized clinical trial with insomnia patients.   Am J Psychiatry. 2019;176(4):315-323. doi:10.1176/appi.ajp.2018.18060699PubMedGoogle ScholarCrossref
    ×