Prognostic Risk Factors in Randomized Clinical Trials of Face-to-Face and Internet-Based Psychotherapy for Depression

This systematic review and meta-regression analysis compares the prognostic risk factors in patient samples reported in 105 randomized clinical trials of face-to-face and internet-based psychotherapy for treating depression.


P
sychotherapy for depression can be offered in different settings and treatment modalities.Traditionally, treatment is delivered face to face, but internet-based therapy (IBT) has gained popularity and the number of randomized clinical trials (RCTs) of IBT has grown during the past 2 decades. 1 Recent direct and indirect comparisons 2,3 suggest that therapist-guided IBT can be as beneficial as face-to-face therapy (FTF).
To draw valid inferences about different treatments for routine care, RCTs should include patients representative of the clinical population.However, while the rationale of RCTs maximizes the internal validity, the external validity or generalizability (ie, whether the effects can be generalized to the patient population in clinical practice) is an issue, with up to 80% of individuals with depression being excluded from depression trials due to their failure to meet inclusion criteria. 4,5This includes patients with unfavorable socioeconomic characteristics and complex clinical presentations 6 (ie, factors known to be associated with a poorer prognosis). 7,8Yet it remains unclear whether the empirical distributions of prognostic risk factors (PRFs) in samples in FTF and IBT trials differ or not. 9,10his study was preregistered with OSF Registries. 11We aimed to compare the samples of RCTs of FTF and IBT for depression with regard to PRFs and explore their association with outcome.

Methods
This systematic review and meta-regression analysis followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline (eTable 1 in Supplement 1) and used the Cochrane revised risk-of-bias tool to assess risk of within-study bias (eMethods in Supplement 1). 12,13We included RCTs that investigated the efficacy of FTF (individual or group) or IBT (guided or selfguided) in adults with acute depressive symptoms compared with either a treatment as usual or waiting list control group.PsycINFO, Cochrane CENTRAL, and the reference lists of published meta-analyses were searched from January 1, 2000, to December 31, 2021 (eTable 2 in Supplement 1).
A prognostic risk index (PROG) was created to quantify in each trial the extent to which participants with PRFs were enrolled.The PROG comprised 12 predefined PRFs (eg, diagnosis of depression, and lower income) from the literature (eMethods 2 and eTable 8 in Supplement 1).Three factors were coded dichotomously indicating the presence (1) or absence (0) of participants with a given prognostic factor, and 9 were coded on a scale including 1, 0.5, and 0. If information was not assessed, not reported, or unclear, the factor was coded as absent (0).The PROG was equal to the sum of the factors ranging from 0 to 12, with higher values representing a sample that comprised patients with a more unfavorable prognosis.The interrater reliability (intraclass coefficient using 2-way randomeffects and absolute agreement) for PROG was 0.87 (95% CI, 0.80-0.91;F = 7.87; P < .001).
We used the Mann-Whitney U and Kruskal-Wallis tests to assess PROG differences between trials investigating FTF and IBT and their submodalities (individual FTF, group FTF, guided IBT, and self-guided IBT).Fisher exact and χ 2 tests were used to assess differences in the frequencies of single prognostic factors.Meta-regression analyses estimated the association between PROG and outcome.The primary outcome was the standardized mean difference (Hedges g effect size) in depressive symptoms at treatment termination, with a positive standardized mean difference indicating larger improvements in the intervention compared with those in the control group.Metaregression analyses were adjusted for type of control group. 14hree preregistered and 2 exploratory sensitivity analyses were conducted.The threshold for statistical significance was 2-tailed (P = .05).

Key Points
Question Do samples of randomized clinical trials (RCTs) of face-to-face therapy (FTF) and internet-based therapy (IBT) for depression differ with regard to the prognostic risk factors (ie, prognosis) of the included patients?
Findings In this systematic review and meta-regression analysis of 105 RCTs comprising 18 363 participants, the prevalence of patients with poor prognosis was higher in RCTs of FTF than in the RCTs of IBT.The quality of reporting of prognostic risk factors was not optimal.
Meaning These results suggest that indirect comparisons of FTF and IBT may be problematic because, in terms of reporting prognostic risk factors, samples of RCTs may not be drawn from the same clinical population.
A random-effects meta-regression analysis suggested that PROG was not associated with effect size (Table ).Betweenstudy heterogeneity of effect sizes was large (I 2 = 93%).After exclusion of 4 outliers (Hedges g > 2.00), PROG was associated with outcome (B = 0.05 [95% CI, 0.02-0.09];SE = 0.02; P = .006).Two sensitivity analyses, which accounted for risk of bias (low risk vs other) and small-study size (N < 100) indicated no association between PROG and outcome.Exploratory sensitivity analyses with outliers removed indicated that the association between PROG and outcome was not significant after controlling for treatment modality (Table ).

Discussion
This systematic review and meta-regression analysis found that samples of RCTs of FTF more often included patients with PRFs than RCTs of IBT.An explanation might be that these trials tend to include self-selected samples from the community or online sources, as opposed to FTF trials, which more frequently enroll patients from clinical settings. 15ur results regarding the association of PROG and outcome are inconclusive, which might be due to the study design,

Limitations
This study has limitations.In total, 90 of 105 trials were judged at a risk of bias or some concern.We assigned equal weights to all indicators, but research has yet to determine whether they are of equal relevance.If no information about a PRF was available, we treated this factor as absent, but we cannot be sure if this was actually the case.Our sample size was probably too small for subgroup analyses to reliably address the associations between PROG and outcome.

Figure
Figure.Prognostic Risk Index (PROG) as a Function of 4 Treatment Submodalities in Samples of Randomized Clinical Trials on Face-to-Face Therapy (FTF) and Internet-Based Therapy (IBT) for Depression10

Table .
Random-Effects Meta-Regression Analysis of Prognostic Risk Factors in Trials of Therapy for Depression vs Control a Self-guided IBT served as the reference group.Risk Factors in Randomized Clinical Trials of Face-to-Face and Internet-Based Psychotherapy for DepressionBrief Report Research sample size, or the composition of the PROG.The reporting quality of PRFs was poor in most trials, particularly in IBT trials.This renders the translation of trial results difficult in clinical practice, where most patients present with complex conditions.Based on these findings, comparisons across different depression treatments are difficult because trial samples are possibly not drawn from the same clinical population.To improve, reporting guidance is needed about which PRFs are deemed relevant, and adherence to guidelines should be encouraged by grant providers and scientific journals.
a Coefficient refers to the meta-regression coefficient; restricted maximum likelihood was used.bThevalue indicates that a 1-point increase in PROG was associated with a 0.05-point increase in the effect size (Hedges g) that compared the intervention against the control group.c jamapsychiatry.com(Reprinted) JAMA Psychiatry Published online October 11, 2023 E3 Downloaded from jamanetwork.comby UZH Hauptbibliothek / Zentralbibliothek Zuerich user on 11/27/2023 its