A, Change in depression least square mean scores by intervention condition over the intervention period (with imputed data). B, Change in anxiety least square mean scores by intervention condition over the intervention period (with imputed data).
eTable 1. GLM model parameter estimates comparing treatment conditions on changes in the primary outcomes among individuals with depression or anxiety
eTable 2. Correlations and 95% confidence intervals between individual app use and outcome
Data Sharing Statement.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Graham AK, Greene CJ, Kwasny MJ, et al. Coached Mobile App Platform for the Treatment of Depression and Anxiety Among Primary Care Patients: A Randomized Clinical Trial. JAMA Psychiatry. 2020;77(9):906–914. doi:10.1001/jamapsychiatry.2020.1011
Is a mobile intervention platform composed of a suite of simple-to-use apps, supported by brief coaching, efficacious for treating depression and anxiety among primary care patients?
In this randomized clinical trial of 146 patients with depression and anxiety, a mobile platform achieved greater reductions in depression and anxiety and higher odds of recovery compared with treatment-as-usual wait list control individuals, and effects were sustained at follow-up. Engagement with apps was high throughout the intervention period.
Results support the efficacy of a platform approach to mobile intervention using apps designed to fit into the fabric of users’ lives for treating patients with depression and anxiety in primary care.
Depression and anxiety are common and disabling. Primary care is the de facto site for treating these mental health problems but is typically underresourced to meet the burden of these demands.
To evaluate the efficacy of a mobile intervention platform, IntelliCare, for addressing depression and anxiety among primary care patients.
Design, Setting, and Participants
Two-arm randomized clinical trial at internal medicine clinics at the University of Arkansas for Medical Sciences. Adult primary care patients (N = 146) who screened positive for depression on the Patient Health Questionnaire-8 (PHQ; score ≥ 10) or anxiety on the Generalized Anxiety Disorder-7 (GAD-7; score ≥ 8) were recruited between July 17, 2018, and December 14, 2018.
The coach-supported platform composed of a suite of apps, was delivered over 8 weeks. Wait list control participants received treatment as usual for 8 weeks, then the mobile platform.
Main Outcomes and Measures
Primary outcomes were changes in depression (PHQ-9) and anxiety (GAD-7) during the intervention period. Secondary outcomes were differences in the proportion of patients who achieved recovery (PHQ-9/GAD-7 <5 or 50% improvement from baseline), sustainment of intervention effects during 2-month follow-up, and app use during the intervention period.
One hundred forty-six patients were included (119 of 146 were women [81.5%]; mean [SD] age, 42.3 [13.8] years). Of the 146 patients, 122 (83.6%) were diagnosed as having depression and 131 (89.7%) were diagnosed as having anxiety. A greater proportion of intervention vs wait list control participants achieved recovery from depression (n = 38 of 64 [59%] vs n = 18 of 58 [31%]; odds ratio, 3.25; 95% CI, 1.54-6.86) and anxiety (n = 37 of 65 [57%] vs n = 25 of 66 [38%]; odds ratio, 2.17; 95% CI, 1.08-4.36). Sustained effects were observed for depression (slope, 0.01; 95% CI, –0.09 to 0.10; P = .92) and anxiety scores (slope, 0.02; 95% CI, –0.08 to 0.12; P = .67) during follow-up. App use was high, with a median of 93 and 98 sessions among participants with depression and anxiety, respectively.
Conclusions and Relevance
In this trial, a mobile intervention app was effective for depression and anxiety among primary care patients. Findings also support designing digital mental health interventions as platforms containing simple, brief apps that can be bundled by users to meet their needs.
ClinicalTrials.gov Identifier: NCT03500536.
Depression and anxiety are prevalent, impairing, and costly health problems, most often treated in primary care.1-3 The US Preventive Services Task Force recommends routine depression screening in primary care4; however, without additional assistance to support treatment, increased identification provides no benefit.5 Unfortunately, demand for mental health services outweighs workforce capacity and resources to meet these needs. Digital mental health interventions (DMHIs), using Internet-based and mobile tools, can be cost-effective and have potential to improve capacity of health care settings to address mental health problems.6 However, implementation in primary care has often failed, primarily because patients do not use the tools.7
Computer-delivered DMHIs are primarily based on evidence-based therapies, rely heavily on psychoeducation, and typically require use for 30 to 45 minutes per week for 6 or more weeks. Despite good efficacy in randomized clinical trials (RCTs),8 real-world engagement is challenging.7,9,10 In contrast to computers, people tend to use smartphone apps for a single purpose, in short bursts of time, sometimes frequently throughout the day.11,12 Thus, app-based DMHIs often focus on 1 to 2 psychological strategies and more on actions than psychoeducation.13 App-based DMHIs have demonstrated efficacy in RCTs,14,15 are increasingly used by the public, and produce reasonable early engagement, but adherence drops dramatically over the first 2 weeks.13
App-based DMHIs may have greater success if they accommodate app use conventions and overcome 2 major challenges for delivering mental health interventions. First, the many effective behavioral strategies may appeal differentially to different people. Second, people do not tend to use mental health apps for long periods. This may be because people seek the novelty of new apps16 or have acquired the app’s benefit and no longer find it useful. Thus, a single app approach for a mental health condition may be less effective.17
The IntelliCare platform was designed to address these challenges. Rather than a single app, the platform contains multiple apps, each of which is brief, simple, and supports a single clinical target. For example, the Daily Feats app targets goal-setting; the sole activity is setting and monitoring completion of daily goals, with scaffolding to increase complexity. Because depression and anxiety are highly comorbid1,2 and respond to similar psychological strategies,18 the platform was designed to address both conditions. This mobile platform has been evaluated in field trials, the first of which assessed symptom changes over 8 weeks and the second of which tested 2 engagement strategies.19,20 In both trials, the platform produced significant reductions in depression and anxiety19,20 and consistent engagement.16,21
Quiz Ref IDThe purpose of this RCT was to evaluate the efficacy of the platform for the treatment of depression and anxiety in primary care patients. We hypothesized that individuals randomized to receive the mobile platform would report greater reductions in depression or anxiety symptoms compared with treatment-as-usual wait list control individuals (WLC) during the 2-month intervention period (primary outcome) and have a higher proportion of patients achieve recovery from depression or anxiety (secondary outcome). Secondary outcomes also included evaluating sustainment of effects over follow-up among individuals in the condition and engagement metrics. To our knowledge, this is the first evaluation of an app platform for the treatment of a mental health condition.
This RCT compared IntelliCare with treatment as usual WLC for depression and anxiety. The University of Arkansas for Medical Sciences (UAMS) institutional review board approved the study. A data safety monitoring board provided oversight. Supplement 1 provides the trial protocol.
The trial protocol in Supplement 1 details entry criteria. In brief, adult UAMS primary care patients were eligible if they had a compatible smartphone (Android or Apple) and elevated symptoms of depression or anxiety at screening (score ≥10 on the Patient Health Questionnaire-8 [PHQ-8]22 or ≥8 on the Generalized Anxiety Disorder-7 scale [GAD-7]23). Individuals were excluded, among others, if they were acutely suicidal,24 had a diagnosis for which the IntelliCare study might not be appropriate, and were currently in or scheduled to initiate psychotherapy to avoid treatment interference. Psychotropic medication was permitted if dose was stable over the past 2 weeks to eliminate medication placebo effects.
Participants were recruited between July 17, 2018, and December 14, 2018, from UAMS internal medicine clinics on the main medical center campus and surrounding communities. These clinics serve a diverse, often low-income population. All clinics provided routine depression screening; some had on-site integrated behavioral health resources. Participants were referred by a clinician or responded to outreach methods. Recruitment strategies have been described25 and are in the protocol in Supplement 1.
Interested individuals completed an online screener; those whose results indicated potential eligibility were automatically invited to provide online written informed consent. After medical record review by the research team, individuals who remained potentially eligible were invited to complete an online baseline assessment.
Following randomization, participants in the intervention condition began IntelliCare via an onboarding telephone call with their assigned coach. Wait list control participants were informed they would have access to IntelliCare after an 8-week delay. All participants were prompted to complete assessments every 4 weeks for 16 weeks, administered online and independent of the intervention to mask outcomes to coaches. Participants could receive up to $100 for completing the research assessments. Data collection ended on April 17, 2019.
An independent statistician created a computer-generated randomization sequence with a 1-to-1 ratio in randomly permutated blocks of 4 and 6. Randomization assignment was concealed until after entry criteria were confirmed.
The intervention was delivered over 8 weeks using the coach-supported IntelliCare platform. IntelliCare is a suite of mobile apps available on iPhone and Android operating systems. The Hub app organized the user experience by supporting access to clinically focused IntelliCare apps, providing a library of psychoeducational material, and administering a weekly symptom assessment.19 In this study, participants received access to 5 clinically focused apps (described in the protocol in Supplement 1).19 Each week, a coach recommended a new app to download and try, based on the participant’s preferences and a recommendation protocol. Participants were encouraged to try the newly recommended app but could download any app at any time and use or discontinue apps as preferred.
Coaches were 2 bachelor’s degree–level individuals who received training in the coaching manual26 and weekly supervision with a clinical psychologist. Coaching was delivered primarily via short message service text messaging; coaches typically initiated 2 contacts per week and responded to participants’ messages. Participants also received a welcome packet via email (developed with health literacy experts), 1 onboarding telephone call (approximately 30-45 minutes), and the offer for an optional midtreatment call (approximately 10-15 minutes). Further details are described in the protocol in Supplement 1.
Coaches managed participants using an online dashboard, which provided information on each participant’s app use and weekly symptom assessment score and enabled sending and receiving text messages. Apps remained available to participants after treatment ended.
The WLC allowed participants to continue treatment-as-usual through primary care. Following the 8-week delay, WLC individuals received the coached intervention.
Quiz Ref IDPrimary outcomes (PHQ-927 and GAD-723) were measured at baseline and each follow-up assessment. App use metrics were captured automatically and stored on a secure server. App use was defined using 3 commonly used metrics20,28: number of app sessions, number of days the intervention was used, and time to last use (ie, number of days between the first and last launch) during the treatment period. An app session was defined as a user-initiated action or event after 5 minutes of no activity. Engagement with coaching was defined as number of messages sent by coaches and participants.
Assuming 80% power and α = .05, a medium effect size of 0.5 between IntelliCare and WLC could be expected to be detected with 128 participants (64 per arm). Eighteen additional participants enrolled before recruitment procedures closed. All analyses were performed at a 2-sided type 1 error rate of .05 in SAS, version 9.4 (SAS Institute Inc).
Analyses followed intention to treat. Five participants did not have any follow-up data during the first phase of the study, evenly split between study arms (IntelliCare, 3; WLC, 2). No evidence indicated these participants were other than missing at random; hence, multiple imputation methods were used to impute follow-up data for these participants.
Primary outcomes were analyzed separately for participants who screened positive for depression and for anxiety to assess change in depression and anxiety, respectively. General linear mixed models were fit for PHQ-9 scores among those with depression and GAD-7 scores among those with anxiety for each imputed data set. Results were combined for those analyses to present valid inferences for the effect of the intervention over time. Primary analyses were unadjusted; secondary analyses adjusted for age, sex, and race.
Secondary outcome analyses were conducted. Univariable logistic regression models were fit to determine whether there was an association between study condition and recovery at the postintervention point, defined as PHQ-9/GAD-7 scores less than 5 or 50% reduction from baseline; additional models were fit adjusting for age, sex, and race. To assess sustained response between posttreatment and 16-week follow-up (among only those in the IntelliCare condition), general linear mixed models were fit for PHQ-9 among those with depression and GAD-7 among those with anxiety; models did not use imputed data. Correlations were used to assess the association between app use metrics and outcomes and engagement with coaching and outcomes at the postintervention point. Because these were not planned comparisons, only confidence intervals are reported.29
Figure 1 presents the CONSORT diagram. Of 435 individuals screened, 146 participants (34%) were enrolled and randomized. At screening, 122 participants presented with depression (64 IntelliCare; 58 WLC) and 131 presented with anxiety (65 IntelliCare; 66 WLC); most (n = 107; 73%) had comorbid depression and anxiety. Table 1 shows pretreatment characteristics of the sample and 2 study conditions among the subsets with depression or anxiety. The lost-to-follow-up rate was low, with 141 of 146 participants (96.6%) completing at least 1 follow-up assessment.
Quiz Ref IDFigure 2A and B depict the least square means (LSM) of outcomes over time for depression and anxiety, with imputed data from 5 multiply imputed data sets. eTable 1 in Supplement 2 provides model parameter estimates. There was a significant group-by-time effect over 8 weeks (F2,33 316 = 15.9; P < .001), meaning those assigned to the mobile platform experienced a faster rate of improvement than WLC. The LSM difference in depression scores between the mobile platform and WLC at week 4 was 2.88 (SE = 0.82; Cohen d effect size [ES] = 0.52) and at week 8 was 4.33 (SE = 0.83; d ES = 0.78). The LSM difference in anxiety scores at week 4 was 2.38 (SE = 0.77; d ES = 0.47) and at week 8 was 3.19 (SE = 0.76; d ES = 0.64).
After adjusting for age, sex, and race, the effect of group by treatment interaction remained significant for depression (F2,34 315 = 15.3; P < .001). The LSM difference in depression scores at week 4 was 2.91 (SE = 0.83; ESd = 0.43) and at week 8 was 4.37 (SE = 0.83; d ES = 0.64). Similarly, the group by treatment interaction remained significant for anxiety (F2,3228.4 = 11.9; P < .001). The LSM difference in anxiety scores at week 4 was 2.51 (SE = 0.78; d ES = 0.41) and at week 8 was 3.33 (SE = 0.76; d ES = 0.55).
The rate of recovery from depression was 59.4% in IntelliCare and 31.0% in WLC. The rate of recovery from anxiety was 56.9% in IntelliCare and 37.9% in WLC. The odds of recovery for depression were 3.25 (95% CI, 1.54-6.86) times greater and for anxiety were 2.17 (95% CI, 1.08-4.36) times greater for IntelliCare compared with WLC. After adjusting for age, sex, and race, odds of recovery increased slightly to 3.42 (95% CI, 1.57-7.48) for depression and 2.24 (95% CI, 1.07-4.68) for anxiety.
Among those randomized to IntelliCare, effects were sustained from posttreatment to 16-week follow-up. There was no evidence of a linear change in slope for change in depression scores (slope, 0.01; 95% CI, –0.09 to 0.10; P = .92), nor were the means different over time (F2,119= 0.02; P = .98). Similarly, there was no evidence of a linear change in slope for change in anxiety scores (slope, 0.02; 95% CI, –0.08 to 0.12; P = .67), nor were the means different over time (F2,118= 0.09; P = .92). Means and standard errors for the study conditions by depression and anxiety scores at each point are presented in Table 2, using observed data for ease of interpretation.
Quiz Ref IDTable 3 presents app use metrics for the full suite and each app. Neither app sessions (r, −0.03; 95% CI, −0.22 to 0.15), time to last use (r, −0.14; 95% CI, −0.31 to 0.05), nor days used (r, −0.05; 95% CI, −0.23 to 0.14) were strongly associated with changes in depression. Neither app sessions (r, 0.01; 95% CI, −0.17 to 0.19), time to last use (r, −0.04; 95% CI, −0.22 to 0.13)], nor days used (r, 0.02; 95% CI, −0.15 to 0.20) were strongly associated with changes in anxiety. eTable 2 in Supplement 2 shows correlations with outcomes for each app.
At 8 weeks following treatment, 119 participants (81.5%) had some app use. For all participants, postintervention median time to last app use was 28 days (range, 0-212 days) and median days used was 7 (range, 0-102 days).
Among those with depression, coaches sent a mean of 32 messages (SD, 9; range, 7-61); participants sent a mean of 19 messages (SD, 12; range, 1-68). There was a negligible correlation between change in PHQ-9 and number of messages from coaches (r, −0.07; 95% CI, −0.25 to 0.13) or participants (r, −0.03; −0.22 to 0.16). Among those with anxiety, coaches sent a mean of 32 messages (SD, 10; range, 7-61); participants sent a mean of 19 messages (SD, 12; range, 1-68). The correlation between change in GAD-7 and number of messages from coaches (r, −0.01; 95% CI, −0.19 to 0.17) or participants (r, −0.02; 95% CI, −0.20 to 0.17) was not meaningful.
There were no study-related adverse events.
This is, to our knowledge, the first RCT of an app-based platform for the treatment of depression and anxiety and of any app-based mental health treatment among primary care patients. Individuals who received the mobile platform had a greater reduction in depression and anxiety symptoms compared with treatment-as-usual WLC individuals, and changes were sustained over 2-month follow-up. Although analysis of the treatment period for the WLC was not planned or performed, once these participants received treatment, their depression and anxiety scores dropped to levels similar to those in the treatment group, further strengthening the conclusion that IntelliCare is effective for depression and anxiety among primary care patients.
Effect sizes of 0.78 and 0.64 for depression and anxiety in this trial are in the range of effect sizes in meta-analyses of face-to-face psychotherapy (0.69 and 0.84, respectively)30,31 and compare favorably with effect sizes for app-based treatments (0.38 and 0.33, respectively)14,15 and web-based interventions evaluated in primary care (0.31 and 0.26, respectively).32,33
Digital mental health interventions have been championed as a solution to access barriers and high dropout rates for psychotherapy among primary care patients.34,35 However, engagement has proven challenging. An effectiveness trial of a web-based intervention found participants logged in for approximately half of the 8 sessions32; in 2 implementation trials of web-based interventions in English primary care settings, median uses were 1-2, with only 10% to 19% completing their intervention.7,9 While we are unaware of mental health app adherence data in primary care, the median number of mental health app sessions is 3 to 9 among active public users.13 In contrast, we found high app use, with a median of 93 to 98 app sessions across all apps over 8 weeks. Our findings may be because the platform approach matches how people use apps: participants could use or ignore apps as preferred, and the novelty of receiving new apps is likely engaging.16,21,36 The sample also comprised a group rarely represented in technology research: in contrast to past research, which has tended to attract those drawn to technology,33 participants were from a diverse, traditionally underserved community seeking help for depression and anxiety. Thus, findings suggest this service appealed to patients more generally and underscore the importance of designing DMHIs to match patients’ app use preferences.33,37,38 App use was not strongly associated with outcomes, consistent with much of the DMHI literature.16,39,40 The association between use and outcomes is complex. Clinically meaningful app engagement may be more important than raw quantity.21 Furthermore, people tend to use DMHIs less as their depression improves.41 Taken together, the relative high engagement suggests that platform approaches, such as IntelliCare, may be more acceptable to patients and likely produce better outcomes than disorder-focused single apps; however, this remains to be tested in a trial.
Results also inform potential for scale. Although routine screening has increased the identification of patients with mental health problems in primary care, this has not necessarily led to improved outcomes.5 Further, despite the success of collaborative care, which embeds behavioral health specialists in primary care,42,43 demand for such services frequently exceeds health care organizations’ capacity. Digital mental health intervention app platforms have potential to enhance behavioral health treatment effectiveness by improving clinicians’ efficiency while maintaining cost-effectiveness. Indeed, in this trial, coaches spent less than 1 hour on the telephone with participants, communicating primarily through text messages. Text messaging is simple, unobtrusive, acceptable to patients, and overcomes the often high rates of disengagement frequently seen in phone-based coaching.7,9
Quiz Ref IDStudy limitations should be considered in interpreting results. Although previous studies20 of the mobile platform have shown maintenance of gains over 6 months,20 this trial did not have a follow-up period sufficient to replicate those findings. By using self-report depression and anxiety measures to align with screening/monitoring methods in primary care,44 findings may not generalize to mental health clinics where thorough clinical assessments are performed. Because 56% of the sample was receiving antidepressant medication, it is possible some of the within-treatment effects were owing to medication; however, these effects should be equivalent across treatment arms and do not affect the validity of the between-treatment effects. We also did not capture data on patients’ use of other apps during the intervention period. Finally, although the research was conducted in partnership with UAMS clinics, study team members delivered the intervention or assessments (performed by different individuals to prevent coaches from knowing outcomes). Future work will be required to design and evaluate implementation strategies that integrate DMHI app platforms into clinic workflows. Further, design work to support translation to adolescent populations and care settings may be beneficial given the prevalence of mental health problems and unmet treatment need among youths.45
The mobile app is effective for depression and anxiety among primary care patients, which represents an important innovation for alleviating the burden of these mental health problems in this population. Findings support the movement of DMHIs from single apps for depression or anxiety to platform approaches containing multiple brief apps that patients can bundle to meet their needs and fit into the fabric of their lives.
Corresponding Author: David C. Mohr, PhD, Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, 750 N Lake Shore Dr, 10th Floor, Chicago, IL 60611 (firstname.lastname@example.org).
Accepted for Publication: March 9, 2020.
Published Online: May 20, 2020. doi:10.1001/jamapsychiatry.2020.1011
Author Contributions: Drs Mohr and Kwasny had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Greene, Lieponis, Mohr.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Graham, Greene, Kaiser, Mohr.
Critical revision of the manuscript for important intellectual content: Graham, Greene, Kwasny, Lieponis, Powell, Mohr.
Statistical analysis: Kwasny, Powell, Mohr.
Obtained funding: Greene, Mohr.
Administrative, technical, or material support: Graham, Greene, Kaiser, Lieponis, Mohr.
Supervision: Graham, Greene, Mohr.
Conflict of Interest Disclosures: Dr Mohr has an ownership interest in Adaptive Health Inc, which has a license from Northwestern University to commercialize IntelliCare. Drs Graham and Kwasny and Ms Kaiser have received consulting fees from Actualize Therapy LLC. Dr Graham reported grants from the National Institute of Diabetes and Digestive and Kidney Diseases during the conduct of the study and from the National Institute of Mental Health outside the submitted work. Dr Greene reported grants from Actualize Therapy during the conduct of the study. Dr Powell reported grants from the National Institutes of Health Small Business Innovation Research during the conduct of the study. Dr Mohr reported grants from the National Institute of Mental Health during the conduct of the study; personal fees from Apple Inc; and other support from Actualize Therapy Inc and Otsuka Pharmaceuticals outside the submitted work; in addition, Dr Mohr had a patent to US Patent 15/654,245, 2018 pending. No other disclosures were reported.
Funding/Support: This work was supported by grants from the National Institutes of Health (R44 MH114725 and K01 DK116925) as well as by the Translational Research Institute (U54 TR001629) through the National Center for Advancing Translational Sciences of the National Institutes of Health.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Data Sharing Statement: See Supplement 3.
Create a personal account or sign in to: