Effectiveness of Self-guided App-Based Virtual Reality Cognitive Behavior Therapy for Acrophobia: A Randomized Clinical Trial

Key Points

Question  Is fully self-guided app-based virtual reality cognitive behavior therapy using low-cost (cardboard) virtual reality goggles user friendly and effective in reducing acrophobia symptoms compared with a wait-list control group?

Findings  In a single-blind randomized clinical trial that included 193 participants with acrophobia symptoms, app-based therapy demonstrated a large and significant reduction in acrophobia symptoms compared with wait-list controls and was rated as user friendly.

Meaning  Acrophobia cognitive behavioral therapy can be effectively delivered without therapist intervention through standard smartphones and low-cost virtual reality goggles at a fraction of the cost of current face-to-face treatment or high-end virtual reality exposure therapy.

Importance  Globally, access to evidence-based psychological treatment is limited. Innovative self-help methods using smartphone applications and low-cost virtual reality have the potential to significantly improve the accessibility and scalability of psychological treatments.

Objective  To examine the effectiveness of ZeroPhobia, a fully self-guided app-based virtual reality cognitive behavior therapy (VR CBT) using low-cost (cardboard) virtual reality goggles compared with a wait-list control group and to determine its user friendliness.

Design, Setting, and Participants  In a single-blind randomized clinical trial, participants were enrolled between March 24 and September 28, 2017, and randomly assigned (1:1) by an independent researcher to either VR CBT app or a wait-list control group. A total of 193 individuals aged 18 to 65 years from the Dutch general population with acrophobia symptoms and access to an Android smartphone participated. The 6 animated modules of the VR-CBT app and gamified virtual reality environments were delivered over a 3-week period in participants’ natural environment. Assessments were completed at baseline, immediately after treatment, and at 3-month follow-up. Analysis began April 6, 2018, and was intention to treat.

Intervention  Self-guided app-based VR CBT.

Main Outcomes and Measures  The primary outcome measure was the Acrophobia Questionnaire. The hypothesis was formulated prior to data collection.

Results  In total, 193 participants (129 women [66.84%]; mean [SD] age, 41.33 [13.64] years) were randomly assigned to intervention (n = 96) or a wait-list control group (n = 97). An intent-to-treat analysis showed a significant reduction of acrophobia symptoms at posttest at 3 months for the VR-CBT app compared with the controls (b = –26.73 [95% CI, −32.12 to −21.34]; P < .001; d = 1.14 [95% CI, 0.84 to 1.44]). The number needed to treat was 1.7. Sensitivity and robustness analysis confirmed these findings. Pretreatment attrition was 22 of 96 (23%) because of smartphone incompatibility. Of the 74 participants who started using the VR-CBT app, 57 (77%) completed the intervention fully.

Conclusions and Relevance  A low-cost fully self-guided app-based virtual reality cognitive behavioral therapy with rudimentary virtual reality goggles can produce large acrophobia symptom reductions. To our knowledge, this study is the first to show that virtual reality acrophobia treatment can be done at home without the intervention of a therapist.

Trial Registration  Trialregister.nl identifier: NTR6442

Access to evidence-based psychological treatment for mental health disorders is a global challenge because of high-treatment costs¹ and the limited availability of mental health professionals.² Treatment coverage is below 50% and usually substantially lower.² Novel technologies may contribute to accessible and affordable treatment options in important ways.

Specific phobias rank among the most prevalent mental health disorders,^3,4 of which acrophobia is the most prevalent of all subtypes.^5,6 Worryingly, specific phobias may increase the risk of developing other anxiety disorders⁷ and major depression.⁸

Several evidence-based therapies exist, most of which use cognitive behavior therapy (CBT), involving exposing individuals to their feared object or situation.⁹ An emergent form of exposure is virtual reality exposure therapy (VRET), in which immersive virtual environments replace real-life exposure settings.¹⁰ Meta-analyses indicate VRET is as effective as conventional exposure therapy^11-13 with large effect sizes^11-15 and similar dropout and deterioration rates.^11,12,16,17 However, VRET has thus far required therapist involvement and uses high-end virtual reality (VR) equipment.¹⁸

Recently, efforts toward reducing therapist involvement using VRET have been undertaken. A randomized clinical trial¹⁹ targeting acrophobia demonstrated a large effect size (d = 2.0) but still required expensive technology, and treatment was delivered in a clinic under the supervision of a therapist. Hence, the relative effect of the intervention itself remains unknown and costs are not reduced. A literature search (eAppendix 1 in Supplement 2) yielded no results on previous studies using a mobile app for acrophobia treatment (except for the present study’s protocol²⁰).

Quiz Ref IDThe present study tested the effectiveness and user friendliness of ZeroPhobia, a fully self-guided VR CBT for acrophobia symptoms that is delivered through a smartphone. To ensure scalability, the VR-CBT app relies on participants’ own smartphone and basic ($10) cardboard VR goggles, while the program can be followed at home. We hypothesized that the app would be associated with greater overall response at posttest compared with a wait-list control group and that the treatment gains would be maintained at 3-month follow-up. For robustness, we included a second questionnaire assessing acrophobia symptoms. Depressive symptoms were examined to assess whether the VR-CBT app could also influence depression levels. We also tested whether variation in perceived user friendliness, general anxiety, and cyber sickness when using the app affected acrophobia symptoms at posttest.

In this single-blind randomized clinical trial, participants were recruited from the Dutch general population through websites, magazines, and local media. Ethical approval was received from the Medical Ethics Committee of the Vrije Universiteit Medical Center (the trial protocol has been previously published²⁰ and is available in Supplement 1). Participants provided written informed consent by email or mail.

Quiz Ref IDTo be considered for inclusion, individuals had to score at least 45.45 on the Acrophobia Questionnaire (AQ)–Anxiety (1 SD below the mean of a previous acrophobic sample),^21,22 have access to an Android smartphone (Android version 5.1 Lollipop or higher, 4.7- to 5.5-inch screen, and gyroscope), be aged 18 to 65 years, and have provided informed consent. Exclusion criteria were insufficient Dutch language skills, current phobia treatment or receiving psychotropic medication, or having severe depression (total score >19 on the Patient Health Questionnaire–9 item²³) or suicidality (score ≥3 on the Web Screening Questionnaire²⁴). Enrollment commenced March 24 and ceased September 28, 2017. Analysis began in April 2018.

A randomization list was created with Random Allocation Software²⁵ using block randomization of 6, 8, 10, and 12 blocks at an allocation ratio of 1:1. Participants were randomized into 2 groups: intervention or wait-list. The randomization list was kept by an independent researcher who revealed the next randomization outcome after every inclusion, thus ensuring that the research team was blind for treatment allocation. All materials were completed online without researcher intervention.

Quiz Ref IDParticipants received 6 animated CBT-based modules using 2-dimensional animations and a voice-over provided by a virtual therapist. The modules took between 5 and 40 minutes to complete. Participants were asked to complete the entire intervention within 3 weeks. Aside from the psychoeducation and CBT techniques, the VR-CBT app included a gamified immersive VR environment and four 360° videos covering the entire exposure spectrum. The participants started using VR and 360° videos from module 3 onwards and navigated through the virtual environment using gaze control. Because previous research found better effects when the treatment was delivered twice a week compared with once a week,²⁶ the 6 app modules were offered over a period of 3 weeks. However, participants could practice with the VR-CBT app as often as they wanted. During the 3-week period, participants received weekly standardized motivational emails with reminders to start or continue with the app (Donker et al²⁰ includes intervention details; eFigure in Supplement 2 includes VR screen shots). The VR environment was created with the Unity game engine (version 2017.3.0f3; Unity Technologies).

All measures at baseline, posttest, and 3-month follow-up were completed online. The participants were asked for demographic information and at each assessment point, whether they received any other professional acrophobia treatment.

The primary outcome was the AQ.²¹ In a previous study, the mean (SD) AQ-Anxiety scores in a Dutch population were 59.06 (17.12).²⁷ Secondary outcomes included the Attitudes Toward Heights Questionnaire,²⁸ the Beck Anxiety Inventory,²⁹ the System Usability Scale³⁰ to assess user friendliness, the Igroup Presence Questionnaire³¹ to assess presence in VR, Mastery³² to assess subjective feelings of control, and the Patient Health Questionnaire–9 item²³ to assess depressive symptoms. The use of professional treatment during the VR-CBT app, the 3-item Simulator Sickness Questionnaire (SSQ),^33,34 and vision impairment (eg, wearing glasses) were included as control variables. The SSQ is commonly used to assess simulator sickness but not in the context of anxiety treatment; the items completely overlap with anxiety symptoms (eg, nausea, dizziness).¹⁹ We used a simple raw score of whether the symptom was present or not. All assessments were programmed with Survalyzer software.³⁵

The primary outcome measure, the AQ, was used for the power calculations. In previous meta-analyses in which VRET vs a wait-list control group was compared using the AQ, effect sizes ranged from 0.93 to 1.23.^12,14,15 However, the unguided nature of the VR-CBT app, the use of low-end equipment, and the lack of comparable studies led us to opt for a conservative estimate of Cohen d = 0.50. With 80% power at a 5% significance level (2-sided), 64 participants in each condition were required (128 participants in total). Because of the high dropout rates often encountered in self-guided interventions,³⁶ we estimated a 40% dropout rate, meaning 180 participants were required.²⁰

Demographic and clinical characteristics were compared between the intervention and the wait-list control groups, with χ² or variance analysis as appropriate. To assess whether pretreatment attrition (participants who failed to commence treatment after randomization³⁷) and dropout (participants who discontinued treatment after commencing treatment³⁷) were nonrandom, a balancing table was constructed comparing background characteristics, prescores, and other covariates between participants with and without missing outcome observations. Missing outcome observations were accounted for by using a single best regression-imputed value. Missing outcome values for the pretreatment attrition sample were imputed using the wait-list control sample, whereas for the dropout sample initial treatment assignment was used. The empirical evaluation was performed on an intention-to-treat (ITT) basis, using ordinary least squares regression models with prescores and background characteristics included. Standardized effect sizes (Cohen d) and confidence intervals were calculated for each end point.

Two additional robustness analyses were performed. First, because nonrandom sample attrition may bias the estimated treatment effects, nonparametric treatment effect bounds were estimated.³⁸ Second, multiple imputation using an iterative Markov chain Monte Carlo method based on initial treatment assignment was performed to infer the statistical significance of the reported differences between the intervention and control groups. Furthermore, potential heterogeneity and mechanisms of effective treatment were addressed by estimating how treatment effects vary with prescores, general anxiety, and experiences when using the app. We also analyzed the predicted reductions in the posttest for 10-point bins of prescores using regression analyses to examine whether severity at baseline AQ predicts outcome. Sensitivity analyses using participants who completed posttest or follow-up after the intervention examined whether there was a difference in the results compared with the ITT analysis.³⁹ We exploratively assessed clinically meaningful change on the AQ using the clinically significant change formula (method C⁴⁰) with an AQ score less than 31.67.⁴¹ The reliable change criterion was assessed based on the AQ pretest SD scores with a Cronbach α of .91. This yielded an SE of change of 7.98, with a corresponding reliable change criterion on the AQ score of 15.65 (7.98 × 1.96).^42,43 We also assessed the number needed to treat using the formula by Furukawa et al⁴⁴ based on the effect size of the AQ. Two-sided P value less than .05 indicated statistical significance. Stata version 14.2 (StataCorp) was used for the analyses except for number needed to treat, which was conducted in R version 3.4.3 (R Project for Statistical Computing). A data monitoring committee was not required by the ethics committee because of the expected low safety risk of the participants.

We deviated on some issues from the ethics protocol (Supplement 1). For details, see eAppendix 2 in Supplement 2.

Of 663 individuals who signed up for participation, 291 were ineligible and excluded from participation. In total, 193 participants filled in the baseline assessment and were randomly assigned to the VR-CBT app (n = 96) or wait-list control (n = 97). The pretreatment attrition rate was 23% in the app condition because of illness (1 [1%]) or an incompatible smartphone (21 [22%]). Several participants with smartphones lacking a gyroscope (required for experiencing VR) were erroneously included. To ensure that this did not jeopardize randomization, these participants were included in the ITT analyses, and Lee bounds³⁸ were estimated to address potential bias. Figure 1 shows the participant flowchart.

Baseline demographics are provided in Table 1. Data were balanced between the groups. None of the participants reported receiving professional acrophobia treatment at baseline, posttest, or follow-up.

There were no significant differences in the demographics between participants (n = 193) and those who were ineligible or chose not to participate (n = 346) (data not shown). In the intervention group, 59% (57 of 96) of the participants completed the posttest and 49% (47 of 96) completed follow-up, compared with 91% (87 of 97) posttest responses for the wait-list group. Attrition (22 of 96 [23%]) and dropout (17 of 96 [18%]) were not related to background characteristics, prescores, and other covariates (eTable 1 in Supplement 2), as indicated by the likelihood ratio test for predicting missing outcomes (χ² (9) = 3.47; P = .94), supporting the assumption that outcomes are missing completely at random.⁴⁵

Quiz Ref IDThe intervention condition showed a significant reduction in acrophobia symptoms compared with the control on the AQ in the ITT analysis at posttest (b = −26.73 [95% CI, −32.12 to −21.34]; t₁₉₁ = −9.79; P < .001; adjusted R² = 0.52) with an effect size of d = 1.14 (95% CI, 0.84-1.44). The number needed to treat was 1.7 (Table 2 and Table 3).

The results of the ITT analyses (Table 3) demonstrated a significant intervention effect compared with the controls on acrophobia symptoms, Attention to Height Questionnaire (b = −12.59; t₁₈₂ = −8.92; P < .001; d = 1.091; 95% CI, 0.787-1.393), general anxiety symptoms, Beck Anxiety Inventory (b = −3.87; t₁₈₂ = −2.62; P < .001; d = 0.37; 95% CI, 0.087-656), and a greater sense of mastery (b = 1.01; t₁₈₂ = −2.00; P = .047; d = −0.11; 95% CI, −0.389 to 0.176). Depressive symptoms did not change significantly between both groups, Patient Health Questionnaire–9 item (b = −0.49; t₁₈₂ = −1.58; P ≥ .99; d = 0.14; 95% CI, −0.143 to 0.422).

When estimating the Lee bounds,³⁸ the resulting upper and lower bounds (model 2) indicated that the treatment effect on the primary outcome measure (AQ) remained statistically significant. The ITT estimate in model 1 (b = −26.73, SE = 2.73) was close to the conservative upper Lee bound (b = −22.34, SE = 4.55) after imputing outcomes for the attrition sample using wait-list control participants. Applying multiple imputation techniques for the attrition and dropout sample, based on initial treatment assignment yielded somewhat bigger SEs (model 3), but statistical significance of the ITT effect on AQ was maintained (d = 1.53; 95% CI, 1.15-1.91; Table 3).

Adding an interaction term to the ITT treatment estimation model of Table 3 revealed that treatment effectiveness related positively to baseline acrophobia symptoms (ie, as a modifier) (b = –0.306; SE = 0.15; P = .04), indicating higher potential benefits for participants with more severe acrophobia symptoms (eTable 3 in Supplement 2).

The experience-related results indicate that a reduction in acrophobia symptoms tends to be larger when the usability of the app was higher (System Usability Scale), when the feeling of being present in the virtual environment was higher (Igroup Presence Questionnaire), and when participants experienced more symptoms of cyber sickness (SSQ) (Figure 2). The robustness analysis further confirmed that the VR-CBT app had an impact on the anxiety for heights and that the general anxiety effect did not drive the results (eTable 4 in Supplement 2). eTable 5 in Supplement 2 provides regression-predicted reductions in posttest for 10-point bins of prescores suggesting that moderation is quantitative. Those with relatively low levels of baseline acrophobia did benefit significantly from the treatment.

For individuals who returned the posttest, the between-group AQ effect size was d = 1.53 (95% CI, 1.15-1.91; number needed to treat: 1.4), and for the app individuals, the within-group effect size was d = 2.68 (95% CI, 2.09-3.22) between baseline and follow-up. Effect sizes on secondary outcomes for app individuals were also large (eTable 2 in Supplement 2). The VR-CBT app participants’ total mean (SD) Igroup Presence Questionnaire score was 42.69 (10.40) (range, 14-70).

For individuals who returned the AQ posttest (n = 56), all participants showed reliable change. Furthermore, 44 of 56 (79%) experienced clinically significant change, a change of 57.97 or more points on the AQ.

The VR-CBT app was rated as user friendly (mean [SD], 75.35 [14.74], n = 55) and can be interpreted as a good and usable system.²² No deterioration or negative effects as defined by Rozental et al⁴⁶ were identified, except for 24 participants who reported 1 or more symptoms of transient cyber sickness.

To our knowledge, this randomized clinical trial tests the first fully self-guided treatment for acrophobia. We showed that an app-based VR-CBT program using low-cost VR goggles is both effective and user friendly. Large reductions in acrophobia symptoms on both the self-reported primary outcome measure AQ and secondary outcome measure Attention to Height Questionnaire were obtained. At 3-month follow-up, the results were maintained for the intervention group as demonstrated by the large within-group effect size. Importantly, these findings are in line with previous meta-analyses of guided VR acrophobia studies using high-end VR equipment^12,14,15 and similar or better compared with therapist-guided exposure in vivo.^47,48 In the current study, dropout rates were similar compared with dropout rates in previous VR studies.¹⁶ The app participants also experienced a greater reduction in general anxiety and an increased sense of mastery compared with the controls. As expected, because the baseline level of depression was low, the symptoms did not decrease; hence, there were no between-group differences. App participants perceived the VR environment as realistic although these results are slightly lower compared with previous research.⁴⁹ This might be because of the quality of the cardboard VR goggles. However, the VR-CBT app was rated as user friendly and a good and usable system. Interestingly, the participants who experienced more cyber sickness also experienced a larger reduction in acrophobia symptoms. This might be an artifact of the SSQ because it overlaps with symptoms of anxiety. For exposure to be beneficial, experiencing anxiety is required. Therefore, the SSQ might not be a valid measure of cyber sickness for treating anxiety.

One of the key strengths of the present study was its ecological validity because the intervention was conducted in the participants’ natural environments instead of a research laboratory. Furthermore, participants received no guidance when using the VR-CBT app or filling in assessments, thereby ruling out any influence of human contact. Another strength was that the missing outcome observations for participants assigned to treatment to the VR-CBT app but unable to use the app because of incompatible smartphones were based on wait-list control group characteristics (ie, no treatment effect was imputed), yielding conservative overall treatment effect estimates. Furthermore, the ITT design has strong credibility in terms of aggregate effects when the intervention is scaled and adopted in real life.

Quiz Ref IDThe current study had several limitations. First, attrition was relatively high in the intervention condition because of incompatible smartphones. However, attrition was statistically unrelated to all observable characteristics at baseline. To retain a balanced experimental sample, the results were imputed for missing participants using regression-based imputation. Robustness and sensitivity analyses confirmed that potential bias concerns and precision concerns did not compromise the statistical significance of the results. Second, the data rely exclusively on self-reported measurements. However, the behavioral measurement effect sizes were similar to those calculated from other self-reported measures.¹³ Because the AQ has no validated cutoff scores, we were unable to provide results on the clinical threshold for acrophobia. We did not include a diagnostic interview because we wanted to isolate the true effect of VR; even in the assessment phase, human involvement may already have influenced the results.^50,51 Third, because our follow-up period was only 3 months and because of dropout, the results are exploratory, and longer-term effects remain unknown. Fourth, the generalizability of the results is limited to Dutch Android smartphone users. However, most of the Dutch population (57%) uses Android.⁵²

Future research is needed to replicate the VR-CBT app to draw firm conclusions about its effectiveness, compare the app directly with high-end VRET and exposure in vivo treatment, examine its long-term effects, determine cost effectiveness compared with treatment as usual, and examine whether the app can also be effective in the treatment of other phobias and mental health disorders.

In sum, our findings support the hypothesis that a fully self-guided app-based VR-CBT, which can be done at home at a fraction of the cost of existing evidence-based treatment options, strongly reduces acrophobia symptoms. The current study adds to the development of innovative and scalable delivery methods of evidence-based treatments and underlines that new technologies have the potential to transform mental health care worldwide.

Corresponding Author: Tara Donker, PhD, Department of Clinical, Neuro- and Developmental Psychology, Vrije Universiteit Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, the Netherlands, (t.donker@vu.nl).

Accepted for Publication: January 7, 2019.

Published Online: March 20, 2019. doi:10.1001/jamapsychiatry.2019.0219

Author Contributions: Drs Cornelisz and van Klaveren had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Donker and van Gelder.

Acquisition, analysis, or interpretation of data: Donker, Cornelisz, van Klaveren, van Straten, Carlbring.

Drafting of the manuscript: Donker, Cornelisz, van Klaveren.

Critical revision of the manuscript for important intellectual content: van Klaveren, van Straten, Carlbring, Cuijpers, van Gelder.

Statistical analysis: Cornelisz, van Klaveren.

Obtained funding: Donker.

Administrative, technical, or material support: van Gelder.

Supervision: Donker, van Straten, Cuijpers.

Conflict of Interest Disclosures: Dr Donker reports grants from NWO Toegepaste en Technische Wetenschappen and NWO Creative Industrie-KIEM during the conduct of the study. Drs Donker and van Gelder have developed the virtual reality application ZeroPhobia, which is used in the present study in collaboration with Vrije Universiteit. ZeroPhobia is intended for commercial release. Hence, Drs Donker and van Gelder have not been involved in data analysis or any decisions related to the publication of findings. No other disclosures were reported.

Funding/Support: This study has been funded by NWO Toegepaste en Technische Wetenschappen (2016/STW/00099738) and NWO Creative Industrie-KIEM (314-98-076).

Role of the Funder/Sponsor: The funders of the study had no role in study design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, writing of the report, review or approval of the manuscript; and the decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 3.

Additional Contributions: We extend our gratitude to all the participants of the study; Bruno de Vos, BA (Studio Barbaar), for designing ZeroPhobia; Wouter Houwen (Fontys Hogescholen Eindhoven), Ruud op den Kelder, MSc (Robo Ruud), Bas van Loon (Fontys Hogescholen Eindhoven), and Doruk Eker, MSc (Orb Amsterdam), for programming ZeroPhobia; Rufus van Baardwijk, MSc, for ZeroPhobia sound; and Stefanie van Esveld, MSc (Vrije Universiteit Amsterdam), and Niclas Fischer, MSc (Vrije Universiteit Amsterdam), for assistance in data recruitment and data collection. Bruno de Vos, BA; Wouter Houwen; Ruud op den Kelder, MSc; Bas van Loon; Doruk Eker, MSc; and Rufus van Baardwijk, MSc, recieved financial compensation.