CBT indicates cognitive behavioral therapy.
ACG indicates attention-control group; wCBT, web-based cognitive behavioral therapy.
wCBT indicates web-based cognitive behavioral therapy.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-Based Cognitive Behavioral Therapy Intervention for the Prevention of Suicidal Ideation in Medical InternsA Randomized Clinical Trial. JAMA Psychiatry. 2015;72(12):1192–1198. doi:10.1001/jamapsychiatry.2015.1880
Copyright 2015 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
In the United States, approximately 1 physician dies by suicide every day. Training physicians are at particularly high risk, with suicidal ideation increasing more than 4-fold during the first 3 months of internship year. Despite this increase, to our knowledge, very few efforts have been made to prevent the escalation of suicidal thoughts among training physicians.
To assess the effectiveness of a web-based cognitive behavioral therapy (wCBT) program delivered prior to the start of internship year in the prevention of suicidal ideation in medical interns.
Design, Setting, and Participants
A randomized clinical trial conducted at 2 university hospitals with 199 interns from multiple specialties during academic years 2009-2010 or 2011-2012. The current study was conducted from May 2009 to June 2010 and May 2011 to June 2012, and data were analyzed using intent-to-treat principles, including last observation carried forward.
Interns were randomly assigned to 2 study groups (wCBT and attention-control group [ACG]), and completed study activities lasting 30 minutes each week for 4 weeks prior to starting internship year. Participants assigned to wCBT completed online CBT modules and those assigned to ACG received emails with general information about depression, suicidal thinking, and local mental health professionals.
Main Outcomes and Measures
The Patient Health Questionnaire–9 was used to assess suicidal ideation (ie, “thoughts that you would be better off dead or hurting yourself in some way”) prior to the start of intern year and at 3-month intervals throughout the year.
A total of 62.2% of interns (199 of 320) agreed to take part in the study; 100 were assigned to the wCBT group and 99 to the ACG. During at least 1 point over the course of internship year, 12% of interns (12 of 100) assigned to wCBT endorsed suicidal ideation compared with 21.2% of interns (21 of 99) assigned to ACG. After adjusting for covariates identified a priori that have previously shown to increase the risk for suicidal ideation, interns assigned to wCBT were less likely to endorse suicidal ideation during internship year (relative risk, 0.40; 95% CI, 0.17- 0.91; P = .03) compared with those assigned to ACG.
Conclusions and Relevance
This study demonstrates that a free, easily accessible, brief wCBT program is associated with reduced likelihood of suicidal ideation among medical interns. Prevention programs with these characteristics could be easily disseminated to medical training programs across the country.
anzctr.org.au Identifier: ACTRN12610000628044
Physicians are at high risk for suicide compared with the general population.1 A meta-analysis of physician suicide revealed that male physicians are 1.41 times more likely and female physicians are 2.27 times more likely to die by suicide compared with their counterparts in the general population.2 According to the American Foundation for Suicide Prevention, 300 to 400 physicians die by suicide each year, equating to approximately 1 physician dying by suicide every day.3
Physicians in training are at high risk for suicide and suicidal ideation.4 A review of prospective studies conducted during 1982-2002 identified high rates of suicidal ideation among physicians during their first postgraduate year, or internship year.5 These findings are consistent with several studies demonstrating elevated rates of suicidal ideation in medical trainees.6-9 In a prospective cohort study of 740 interns from 13 institutions and multiple specialties across the United States, our research group found that suicidal ideation increased 370% over the first 3 months of internship year.10 These findings were replicated in subsequent cohorts of interns.11 Suicidal ideation in this study was defined as endorsement of item 9 of the Patient Health Questionnaire–9 (PHQ-9) (ie, “Thoughts that you would be better off dead, or hurting yourself in some way” over the past 2 weeks). These findings are concerning given that a positive response to this item increases the cumulative risk for a suicide attempt or completion over the next year by 10- and 100-fold, respectively.12
Despite the increase in suicidal ideation and other mental health problems among interns,9-13 very few seek mental health treatment. Lack of time, preference to manage problems on their own, lack of convenient access to care, and concerns about confidentiality have been identified as significant barriers to mental health treatment among training physicians.5,14-16 To reduce rates of suicidal ideation among training physicians, interventions that overcome these obstacles are necessary.
One promising approach to surmount these obstacles is the use of web-based tools. Specifically, web-based tools have important benefits over in-person treatment that correspond to the identified barriers to mental health care for physicians including (1) enhanced confidentiality; (2) low cost; (3) easy accessibility; (4) flexibility in timing of access; and (5) self-management tools.5,14-16 To date, 5 randomized clinical trials have demonstrated a reduction in suicidal thoughts among adult individuals in the general population and primary care settings using web-based cognitive behavioral therapy programs.17 Previous work has also demonstrated the efficacy of web-based preventive interventions for populations at high risk for mental health problems18-20; however, this preventive approach has not yet been tested in training physicians.
This study aimed to determine the feasibility, acceptability, and effectiveness of a web-based cognitive behavioral therapy (wCBT) program for the reduction of suicidal thoughts among medical interns.
A total of 352 interns entering traditional and primary care internal medicine, general surgery, pediatrics, obstetrics/gynecology, emergency medicine, and combined medicine and pediatrics and psychiatry residency programs at 2 university hospitals in the United States (Yale University and University of Southern California) during the 2009-2010 and 2011-2012 academic years were sent an email 3 months prior to commencing internship inviting them to participate in the study. Email invitations were returned as undeliverable for 9% (32 of 352) of potential participants and 62.2% (199 of 320) of the remaining invited interns agreed to participate in the study (Figure 1). The full study protocol can be found in the Supplement.
Participants were eligible for the study if they were beginning internship in July 2009 or July 2011 at 1 of 2 participating university hospitals. Data were collected through a secure online website designed to maintain confidentiality, with participants identified only by a nondecodable identification number. All participants were given information about symptoms of depression and encouraged to seek treatment locally if necessary. The interns received $100 ($60 prior to the start of intern year and $40 during intern year) in online gift certificates for study participation. Study approval and a waiver of written or oral consent were granted by the institutional review boards at Yale University and the University of Southern California. Patient consent was waived because the study was otherwise not feasible and it was determined that study participation presented no more than minimal risk.
Participants completed a secure web-based baseline survey 3 months prior to commencing internship year including (1) demographic characteristics (sex, age, and race/ethnicity); (2) medical specialty; (3) current depressive symptoms measured via the PHQ-921; (4) neuroticism22; and (5) early family environment.23
Suicidal ideation was assessed 3 months prior to the start of intern year and at 3, 6, 9, and 12 months of intern year. Suicidal ideation was measured using item 9 of the PHQ-9.21 Endorsement of suicidal ideation was positive if participants reported having “thoughts that you would be better off dead, or hurting yourself” for “several days,” “more than half the days,” or “nearly every day” over the past 2 weeks (ie, a score of ≥1 on the PHQ-9).
Two months prior to starting internship year, 199 eligible participants were randomly assigned to wCBT or the attention-control group (ACG). Participant email addresses were randomly assigned to conditions using complete randomization with equal allocation24 by a person independent of the research. Randomization successfully balanced key variables between conditions (Table).
Following randomization, participants assigned to the ACG received an email once a week for 4 weeks containing information about mental illness including symptoms of depression, suicide, and where to obtain local mental health treatment. Participants assigned to the intervention were directed via email each week for 4 weeks to the intervention website (http://moodgym.anu.edu.au) to complete a wCBT module. Participants gained access to the secure website via a user name provided within the email. Once participants accessed the website, they created a unique password known only by them, allowing the content provided within the modules (eg, individualized CBT exercises) to remain anonymous but also allowing website developers to track module completion for verification of participation based on assigned user name. At months 2, 5, 8, and 11 of internship year, the wCBT group was asked via email to return to the website to review a module of their choice, while the ACG received an email containing information about the symptoms of depression, suicide, and where to obtain local mental health treatment. Subsequent quarterly assessments for depression and suicidal ideation were completed at least 6 weeks after any wCBT- or ACG-related activities or email contact (Figure 2).
The wCBT program, MoodGYM, was developed by staff at the National Institute for Mental Health Research at the Australian National University. The program consisted of 4 weekly web-based sessions lasting approximately 30 minutes each. The interactive program aims to facilitate an understanding of the interplay between thoughts, emotions, and behaviors (module 1) and teaches cognitive restructuring techniques that promote the ability to identify and challenge inaccurate, unrealistic, or overly negative thoughts (modules 2 and 3). The program also includes problem-solving strategies (module 4).
The control participants received 4 weekly emails. All emails included information about the prevalence of depression and suicide among physicians, as well as described symptoms of depression and suicide and encouraged participants to seek treatment locally, if necessary. Contact information for local, confidential, and free mental health services was included in each email. Additional information about anxiety, substance abuse, and other mood disorders were included in the second, third, and fourth emails, respectively.
Power analysis of suicidal ideation was conducted using Power and Precision statistical software (Biostat). The target sample size (N = 200) was designed to have 90% power to detect a difference between groups of a 50% reduction in cases of suicidal ideation at α = .05.
Data were analyzed using intent-to-treat principles including last observation carried forward. Analyses were performed using a generalized estimating equation to account for the correlated repeated measures within participants. All analyses were performed using SAS version 9.3 (SAS Institute). We assessed the adequacy of randomization by comparing the wCBT and ACG on demographic and baseline clinical characteristics, as well as psychological factors previously shown to predict suicidal ideation in medical interns10,11 using χ2 tests for categorical variables and independent-sample t tests for continuous variables.
The point prevalence of suicidal ideation during internship year was determined through analysis of item 9 of the PHQ-9 at the 3-, 6-, 9-, and 12-month assessments. Endorsement of suicidal ideation was positive if participants reported having “thoughts that you would be better off dead, or hurting yourself” for “several days,” “more than half the days,” or “nearly every day” over the past 2 weeks (ie, a score of ≥1 on the PHQ-9 during at least 1 follow-up assessment). Treatment assignment, institution, class year, and their interactions, as well as variables identified a priori that have been shown to increase the risk for suicidal ideation (ie, sex, history of depression, preinternship PHQ-9 scores, neuroticism, and early family environment),10,11 were entered into a stepwise logistic regression model to identify significant predictors while accounting for collinearity among variables. Intervention assignment was included in the model to determine the impact of the intervention on suicidal ideation during internship year. Covariates included in the final model were identified a priori based on prior work. No post hoc testing of covariates were performed. The dependent variable was not used in the construction of the independent variable.
Participants’ mean (SD) age was 25.2 (8.1) years. Overall, 49.3% of participants were female and 43.0% were self-identified members of an ethnic/racial minority group. Participants did not differ significantly by study intervention condition on demographic, clinical characteristics (eg, suicidal ideation and use of psychotropic medication), or previously identified baseline variables shown to predict suicidal ideation in medical interns10,11 (Table). A total of 83% of all participants (165 of 199) completed at least 1 within-internship follow-up assessment. There were no significant differences in demographic characteristics (age, sex, and race/ethnicity), specialty, university, clinical characteristics (history of depression and preinternship current depressive symptoms), or other previously identified variables shown to be associated with depression or suicidal ideation in medical interns10,11 between retained participants and those who did not complete within-internship follow-up assessments. Over the course of the internship year, there were no significant differences between participants assigned to wCBT or the ACG in seeking mental health treatment (7% vs 8%, respectively) or starting a psychotropic medication (4% vs 7%, respectively).
A total of 88% of participants (88 of 100) assigned to the intervention completed at least 1 wCBT module. A total of 78% (78 of 100), 65% (65 of 100), and 51% (51 of 100) of participants completed 2, 3, or all 4 CBT modules, respectively. Completers of the 4 CBT modules did not differ from participants who completed fewer than 4 modules in age, sex, specialty, race/ethnicity, university, or previous variables shown to be associated with depression and suicidal ideation.10,11 Over the course of internship year, 82% (82 of 100) of participants reviewed at least 1 wCBT module.
Over the course of internship year, 12% of interns (12 of 100) in the wCBT group endorsed suicidal ideation during at least 1 follow-up assessment compared with 21.2% of interns (21 of 99) in the ACG. Treatment assignment, institution, class year, and their interactions did not associate with endorsement of suicidal ideation during internship year. After accounting for a priori baseline factors associated with the development of suicidal ideation and depression (sex, preinternship PHQ-9 scores, history of depression, neuroticism, and early family environment), interns assigned to the wCBT group were 60% less likely to endorse suicidal ideation during internship year compared with those assigned to the ACG (relative risk, 0.40; 95% CI, 0.17- 0.91; P = .03). The effect size of wCBT was 1.97. The number needed to treat was 11 (Figure 3). Analyses stratified by sex did not reveal significant sex effects.
To our knowledge, this is the first study to evaluate the feasibility, acceptability, and efficacy of an intervention that targets the high rate of suicidal ideation among medical interns. First, this study demonstrated the acceptability of a web-based preventive intervention for medical trainees. While only 10% to 20% of interns use traditional mental health resources,5,14 62% agreed to participate in the current intervention, with 88% completing at least 1 wCBT module and 82% returning for at least 1 wCBT review during internship year. It is possible that prevention programs for healthy interns are less stigmatizing than treatment programs for mental health problems25 and, as a result, achieve higher participation rates. This acceptability should encourage the development and implementation of other preventive interventions for physicians that accommodate the desire for confidentiality, time flexibility, and autonomy present in web-based programs.
Further, this 2-site randomized prevention trial demonstrated that a wCBT program was significantly associated with reducing the development of suicidal ideation during internship year. Interns assigned to the wCBT group were 60% less likely to endorse suicidal ideation during internship year compared with those assigned to the ACG. The suicide risk reduction in our study translated into a number needed to treat of 11; that is, for every 11 interns taking part in the intervention, we would expect to prevent 1 intern from developing suicidal ideation. Importantly, the effect of the intervention was sustained over an entire year. Equipping medical interns early on in their careers with evidence-based strategies to better manage their mental health can potentially have long-lasting effects on their mental health and the health of their patients.
Our findings are consistent with previous research examining the effect of wCBT on the reduction of suicidal ideation.17 Studies using wCBT have found statistically significant reductions in suicidal ideation.26-30 Our findings differ from prior studies examining the effect of a web-based depression intervention on suicidal ideation in that prior studies included populations with current suicidal ideation. Our findings extend the current literature and demonstrate that wCBT appears to prevent the onset of suicidal ideation over the course of an entire year, as opposed to decreasing suicidal ideation after it occurs. These findings are exciting given the high likelihood that interns will experience suicidal ideation and low likelihood that they will seek mental health treatment.
There are a number of aspects of this prevention program that increase the likelihood of successful dissemination and implementation. First, the program is free to the public and available to anyone with access to the internet. Second, all interactions with the participants (eg, invitation to take part in the program and reminders about completing the program) occurred via an automated email system and required very little personnel time and attention. Third, the program allowed trainees to complete much of the program before internship year, when their time is more abundant. Last, interns are willing to take part in this sort of intervention. Pragmatic programs with successful participant adherence are likely to be disseminated and easily implemented in real-world settings.
Interpretation of these data are limited by several considerations. First, we assessed suicidal ideation through a self-report inventory rather than a diagnostic interview. We chose this method, as opposed to an in-person assessment, based on previous data demonstrating that anonymity is necessary to accurately ascertain mental health problems among medical students.31 Nonetheless, it would be important to validate these findings using structured clinical interviews. Second, this study assessed suicidal ideation and not suicide or suicidal behaviors. Therefore, it is unknown whether the intervention has any impact on physician suicide. The ability to detect the intervention’s effect on suicide would require a much larger sample size with a longer follow-up period. Third, the actual numbers of interns endorsing suicidal ideation during at least 1 follow-up assessment over the course of internship year in the wCBT group and ACG is small, at 12% vs 21%, respectively. Thus, replication of these findings is important. Last, this study was conducted at 2 universities and findings may not generalize to other academic institutions or community hospitals.
Approximately half of interns assigned to wCBT completed all 4 of the assigned CBT modules. Future studies aimed at better understanding which modules are most effective at reducing suicidal ideation would be of great benefit. Further research is needed to better understand the mechanism by which wCBT reduces suicidal ideation among interns and to determine whether the positive benefits of wCBT on suicidal ideation are sustained over time.
Taken together, our study supports the feasibility, acceptability, and efficacy of a wCBT program for the prevention of suicidal ideation among medical interns during internship year. With approximately 24 000 medical trainees beginning internship each year,32 dissemination of a pragmatic, no-cost, feasible, and efficacious prevention program could have substantial public health benefits.1,33,34 This work further supports web-based interventions as promising tools to enhance physician mental health and decrease their high risk for suicide.
Corresponding Author: Constance Guille, MD, Department of Psychiatry and Behavioral Science, Medical University of South Carolina, 67 President St, 5 South, MSC861, Charleston, SC 29425 (firstname.lastname@example.org).
Submitted for Publication: May 13, 2015; final revision received August 11, 2015; accepted August 14, 2015.
Published Online: November 4, 2015. doi:10.1001/jamapsychiatry.2015.1880.
Author Contributions: Drs Guille and Mrs Zhao had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Guille, Krystal, Sen.
Acquisition, analysis, or interpretation of data: Guille, Zhao, Nichols, Brady, Sen.
Drafting of the manuscript: Guille, Nichols.
Critical revision of the manuscript for important intellectual content: Guille, Zhao, Krystal, Brady, Sen.
Statistical analysis: Guille, Zhao, Sen.
Obtained funding: Guille, Sen.
Administrative, technical, or material support: Guille, Zhao, Krystal, Nichols.
Study supervision: Brady, Sen.
Conflict of Interest Disclosures: None reported.
Funding/Support: This project was supported by grant K12HD055885 from the National Institute of Child Health and Human Development and the Office of Research on Women’s Health, as well as grant 1K23DA039318-01 from the National Institutes of Health, National Institute on Drug Abuse. Federal support was also provided by grant R01 MH101459 from the National Institutes of Health, National Institute of Mental Health. Foundation funding was provided by the American Foundation for Suicide Prevention grant 2YIG-00054-1207.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The content of this work is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, National Institute of Child Health and Human Development, Office of Research on Women’s Health, National Institute on Drug Abuse, National Institute of Mental Health, and American Foundation for Suicide Prevention.
Additional Contributions: We acknowledge and thank the interns who took part in this study. Without their participation, this study would not have been possible. We also thank and acknowledge Helen Christensen, PhD (professor at the Black Dog Institute, University of New South Wales, Sydney, Australia), for providing access to MoodGYM to allow the conduct of this study. We also acknowledge the assistance of Kylie Bennett, BSc, e-hub manager, and Anthony Bennett, BAppSc, software engineer, at the Centre for Mental Health Research, for creating the software to deploy this version of MoodGYM and tracking user completion. Dr Christensen, Ms Bennett, and Mr Bennett received no financial support and were not involved in the design, analysis, or interpretation of the study results.
Create a personal account or sign in to: