[Skip to Content]
[Skip to Content Landing]
Figure 1.  Suicidal Behavior in 6-Month Follow-up for Safety Planning Intervention With Structured Follow-up Telephone Contact (SPI+) and Usual Care
Suicidal Behavior in 6-Month Follow-up for Safety Planning Intervention With Structured Follow-up Telephone Contact (SPI+) and Usual Care

Proportion of patients with suicidal behavior in the 6 months following emergency department discharge in SPI+ compared with usual care patients. Error bars denote the standard error of the proportion.

Figure 2.  Treatment Engagement in 6-Month Follow-up for Safety Planning Intervention With Structured Follow-up Telephone Contact (SPI+) and Usual Care
Treatment Engagement in 6-Month Follow-up for Safety Planning Intervention With Structured Follow-up Telephone Contact (SPI+) and Usual Care

Proportion of patients with at least 1 outpatient behavioral health appointment in the 6 months following emergency department discharge in SPI+ compared with usual care patients. Error bars denote the standard error of the proportion.

Table.  Demographic and Clinical Characteristics of SPI+ and Usual Care Groups
Demographic and Clinical Characteristics of SPI+ and Usual Care Groups
1.
The Centers for Disease Control and Prevention. Leading causes of death reports, national and regional, 1999-2015: web-based injury statistics query and reporting system. https://webappa.cdc.gov/sasweb/ncipc/leadcaus10_us.html. Published 2017. Accessed February 26, 2017.
2.
Office of Mental Health and Suicide Prevention. Suicide among veterans and other Americans 2001-2014. https://www.mentalhealth.va.gov/docs/2016suicidedatareport.pdf. Published 2016. Accessed December 5, 2017.
3.
Claassen  CA, Larkin  GL.  Occult suicidality in an emergency department population.  Br J Psychiatry. 2005;186(4):352-353. doi:10.1192/bjp.186.4.352PubMedGoogle ScholarCrossref
4.
Fields  WW, Asplin  BR, Larkin  GL,  et al.  The Emergency Medical Treatment and Labor Act as a federal health care safety net program.  Acad Emerg Med. 2001;8(11):1064-1069. doi:10.1111/j.1553-2712.2001.tb01116.xPubMedGoogle ScholarCrossref
5.
Larkin  GL, Beautrais  AL.  Emergency departments are underutilized sites for suicide prevention.  Crisis. 2010;31(1):1-6. doi:10.1027/0227-5910/a000001PubMedGoogle ScholarCrossref
6.
Owens  P, Mutter  R, Stocks  C. Mental health and substance abuse-related emergency department visits among adults, 2007: statistical brief #92: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs 2010. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb92.pdf. Accessed February 20, 2017.
7.
Ting  SA, Sullivan  AF, Boudreaux  ED, Miller  I, Camargo  CA  Jr.  Trends in US emergency department visits for attempted suicide and self-inflicted injury, 1993-2008.  Gen Hosp Psychiatry. 2012;34(5):557-565. doi:10.1016/j.genhosppsych.2012.03.020PubMedGoogle ScholarCrossref
8.
Olfson  M, Marcus  SC, Bridge  JA.  Focusing suicide prevention on periods of high risk.  JAMA. 2014;311(11):1107-1108. doi:10.1001/jama.2014.501PubMedGoogle ScholarCrossref
9.
Ahmedani  BK, Simon  GE, Stewart  C,  et al.  Health care contacts in the year before suicide death.  J Gen Intern Med. 2014;29(6):870-877. doi:10.1007/s11606-014-2767-3PubMedGoogle ScholarCrossref
10.
Da Cruz  D, Pearson  A, Saini  P,  et al.  Emergency department contact prior to suicide in mental health patients.  Emerg Med J. 2011;28(6):467-471. doi:10.1136/emj.2009.081869PubMedGoogle ScholarCrossref
11.
Gairin  I, House  A, Owens  D.  Attendance at the accident and emergency department in the year before suicide: retrospective study.  Br J Psychiatry. 2003;183:28-33. doi:10.1192/bjp.183.1.28PubMedGoogle ScholarCrossref
12.
Kurz  A, Möller  HJ.  Help-seeking behavior and compliance of suicidal patients.  Psychiatr Prax. 1984;11(1):6-13.PubMedGoogle Scholar
13.
Piacentini  J, Rotheram-Borus  MJ, Gillis  JR,  et al.  Demographic predictors of treatment attendance among adolescent suicide attempters.  J Consult Clin Psychol. 1995;63(3):469-473. doi:10.1037/0022-006X.63.3.469PubMedGoogle ScholarCrossref
14.
Lizardi  D, Stanley  B.  Treatment engagement: a neglected aspect in the psychiatric care of suicidal patients.  Psychiatr Serv. 2010;61(12):1183-1191. doi:10.1176/ps.2010.61.12.1183PubMedGoogle ScholarCrossref
15.
Monti  K, Cedereke  M, Öjehagen  A.  Treatment attendance and suicidal behavior 1 month and 3 months after a suicide attempt: a comparison between two samples.  Arch Suicide Res. 2003;7(2):167-174. doi:10.1080/13811110301581Google ScholarCrossref
16.
Kelly  KT, Knudson  MP.  Are no-suicide contracts effective in preventing suicide in suicidal patients seen by primary care physicians?  Arch Fam Med. 2000;9(10):1119-1121. doi:10.1001/archfami.9.10.1119PubMedGoogle ScholarCrossref
17.
Kroll  J.  Use of no-suicide contracts by psychiatrists in Minnesota.  Am J Psychiatry. 2000;157(10):1684-1686. doi:10.1176/appi.ajp.157.10.1684PubMedGoogle ScholarCrossref
18.
Simon  RI.  The suicide prevention contract: clinical, legal, and risk management issues.  J Am Acad Psychiatry Law. 1999;27(3):445-450.PubMedGoogle Scholar
19.
Reid  WH.  Promises, promises: don’t rely on patients’ no-suicide/no-violence “contracts”.  J Pract Psychiatry Behav Health. 1998;4:316-318.Google Scholar
20.
Shaffer  D, Pfeffer  CR; American Academy of Child and Adolescent Psychiatry.  Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior.  J Am Acad Child Adolesc Psychiatry. 2001;40(7)(suppl):24S-51S. doi:10.1097/00004583-200107001-00003PubMedGoogle ScholarCrossref
21.
Stanford  EJ, Goetz  RR, Bloom  JD.  The No Harm Contract in the emergency assessment of suicidal risk.  J Clin Psychiatry. 1994;55(8):344-348.PubMedGoogle Scholar
22.
The Joint Commission. Sentinel Event Alert 56: Detecting and treating suicide ideation in all settings 2016; https://www.jointcommission.org/assets/1/18/SEA_56_Suicide.pdf. Accessed March 11, 2017.
23.
Rudd  MD, Mandrusiak  M, Joiner  TE  Jr.  The case against no-suicide contracts: the commitment to treatment statement as a practice alternative.  J Clin Psychol. 2006;62(2):243-251. doi:10.1002/jclp.20227PubMedGoogle ScholarCrossref
24.
Stanley  B, Brown  GK.  Safety planning intervention: a brief intervention to mitigate suicide risk.  Cognit Behav Pract. 2012;19(2):256-264. doi:10.1016/j.cbpra.2011.01.001Google ScholarCrossref
25.
Stanley  B, Chaudhury  SR, Chesin  M,  et al.  An emergency department intervention and follow-up to reduce suicide risk in the VA: acceptability and effectiveness.  Psychiatr Serv. 2016;67(6):680-683. doi:10.1176/appi.ps.201500082PubMedGoogle ScholarCrossref
26.
Chesin  MS, Stanley  B, Haigh  EA,  et al.  Staff views of an emergency department intervention using safety planning and structured follow-up with suicidal veterans.  Arch Suicide Res. 2017;21(1):127-137. doi:10.1080/13811118.2016.1164642PubMedGoogle ScholarCrossref
27.
Bryan  CJ, Mintz  J, Clemans  TA,  et al.  Effect of crisis response planning vs. contracts for safety on suicide risk in U.S. Army soldiers: a randomized clinical trial.  J Affect Disord. 2017;212:64-72. doi:10.1016/j.jad.2017.01.028PubMedGoogle ScholarCrossref
28.
Brown  GK, Green  KL.  A review of evidence-based follow-up care for suicide prevention: where do we go from here?  Am J Prev Med. 2014;47(3)(suppl 2):S209-S215. doi:10.1016/j.amepre.2014.06.006PubMedGoogle ScholarCrossref
29.
Fleischmann  A, Bertolote  JM, Wasserman  D,  et al.  Effectiveness of brief intervention and contact for suicide attempters: a randomized controlled trial in five countries.  Bull World Health Organ. 2008;86(9):703-709. doi:10.2471/BLT.07.046995PubMedGoogle ScholarCrossref
30.
Miller  IW, Camargo  CA  Jr, Arias  SA,  et al; ED-SAFE Investigators.  Suicide prevention in an emergency department population: the ED-SAFE study.  JAMA Psychiatry. 2017;74(6):563-570. doi:10.1001/jamapsychiatry.2017.0678PubMedGoogle ScholarCrossref
31.
Spirito  A, Boergers  J, Donaldson  D, Bishop  D, Lewander  W.  An intervention trial to improve adherence to community treatment by adolescents after a suicide attempt.  J Am Acad Child Adolesc Psychiatry. 2002;41(4):435-442. doi:10.1097/00004583-200204000-00016PubMedGoogle ScholarCrossref
32.
Vaiva  G, Vaiva  G, Ducrocq  F,  et al.  Effect of telephone contact on further suicide attempts in patients discharged from an emergency department: randomised controlled study.  BMJ. 2006;332(7552):1241-1245. doi:10.1136/bmj.332.7552.1241PubMedGoogle ScholarCrossref
33.
Carter  GL, Clover  K, Whyte  IM, Dawson  AH, D’Este  C.  Postcards from the EDge project: randomised controlled trial of an intervention using postcards to reduce repetition of hospital treated deliberate self poisoning.  BMJ. 2005;331(7520):805-807. doi:10.1136/bmj.38579.455266.E0PubMedGoogle ScholarCrossref
34.
Morgan  HG, Jones  EM, Owen  JH.  Secondary prevention of non-fatal deliberate self-harm: the green card study.  Br J Psychiatry. 1993;163:111-112. doi:10.1192/bjp.163.1.111PubMedGoogle ScholarCrossref
35.
Motto  JA, Bostrom  AG.  A randomized controlled trial of postcrisis suicide prevention.  Psychiatr Serv. 2001;52(6):828-833. doi:10.1176/appi.ps.52.6.828PubMedGoogle ScholarCrossref
36.
Beautrais  AL, Gibb  SJ, Faulkner  A, Fergusson  DM, Mulder  RT.  Postcard intervention for repeat self-harm: randomised controlled trial.  Br J Psychiatry. 2010;197(1):55-60. doi:10.1192/bjp.bp.109.075754PubMedGoogle ScholarCrossref
37.
Currier  GW, Brown  GK, Brenner  LA,  et al.  Rationale and study protocol for a two-part intervention: safety planning and structured follow-up among veterans at risk for suicide and discharged from the emergency department.  Contemp Clin Trials. 2015;43:179-184. doi:10.1016/j.cct.2015.05.003PubMedGoogle ScholarCrossref
38.
Stanley  B, Brown  GK.  Safety Planning Intervention: Brief Instructions. Washington, DC: US Department of Veterans Affairs; 2008.
39.
Stanley  B, Brown  GK, Currier  GW, Lyons  C, Chesin  M, Knox  KL.  Brief intervention and follow-up for suicidal patients with repeat emergency department visits enhances treatment engagement.  Am J Public Health. 2015;105(8):1570-1572. doi:10.2105/AJPH.2015.302656PubMedGoogle ScholarCrossref
40.
Hoffmire  C, Stephens  B, Morley  S, Thompson  C, Kemp  J, Bossarte  RM.  VA suicide prevention applications network: a national health care system-based suicide event tracking system.  Public Health Rep. 2016;131(6):816-821. doi:10.1177/0033354916670133PubMedGoogle ScholarCrossref
41.
Bates  D, Maechler  M, Bolker  B, Walker  S.  Fitting linear mixed-effects models using lme4.  J Stat Softw. 2015;67(1):1-48. doi:10.18637/jss.v067.i01Google ScholarCrossref
42.
Bryan  CJ, Mintz  J, Clemans  TA,  et al.  Effect of crisis response planning on patient mood and clinician decision making: a clinical trial with suicidal U.S. soldiers.  Psychiatr Serv. 2018;69(1):108-111. doi:10.1176/appi.ps.201700157PubMedGoogle ScholarCrossref
43.
Brown  GK, Ten Have  T, Henriques  GR, Xie  SX, Hollander  JE, Beck  AT.  Cognitive therapy for the prevention of suicide attempts: a randomized controlled trial.  JAMA. 2005;294(5):563-570. doi:10.1001/jama.294.5.563PubMedGoogle ScholarCrossref
44.
Linehan  MM, Armstrong  HE, Suarez  A, Allmon  D, Heard  HL.  Cognitive-behavioral treatment of chronically parasuicidal borderline patients.  Arch Gen Psychiatry. 1991;48(12):1060-1064. doi:10.1001/archpsyc.1991.01810360024003PubMedGoogle ScholarCrossref
Original Investigation
September 2018

Comparison of the Safety Planning Intervention With Follow-up vs Usual Care of Suicidal Patients Treated in the Emergency Department

Author Affiliations
  • 1Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, New York
  • 2New York State Psychiatric Institute, New York
  • 3Department of Psychiatry, Perelman School of Medicine of the University of Pennsylvania, Philadelphia
  • 4Department of Veterans Affairs, Rocky Mountain Mental Illness, Research, Education and Clinical Care, Denver, Colorado
  • 5University of Colorado, Anschutz Medical Campus, Aurora
  • 6Department of Psychiatry, University of South Florida, Tampa
  • 7Department of Psychiatry, University of Rochester, Rochester, New York
  • 8Department of Veterans Affairs, Veterans Integrated Services Networks 2 Center of Excellence for Suicide Prevention, Rochester, New York
JAMA Psychiatry. 2018;75(9):894-900. doi:10.1001/jamapsychiatry.2018.1776
Key Points

Question  Can a brief suicide prevention intervention reduce suicidal behaviors and improve treatment engagement among patients who present to the emergency department for suicide-related concerns?

Findings  In this cohort comparison study, patients who visited the emergency department for suicide-related concerns and received the Safety Planning Intervention with structured follow-up telephone contact were half as likely to exhibit suicidal behavior and more than twice as likely to attend mental health treatment during the 6-month follow-up period compared with their counterparts who received usual care following their ED visit.

Meaning  The Safety Planning Intervention with structured follow-up telephone contact may be an effective brief suicide prevention intervention that can be implemented in emergency departments.

Abstract

Importance  Suicidal behavior is a major public health problem in the United States. The suicide rate has steadily increased over the past 2 decades; middle-aged men and military veterans are at particularly high risk. There is a dearth of empirically supported brief intervention strategies to address this problem in health care settings generally and particularly in emergency departments (EDs), where many suicidal patients present for care.

Objective  To determine whether the Safety Planning Intervention (SPI), administered in EDs with follow-up contact for suicidal patients, was associated with reduced suicidal behavior and improved outpatient treatment engagement in the 6 months following discharge, an established high-risk period.

Design, Setting, and Participants  Cohort comparison design with 6-month follow-up at 9 EDs (5 intervention sites and 4 control sites) in Veterans Health Administration hospital EDs. Patients were eligible for the study if they were 18 years or older, had an ED visit for a suicide-related concern, had inpatient hospitalization not clinically indicated, and were able to read English. Data were collected between 2010 and 2015; data were analyzed between 2016 and 2018.

Interventions  The intervention combines SPI and telephone follow-up. The SPI was defined as a brief clinical intervention that combined evidence-based strategies to reduce suicidal behavior through a prioritized list of coping skills and strategies. In telephone follow-up, patients were contacted at least 2 times to monitor suicide risk, review and revise the SPI, and support treatment engagement.

Main Outcomes and Measures  Suicidal behavior and behavioral health outpatient services extracted from medical records for 6 months following ED discharge.

Results  Of the 1640 total patients, 1186 were in the intervention group and 454 were in the comparison group. Patients in the intervention group had a mean (SD) age of 47.15 (14.89) years and 88.5% were men (n = 1050); patients in the comparison group had a mean (SD) age of 49.38 (14.47) years and 88.1% were men (n = 400). Patients in the SPI+ condition were less likely to engage in suicidal behavior (n = 36 of 1186; 3.03%) than those receiving usual care (n = 24 of 454; 5.29%) during the 6-month follow-up period. The SPI+ was associated with 45% fewer suicidal behaviors, approximately halving the odds of suicidal behavior over 6 months (odds ratio, 0.56; 95% CI, 0.33-0.95, P = .03). Intervention patients had more than double the odds of attending at least 1 outpatient mental health visit (odds ratio, 2.06; 95% CI, 1.57-2.71; P < .001).

Conclusions and Relevance  This large-scale cohort comparison study found that SPI+ was associated with a reduction in suicidal behavior and increased treatment engagement among suicidal patients following ED discharge and may be a valuable clinical tool in health care settings.

Introduction

Quiz Ref IDThe suicide rate in the United States increased dramatically over the past 2 decades, with more than 44 000 suicides in 2016.1 The rate among veterans is 21% higher than nonveterans.2 Development of effective treatments has not kept pace. Often the only treatment patients receive during a suicidal crisis is in hospital emergency departments (EDs),3-5 in which care usually consists of risk assessment and triage to inpatient or outpatient treatment. More than 4% of ED visits are attributable to psychiatric conditions,6 with approximately 420 000 visits annually for intentional self-harm.7 Patients are at increased risk for suicide attempts and suicide following an ED visit.8-11 Unfortunately, nearly half of suicidal ED patients do not attend treatment12,13 or discontinue quickly.14 Poor engagement is troubling because risk for subsequent suicidal behavior is greatest during the 6 months following a suicidal crisis.15 Given this picture, the ED visit is an important opportunity for brief, targeted interventions to prevent further suicidal crises.

Up to 10 years ago, no-suicide contracts, in which patients promised to not engage in suicidal behavior,16-18 were frequently used. However, these contracts are ineffective.16,19-21 The Joint Commission22 recommended development of a collaborative safety plan as an alternative to no-suicide contracts.23,24 Safety planning has been found to be acceptable to and feasible by both suicidal patients and staff. Patients report it helps maintain their safety.25,26 In a randomized clinical trial, crisis response planning, a form of safety planning, resulted in fewer suicide attempts, lower suicidal ideation, and greater treatment engagement than no-suicide contracts during a 6-month follow-up with active duty military members.27

Postdischarge follow-up interventions, particularly those that involve in-person and telephone contact, are effective.28-32 One ED study found a 30% lower suicide attempt rate with an intervention consisting of screening, providing basic written material on safety planning, and several calls to suicidal patients and their significant others.30 Some33-35 but not all36 studies have reported similar findings with mail interventions.

Given the potential of safety planning and follow-up contact in reducing suicidal behavior, we evaluated a combination intervention, SPI+, which consisted of (1) the Safety Planning Intervention (SPI)24 administered in the ED and (2) structured follow-up37 following ED discharge to prevent suicidal behavior and enhance treatment engagement. The SPI is a brief clinical intervention24,38 widely used in health systems including the Veterans Health Administration.38 The SPI combines evidence-based strategies to reduce suicidal behavior by providing prioritized coping strategies including lethal means counseling to reduce access to potential suicide methods. Coping strategies were prioritized, ranging from strategies that can be done alone to those involving social contacts (ie, family and friends), followed by outreach to professionals and the ED. The follow-up component (SPI+) consisted of at least 2 brief telephone calls following ED discharge to assess risk, review and revise the safety plan, and support treatment engagement. Calls continued on a weekly basis until the patient began treatment or withdrew. In a pilot study of repeated ED patients, SPI+ was associated with improved treatment attendance at 3-month follow-up compared with a prior ED visit for suicidal behavior.39 We also found that, among suicidal patients provided with a safety plan, nearly two-thirds reported using the plan to mitigate risk.25 The purpose of this cohort comparison study was to evaluate the association of SPI+ compared with usual care with suicidal behavior and treatment engagement in the 6 months following discharge from the ED. We hypothesized that SPI+ would be associated with fewer suicidal behavior reports and that SPI+ would be associated with enhanced treatment engagement in the 6 months following ED discharge.

Methods
Procedure

Quiz Ref IDA total of 1640 patients (1186 in the intervention group and 454 in the comparison group) meeting eligibility criteria from 5 Veterans Affairs (VA) ED intervention sites where SPI+ was implemented as standard care and 4 VA ED comparison usual care sites were included in the analyses. Patients were not prospectively assigned to intervention or comparison groups. Instead, in the intervention site EDs, all eligible patients were provided the SPI+ as standard care. Matched sites that did not have the intervention were identified later to obtain electronic health record data during the same time as a comparison. Sites were matched on geographic location, approximate number of psychiatric ED evaluations per year, and presence of a psychiatric inpatient unit. Each ED treated a mean of approximately 10 000 patients per year; about 10% of these patients were admitted for mental health/substance abuse treatment.

Patients who came to the ED for a suicidal crisis and were determined to not require inpatient hospitalization were included. Usual care for suicidal patients not requiring inpatient hospitalization typically consisted of evaluation and outpatient referral. Emergency department patients who met eligibility criteria were included in the project. Inclusion criteria were (1) 18 years or older; (2) ED visit for a suicide-related concern; (3) inpatient hospitalization not clinically indicated; and (4) able to read and understand English. This project was reviewed and approved by each of the VA institutional review boards at the VA Medical Center of Denver, Colorado, the Manhattan VA Medical Center, the Portland Corporal Michael J. Crescenz VA Medical Center, and the VA Medical Center of Philadelphia, Pennsylvania. Because this was a clinical demonstration project, the institutional review boards determined that informed consent was not required. Patients were able to opt out of the SPI+ during the ED visit or during follow-up calls.

Intervention Condition

Quiz Ref IDIn addition to usual care, SPI+ included the original SPI, a brief, structured intervention24,38 and a best practice on the Suicide Prevention Resource Center/American Foundation for Suicide Prevention Best Practices Registry (http://www.sprc.org), designed to mitigate future risk by providing suicidal individuals with a written, personalized safety plan to be used in the event of a suicidal crisis. The SPI has 6 key steps: (1) identify personalized warning signs for an impending suicide crisis; (2) determine internal coping strategies that distract from suicidal thoughts and urges; (3) identify family and friends who are able to distract from suicidal thoughts and urges and social places that provide the opportunity for interaction; (4) identify individuals who can help provide support during a suicidal crisis; (5) list mental health professionals and urgent care services to contact during a suicidal crisis; and (6) lethal means counseling for making the environment safer (http://www.suicidesafetyplan.com).

The SPI+ adds a component that consists of telephone contact after discharge from the ED, usually done by project staff who were social workers or psychologists and trained and supervised by senior project staff. Contacts were attempted within 72 hours of discharge and included 3 components: (1) brief risk assessment and mood check; (2) review and revision of the SPI, if needed; and (3) facilitation of treatment engagement. Follow-up outreach continued weekly and generally discontinued after at least 2 calls if the patient had at least 1 outpatient behavioral health appointment or no longer wished to be contacted.

Usual Care Condition

Usual care varied somewhat between sites because it was not protocol driven; it generally consisted of an initial assessment by a nurse or social worker followed by a secondary evaluation by an ED physician. Medical care was provided if indicated and the patient was medically stabilized. Medications were initiated or adjusted as indicated. Patients discharged were typically provided with either a specific outpatient appointment or information about how to seek psychiatric care if they declined a referral at discharge. Usual care patients did not receive a safety plan during the ED visit, although some may have received one in the past if they were at risk for suicide. Patients in the usual care condition were identified retrospectively by clinical medical record review for the same time that SPI+ was implemented in the intervention EDs.

Assessments

Medical records were reviewed to obtain demographic information, diagnoses, health service use, and physician-rated Global Assessment of Functioning scores at the index ED visit. Suicide Behavior Reports for suicide behaviors in the postdischarge 6 months were also retrieved from the medical records at each hospital. These reports are mandated in the VA and include descriptions of all suicide attempts, suicide deaths, and other suicidal behaviors including interrupted attempts.2,40

Statistical Analyses

All statistical analyses were performed in the statistical language R, version 3.1.2 (2014-10-31, The R Foundation for Statistical Computing). All tests performed were 2-sided and used α = .05 for significance level cutoff. All eligible patients were included in the analyses. Comparisons of demographic and clinical characteristics between patients receiving SPI+ and usual care were conducted using t tests for quantitative measures and Pearson χ2 tests for categorical measures. To account for intrasite correlations in the outcome measures, mixed-effect logistic regression models were run using the “glmer” function41 in R, with intervention group as a fixed effect and with site-specific random intervention effects to account for differences in treatment outcome by site to test for differences in suicidal behavior reports between the intervention and comparison groups. This generalized linear mixed model uses the logit link and was estimated using maximum likelihood methods through the Laplace approximation. We performed 2 analyses using this model to examine whether treatment condition predicted suicidal behavior during the 6-month follow-up, 1 with only treatment condition as the independent variable and 1 where we controlled for patient suicidal behavior in the 6 months preintervention (not including suicidal behavior that was the reason for the ED visit). Because patients were not randomized, we next calculated propensity scores based on variables that differed by condition. Significant predictors of group membership (Table) with less than 5% missing data, namely age, homelessness status, service period Operation Enduring Freedom or Operation Iraqi Freedom vs other service periods (World War II, Vietnam-era, post-Vietnam, and Gulf War), indicator variables for a history of more than 5 mental health visits, bipolar, depression, posttraumatic stress disorder (PTSD), and substance abuse diagnoses and suicidal behavior 6 months preintervention, were entered into a binary logistic regression model with group as outcome, and the propensity score was calculated as the patient-level predicted values of likelihood of belonging to the intervention vs control group. We used propensity scores to perform covariate adjustment: first with a categorical propensity variable by breaking the propensity score into quartiles (ie, 4 strata) and then with a continuous covariate. We adjusted the mixed-effect logistic model with postintervention suicidal behavior as the outcome variable by the strata and strata by condition interaction, removing the interaction when found to be not significant. We repeated the analysis with the quantitative propensity score to test for linear effect. Owing to the relatively low event rate, we did not perform separate analyses by propensity score strata. In exploratory analyses, the effect of baseline diagnosis on treatment outcome was tested using separate mixed-effect logistic regression models as described here, with randomization group, baseline diagnosis, and their interaction as fixed predictors.

Results
Enrollment

A total of 1640 patients with a mean (SD) age of 48 (14) years, 88% men, were included in the analyses. Of the eligible patients at the intervention sites, 99.4% (n = 1179) agreed to receive the SPI. Fewer patients (89.6%; n = 1063) were able to be contacted for at least 1 follow-up call. Patients completed a mean (SD) of 3.7 (3.3) calls (median, 4; range, 0-26).

Demographic Characteristics

Patients in the SPI+ and usual care conditions did not differ on race/ethnicity or sex. However, there were some baseline differences between conditions. Specifically, those in SPI+ were significantly younger (t1625 = 2.716; P = .007), less educated (χ21 = 4.627; P = .03), and less likely to be homeless (χ21 = 8.453; P = .004) than those in usual care (Table). Those in the SPI+ group were more likely to have more than 5 mental health visits in the prior 3 years than those in the usual care (Table). Also, patients in SPI+ were also more likely to have served in the 2 most recent military operations (Operation Enduring Freedom or Operation Iraqi Freedom) vs other service periods (Table).

Clinical Characteristics

Patients in SPI+ did not differ significantly from those in usual care on Global Assessment of Functioning at the index ED visit. Patients in the SPI+ condition were more likely to have had suicidal behavior in the 6 months previous (χ21 = 8.31; P = .004). Patients in the SPI+ condition were significantly less likely to have had the following lifetime diagnoses: major depression, bipolar disorder, PTSD, and substance abuse (Table).

Suicidal Behavior Report Outcomes

Quiz Ref IDThe SPI+ was associated with 45% fewer suicidal behaviors in the 6-month period following the ED visit compared with usual care (Figure 1). Patients in the SPI+ condition were less likely to engage in suicidal behavior (n = 36 of 1186; 3.03%) than those receiving usual care (n = 24 of 454; 5.29%) during the 6-month follow-up period, yielding a number needed to treat of 44.43. Mixed-effect logistic regression analysis examining whether treatment condition was related to suicidal behavior found that when we included treatment condition as the independent variable and adjusted for random intervention effects by site, SPI+ had approximately half the odds of suicidal behavior reports during the study period (OR, 0.50; 95% CI, 0.25-0.99; P = .05). In a secondary analysis using the mixed-effect logistic model, when we controlled for whether the patient had a history of suicidal behavior in the 6 months preintervention, we found that treatment condition remained significantly associated with suicidal behavior 6 months following intervention (odds ratio [OR], 0.48; 95% CI, 0.24-0.93; P = .03). Preintervention suicidal behavior was found to be a significant factor in this model (OR, 2.93; 95% CI, 1.16-7.37; P = .02). In another secondary analysis, we used the propensity score to adjust for the significant differences between the groups. The propensity score for each individual in SPI+ vs comparison groups was calculated based on significant factors from the Table with less than 5% missing values, as described previously. Patients were stratified into 4 groups based on its quartiles. In a mixed-effect logistic regression with condition, propensity-based strata and their interaction as predictors, we found no evidence for differential treatment effect by strata (χ23 = 1.38; P = .71), and after removing the interaction, we found that treatment condition remained significant (OR, 0.47; 95% CI, 0.24-0.92; P = .03), while the propensity strata was also significant (χ23 = 17.88; P < .001). Adjusting for the propensity score as a linear covariate yielded a significant treatment effect (OR, 0.46; 95% CI, 0.23-0.91; P = .03) and a nonsignificant propensity score effect (OR, 1.08; 95% CI, 0.81-1.43; P = .60).

Baseline Diagnosis and Suicidal Behavior Outcomes

In exploratory analyses, we tested the association of baseline diagnosis with suicidal behavior outcomes using separate mixed-effect logistic regression models. Depression, PTSD, and substance use disorder had no moderating effect on the treatment (interaction with treatment: depression OR, 0.62; 95% CI, 0.21-1.85; P = .39; PTSD OR, 0.37; 95% CI, 0.11-1.28; P = .12; and substance abuse OR, 2.72; 95% CI, 0.87-8.51; P = .09) nor were they significantly associated with suicidal behavior during follow-up overall (depression: z = −0.71; P = .48; PTSD: z = −1.05; P = .29; substance abuse: z = 0.22; P = .83). However, bipolar disorder was a significant moderator of treatment effect (interaction OR, 7.05; 95% CI, 1.30-38.18; P = .02), whereby patients with bipolar disorder who received SPI+ did not differ in the risk of suicidal behavior from those in the usual care condition (SPI: OR, 2.1; 95% CI, 0.28-15.73; P = .47). For patients with all other diagnoses, those in the SPI+ intervention had fewer suicidal behaviors in the 6-month postintervention period (OR, 0.40; 95% CI, 0.22-0.75; P = .05).

Treatment Engagement Outcomes

Using mixed-effect logistic models, we tested whether the intervention was associated with the likelihood of attending mental health and/or substance abuse treatment (at least 1 visit in the 6 months following ED discharge). Patients in SPI+ had more than double the odds of attending mental health treatment (OR, 2.20; 95% CI, 1.35-2.20; P = .002) (Figure 2), and the difference stayed significant after adjusting for the propensity score strata (OR, 1.74; 95% CI, 1.08-2.81; P = .02). The likelihood of attending substance abuse treatment did not differ between the groups (OR, 1.23; 95% CI, 0.88-1.71; P = .22), even after adjusting for the propensity score strata (OR, 1.44; 95% CI, 0.90-2.28; P = .13).

Components of SPI+ and Suicidal Behavior Outcomes

When entering both treatment group and mental health treatment attendance as predictors of posttreatment suicidal behavior, the treatment group effect stayed significant, and the effect size did not decrease (OR, 0.46; 95% CI, 0.23-0.91; P = .03), indicating that the protective effect of the intervention was not mediated by the increased likelihood of attending mental health treatment. We also examined the association between the number of follow-up calls and suicidal behavior within the SPI+ group and found that the number of calls was not associated with whether the patient had at least 1 suicide event (Wilcoxon W = 23 711.5; P = .13), possibly pointing to a key role of the safety plan.

Discussion

To our knowledge, this is the first large-scale study to demonstrate the association of SPI+ with decreased suicidal behavior and increased behavioral health treatment engagement following ED discharge for suicide-related concerns. The SPI+ was associated with about 50% fewer suicidal behaviors over a 6-month follow-up and more than double the odds of engaging in outpatient behavioral health care. The significant decrease in suicidal behavior underscores the utility of SPI+ as an effective prevention strategy in EDs. Interestingly, our mediation analysis demonstrated that the association with suicidal behavior was not attributable to the increased treatment engagement in the intervention group nor was it accounted for by the number of follow-up calls. This finding suggests that SPI+ has a positive association with suicidal behavior apart from attendance in outpatient behavioral health care. We chose attendance at more than 1 outpatient appointment as our outcome because the principal difficulty with suicidal patients is that they tend to not attend a first appointment. Because patients admitted to inpatient units were not included, the range of suicidality in our sample is restricted to a lower-risk population. The reach of this intervention may be greater if the range of suicide risk is not restricted and the intervention is provided to all suicidal patients including those admitted to inpatient units.

Our results are consistent with findings that crisis response planning reduced suicide attempts during a 6-month follow-up period in high-risk active duty soldiers.42 Crisis response planning is an abbreviated form of safety planning that uses 4 of 6 elements of the SPI (without social interaction as a means of distracting from suicidal thoughts and lethal means counseling). Our results are also consistent with past research30 that found that a combination of in-person intervention using screening and a safety plan coupled with telephone phone follow-up reduced suicide attempts.

Limitations

Quiz Ref IDThere are several limitations to consider. Although intervention sites were matched with control sites and the analyses used propensity scoring, this was not a randomized trial, so there is a potential for confounding. Given that this was a clinical demonstration project, limited information is available about individual patients. We were reliant on medical records for suicidal behavior reports and diagnostic and demographic information. Some suicidal behaviors may have occurred that were not entered into the medical record because staff were unaware of them. However, we have no reason to believe that this occurred more often in either condition. Also, some patients in usual care may have had a safety plan in prior treatment. This could have diminished the difference between the SPI+ and usual care groups. Furthermore, this project was conducted in Veterans Health Administration hospital EDs with predominantly men; therefore, we do not know how well our findings generalize to civilian settings and women.

Additional limitations are the low observed suicide event rate in both groups and the lower proportion of eligible patients in the comparison group. This low rate probably resulted from excluding patients admitted to inpatient units from the ED. Thus, the study likely included patients at lower risk for suicide. Interestingly, this lower-than-expected rate may indicate that ED clinicians are correctly hospitalizing higher-risk patients. Another possibility is that some suicide events were not captured by suicide behavior reports either because reports were not written or patients did not reveal all events to clinicians. Given that our sample was limited to patients who presented to an ED for suicide-related concerns but were not hospitalized, the efficacy of SPI+ is unknown for patients who required immediate psychiatric hospitalization. Further randomized trials with the full range of suicidal patients are required to assess the effect of SPI+ on suicidal behavior.

Conclusions

Our findings are promising and indicate that safety planning and active outreach, a set of low-burden strategies, are useful components of effective suicide prevention. Importantly, using the low-burden intervention in this project and others30 was associated with about the same reduction in suicidal behavior as more intensive and costly psychosocial interventions.43,44 If implemented broadly, SPI+ has the potential to reduce suicidal behavior and enhance behavioral health treatment engagement, particularly during high-risk periods following ED discharge. This strategy may help decrease suicide risk in the long term.

Back to top
Article Information

Corresponding Author: Barbara Stanley, PhD, Columbia University, New York State Psychiatric Institute, 1051 Riverside Dr, Unit 42, New York, NY 10032 (bhs2@cumc.columbia.edu).

Accepted for Publication: August 18, 2018.

Published Online: July 11, 2018. doi:10.1001/jamapsychiatry.2018.1776

Author Contributions: Drs Stanley and Brown had full access to all data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Stanley, Brown, Brenner, Currier, Knox, Green.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Stanley, Brown, Brenner, Galfalvy, Currier, Knox, Chaudhury, Bush.

Critical revision of the manuscript for important intellectual content: Stanley, Brown, Brenner, Galfalvy, Green.

Statistical analysis: Stanley, Galfalvy, Currier.

Obtained funding: Stanley, Brown, Currier.

Administrative, technical, or material support: Stanley, Brown, Brenner, Currier, Bush, Green.

Supervision: Brown, Knox.

Conflict of Interest Disclosures: None reported.

Funding/Support: This project was supported by the Office of Mental Health Services, Department of Veterans Affairs.

Role of the Funder/Sponsor: The Department of Veterans Affairs assisted in the design and conduct of the study but did not have a role in the data analysis; the interpretation of the data; preparation, review or approval of the manuscript; and the decision to submit the manuscript for publication.

Additional Contributions: Site leaders were Lisa Brenner, PhD, Denver VA Medical Center; Joan Chipps, LCSW, Buffalo VA Medical Center; Joshua Hooberman, PhD, Manhattan VA Medical Center; Christine Jackson, PhD, Manhattan VA Medical Center; Mitchell Kling, MD, Corporal Michael J. Crescenz VA Medical Center; and Keith Rogers, MD, Portland VA Medical Center. Acute services coordinators were Patricia Alexander, PhD, Denver VA Medical Center; Laura Blandy, PsyD, Corporal Michael J. Crescenz VA Medical Center; Aimee Coughlin, MSW, Portland VA Medical Center; John Dennis, PhD, Corporal Michael J. Crescenz VA Medical Center; Michael Miello, PhD, Buffalo VA Medical Center; Katherine Mostkoff, LCSW, Manhattan VA Medical Center; and Jarrod Reisweber, PsyD, Corporal Michael J. Crescenz VA Medical Center. Site lead psychiatrists were Steven Dobscha, MD, Portland VA Medical Center; Keith Rogers, MD, Portland VA Medical Center; Donald Tavakoli, MD, Corporal Michael J. Crescenz VA Medical Center; and Adam Wolkin, MD, Manhattan VA Medical Center. Site leads and site psychiatrists were not compensated, and acute service coordinators were compensated through their work as project staff employees. We thank Ira Katz, MD, and Janet Kemp, PhD, who were instrumental in facilitating the implementation of this project, and Marjan Ghahramanlou Holloway, PhD, who provided guidance. We would also like to acknowledge Walter Matweychuk, PhD and Lauren Denneson, PhD, for their contributions in on-site implementation and staff supervision.

References
1.
The Centers for Disease Control and Prevention. Leading causes of death reports, national and regional, 1999-2015: web-based injury statistics query and reporting system. https://webappa.cdc.gov/sasweb/ncipc/leadcaus10_us.html. Published 2017. Accessed February 26, 2017.
2.
Office of Mental Health and Suicide Prevention. Suicide among veterans and other Americans 2001-2014. https://www.mentalhealth.va.gov/docs/2016suicidedatareport.pdf. Published 2016. Accessed December 5, 2017.
3.
Claassen  CA, Larkin  GL.  Occult suicidality in an emergency department population.  Br J Psychiatry. 2005;186(4):352-353. doi:10.1192/bjp.186.4.352PubMedGoogle ScholarCrossref
4.
Fields  WW, Asplin  BR, Larkin  GL,  et al.  The Emergency Medical Treatment and Labor Act as a federal health care safety net program.  Acad Emerg Med. 2001;8(11):1064-1069. doi:10.1111/j.1553-2712.2001.tb01116.xPubMedGoogle ScholarCrossref
5.
Larkin  GL, Beautrais  AL.  Emergency departments are underutilized sites for suicide prevention.  Crisis. 2010;31(1):1-6. doi:10.1027/0227-5910/a000001PubMedGoogle ScholarCrossref
6.
Owens  P, Mutter  R, Stocks  C. Mental health and substance abuse-related emergency department visits among adults, 2007: statistical brief #92: Healthcare Cost and Utilization Project (HCUP) Statistical Briefs 2010. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb92.pdf. Accessed February 20, 2017.
7.
Ting  SA, Sullivan  AF, Boudreaux  ED, Miller  I, Camargo  CA  Jr.  Trends in US emergency department visits for attempted suicide and self-inflicted injury, 1993-2008.  Gen Hosp Psychiatry. 2012;34(5):557-565. doi:10.1016/j.genhosppsych.2012.03.020PubMedGoogle ScholarCrossref
8.
Olfson  M, Marcus  SC, Bridge  JA.  Focusing suicide prevention on periods of high risk.  JAMA. 2014;311(11):1107-1108. doi:10.1001/jama.2014.501PubMedGoogle ScholarCrossref
9.
Ahmedani  BK, Simon  GE, Stewart  C,  et al.  Health care contacts in the year before suicide death.  J Gen Intern Med. 2014;29(6):870-877. doi:10.1007/s11606-014-2767-3PubMedGoogle ScholarCrossref
10.
Da Cruz  D, Pearson  A, Saini  P,  et al.  Emergency department contact prior to suicide in mental health patients.  Emerg Med J. 2011;28(6):467-471. doi:10.1136/emj.2009.081869PubMedGoogle ScholarCrossref
11.
Gairin  I, House  A, Owens  D.  Attendance at the accident and emergency department in the year before suicide: retrospective study.  Br J Psychiatry. 2003;183:28-33. doi:10.1192/bjp.183.1.28PubMedGoogle ScholarCrossref
12.
Kurz  A, Möller  HJ.  Help-seeking behavior and compliance of suicidal patients.  Psychiatr Prax. 1984;11(1):6-13.PubMedGoogle Scholar
13.
Piacentini  J, Rotheram-Borus  MJ, Gillis  JR,  et al.  Demographic predictors of treatment attendance among adolescent suicide attempters.  J Consult Clin Psychol. 1995;63(3):469-473. doi:10.1037/0022-006X.63.3.469PubMedGoogle ScholarCrossref
14.
Lizardi  D, Stanley  B.  Treatment engagement: a neglected aspect in the psychiatric care of suicidal patients.  Psychiatr Serv. 2010;61(12):1183-1191. doi:10.1176/ps.2010.61.12.1183PubMedGoogle ScholarCrossref
15.
Monti  K, Cedereke  M, Öjehagen  A.  Treatment attendance and suicidal behavior 1 month and 3 months after a suicide attempt: a comparison between two samples.  Arch Suicide Res. 2003;7(2):167-174. doi:10.1080/13811110301581Google ScholarCrossref
16.
Kelly  KT, Knudson  MP.  Are no-suicide contracts effective in preventing suicide in suicidal patients seen by primary care physicians?  Arch Fam Med. 2000;9(10):1119-1121. doi:10.1001/archfami.9.10.1119PubMedGoogle ScholarCrossref
17.
Kroll  J.  Use of no-suicide contracts by psychiatrists in Minnesota.  Am J Psychiatry. 2000;157(10):1684-1686. doi:10.1176/appi.ajp.157.10.1684PubMedGoogle ScholarCrossref
18.
Simon  RI.  The suicide prevention contract: clinical, legal, and risk management issues.  J Am Acad Psychiatry Law. 1999;27(3):445-450.PubMedGoogle Scholar
19.
Reid  WH.  Promises, promises: don’t rely on patients’ no-suicide/no-violence “contracts”.  J Pract Psychiatry Behav Health. 1998;4:316-318.Google Scholar
20.
Shaffer  D, Pfeffer  CR; American Academy of Child and Adolescent Psychiatry.  Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior.  J Am Acad Child Adolesc Psychiatry. 2001;40(7)(suppl):24S-51S. doi:10.1097/00004583-200107001-00003PubMedGoogle ScholarCrossref
21.
Stanford  EJ, Goetz  RR, Bloom  JD.  The No Harm Contract in the emergency assessment of suicidal risk.  J Clin Psychiatry. 1994;55(8):344-348.PubMedGoogle Scholar
22.
The Joint Commission. Sentinel Event Alert 56: Detecting and treating suicide ideation in all settings 2016; https://www.jointcommission.org/assets/1/18/SEA_56_Suicide.pdf. Accessed March 11, 2017.
23.
Rudd  MD, Mandrusiak  M, Joiner  TE  Jr.  The case against no-suicide contracts: the commitment to treatment statement as a practice alternative.  J Clin Psychol. 2006;62(2):243-251. doi:10.1002/jclp.20227PubMedGoogle ScholarCrossref
24.
Stanley  B, Brown  GK.  Safety planning intervention: a brief intervention to mitigate suicide risk.  Cognit Behav Pract. 2012;19(2):256-264. doi:10.1016/j.cbpra.2011.01.001Google ScholarCrossref
25.
Stanley  B, Chaudhury  SR, Chesin  M,  et al.  An emergency department intervention and follow-up to reduce suicide risk in the VA: acceptability and effectiveness.  Psychiatr Serv. 2016;67(6):680-683. doi:10.1176/appi.ps.201500082PubMedGoogle ScholarCrossref
26.
Chesin  MS, Stanley  B, Haigh  EA,  et al.  Staff views of an emergency department intervention using safety planning and structured follow-up with suicidal veterans.  Arch Suicide Res. 2017;21(1):127-137. doi:10.1080/13811118.2016.1164642PubMedGoogle ScholarCrossref
27.
Bryan  CJ, Mintz  J, Clemans  TA,  et al.  Effect of crisis response planning vs. contracts for safety on suicide risk in U.S. Army soldiers: a randomized clinical trial.  J Affect Disord. 2017;212:64-72. doi:10.1016/j.jad.2017.01.028PubMedGoogle ScholarCrossref
28.
Brown  GK, Green  KL.  A review of evidence-based follow-up care for suicide prevention: where do we go from here?  Am J Prev Med. 2014;47(3)(suppl 2):S209-S215. doi:10.1016/j.amepre.2014.06.006PubMedGoogle ScholarCrossref
29.
Fleischmann  A, Bertolote  JM, Wasserman  D,  et al.  Effectiveness of brief intervention and contact for suicide attempters: a randomized controlled trial in five countries.  Bull World Health Organ. 2008;86(9):703-709. doi:10.2471/BLT.07.046995PubMedGoogle ScholarCrossref
30.
Miller  IW, Camargo  CA  Jr, Arias  SA,  et al; ED-SAFE Investigators.  Suicide prevention in an emergency department population: the ED-SAFE study.  JAMA Psychiatry. 2017;74(6):563-570. doi:10.1001/jamapsychiatry.2017.0678PubMedGoogle ScholarCrossref
31.
Spirito  A, Boergers  J, Donaldson  D, Bishop  D, Lewander  W.  An intervention trial to improve adherence to community treatment by adolescents after a suicide attempt.  J Am Acad Child Adolesc Psychiatry. 2002;41(4):435-442. doi:10.1097/00004583-200204000-00016PubMedGoogle ScholarCrossref
32.
Vaiva  G, Vaiva  G, Ducrocq  F,  et al.  Effect of telephone contact on further suicide attempts in patients discharged from an emergency department: randomised controlled study.  BMJ. 2006;332(7552):1241-1245. doi:10.1136/bmj.332.7552.1241PubMedGoogle ScholarCrossref
33.
Carter  GL, Clover  K, Whyte  IM, Dawson  AH, D’Este  C.  Postcards from the EDge project: randomised controlled trial of an intervention using postcards to reduce repetition of hospital treated deliberate self poisoning.  BMJ. 2005;331(7520):805-807. doi:10.1136/bmj.38579.455266.E0PubMedGoogle ScholarCrossref
34.
Morgan  HG, Jones  EM, Owen  JH.  Secondary prevention of non-fatal deliberate self-harm: the green card study.  Br J Psychiatry. 1993;163:111-112. doi:10.1192/bjp.163.1.111PubMedGoogle ScholarCrossref
35.
Motto  JA, Bostrom  AG.  A randomized controlled trial of postcrisis suicide prevention.  Psychiatr Serv. 2001;52(6):828-833. doi:10.1176/appi.ps.52.6.828PubMedGoogle ScholarCrossref
36.
Beautrais  AL, Gibb  SJ, Faulkner  A, Fergusson  DM, Mulder  RT.  Postcard intervention for repeat self-harm: randomised controlled trial.  Br J Psychiatry. 2010;197(1):55-60. doi:10.1192/bjp.bp.109.075754PubMedGoogle ScholarCrossref
37.
Currier  GW, Brown  GK, Brenner  LA,  et al.  Rationale and study protocol for a two-part intervention: safety planning and structured follow-up among veterans at risk for suicide and discharged from the emergency department.  Contemp Clin Trials. 2015;43:179-184. doi:10.1016/j.cct.2015.05.003PubMedGoogle ScholarCrossref
38.
Stanley  B, Brown  GK.  Safety Planning Intervention: Brief Instructions. Washington, DC: US Department of Veterans Affairs; 2008.
39.
Stanley  B, Brown  GK, Currier  GW, Lyons  C, Chesin  M, Knox  KL.  Brief intervention and follow-up for suicidal patients with repeat emergency department visits enhances treatment engagement.  Am J Public Health. 2015;105(8):1570-1572. doi:10.2105/AJPH.2015.302656PubMedGoogle ScholarCrossref
40.
Hoffmire  C, Stephens  B, Morley  S, Thompson  C, Kemp  J, Bossarte  RM.  VA suicide prevention applications network: a national health care system-based suicide event tracking system.  Public Health Rep. 2016;131(6):816-821. doi:10.1177/0033354916670133PubMedGoogle ScholarCrossref
41.
Bates  D, Maechler  M, Bolker  B, Walker  S.  Fitting linear mixed-effects models using lme4.  J Stat Softw. 2015;67(1):1-48. doi:10.18637/jss.v067.i01Google ScholarCrossref
42.
Bryan  CJ, Mintz  J, Clemans  TA,  et al.  Effect of crisis response planning on patient mood and clinician decision making: a clinical trial with suicidal U.S. soldiers.  Psychiatr Serv. 2018;69(1):108-111. doi:10.1176/appi.ps.201700157PubMedGoogle ScholarCrossref
43.
Brown  GK, Ten Have  T, Henriques  GR, Xie  SX, Hollander  JE, Beck  AT.  Cognitive therapy for the prevention of suicide attempts: a randomized controlled trial.  JAMA. 2005;294(5):563-570. doi:10.1001/jama.294.5.563PubMedGoogle ScholarCrossref
44.
Linehan  MM, Armstrong  HE, Suarez  A, Allmon  D, Heard  HL.  Cognitive-behavioral treatment of chronically parasuicidal borderline patients.  Arch Gen Psychiatry. 1991;48(12):1060-1064. doi:10.1001/archpsyc.1991.01810360024003PubMedGoogle ScholarCrossref
×