Ask Suicide-Screening Questions (ASQ): A Brief Instrument for the Pediatric Emergency Department

Objective To develop a brief screening instrument to assess the risk for suicide in pediatric emergency department patients.

Design A prospective, cross-sectional instrument-development study evaluated 17 candidate screening questions assessing suicide risk in young patients. The Suicidal Ideation Questionnaire served as the criterion standard.

Setting Three urban, pediatric emergency departments associated with tertiary care teaching hospitals.

Participants A convenience sample of 524 patients aged 10 to 21 years who presented with either medical/surgical or psychiatric chief concerns to the emergency department between September 10, 2008, and January 5, 2011.

Main Exposures Participants answered 17 candidate questions followed by the Suicidal Ideation Questionnaire.

Main Outcome Measures Sensitivity, specificity, predictive values, likelihood ratios, and area under the receiver operating characteristic curves of the best-fitting combinations of screening questions for detecting elevated risk for suicide.

Results A total of 524 patients were screened (344 medical/surgical and 180 psychiatric). Fourteen of the medical/surgical patients (4%) and 84 of the psychiatric patients (47%) were at elevated suicide risk on the Suicidal Ideation Questionnaire. Of the 17 candidate questions, the best-fitting model comprised 4 questions assessing current thoughts of being better off dead, current wish to die, current suicidal ideation, and past suicide attempt. This model had a sensitivity of 96.9% (95% CI, 91.3-99.4), specificity of 87.6% (95% CI, 84.0-90.5), and negative predictive values of 99.7% (95% CI, 98.2-99.9) for medical/surgical patients and 96.9% (95% CI, 89.3-99.6) for psychiatric patients.

Conclusions A 4-question screening instrument, the Ask Suicide-Screening Questions (ASQ), with high sensitivity and negative predictive value, can identify the risk for suicide in patients presenting to pediatric emergency departments.

Youth suicide is an international public health problem. In 2007, suicide was the third leading cause of death among youth ages 10 to 24 years, accounting for 4320 deaths in the United States.¹ Nonfatal suicide attempts are more prevalent, affecting as many as 5% to 8% of children and adolescents annually^2-5 and resulting in significant morbidity and increased use of emergency departments (EDs) and hospitals.

Early identification and treatment of patients at elevated risk for suicide is a key suicide prevention strategy,⁶ yet high-risk patients are often not recognized by health care providers.⁷ In fact, most individuals who die by suicide have visited a health care provider in the year before their death, most within the prior 3 months.^7,8 Whereas medical visits afford clinicians an opportunity to identify and refer patients at risk for suicide,⁹ individuals often present solely with somatic concerns and infrequently discuss suicidal thoughts and plans unless asked directly.¹⁰

In 2010, the Joint Commission issued a Sentinel Event Alert, suggesting suicide-screening for all patients visiting health care settings.¹¹ Additionally, the American Academy of Pediatrics called for rapid, easy to administer suicide screening tools to guide health care clinicians in the assessment of suicide risk among young people in medical settings.¹²

The ED is a promising venue for identifying young people at risk for suicide.¹³ Emergency department clinicians are often the sole connection with the health care system for millions of youth and their families; they are uniquely positioned to screen for suicide risk in patients and assist in the process of making clinically appropriate referrals for mental health treatment.^14-16 Nevertheless, most patients presenting to the ED are not currently assessed for the risk for suicide.¹⁷ Time constraints, inadequate training, and the lack of proper screening instruments are reported as reasons why ED clinicians do not routinely screen for suicide risk.¹⁸

Emergency department clinicians require tools that do not assume extensive psychiatric training to administer.¹⁹ Instruments to guide these clinicians, such as the 4-item Risk of Suicide Questionnaire, were developed and validated on pediatric ED psychiatric populations.^20-22 However, to our knowledge, brief instruments to assess the risk for suicide in patients who present to EDs for medical or surgical reasons do not yet exist.

The primary aim of this study was to develop a brief valid screening instrument that could assess the risk for suicide in pediatric and young adult patients evaluated in EDs for medical/surgical reasons. To avoid burdening ED workflow, we sought to include the smallest number of questions in our instrument that could identify youth with suicidal thoughts, yet maintain high sensitivity, specificity, and negative predictive value (NPV). Psychiatric patients were included in the sample to determine whether 1 screening instrument could be valid for all pediatric patients evaluated in the ED, regardless of their chief concern.

Between September 10, 2008, and January 5, 2011, we prospectively enrolled convenience samples of patients aged 10 to 21 years who presented with either medical/surgical or psychiatric concerns to 1 of 3 large pediatric EDs associated with major urban teaching hospitals.

Exclusion criteria included (1) developmental disability, cognitive impairment, or communication disorder such that the patient was not able to comprehend questions or communicate their answers; (2) triage level 1 (for the medical/surgical patients only),²³ suggesting that the patient was not physiologically stable enough to be approached; (3) parent/legal guardian unavailable for patients younger than 18 years; and (4) parents/guardians and/or patients were non-English speaking. No exclusions were based on sex, race, or ethnicity.

Patients with psychiatric concerns were included to achieve our secondary aim of creating a screening instrument for all patients in a pediatric ED and to ensure that enough subjects with the outcome of interest (at risk for suicide) were included in the total sample. Given that patients with psychiatric concerns accounted for less than 5% of total annual ED visits across sites, we adopted a strategy of approaching every eligible psychiatric patient and every other patient with a medical/surgical concern for recruitment into the study.

The institutional review boards at the participating institutions and the National Institute of Mental Health approved this study. For participants younger than 18 years of age, written informed consent was obtained from the parent/guardian, and written informed assent was obtained from the patient. All participants 18 years of age and older provided written informed consent.

Seventeen candidate screening questions were assembled based on risk factors for suicide in adolescents, including suicide attempt history, suicidal ideation, depression, hopelessness, substance abuse, and social isolation. The 17 candidate questions were identified from several sources, including published literature on adolescent suicide risk, interviews with adolescent suicide experts and senior pediatric mental health clinicians, items from the Centers for Disease Control and Prevention Youth Risk Behavior Survey,² and the existing Risk of Suicide Questionnaire.²⁰ The 17 candidate questions for the new measure were reviewed and revised by a panel of mental health clinicians, health services researchers, and survey methodologists for use among young patients presenting to an ED. The adapted questions were then pilot tested by several pediatric ED clinicians and mental health specialists in a sample of adolescent psychiatric inpatients and healthy youths for appropriateness, comprehensibility, and ease of administration. All items were phrased in the form of a question with possible responses of yes, no, or no response. Nine of the items were considered trigger items because positive endorsement represented potential significant emotional distress; if a subject responded positively, further psychiatric assessment would be triggered automatically, regardless of other answers. These questions asked about severe depression, suicidal ideation, and suicidal behavior.

The Suicidal Ideation Questionnaire (SIQ),²⁴ a self-reported measure of the severity of suicidal ideation in adolescents, was used as the criterion standard to validate the 17 candidate questions. Two versions of the SIQ are available, depending on the participant's age. For this study, the 30-item SIQ was administered to participants 15 years of age and older; the 15-item SIQ-Junior (JR) was administered to participants 14 years of age and younger. In both versions of the SIQ, individuals are asked to rate the frequency with which a thought occurs on a 7-point scale ranging from almost every day to never. A cutoff score is used to judge the severity level of suicidal ideation warranting additional psychiatric evaluation. A total score of 41 or greater was considered clinically significant (31 or greater for the SIQ-JR). In addition, the SIQ has 8 items (6 on the SIQ-JR) deemed critical items because they directly assess serious self-destructive behavior. If a person responds positively to 3 or more of those items (2 or more on the SIQ-JR), they are also considered to have clinically significant suicidal ideation.²⁴ The SIQ has demonstrated a high reliability (SIQ: r = 0.97; SIQ-JR: r = 0.94), validity, and predictive ability.^24,25

After the initial triage assessment and room assignment, participants were administered the 17 candidate questions followed by the SIQ by trained bachelor's degree–level or master's degree–level research assistants. A survey containing questions about sociodemographic information, history of medical and psychiatric illness, prior health care usage, and a screening evaluation was also administered. Interviews were conducted without the parent/guardian in the room, but participants were told that if the research assistants had any concerns about their safety, their parents would be notified and pertinent information would be shared with the ED clinical staff.

As a safety measure, any patient who responded positively to any 1 of 9 trigger screening questions on the 17-item questionnaire (eg, any level of current suicidal ideation, past suicidal behavior, or severe depression), scored positive on the SIQ, or responded positively to any of the 8 SIQ critical items required a further psychiatric assessment, which was conducted as per standard of care for suicidal risk in each ED.

We calculated the frequencies of positive responses to all 17 candidate and SIQ items as well as the frequencies of all demographic, clinical, and health care usage characteristics for our sample. To measure the validity of the candidate questions, we examined the agreement between individual candidate questions and the risk for suicide as assessed by the SIQ, with the chance-corrected kappa statistic (κ).²⁶ We then constructed logistic regression models to examine the ability of different combinations of the candidate questions to predict the risk for suicide as assessed by the SIQ; only candidate questions with κ of 0.40 or greater were considered in these models.

Models were estimated using the logistic procedure of SAS version 9.2 (SAS Institute).²⁷ Because a prior suicide attempt is the most potent predictor of future suicidal behavior,²⁸ we made an a priori decision to retain question 16 (“Have you ever tried to kill yourself?”) in all regression models. Initially, we used a backward stepwise procedure to arrive at a reasonably parsimonious model that retained a high level of sensitivity. Next, we evaluated all possible combinations of the candidate questions remaining after the stepwise procedure. A positive response to any 1 of the candidate questions constituted a positive screen for the risk for suicide. The predictive ability of each model was assessed using the C statistic, which represents the area under the receiver operating characteristic curve.^29,30

We calculated the sensitivity (the probability of a positive result when given to youth who are at risk for suicide), the specificity (the probability of a negative result when given to youth who are not at risk for suicide), the positive predictive value (PPV; the probability that a child who screened positive actually is at risk for suicide), and the NPV (the probability that a child who screened negative actually is not at risk for suicide). Likelihood ratios (LRs) were calculated to summarize the diagnostic accuracy of the best-fitting combination of screening questions. Ninety-five percent confidence intervals for the sensitivity, specificity, PPV, and NPV were calculated using exact binomial methods.³¹ Because of the clinical significance and relative importance of not misclassifying suicidal youths as false negatives, we also identified the proportion of youth at risk for suicide (as determined by the SIQ) who would have been undetected by each combination of the 17 candidate questions. We arrived at the final model by choosing the candidate items that maximized sensitivity, specificity, and NPV such that the minimum number of suicide-positive patients would be misclassified and ED clinicians would not be overburdened managing false-positive patients.

The sample size calculation was based on sensitivity (98%) and specificity (37%) results reported in a previous study by Horowitz et al¹³ and on the expectation that two-thirds of our participants would present to the ED with medical/surgical concerns (by design, medical/surgical patients were oversampled). α and β were set at 0.05 (2-tailed) and 0.10 (90% power), respectively. Using the McNemar test of equality of paired proportions, we calculated a minimum sample size of 388 participants, which we rounded up to 450 (approximately 150 participants/site).

A total of 1170 patients (783 medical/surgical, 364 psychiatric, and 23 undetermined patients) were approached during the study period across the 3 sites; 803 patients (69%) were eligible for participation; 529 (66%) consented to participate, of whom 524 (344 medical/surgical and 180 psychiatric patients) completed the screening protocol (Figure). There were no significant differences in age, race/ethnicity, sex, or presenting concern (medical/surgical or psychiatric) between those who did and did not participate in this study. Characteristics of study participants are shown in Table 1. The mean (SD) age at enrollment was 15.2 (2.6) years. Most of the sample was female (57.0%), white (50.4%), and privately insured (53.2%).

Ninety-eight of 524 participants (84 psychiatric, 14 medical/surgical; 18.7%) were at elevated risk for suicide based on the criterion standard SIQ. The chance-corrected agreement between individual candidate questions and suicidal risk as determined by the SIQ ranged from κ of 0.78 (95% CI, 0.72 to 0.85) to −0.06 (95% CI, −0.09 to 0.02) (Table 2). Logistic regression analyses revealed that there was little improvement in the model properties obtained beyond the inclusion of 4 candidate questions. The top 6 combinations of candidate questions are provided in Table 3. These models represented the best-fitting combinations of screening questions that maximized sensitivity, specificity, and NPV.

While the 6-question and 5-question models had a higher sensitivity, model 4 showed a comparable NPV and a 3-point improvement in PPV from the 5-question model, resulting in fewer false positives. Additionally, model 4 correctly classified all but 1 SIQ-positive patient with medical/surgical concerns. Therefore, we chose model 4 as the final Ask Suicide-Screening Questions (ASQ) instrument (eTable and eAppendix). The selection of the best-fitting models was based on both clinical and statistical significance.

The performance of the ASQ in detecting elevated suicidal risk (Table 4) consisted of responses to the combination of candidate questions 9 (current thoughts of being better off dead), 11 (current wish to die), 15 (current suicidal ideation), and 16 (past suicide attempt). The sensitivity and specificity of having 1 or more positive responses on the ASQ in identifying either patients with medical/surgical or psychiatric concerns at elevated suicidal risk were 96.9% (95% CI, 91.3-99.4) and 87.6% (95% CI, 84.0-90.5), respectively. Of the 311 patients with medical/surgical concerns who screened negative on the ASQ, only 1 (0.3%) screened positive on the SIQ (NPV, 99.7%; 95% CI, 98.2-99.9). The positive LR for patients with medical/surgical concerns was 15.2 (95% CI, 7.2-27.0), indicating that a positive screen on the ASQ was 15.2 times more likely to be seen in someone actually at suicidal risk than in someone not at risk. The negative LR was 0.08 (95% CI, 0.002-0.37), meaning that a negative 4-item ASQ was 0.08 times as likely to be seen in someone at suicidal risk than in someone not at risk. The corresponding NPV, positive LR, and negative LR in patients with psychiatric concerns were 96.9% (95% CI, 89.3-99.6), 2.8 (95% CI, 2.1-4.0), and 0.04 (95% CI, 0.004-0.15), respectively.

We examined the performance of the 4-item ASQ in subgroups defined by age, sex, and race (eAppendix). There were no statistical differences in the sensitivity, specificity, PPV, and NPV of the ASQ results in males and females and in younger and older participants. However, the sensitivity of the ASQ was significantly lower in black participants compared with white participants and participants of other races (P = .02).

The ED is a viable medical setting for implementing routine suicide screening among youth. With the longer SIQ as the criterion standard, our 4-question screening instrument, the ASQ (eAppendix), accurately assessed the risk for suicide in young ED patients with medical/surgical or psychiatric chief concerns.

The ASQ appeared to have good content validity. The 4 questions together assessed major facets of established suicide risk factors, including 3 questions that targeted current suicidal ideation in a manner in which youth with medical concerns in particular can relate: current thoughts of being better off dead, current wish to die, and current suicidal ideation; a fourth question inquired about the most critical risk factor for future suicidal behavior—a history of suicide attempt. Positive responses to 1 or more of these 4 questions identified 97% of the youth at risk for suicide, as assessed by the SIQ, a much longer criterion standard instrument typically administered by mental health clinicians. In addition, the high specificity demonstrated by the 4 questions, the ability to correctly identify young patients who are currently not at elevated risk for suicide (87.6%), is of paramount importance in not overburdening a busy ED setting with limited mental health resources. Given the consequences of failing to detect an increased risk for suicide, the high NPV (99.7%), or the probability that the young person who screened negative is not at elevated risk for suicide, is also an important attribute of an instrument used by ED clinicians.

Overall, 18.7% of the ED patients (98 of the 524) screened positive for the risk for suicide; most of whom were patients with psychiatric concerns. Elevated risk for suicide was detected in 4.1% of the ED patients (14 of the 344) with medical/surgical concerns. Had it not been for the screening, risk for suicide in these 14 patients would have perhaps been undetected, as their chief concerns were medical in nature (eg, ankle injury, abdominal pain, headaches). This is a relatively small number of patients in terms of overburdening a busy ED; yet, a notable number of youth could be identified with a screening instrument that takes less than 2 minutes to administer. These data are consistent with King et al,³² who examined suicide screening in nonpsychiatric patients in the ED with a much longer battery of assessments and concluded that risk identification may be a critical step in reducing the youth suicide rate.

These findings are subject to the following limitations. Participating EDs were all in urban, tertiary care teaching hospitals and may not generalize to other ED settings. We used convenience sampling, which could have introduced bias into our findings, and we did not administer the ASQ to a validation cohort. There may also have been a fatigue effect by asking the participants repeatedly about suicidal thoughts; however, this did not arise as a concern in study evaluation interviews. In addition, although the criterion standard SIQ has sound psychometric properties, including good validity and reliability, it primarily identifies youth at risk for clinically significant suicidal ideation and may not necessarily be predictive of suicidal behavior. Ideally, our protocol would have included a longitudinal follow-up component to determine whether patients who screened positive for the risk for suicide were more likely than others to attempt suicide after ED discharge.

Future studies measuring the impact of suicide screening in pediatric EDs on such critical outcomes as linkage with mental health services and future suicidal behavior are warranted. Potential racial differences in sensitivity of suicide screening instruments should also be examined. In addition, evaluating the acceptance of such screening instruments by clinicians and the costs associated with implementing universal suicide screening in EDs would inform implementation strategies.

In conclusion, youth presenting to pediatric EDs can be rapidly assessed for the risk for suicide with a brief 4-question screening instrument, the ASQ, which demonstrates high sensitivity, specificity, and NPV. The ASQ may be an appropriate tool for implementation in this venue as part of the Joint Commission¹¹ and the American Academy of Pediatrics¹² recommendations.

Correspondence: Lisa M. Horowitz, PhD, MPH, National Institute of Mental Health, Clinical Research Center, Bldg 10, Room 6-5362, Bethesda, MD 20892 (horowitzl@mail.nih.gov).

Accepted for Publication: May 1, 2012.

Published Online: October 1, 2012. doi:10.1001/archpediatrics.2012.1276

Author Contributions: Drs Horowitz, Bridge, and Klima had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Horowitz, Bridge, Ballard, Rosenstein, and Pao. Acquisition of data: Horowitz, Bridge, Teach, Ballard, Wharff, Ginnis, Cannon, Joshi, and Pao. Analysis and interpretation of data: Horowitz, Bridge, Teach, Ballard, Klima, Rosenstein, Ginnis, Cannon, and Pao. Drafting of the manuscript: Horowitz, Bridge, Teach, Klima, Rosenstein, Cannon, and Pao. Critical revision of the manuscript for important intellectual content: Horowitz, Bridge, Teach, Ballard, Klima, Rosenstein, Wharff, Ginnis, Joshi, and Pao. Statistical analysis: Bridge and Klima. Obtained funding: Bridge, Rosenstein, Wharff, Ginnis, and Pao. Administrative, technical, and material support: Horowitz, Bridge, Teach, Ballard, Rosenstein, and Pao. Study supervision: Horowitz, Bridge, Rosenstein, Joshi, and Pao.

Funding/Support: The research in this article was supported by the Intramural Research Program of the National Institutes of Health and the National Institute of Mental Health. Dr Bridge's work was supported by institutional research funds from the Research Institute at Nationwide Children's Hospital and grant K01 MH-69948 from the National Institute of Mental Health. Dr Wharff and Ms Ginnis' work was supported by institutional research funds from the Program for Patient Safety and Quality at Children's Hospital Boston.

Disclaimer: The opinions expressed in the article are the views of the authors and do not reflect the views of the Department of Health and Human Services or the US government.

Additional Contributions: We thank David Jobes, PhD (Catholic University of America); Abigail Bosk, BA, Deborah Snyder, MSW, Rebecca Prengler, BS, Amy Goldstein, PhD, Erica Ludi, BS, Sarah Westen, BS, David Luckenbaugh, MA, and Jeanne Radcliffe, RN, MPH (National Institutes of Mental Health); Janet Heekin, MLS (National Institutes of Health); Arielle Sheftall, PhD, Sandra M. McBee-Strayer, PhD, Rachel Jones, BS, Kristen Mendoza, BS (Nationwide Children's Hospital); Eliza Hutchinson, BA (Children's Hospital Boston); Angelica Kloos, MD, Khristine Heflin, MSW, Amanda Lewis, RN, Marc Dalton, MD (Children's National Medical Center); and the emergency department social workers, nurses, and physicians at Children's National Medical Center, Nationwide Children's Hospital, and Children's Hospital Boston; as well as the patients and their families for their participation.