Effect of the Garrett Lee Smith Memorial Suicide Prevention Program on Suicide Attempts Among Youths

Importance  Youth suicide prevention is a major public health priority. Studies documenting the effectiveness of community-based suicide prevention programs in reducing the number of nonlethal suicide attempts have been sparse.

Objective  To determine whether a reduction in suicide attempts among youths occurs following the implementation of the Garrett Lee Smith Memorial Suicide Prevention Program (hereafter referred to as the GLS program), consistent with the reduction in mortality documented previously.

Design, Setting, and Participants  We conducted an observational study of community-based suicide prevention programs for youths across 46 states and 12 tribal communities. The study compared 466 counties implementing the GLS program between 2006 and 2009 with 1161 counties that shared key preintervention characteristics but were not exposed to the GLS program. The unweighted rounded numbers of respondents used in this analysis were 84 000 in the control group and 57 000 in the intervention group. We used propensity score–based techniques to increase comparability (on background characteristics) between counties that implemented the GLS program and counties that did not. We combined information on program activities collected by the GLS national evaluation with information on county characteristics from several secondary sources. The data analysis was performed between April and August 2014. P < .05 was considered statistically significant.

Exposures  Comprehensive, multifaceted suicide prevention programs, including gatekeeper training, education and mental health awareness programs, screening activities, improved community partnerships and linkages to service, programs for suicide survivors, and crisis hotlines.

Main Outcomes and Measures  Suicide attempt rates for each county following implementation of the GLS program for youths 16 to 23 years of age at the time the program activities were implemented. We obtained this information from the National Survey on Drug Use and Health administered to a large national probabilistic sample between 2008 and 2011.

Results  Counties implementing GLS program activities had significantly lower suicide attempt rates among youths 16 to 23 years of age in the year following implementation of the GLS program than did similar counties that did not implement GLS program activities (4.9 fewer attempts per 1000 youths [95% CI, 1.8-8.0 fewer attempts per 1000 youths]; P = .003). More than 79 000 suicide attempts may have been averted during the period studied following implementation of the GLS program. There was no significant difference in suicide attempt rates among individuals older than 23 years during that same period. There was no evidence of longer-term differences in suicide attempt rates.

Conclusions and Relevance  Comprehensive GLS program activities were associated with a reduction in suicide attempt rates. Sustained suicide prevention programming efforts may be needed to maintain the reduction in suicide attempt rates.

Suicide prevention is a major public health priority, as recognized by the Prioritized Research Agenda for Suicide Prevention,¹ the Revised National Strategy for Suicide Prevention,² the formation in 2010 of the National Action Alliance for Suicide Prevention, and continued funding of the Garrett Lee Smith Memorial Suicide Prevention Program (hereafter referred to as the GLS program).³ Since 2005, the GLS program has funded competitive grants for suicide prevention activities that are awarded to states, tribal communities, and college campuses throughout the United States.⁴ Nonetheless, until recently, there have been few data documenting the effectiveness of the GLS program in reducing the number of deaths by suicide and the number of nonlethal suicide attempts.⁵

Suicide prevention programs supported by the GLS program generally have been comprehensive in scope and multifaceted. Their activities have included education and mental health awareness programs, screening activities, gatekeeper training events, improved community partnerships and linkages to service, programs for suicide survivors, and crisis hotlines. Because of this diversity of activities, the specific mechanisms of action may differ across sites but likely include some combination of increased awareness of suicide and suicide prevention resources, increased identification and referral of youths at risk, and greater availability of support for these youths. As recently reported in Walrath et al,⁶ these suicide prevention programs have been found to be associated with a reduction in suicide mortality among young people in communities where such programs were implemented, compared with communities with similar background characteristics. In the same study⁶ and lending greater confidence to the primary finding, reductions in suicide mortality were not found among older age groups not targeted by suicide prevention efforts, and GLS program activities did not result in reductions in mortality for reasons other than suicide among young people in the affected communities.

Although these suicide mortality findings are encouraging, suicide attempts among young people occur at much higher frequency than deaths by suicide. For example, according to data from the Centers for Disease Control and Prevention Youth Risk Behavior Survey administered in secondary schools in 2013, 7.5% of 11th-grade students and 6.2% of 12th-grade students reported that they made a suicide attempt within the last year.^7(p76) Moreover, costs for emergency department visits and hospitalizations associated with self-injury among youths and young adults through the age of 23 years in 2010 were estimated to be $238 million and $2.6 billion, respectively.⁸ These costs are likely underestimates because they do not include costs associated with suicidal behavior that does not result in emergency department visits or hospitalizations. Suicide attempts can result in significant injury, and previous suicide attempts are one of the best predictors of future nonlethal suicide attempts, as well as deaths by suicide.^9-11 Nonetheless, very few data have been published to indicate that community-based suicide prevention efforts, such as those funded through the GLS program, have affected rates of nonlethal suicide attempts.

A suicide prevention activity common to virtually all of the state and tribal suicide prevention programs is gatekeeper training, with an average of 32% of the expenditures for GLS programs for gatekeeper training.⁶ In gatekeeper training events, individuals (known as gatekeepers) who may have contact with others who are suicidal are trained to better recognize risk for suicide, inquire about risk, intervene appropriately, and/or help the suicidal individual obtain needed assistance.¹² Because of their ubiquity in the suicide prevention programs funded by the GLS program, gatekeeper training events can be used as an indicator of local implementation of the GLS program.

As a complement to our earlier report regarding the effects of the GLS program’s suicide prevention activities on suicide mortality among young people, the main purpose of the present investigation was to examine the effects of this same program, operationalized as communities with gatekeeper training funded by the GLS program, on rates of nonlethal suicide attempts. Specifically, we tested the hypothesis that communities that implemented the GLS program would show greater reductions in rates of suicide attempts compared with communities with similar characteristics that did not implement the GLS program. Unlike the prior mortality analysis and because of limits in the availability of information, we focused on suicide attempts among older youths within the age range targeted by the GLS program (10-24 years): those who were between 16 and 23 years of age during the implementation. These effects were expected to be specific to this group and were not expected to be evident in older populations. Given our recent finding that reductions in suicide mortality were evident in the year following GLS program activities but were not found to persist beyond the 1-year period after GLS program activities were discontinued, the durability of any beneficial effects of the GLS program on suicide attempts was also examined.

A total of 466 counties exposed to the suicide prevention efforts of the GLS program at some point between 2006 and 2009 (intervention counties) and a total of 1161 counties that shared key characteristics but were not exposed to these suicide prevention efforts (control counties) were included in the sample. The present analysis is based on information from all the counties in this sample that also participated in the National Survey on Drug Use and Health (NSDUH) any time between 2008 and 2011. Approximately 80% of the intervention counties and approximately 96% of the control counties were represented in the NSDUH at least 1 year during that period, and approximately 73% of the total number of county-years were represented. Sample selection procedures are shown in Figure 1 and further described in the eAppendix in the Supplement. The present study is based on deidentified secondary data and was determined to be exempt from institutional review board approval by ICF International. The unweighted numbers of respondents in the intervention and control counties were rounded to the nearest thousand due to NSDUH disclosure restrictions. P < .05 was considered statistically significant.

The primary outcome was the suicide attempt rate for each county following the implementation of the GLS program for the population that was approximately 16 to 23 years of age during implementation. Suicide attempt rates among adults 24 year of age or older were analyzed as a control outcome and were not expected to be affected by GLS program activities. Suicide attempt rates are based on self-report measures collected in the NSDUH between 2008 and 2011.¹³ We examined outcome data corresponding to the years following implementation of the GLS program.

Beginning in 2008, NSDUH respondents 18 years of age or older were asked whether they had thought seriously about trying to kill themselves at any time during the past 12 months. Those respondents who reported having had serious suicidal thoughts were then asked whether, in the past 12 months, they had tried to kill themselves. While similar survey questions that were administered among high school students demonstrated good test-retest reliability,¹⁴ no reliability or validity information is available in relation to these NSDUH respondents. For our study, we focused on NSDUH suicide attempt rates by county in the first and subsequent year following implementation of the GLS program (ie, operationalized as a gatekeeper training event).

The NSDUH was based on a large probabilistic sample design to support national and state-level estimations. While a large number of counties are included in the sample each year, county identifiers are not publicly released but are accessible through the “restricted use” data set for research purposes, and, as such, analyses operate under a controlled access model.

While GLS program grants are received at the campus, state, and tribal levels, campus grantees were not included in the analysis. Because the grantee determines the communities and geographic regions to focus on for suicide prevention programming, as well as the pace at which the program is rolled out across these areas, not all communities within a funded state or tribal territory implement suicide prevention programming, and the implementation does not necessarily occur simultaneously. For the primary analyses, the implementation of at least 1 gatekeeper training event funded by the GLS program was used as an indicator of implementation of the GLS program in the county during that year. Subsequent analysis focused on the number of gatekeepers trained as an indication of the intensity of GLS program activities. To examine the effect beyond the year after implementation, indicators of any training implementation in prior years were also included in the analysis. Information on each training event, such as location of the training and number of participants, has been systematically collected since the inception of the GLS program.

A set of county-level covariates was used for sample selection and weighting prior to the main analysis. Covariates included the county’s total population, age-group composition, racial/ethnic composition (percentages of Hispanic and non-Hispanic white residents, African American residents, American Indian or Alaskan Native residents, Asian residents, and residents of other races), percentage of female residents, median household income, poverty rate, unemployment rate, and percentage of rural residents. Although suicide attempt rates prior to initiating the GLS program were not available (the NSDUH began collecting this information in 2008), preintervention suicide mortality rates were also included as covariates. Both the average covariate value between 2000 and 2006 (time-fixed covariates) and the value in each of the 4 years preceding the implementation of the GLS program (time-varying covariates) were considered. The source of the demographic information was the US Census Bureau of Statistics’ intercensal estimates.¹⁵ Income, poverty, and unemployment rates were based on small area estimations by the US Census Bureau of Statistics¹⁶ and the Bureau of Labor Statistics.¹⁷ The analysis further included respondent-level basic demographic characteristics from the NSDUH, such as sex, age, and race/ethnicity (Hispanic and non-Hispanic white, African American, American Indian or Alaskan Native, Asian, and other race) to adjust each county’s suicide attempt rate.

The main goal of the analysis was to examine the effects of the implementation of the GLS program on rates of suicide attempts among youths, providing an extension of earlier findings regarding the effects of the GLS program on deaths by suicide among youths. The analytic strategy consisted of multiple steps and was very similar to that used in Walrath et al.⁶ Initially, we used propensity score–based techniques to help us select comparison counties that were similar in characteristics to the intervention counties (trimming). To further reduce variability, propensity score–based techniques were used to identify 5 subgroups of intervention and comparison counties that were relatively homogeneous in characteristics other than the intervention (subclassification). As a final step, these characteristics were incorporated as weights in analyses examining differences in suicide attempt rates between intervention counties and control counties (weighting). Additional technical information regarding the propensity score–based techniques and associated steps are described in the eAppendix in the Supplement.

The unweighted rounded numbers of respondents used in this analysis were 84 000 in the control group and 57 000 in the intervention group. In Table 1, we present the characteristics of the intervention and control counties before implementation of the GLS program as estimated using the NSDUH between 2004 and 2006, prior to and after trimming/subclassification. Even though these procedures were based on county-level variables from sources other than the NSDUH, they succeeded in balancing these characteristics. Specifically, mean levels were not significantly different between intervention and control counties for any of the variables. The characteristics of the intervention and control samples for all 5 subclasses are reported in eTable 1 in the Supplement.

Table 2 and Figure 2 present the results of the models testing the estimated average effect of implementation of the GLS program. Counties implementing GLS program activities (operationalized as a gatekeeper training event) exhibited significantly lower suicide attempt rates the year following implementation among youths and young adults 16 to 23 years of age when compared with similar counties that did not implement GLS program activities (4.9 fewer attempts per 1000 youths [95% CI, 1.8-8.0 fewer attempts per 1000 youths]; P = .003). Simultaneously, there was no significant difference in the rates of suicide attempts among adults older than 23 years (P = .53). When testing the sensitivity of the results to the influence of extreme (inverse probability of exposure) weights, the results appeared fairly robust (eTable 2 in the Supplement). There was no significant effect on the suicide attempt rates among the population of interest 2 or more years after implementation of the GLS program (ie, operationalized as a gatekeeper training event).

In Table 3, we present the results of the model of the average effect of the number of GLS program trainees on suicide attempt rates. Table 3 includes 2 sets of estimates corresponding with the truncation of 1% and 5% of the most extreme weights (inverse probability of exposure weights). It was not feasible to obtain estimates for the model using untruncated weights. The first set of estimates reflected a positive association between the number of gatekeepers trained and the decrease in suicide rates among youths 16 to 23 years of age in the subsequent year (P = .03). However, it also indicated a trend toward an increase in adult suicide rates (P = .05). Neither result was confirmed when 5% of the most extreme weights were truncated (P = .05 and .19, respectively). Therefore, we concluded that the findings regarding the effect of the number of GLS program trainees were not reliable and should be interpreted with great caution.

The estimates of the effect of the implementation of the GLS program on suicide attempts among youth for each of the 5 subclasses of counties with homogeneous propensity scores are presented in eTable 3 in the Supplement. Results suggest that the effects of the implementation of the GLS program may have been heterogeneous across subclasses. In particular, the point estimate of the effect in 2 subclasses is considerably higher. The counties in these 2 subclasses were relatively smaller and more rural (subclasses 2 and 3), although they were not the smallest counties in the sample (that would be subclass 1). Separate estimates of the average effect of the implementation of the GLS program by age subgroup (16-19 years and 20-23 years) are presented in eTable 4 in the Supplement. While point estimates suggest a somewhat larger effect on the older age group, the 95% CIs around those estimates would largely overlap.

These results indicate a decrease in the rates of suicide attempts among youths 16 to 23 years of age following implementation of the GLS program funded by the Substance Abuse and Mental Health Services Administration. This finding is consistent with the previously reported reduction in suicide mortality among youths 10 to 24 years of age following implementation of the GLS program.⁶ Although training may have beneficial effects for adult participants and for others in their networks and communities through diffusion of information,¹⁸ the absence of a change in rates of suicide attempts among adults 24 years of age or older further supports the hypothesis that these changes are due to GLS program activities. The estimated difference in the attempt rates represents 39% of the estimated average rate in the absence of the intervention. Although causality cannot be definitively inferred from our study owing to a lack of random assignment, these results suggest that more than 79 000 attempts were avoided between 2008 and 2011 following implementation of the GLS program (based on the point estimate of the decrease in the yearly rate, the number of counties and of years of implementation, and the number of youths in those counties during that period). The findings are consistent with the results of other work documenting the utility of comprehensive approaches to suicide prevention, including findings from interventions for members of the US Air Force¹⁹ and for Native Americans.²⁰

Similar to the previous finding regarding suicide mortality, we also found that the positive effects of the GLS program on suicide attempt rates were not sustained 1 year after implementation of the program activities. Many factors not directly addressed by the present study likely contribute to these results. Assessments of comprehensive suicide prevention programs in other contexts have suggested that adherence and focus on comprehensive suicide prevention activities may fade over time.²¹ The present results do not suggest that the GLS program activities were effective only once but, rather, that effectively preventing suicides requires continued implementation of program activities.

The findings of the present analysis need to be interpreted in the context of existing limitations. First, causal claims outside the context of the ideal randomized experiment are intrinsically more tentative. In particular, despite the use of an extensive set of covariates, as well as the analysis of control outcomes, there could be unaccounted differences between intervention and control counties that are influencing the results. For example, it is plausible that the location and timing of implementation may have been influenced by the level of readiness of the local child-serving agencies and administrative entities to participate in the GLS program. In such scenarios, the estimated effect may overstate the potential effect of implementation. Second, and in contrast to the mortality analysis, information on attempts was only available for a segment of the target population, and, therefore, we did not examine the effect on the younger age groups. Also, some counties in the original sample were not present in the NSDUH. While the documented similarities between the included intervention and control counties suggest that those exclusions may not have affected the internal validity of the results, the extent to which the results may be generalized to the excluded counties is uncertain. In addition, the NSDUH data are based on self-reports, not on observations of actual incidents, and the extent of underreporting or overreporting of behaviors has not been determined. Furthermore, the data on lifetime history and number of suicide attempts were not available, and as such it was not possible to determine whether the GLS program differentially affected youths with different histories of suicidal behavior. Finally, the findings from the present analysis do not shed light on which aspects of the GLS program may be the most effective. Strategies to further disentangle the effect of these important variations are currently being explored and will be the focus of future research.

Despite the limitations already noted, our study contributes to the evidence base regarding the effectiveness of comprehensive suicide prevention programs in reducing the number of deaths by suicide and the number of nonlethal suicide attempts. These findings have significant implications for public health policy—specifically, suicide prevention programs and the ways to reduce morbidity and mortality associated with suicidal behaviors—and suggest that community-based programs (such as the GLS program) provide a pathway toward fewer suicide attempts and deaths.

Submitted for Publication: June 2, 2015; final revision received August 10, 2015; accepted August 23, 2015.

Corresponding Author: Christine Walrath, PhD, Public Health Division, ICF International, 40 Wall St, Ste 3400, New York, NY 10005 (christine.walrath@icfi.com).

Published Online: October 14, 2015. doi:10.1001/jamapsychiatry.2015.1933.

Author Contributions: Mr Godoy Garraza had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Godoy Garraza, Walrath, McKeon.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Godoy Garraza.

Obtained funding: McKeon.

Administrative, technical, or material support: Walrath, Reid, McKeon.

Study supervision: Walrath, McKeon.

Conflict of Interest Disclosures: Dr McKeon is employed by the Substance Abuse and Mental Health Services Administration. No other disclosures were reported.

Funding/Support: The cross-site evaluation was supported through a Substance Abuse and Mental Health Services Administration contract to ICF Macro (grant 280-03-1606).

Role of the Funder/Sponsor: The Substance Abuse and Mental Health Services Administration had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge the GLS program grantees for their participation in the cross-site evaluation and the ICF International staff on the evaluation team, particularly Erin Maher, MPH, Anne Marie Schipani, MPH, Hope Sommerfeldt, MA, and Ye Xu, MA, MS, for their contributions to the discussion group for this analysis. Contributors were not compensated beyond what they received as part of their stated roles (GLS program grantees and ICF International staff).