[Skip to Navigation]
Sign In
April 5, 2022

Evaluating Extreme Risk Protection Order Laws: When Is It Premature to Expect Population-Level Effects?

Author Affiliations
  • 1Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, North Carolina
JAMA Netw Open. 2022;5(4):e224909. doi:10.1001/jamanetworkopen.2022.4909

It is easy to describe the scale of gun violence in the US with large numbers and hyperbolic comparisons; more civilians have died from firearm injuries since 1968 than all the soldiers who perished in all our wars since 1775.1,2 But gun violence is also the sum of a million heartbreaking stories—of preventable tragedies, young lives cut short, loved ones left behind. In a huge and diverse nation with more guns than people and a constitutionally protected individual right to bear arms, the complex variety of circumstances and factors driving gun violence poses a daunting challenge to policies and laws designed to prevent it. A new study by Veronica Pear and colleagues3 addresses the question of whether a promising legal tool—a civil restraining order to temporarily remove guns from people behaving dangerously—has had any detectable effect on the population rate of firearm-related injury and mortality in a single jurisdiction, San Diego County, in the years since California’s gun violence restraining order (GVRO) law went into effect in 2016.4

These researchers’ answer to the GVRO-effectiveness question is essentially, “No, but…” Using a sophisticated statistical ecological analysis to compare trends in the rates of firearm injury and death in San Diego County and a synthetic control county, they found no statistically significant association with GVRO implementation. But a telling caveat appears near the end of the article: “Our results could reflect a true absence of association or limitations of our study; further research is needed to determine which of these is the case.” One can only hope this will deter gun industry lobbyists from spinning the study’s results as definitive evidence that “gun laws do not work.” Previous studies from Connecticut5 and Indiana6 that examined individual-level outcomes have reported that similar laws, generically called extreme risk protection orders (ERPOs), indeed saved lives by preventing suicides. What might be different about San Diego County? Or, what is different about this study in particular, and what lessons might it teach for future implementation of ERPO laws and the emerging body of research to evaluate their impact?

We offer 3 broad considerations. First, we must acknowledge that GVROs represent an infrequently implemented intervention to prevent a rare outcome in the population. A challenge for a risk-based firearm policy like the GVRO is that gun injuries and deaths are caused by a relatively small number of people who are not easily distinguishable from others in a large pool of putatively dangerous individuals. A California court may issue a GVRO if it finds a “substantial likelihood” that a person “poses a significant danger in the near future of causing personal injury” to self or others by having access to a firearm.7 But the indicia of likelihood that the court may consider include nonspecific risk factors, such as communicating a suicide threat.4,8 In relative terms, people who disclose that they have a plan for suicide are statistically more likely than otherwise to die from suicide, but at the same time the large majority of them—about 98%—do not, in fact, go on to die from suicide.9

For these reasons, the GVRO legal criteria for gun removal will unavoidably apply to many more people who would not actually harm themselves or others than to those who would. Accordingly, the court must carefully balance risk and rights to ensure that the GVRO policy is targeted effectively, but not too broadly. But this is why a substantial number of GVROs would be needed in order to see a meaningful effect on reduced gun violence at the population level. And herein lies a key limitation of the study by Pear and colleagues: the number of GVROs issued during the observation period was probably too small a fraction of the population at risk to expect to see a meaningful change in the population gun violence rates. To illustrate, by extrapolating from national survey estimates of the prevalence of impulsive angry behavior combined with gun carrying in public,10 or the prevalence of acknowledged suicide planning among people with firearms at home,11,12 we estimate the size of the GVRO-eligible population in San Diego County in the tens of thousands; the number possibly exceeds 50 000, or about 2% of the adult population of 2.6 million, with some overlap between risk for suicidality and assaultive behavior.

By comparison, during the 4 years covered by the Pear study, a total of only 355 GVROs were issued—a number equating to less than half of 1 percent of our theoretical risk pool of suicide-planners in gun-owning households and angry gun-carriers in San Diego County. Moreover, 70% of these GVROs were issued in 2019 alone; fewer than 10 were activated in 2016 and 2017. This is too small a pebble to make much of a ripple in such a big pond.

Second, unobserved contextual factors specific to San Diego County could have affected the findings. The authors note that GVRO implementation in San Diego County was driven mainly by the enthusiasm of the local city attorney, which suggests that GVROs may have been issued primarily to people encountering the criminal legal system. If it were the case that few of the GVROs addressed suicide risk while most were issued in conjunction with criminal charges, we might expect a blunted deterrent effect of GVRO as a legal tool, for 2 reasons. People inclined to commit violent crimes could have been more likely to acquire firearms through illegal markets despite a GVRO, while people inclined to self-injury with a gun—those hypothetically more amenable to deterrence by the intervention—did not get a GVRO.

In line with this observation, the study noted that the GVRO legislation coincided with a decline in the rate of violent crime but not in self-directed violence. By contrast, in Connecticut, individuals receiving ERPOs had a rate of suicide mortality 40 times higher than that of the general population, and the study estimated that 1 suicide death was averted for every 10 to 20 gun removal actions.5 Such targeting may help explain why ERPOs could be associated with suicide risk reduction in 1 location and not another; a policy’s impact depends largely on how it is implemented and who is touched by it. Understanding the intended purpose of the GVRO in individual cases—whether to prevent self- or other-directed violence—is an important consideration for outcome evaluation and should be addressed by future studies.

The centrality of the San Diego city attorney in GVRO implementation also elevates the importance of demographic information related to GVROs, which was not available for this study. For example, individual-level demographic data would help establish whether racial disparities that are common in the criminal legal system are also reflected in the distribution of GVROs. Using the information available to them, the authors conducted stratification analyses to examine the overall association of GVROs with assaults and suicides within specific racial and ethnic groups. However, without knowing how GVROs are distributed across demographic groups, these analyses do not tell a complete story. A stratified analysis of GVROs’ possible association with suicide rates in the White, non-Hispanic population would be uninformative if most of the GVROs in San Diego were issued to Black and Hispanic individuals, and vice versa; lack of implementation would be indistinguishable from ineffectiveness within groups.

Another important contextual consideration for San Diego is veteran status, which was unknown and uncontrolled. The proportion of former military service members in the San Diego County population is 50% higher than that of the rest of the state.13 Approximately 78% of firearm deaths in the county were suicides,14,15 and more than a third of those who used a gun to end their life were veterans; San Diegans who served in the military were approximately 3 times more likely to die of suicide than their counterparts in the general population of the county.16 What proportion of the GVROs were issued for veterans at risk for suicide? Establishing how common and impactful GVROs are among veterans will be an important contribution for future studies.

Third, we offer a word of caution apropos of “statistically significant” results. In studies of population subsamples, statistical significance allows us to understand how robust the study’s results are to random variation in a particular sample. Although measures of statistical significance are frequently applied to whole populations, their purpose is less clear in this context. Alexander17 suggests that statistical significance should only be applied to a whole population if it can be considered representative of a broader “superpopulation” (for example, if San Diego is representative of all of California or the US) or if it is considered representative of future populations. Given the aforementioned unmeasured contextual factors, there is some question whether statistical significance is relevant to this study. In this light, it would be short-sighted to suggest that the authors found no associations. Instead, we would note that the study does show some association with reduction in firearm-related assaults in San Diego County, which might possibly be attributable to GVROs if we assume they were targeted primarily to individuals known to be at high risk for other-directed gun violence. However, the study only includes 2 years of observation with more than a handful of active GVROs in the community. It is likely too early to be making any confident statements about the law’s potential impact. With all of this in mind, we believe that individual-level, rather than ecological comparisons, are preferred for ERPO evaluations.

To summarize, for such a small cluster of GVROs to have prevented a substantial number of firearm deaths in San Diego County—enough to move the needle on an annual rate of about 6 per 100 000 in a county population of over 3 million—an almost surgical precision would have been required in targeting the intervention to the people most likely to shoot themselves or someone else. In the absence of a crystal ball, many more GVROs would have been needed to produce a meaningful reduction in the population rate of gun violence. There are many roads to fatal gun-related outcomes; GVROs impose a roadblock for some, but the easy availability of guns creates alternative routes to the same tragic destination. Other pieces of the policy puzzle of gun violence prevention need to be in place.

Pear and colleagues’ finding of no statistically significant association between GVROs and the rate of firearm-related injury and death in San Diego County’s population is interesting, but it raises as many questions as it answers. Stakeholders in gun violence prevention scholarship and policy—and perhaps especially the funders of such research studies—are understandably eager to see timely and concrete results. But investigators must balance the value of publishing early reports with the downside of releasing less than definitive information. In the continuing story of ERPOs, this article by Pear and colleagues reads a bit like an early chapter published ahead of the whole novel. From this we cannot tell how the story will turn out. It leaves us wanting more.

Back to top
Article Information

Published: April 5, 2022. doi:10.1001/jamanetworkopen.2022.4909

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Swanson JW et al. JAMA Network Open.

Corresponding Author: Jeffrey W. Swanson, PhD, Professor in Psychiatry and Behavioral Sciences, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, 905 W Main St, Durham, NC 27710 (jeffrey.swanson@duke.edu).

Conflict of Interest Disclosures: The authors reported receiving grant support from the Elizabeth K. Dollard Charitable Trust. The authors are also collaborators in an ongoing multisite study of the implementation of extreme risk protection orders, funded by the National Collaborative on Gun Violence Research, which includes Drs Pear and Wintemute. Dr Swanson reported service as founding member of the Consortium for Risk Based Firearm Policy, and currently serves on its Executive Steering Committee. The consortium was influential in developing the Gun Violence Restraining Order (GVRO) law in California, the subject of evaluation in the article about which Swanson and colleagues wrote the current editorial. The GVRO was first described in the consortium’s published recommendations.18 No other disclosures were reported.

Centers for Disease Control and Prevention (CDC). Injury Prevention & Control Data & Statistics: Fatal Injury Data. Web-based Injury Statistics Query and Reporting System database. Reviewed February 10, 2022. Accessed February 23, 2022. https://www.cdc.gov/injury/wisqars/fatal.html
Department of Veterans Affairs. America's Wars. VA Office of Public Affairs. August 2021. Accessed February 23, 2022. https://www.va.gov/opa/publications/factsheets/fs_americas_wars.pdf
Pear  VA, Wintemute  GJ, Jewell  NP, Ahern  J.  Firearm violence following the implementation of California’s gun violence restraining order law.   JAMA Netw Open. 2022;5(4):e224216. doi:10.1001/jamanetworkopen.2022.4216Google Scholar
Giffords Law Center to Prevent Gun Violence. Extreme Risk Protection Orders in California. Giffords Law Center to Prevent Gun Violence website. Updated October 21, 2021. Accessed February 23, 2022. https://giffords.org/lawcenter/state-laws/extreme-risk-protection-orders-in-california/
Swanson  JW, Norko  MA, Lin  H-J,  et al.  Implementation and effectiveness of Connecticut’s risk-based gun removal law: does it prevent suicides.  Law Contemp Probl. 2017;80(2):179-208.
Swanson  JW, Easter  MM, Alanis-Hirsch  K,  et al.  Criminal justice and suicide outcomes with Indiana’s risk-based gun seizure law.   J Am Acad Psychiatry Law. 2019;47(2):188-197.PubMedGoogle Scholar
Cal Penal Code §18150(b) (2014).
Cal Penal Code §18155(b) (2014).
Substance Abuse and Mental Health Services Administration (SAMHSA). Key Substance Use and Mental Health Indicators in the United States: Results from the 2020 National Survey on Drug Use and Health. Published online October 2021. Accessed February 23, 2022. https://www.samhsa.gov/data/
Swanson  JW, Sampson  NA, Petukhova  MV,  et al.  Guns, impulsive angry behavior, and mental disorders: results from the National Comorbidity Survey Replication (NCS-R).   Behav Sci Law. 2015;33(2-3):199-212. doi:10.1002/bsl.2172PubMedGoogle ScholarCrossref
Kravitz-Wirtz  N, Pallin  R, Miller  M, Azrael  D, Wintemute  GJ.  Firearm ownership and acquisition in California: findings from the 2018 California Safety and Well-being Survey.   Inj Prev. 2020;26(6):516-523. doi:10.1136/injuryprev-2019-043372Google ScholarCrossref
Centers for Disease Control and Prevention. Facts About Suicide. Centers for Disease Control and Prevention. Last updated February 24, 2022. Accessed February 23, 2022. https://www.cdc.gov/suicide/facts/index.html
San Diego Association of Regional Governments. Veterans in the San Diego Region. Published online May 2016. Accessed February 23, 2022. https://www.sandag.org/uploads/publicationid/publicationid_2037_20703.pdf
San Diego Association of Regional Governments. Homicides and Other Violent Crimes that Involve the Use of A Firearm in the San Diego Region. Published online April 2021. Accessed February 23, 2022. https://www.sandag.org/uploads/publicationid/publicationid_4769_29035.pdf
San Diego Suicide Prevention Council. An Annual Report on the Status of Suicides & Suicide Prevention Efforts in San Diego 2020. Published online September 2021. Accessed February 23, 2022. https://www.sdchip.org/wp-content/uploads/2021/09/FINAL_2021-SPC-Report-to-the-Community_090121.pdf
Walsh  S. San Diego tops the state for Veteran suicides. KPBS. December 14, 2018. Accessed February 24, 2022. https://www.kpbs.org/news/military/2018/12/14/san-diego-tops-state-veteran-suicides
Alexander  N.  What’s more general than a whole population?   Emerg Themes Epidemiol. 2015;12(11):1-5. doi:10.1186/s12982-015-0029-4PubMedGoogle Scholar
Consortium for Risk-Based Approach for State Policy. Guns, Public Health, and Mental Illness. Published December 2, 2013. Accessed February 24, 2022. http://efsgv.org/wp-content/uploads/2014/10/Final-State-Report.pdf