[Skip to Navigation]
Original Investigation
March 17, 2021

Accuracy Requirements for Cost-effective Suicide Risk Prediction Among Primary Care Patients in the US

Author Affiliations
  • 1Department of Psychiatry, McLean Hospital, Belmont, Massachusetts
  • 2Department of Psychiatry, Massachusetts General Hospital, Boston
  • 3Department of Psychiatry, Harvard Medical School, Boston, Massachusetts
  • 4Department of Psychology, Harvard University, Cambridge, Massachusetts
  • 5Computational Health Informatics Program, Boston Children’s Hospital, Boston, Massachusetts
  • 6Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
  • 7Psychiatric and Neurodevelopmental Genetics Unit, Massachusetts General Hospital, Boston
JAMA Psychiatry. 2021;78(6):642-650. doi:10.1001/jamapsychiatry.2021.0089
Key Points

Question  How accurate must suicide risk prediction models be to be cost-effective in targeting interventions to high-risk individuals in a US primary care population?

Findings  This economic evaluation found that suicide risk prediction could be cost-effective for targeting a safety planning and telephone call intervention if its specificity was 95% or higher and its sensitivity was 17% or higher, corresponding to a positive predictive value of 1% or greater. For a more expensive cognitive behavioral therapy intervention, the required positive predictive value was 2% or greater.

Meaning  Existing suicide risk prediction models may be accurate enough to be cost-effective in US health care settings.


Importance  Several statistical models for predicting suicide risk have been developed, but how accurate such models must be to warrant implementation in clinical practice is not known.

Objective  To identify threshold values of sensitivity, specificity, and positive predictive value that a suicide risk prediction method must attain to cost-effectively target a suicide risk reduction intervention to high-risk individuals.

Design, Setting, and Participants  This economic evaluation incorporated published data on suicide epidemiology, the health care and societal costs of suicide, and the costs and efficacy of suicide risk reduction interventions into a novel decision analytic model. The model projected suicide-related health economic outcomes over a lifetime horizon among a population of US adults with a primary care physician. Data analysis was performed from September 19, 2019, to July 5, 2020.

Interventions  Two possible interventions were delivered to individuals at high predicted risk: active contact and follow-up (ACF; relative risk of suicide attempt, 0.83; annual health care cost, $96) and cognitive behavioral therapy (CBT; relative risk of suicide attempt, 0.47; annual health care cost, $1088).

Main Outcomes and Measures  Fatal and nonfatal suicide attempts, quality-adjusted life-years (QALYs), health care sector costs and societal costs (in 2016 US dollars), and incremental cost-effectiveness ratios (ICERs) (with ICERs ≤$150 000 per QALY designated cost-effective).

Results  With a specificity of 95% and a sensitivity of 25%, primary care–based suicide risk prediction could reduce suicide death rates by 0.5 per 100 000 person-years (if used to target ACF) or 1.6 per 100 000 person-years (if used to target CBT) from a baseline of 15.3 per 100 000 person-years. To be cost-effective from a health care sector perspective at a specificity of 95%, a risk prediction method would need to have a sensitivity of 17.0% or greater (95% CI, 7.4%-37.3%) if used to target ACF and 35.7% or greater (95% CI, 23.1%-60.3%) if used to target CBT. To achieve cost-effectiveness, ACF required positive predictive values of 0.8% for predicting suicide attempt and 0.07% for predicting suicide death; CBT required values of 1.7% for suicide attempt and 0.2% for suicide death.

Conclusions and Relevance  These findings suggest that with sufficient accuracy, statistical suicide risk prediction models can provide good health economic value in the US. Several existing suicide risk prediction models exceed the accuracy thresholds identified in this analysis and thus may warrant pilot implementation in US health care systems.

Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Cost effectiveness alone cannot justify suicide risk assessment
    Matthew Large |
    Dear Editor

    In a welcome analysis that shifts arguments about suicide risk assessment from suicide prediction to issues of resource allocation, Ross and associates argue that active care and follow-up and cognitive behavioural therapy (CBT) are both cost effective interventions when delivered to higher-risk primary health care patients. The authors reach this conclusion by balancing the expense of risk assessment and the resulting indicated treatments and the costs suicide behaviours among those judged to be high-risk. In one example they conclude that almost half (as prevented by CBT) of one quarter of all potential suicides (as identified by
    risk assessment) might be prevented by providing CBT to 5% of the population identified as higher-risk with no overall cost to the community. [1]

    It has previously been argued that suicide risk assessment can only be justified if an intervention delivered to higher risk people is both inexpensive and benign enough to be tolerated by the vast majority of false positives and when there are also sound reasons for not providing these same interventions to lower risk people, among whom many if not the majority of suicides occur. [2]

    Noting the generous assumptions made by Ross and associates (including that CBT can reduce suicide attempts by 53%) and that the CBT designed to protect against repeated suicide attempts is equally effective in reducing suicides in primary care, the authors have made an economic case for the treatment of approximately 1000 higher risk people in order to prevent one primary care suicide.

    However, authors do not seem to have realistically examined the broader arguments against offering such suicide reducing treatment to lower risk people who included three quarters of all suicides in their models. Healthcare is not usually allocated on the basis of profitability and any measure that could potentially save about 25,000 American lives a year and halve the national suicide rate should be seriously considered as a universal precaution rather than a risk indicated intervention.


    1. Ross EL, Zuromski KL, Reis BY, Nock MK, Kessler RC, Smoller JW.
    Accuracy Requirements for Cost-effective Suicide Risk Prediction Among Primary Care Patients in the US. JAMA Psychiatry 2021 doi:10.1001/jamapsychiatry.2021.0089

    2. Large MDoes the emphasis on risk in psychiatry serve the interests of patients or the public? BMJ 2013 346:f857. doi: 10.1136/bmj.f857.