Development and Validation of a Machine Learning–Based Model of Mortality Risk in First-Episode Psychosis

Key Points Question Is it feasible to develop a machine learning model that can predict mortality risk in first-episode psychosis? Findings This prognostic study developed and validated a machine-learning model using extensive Swedish and Finnish databases, identifying mortality risk in first-episode psychosis. For patients with predicted high risk, only long-acting injectable antipsychotics and mood stabilizers were associated with decreased mortality risk; among those with predicted low risk, oral aripiprazole and risperidone were associated with decreased mortality risk. Meaning If further validated, this model may help to develop personalized interventions to mitigate mortality risk in first-episode psychosis.


Introduction
2][3] Moreover, epidemiologic research underscores the disproportionately high susceptibility of mortality due to suicidal behavior, injuries, and poisoning. 2dividuals with schizophrenia are more likely to succumb to somatic deaths (eg, due to cardiovascular incidents or malignant neoplasms) over the duration of the disorder compared with the general population. 1,4evious epidemiologic research has outlined factors associated with elevated mortality risk in psychotic disorders, such as substance abuse, nonadherence to antipsychotics, and psychotic symptoms. 1,5,6Despite a solid epidemiologic understanding, we lack predictive models to identify premature mortality.Building such a model requires comprehensive, unselected, longitudinal data collection endeavors, enabling a detailed examination of individual risk trajectories across a long follow-up time.A pertinent question is what variables should constitute a model for mortality risk in FEP, given the complexity and heterogeneity of the disorder and its course. 7,8Machine learning (ML) algorithms have emerged as a promising solution for such data mining tasks due to their ability to predict individual patient outcomes by analyzing many predictor variables and their intricate, highdimensional interactions. 9To our knowledge, no previous studies have used ML to predict mortality risk in FEP.
A prognostic tool could help to develop personalized treatment, reducing the mortality gap between FEP and the general population.Although oral antipsychotics remain the predominant firstline pharmacotherapy for FEP in clinical practice, 10 long-acting injectable (LAI) antipsychotics are associated with a 33% mortality risk reduction when juxtaposed with their oral counterparts in schizophrenia. 11One lingering question is whether LAI antipsychotics should be broadly prescribed to all patients with FEP instead of oral agents or whether their use could be optimized by targeting specific patient risk profiles.Current clinical guidelines recommend LAI antipsychotics for instances of nonadherence or when favored by the patient, but they provide no direction regarding further stratification. 12,13re, we developed an ML model for mortality risk in FEP using large, unselected nationwide register databases.5][16] We also examined whether the ML model-based risk groups have differences in associations between the use of specific pharmacotherapies and mortality risk over 15 years after FEP.For these purposes, we developed and validated an ML model using a nationwide Swedish cohort of patients with FEP and externally validated its generalizability using an independent Finnish cohort.

Study Design and Data Acquisition
For this prognostic study, we used 2 nationwide register datasets from Sweden and Finland with identical exclusion criteria (eMethods in Supplement 1).Ethical permission for this research project was granted by the Regional Ethics Board of Stockholm.The Finnish National Institute for Health and

ML Analysis
We trained an ML model to predict mortality risk immediately after a FEP diagnosis.We used a large Computing).We selected XGBoost over other ML algorithms due to its superior performance in analyzing tabular data. 18 then identified the 5 most important variables (ie, top 10%) and retrained a model using these variables in the discovery sample.This model was then applied to the Swedish and Finnish validation samples.The contributions of the variables to the model's predictions are presented as Shapley Additive Explanation (SHAP) values, which are described in more detail elsewhere. 19Briefly, SHAP is a game theoretic approach that helps delineate the directional association (positive or negative) of each variable within the ML model, enhancing insight into the model's decision-making process.We also assessed model calibration using the Brier score and the calibration slope (ie, alignment between predictions and true outcome probabilities).The recalibration model was trained in the discovery sample and applied to the Swedish and Finnish validation samples.We also created Kaplan-Meier survival curves for the predicted groups over the total follow-up.

Pharmacoepidemiologic Analysis
In the discovery sample, we investigated whether the associations of different pharmacotherapies (with antipsychotics as the main exposure) and mortality risk varied between patients predicted to die after FEP diagnosis and those predicted to survive up to 15 years of follow-up after FEP.As supplementary analyses, we also conducted these pharmacoepidemiologic analyses without the ML-based stratification.For these purposes, we conducted between-individual Cox regression analyses using SAS, version 9.4 (SAS Institute Inc), for all available follow-up (Յ15 years).The models were adjusted for time, concomitant pharmacotherapy and medications used to treat substance use disorders (SUDs; Anatomical Therapeutic Chemical [ATC] codes N07BB and N07BC), and sequence of antipsychotics.We investigated the following pharmacotherapies: antipsychotics (ATC code N05A excluding lithium N05AN01), antidepressants (ATC code N06A), mood stabilizers (ATC codes N03AF01, N03AG01, N03AX09, and N05AN01), and benzodiazepines and similar compounds (ATC codes N05BA, N05CD, and N05CF).We obtained the drug usage periods for time-varying exposure by analyzing prescription drug purchases using the prescription drug purchases to drug use periods (PRE2DUP) method, which has been described elsewhere. 20The PRE2DUP method calculates sliding averages of defined daily doses, drug purchase amounts, and individual drug use patterns and also takes into account hospital stays and medicine stockpiling of drugs.
Statistical significance was set at P < .05(2-tailed).Data analyses were completed between December 2022 and December 2023.

Group-Level Sociodemographic and Clinical Differences at Baseline
We gathered data on 24 052 patients with FEP from the Swedish cohort (20 000 in the discovery sample and 4052 in the validation sample) and 1490 patients (in the validation sample) with FEP from the Finnish cohort, as presented in the Table .The Swedish cohort had a mean (SD) age of 29.1

ML Results
The out-of-training classification of mortality within 2 years in the Swedish model discovery sample (n = 20 000) resulted in an area under the receiver operating characteristic curve (AUROC) of 0.71 (95% CI, 0.68-0.74;P < .001),with 59.7% sensitivity and 71.3% specificity (Figure 1A).In this analysis, we included post-FEP variables such as pharmacotherapy within 30 days and early outpatient visits that were used as a proxy for treatment plans at the time of FEP diagnosis because we did not have access to patient records.However, a reanalysis excluding these variables yielded similar performance, with an AUROC of 0.70, 58.3% sensitivity, and 71.4% specificity.Of the different causes of death (eFigure 2 in Supplement 1), the best performance was observed for substance-related deaths, including accidental substance-or drug-related poisonings (80.7% accuracy) and any substance-or drug-related death (73.8% accuracy).Among suicides, the best performance was observed for substance-or drug-related suicides (63.8% accuracy) and the worst for suicide by hanging (42.9% accuracy).Among the 5858 patients predicted to die, 2-year mortality was 5.0% without antipsychotic treatment (n = 1153) vs 3.2% with treatment (n = 4705) (χ 2 = 8.40, P = .004).Among the
Benzodiazepine use was associated with increased mortality risk in both patients predicted to die

Discussion
Using extensive Swedish and Finnish nationwide databases, we thoroughly developed and validated 22 an ML model to predict future mortality risk among patients with FEP.The final model, which is available online, 21 was based on just 5 variables: previous SUD comorbidities, duration of first hospitalization due to FEP, male sex, number of previous somatic hospitalizations, and age.This parsimony, without substantial performance sacrifice, and scalability to assess these variables quickly and cost-effectively is crucial for clinical applicability.The model's discrimination performance (AUROC, 0.70) parallels risk calculators used in other medical disciplines (eg, surgery and cardiology [14][15][16] ), providing robust generalizability and good calibration.After 2 decades following FEP diagnosis, the model's predictions equated to about 30.0% of deaths in the group with high mortality risk, in contrast with less than 8.0% among those predicted to survive.
Our model effectively flagged patients with a high risk of poor outcomes, particularly those with SUD comorbidities, confirming the known negative effects of SUD on recovery from psychosis. 5,23e model reached its utmost predictive accuracy for substance-and drug-related deaths, accurately  Although the model is promising, augmentation with additional modalities could enhance its performance.5][26] However, relying on these more expensive and less accessible modalities is not financially prudent for widespread risk assessments in FEP, even if they may offer enhanced prognostic precision.A pragmatic approach would be to commence screening with a model utilizing readily accessible variables, allocating high-cost methodologies for uncertain cases requiring more refined prognostic accuracy.Previous research indicates the feasibility of achieving cost-efficient workflows by integrating clinician risk estimates with predictions from multimodal ML algorithms. 25Nevertheless, pursuing a flawless model remains elusive in clinical practice, with clinicians already addressing a portion of the mortality risk (eg, through pharmacotherapeutic and psychosocial inventions), thereby affecting the model's perceived predictive performance.In fact, among individuals predicted to die, we observed a substantially higher 2-year mortality rate among those not receiving antipsychotic medication compared with their counterparts taking medication.
In disciplines like oncology, treatment is routinely stratified to individual clinical profiles. 27In this context, we examined the association between the use of pharmacotherapies and mortality in different ML model-based risk groups.We found that among individuals with predicted high mortality risk, the use of LAI antipsychotics and mood stabilizers after FEP diagnosis was associated with decreased mortality risk over the subsequent 15 years.However, these associations were not observed among patients predicted to survive.For the latter group, only the use of oral aripiprazole and oral risperidone was associated with decreased mortality risk during the same period.The effectiveness of LAI antipsychotics, recognized as pivotal in enhancing medication adherence, 28 underscores the importance of ensuring their availability to individuals at heightened mortality risk.
The effectiveness of mood stabilizers might stem from their ability to mitigate impulsivity, a known risk factor for impulsive suicidal behavior and accidents. 29Finally, in both prediction groups, benzodiazepine use was associated with increased mortality risk, which suggests a need for careful consideration when using these medications in FEP.
Conventional FEP treatment protocols may delay optimal treatment response in approximately 23% of individuals with treatment-resistant symptoms present at illness onset, 30,31 due to a lack of refined tools for identifying this subgroup.Without ML-based stratification (eFigure 4 in Supplement 1), a broad recommendation of oral aripiprazole or risperidone might seem logical given the association of these medications with the lowest mortality risk.Nevertheless, our findings suggest that approximately 30.0% of patients with FEP might need an alternative therapeutic approach, because a proportion of this subgroup may have treatment-resistant symptoms from the onset.Early initiation of LAIs and mood stabilizers is likely critical for this high-risk subgroup, whereas oral antipsychotics might be adequate as first-line treatment for the majority of patients.However, it is crucial to acknowledge that some patients with low mortality risk may still require treatment with LAI antipsychotics when adherence is poor.

( 8 . 1 )
years, and more than half (62.1%) were men.A total of 418 Swedish patients died within 2 years (69 due to natural causes and 281 to unnatural causes).There were 27 individuals receiving disability pensions or on sick leave at baseline.The mean (SD) follow-up of Swedish patients was 8.4 (3.9) years.The Finnish cohort had a mean (SD) age of 29.7 (8.0) years, and more than half (61.7%) were men.A total of 31 Finnish patients died within 2 years (3 due to natural causes and 28 to unnatural causes).The mean (SD) follow-up of Finnish patients was 14.2 (4.5) years.The Finnish cohort comprised only those treated with inpatient care, whereas 70.0% of the Swedish cohort were diagnosed in outpatient care.Compared with the Swedish sample, the Finnish cohort included only patients with either schizophrenia or schizoaffective disorder.

Figure 1 .
Figure 1.Receiver Operating Characteristic (ROC) Curves for the Prediction of 2-Year Mortality in First-Episode Psychosis (FEP)

Figure 4 .
Figure 4. Associations of Different Pharmacotherapies and Risk of Death Over the 15-Year Follow-Up

eTable 3 . 4 . 1 . 2 . 3 . 4 .
Differences in the Patterns of Use of Different Oral Antipsychotics and Long-Acting Injectable Antipsychotics (LAIs; Grouped Together) Among Those Predicted to Die and to Survive in the Discovery Sample eTable Association Between Use vs Nonuse of Medications and Risk of Death in Between-Individual Analysis in the Swedish Discovery Sample (n = 20 000) eFigure Flowchart Depicting the Model Development and Validation Analyses in the Present Study eFigure Proportion of Correct Out-of-Training Predictions Among Different Causes of Death in the Discovery Sample eFigure Calibration Plots for the Model Predictions in the 2 Validation Samples eFigure Association of Different Pharmacotherapies and the Risk of Death in the Discovery Sample (n = 20 000) Without Machine Learning-Based Stratification Machine Learning-Based Model of Mortality Risk in First-Episode Psychosis Welfare, the Social Insurance Institution of Finland, and Statistics Finland also granted permission.Because this study was registry based without direct contact with participants, informed consent was not required by Swedish and Finnish legislation.Throughout our study, we followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.The Finnish cohort comprised individuals aged 16 to 45 years who received inpatient care for first-episode schizophrenia (available data for ICD-10 codes F20 and F25) in Finland from 1998 to 2014, and were followed between January 1, 1998, and December 31, 2017.The Finnish cohort was identified from the Hospital Discharge Register maintained by the Finnish Institute for Health and Welfare.Both samples had a 1-year washout period for antipsychotics (ie, no antipsychotic treatment) before diagnosis to ensure the composition of true FEP cases.
A, Patients predicted to die (5858[29.3%]inthewholesample).B, Patients predicted to survive(14 142 [70.7%]).Analyses were stratified based on the out-of-training predictions in the discovery sample (n = 20 000).Antipsychotics were oral except for any long-acting injectable (LAI), which includes all LAI antipsychotics.Polypharmacy refers to concomitant use of 2 or more antipsychotics.HR indicates hazard ratio.identifyingtwo-thirds of substance-induced suicides.Consequently, effective interventions addressing SUD comorbidities are crucial in potentially mitigating the associated mortality risks in this high-risk group.Note, however, that the model failed to identify suicides not induced by substances or drugs.Although the model produced several false-positive predictions for the 2-year follow-up, a closer analysis indicated that these individuals exhibited more hospitalizations for somatic conditions and suicide attempts within this time frame.This underlines the need for a holistic approach in treating individuals at risk, addressing potential health complications and self-harm tendencies even if immediate mortality risk appears overstated, at least for the subsequent 2 years after FEP diagnosis.