Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis

Key Points

Question  Can we develop accurate prediction models for future social and occupational disability in individuals in clinical high-risk states of psychosis or with recent-onset depression?

Findings  Machine-learning prediction models trained on functional, neuroimaging, and combined baseline data correctly determined social outcomes at 1 year in up to 83% of patients in clinical high-risk states and 70% of patients with recent-onset depression across geographically distinct populations but could not accurately determine role-functioning outcomes. Models outperformed human prognostication and provided a prognostic proxy for broader psychiatric morbidity in patients in clinical high-risk states for psychosis.

Meaning  If further validated, these predictive models could inform the personalized prevention of functional impairment in patients in clinical high-risk states and patients with recent-onset depression.

Importance  Social and occupational impairments contribute to the burden of psychosis and depression. There is a need for risk stratification tools to inform personalized functional-disability preventive strategies for individuals in at-risk and early phases of these illnesses.

Objective  To determine whether predictors associated with social and role functioning can be identified in patients in clinical high-risk (CHR) states for psychosis or with recent-onset depression (ROD) using clinical, imaging-based, and combined machine learning; assess the geographic, transdiagnostic, and prognostic generalizability of machine learning and compare it with human prognostication; and explore sequential prognosis encompassing clinical and combined machine learning.

Design, Setting, and Participants  This multisite naturalistic study followed up patients in CHR states, with ROD, and with recent-onset psychosis, and healthy control participants for 18 months in 7 academic early-recognition services in 5 European countries. Participants were recruited between February 2014 and May 2016, and data were analyzed from April 2017 to January 2018.

Main Outcomes and Measures  Performance and generalizability of prognostic models.

Results  A total of 116 individuals in CHR states (mean [SD] age, 24.0 [5.1] years; 58 [50.0%] female) and 120 patients with ROD (mean [SD] age, 26.1 [6.1] years; 65 [54.2%] female) were followed up for a mean (SD) of 329 (142) days. Machine learning predicted the 1-year social-functioning outcomes with a balanced accuracy of 76.9% of patients in CHR states and 66.2% of patients with ROD using clinical baseline data. Balanced accuracy in models using structural neuroimaging was 76.2% in patients in CHR states and 65.0% in patients with ROD, and in combined models, it was 82.7% for CHR states and 70.3% for ROD. Lower functioning before study entry was a transdiagnostic predictor. Medial prefrontal and temporo-parieto-occipital gray matter volume (GMV) reductions and cerebellar and dorsolateral prefrontal GMV increments had predictive value in the CHR group; reduced mediotemporal and increased prefrontal-perisylvian GMV had predictive value in patients with ROD. Poor prognoses were associated with increased risk of psychotic, depressive, and anxiety disorders at follow-up in patients in the CHR state but not ones with ROD. Machine learning outperformed expert prognostication. Adding neuroimaging machine learning to clinical machine learning provided a 1.9-fold increase of prognostic certainty in uncertain cases of patients in CHR states, and a 10.5-fold increase of prognostic certainty for patients with ROD.

Conclusions and Relevance  Precision medicine tools could augment effective therapeutic strategies aiming at the prevention of social functioning impairments in patients with CHR states or with ROD.

Recent research has extended the scope of early recognition and prevention of psychosis beyond disease transition to poor outcomes more broadly.^1-5 This is because the clinical high-risk (CHR) state for psychosis, present in 2% to 10% of youth,⁶ may be linked with nonpsychotic morbidity encompassing mood, anxiety, and substance use disorders.³ Moreover, the CHR state frequently entails persistent neurocognitive and functional deficits, which may cause the affected young persons to increasingly lag behind their peers in a critical phase of personal development.^7-9 Prospective research has demonstrated that these deficits, in combination with clinical and sociodemographic risk factors, foreshadow poor clinical outcomes.^9-13 Similarly, adolescents and young adults who experience a first major affective episode are not only at risk for relapse but frequently have persistent functional deficits, depressive symptoms, and reduced quality of life, as is often seen in patients in the CHR state.¹⁴ These shared impairments^6,15 may point to a common neurobiological surrogate predating the disabling outcomes of these conditions. Such a marker of brain pathology could accurately estimate the risk for functional deficits, thus informing risk-adapted preventive interventions in these vulnerable persons.

Previous research suggested that psychosis can be predicted in individual patients in CHR states who are recruited in research contexts^12,16-20 and secondary health care settings²¹ through the use of clinical, neurocognitive, neurophysiological, and magnetic resonance imaging (MRI) data. This precision medicine approach has been further strengthened by machine-learning studies showing that clinical baseline data may be associated with predictors of functional and treatment outcomes in first-episode psychosis and depression across multiple sites.^22-24 Proof-of-concept studies have also suggested that global functioning of the CHR state can be individually approximated by MRI-based models.^25,26 However, the predictability of functional outcomes should be separately assessed for the social and role functioning domains because these may be differentially linked to symptoms, neurocognitive deficits, and adverse outcomes; for instance, disorganization and processing speed may be associated with predictors of social disability (which in turn is associated with transition to psychosis), while motor disturbances and verbal fluency may be linked with occupational disability.^7,12,27,28 Furthermore, predictability should be compared across partly overlapping clinical syndromes, such as the CHR state and major depressive disorder, and benchmarked in large, geographically diverse cohorts of vulnerable persons.²⁹

Although it has been conceptually suggested, the question of whether behavioral and MRI-based data could be efficiently combined within sequential prognostic algorithms to optimize predictive power has yet to be empirically tested.³⁰ The clinical implementation of such algorithms does not only depend on the evidence for their generalizability, but also on the accuracy margin between models and the practices of health care professionals.³¹ Only if a conservative estimate of this margin indicates that clinical reasoning could be enhanced in terms of precision, time, and costs would computer-aided decision support in clinical settings be justified.^31-33 Finally, validated predictors associated with specific outcomes could provide resource allocation tools for existing psychosocial interventions,^34-36 and foster biobehavioral mechanistic research leading to new personalized and preventive treatments for social and occupational disability.³⁷

To develop such prognostic signatures, the Personalized Prognostic Tools for Early Psychosis Management (PRONIA; https://www.pronia.eu/) study is collecting multimodal data from healthy control participants and young patients who meet criteria for the CHR state, recent-onset psychosis, or recent-onset depression (ROD). In this first study, we test the geographic generalizability of functional, neuroanatomical, and combined machine-learning models, tasked with predicting the 1-year social and role functioning of patients in the CHR state and patients with ROD recruited in 5 European countries. We estimate the models’ transdiagnostic transferability and their associations with prognostic generalizability across diagnostic and psychometric outcome domains, compare them with the prognostic estimates provided by clinical raters, and explore sequential prognostic algorithms combing clinical and imaging-based models.

The study methods are detailed in the eMethods in the Supplement. We analyzed patients with CHR or ROD who were recruited using internationally established diagnostic criteria, and for whom baseline MRI and follow-up social and role functioning scores were available between the 3-month and 12-month points of the study (eFigure 1 in the Supplement). Additionally, we matched health control participants individually for site, age, and sex to the participants in the clinical groups (eTable 1 in the Supplement). Recruitment took place at 7 sites in 5 countries: the Departments of Psychiatry of the Ludwig-Maximilian-University in Munich, Bavaria, Germany; University of Cologne in Cologne, North Rhineland–Westphalia, Germany; University of Turku, Turku, Finland; University of Basel, Basel, Switzerland; University of Udine, Udine, Italy; the Institute of Mental Health at University of Birmingham, Birmingham, England; and 4 recruitment hospitals associated with the University of Milan, Milan, Italy (Niguarda, Policlinico, San Paolo, and Villa San Benedetto Menni in Albese con Cassano; eTable 2 in the Supplement). Participants were recruited between February 2014 and May 2016 and followed up using a standarized longitudinal study protocol (eFigure 1 in the Supplement). Based on this protocol, they were examined using a comprehensive clinical, neuropsychological, and neuroimaging protocol (eTables 3 and 4 in the Supplement). Inclusion and exclusion criteria are detailed in the eMethods and eTable 5 of the Supplement. A Consolidated Standards of Reporting Trials flowchart of the patients studied here is provided in eFigure 2 of the Supplement.

All adult participants provided their written informed consent prior to study inclusion. Minor participants (defined at all sites as those younger than 18 years) provided written informed assent and their guardians, written informed consent. The study was registered at the German Clinical Trials Register (DRKS00005042) and approved by the local research ethics committees in each location.

The Global Functioning: Social and Global Functioning: Role scales^7,10 were used to define good vs impaired social and role functioning at a threshold of more than 7 points (good) vs 7 or fewer points (impaired), using the participants’ latest examination within the 3-month to 12-month follow-up period. This cutoff demarcates a mild but already persistent or frequent social-functioning or role-functioning impairment. Regular interrater reliability tests were performed to calibrate the Global Functioning scales across study sites (eTable 6 in the Supplement). The distributions of Global Functioning Social and Role scores at baseline and follow-up are shown in eFigure 3 in the Supplement, and their changes over time were analyzed in eFigure 4 in the Supplement.

Our machine-learning software, NeuroMiner (version 0.998; https://www.pronia.eu/neurominer/), was used to train 3 types of models to predict these outcomes (eFigure 2 in the Supplement). The first used the participants’ 8 baseline global functioning social and role scores (including each patient’s highest lifetime score, highest or lowest score in the past year, and current social and role Global Functioning Scale scores). The second model analyzed gray matter volume (GMV) images. The third model combined the former 2 models’ outputs into a single result.³⁸ The models were geographically validated using nested leave-site-out cross-validation. Further validation analyses assessed the influence of image quality (eFigure 5 in the Supplement), follow-up interval variations (eFigure 6 in the Supplement), site-associated variations (eFigure 7 and eTable 7 in the Supplement), and baseline social functioning variations (eTable 8 in the Supplement). Voxel-based morphometric (VBM) analyses compared the patterns of association of predictors in patient groups with the neuroanatomical variation of the samples from the matched healthy control participants (eFigures 8 and 9 in the Supplement). Details of structural MRI (sMRI) sequence parameters are in eTable 4 in the Supplement.

The reliability of MRI-based and clinical predictions feeding into combined models was measured via a cross-validation ratio profile (CVR = mean(w)/standard error(w), where w is the normalized weight vector of the support-vector machine models generated in the study’s nested leave-site-out cross-validation setup. Normalization was performed using the Euclidean norm of w, defined as s=w/||w||₂.³⁹

Models were compared with each other and with expert raters’ prognostic performance (eTable 9 in the Supplement). The number of raters is indeterminate, because the health care professionals who rated each patient were not recorded in the central database for security reasons.

Furthermore, we tested model transferability between the 2 clinical groups and assessed their prognostic generalizability to outcomes beyond social functioning, including (1) transition to psychosis, (2) mood, anxiety, and substance-associated DSM-IV-TR diagnoses at the follow-up examination 9 months after study entry (eTable 10 in the Supplement), and (3) multivariate patterns⁴⁰ of clinical and functional changes between baseline and follow-up (eFigure 10 in the Supplement). We also explored whether the outcome probability estimates provided by the sMRI-based social functioning predictor could be used for the prediction of the ordinal GF scores (eMethods and eFigure 11 in the Supplement). Finally, we assessed prognostic algorithms sequentially combining clinical and sMRI-based models (eFigures 12 and 13 in the Supplement).

Permutation testing was used to assess the models’ statistical significance, which was defined as a P value less than .05 (further details in the eMethods in the Supplement). The false-detection rate was used to correct the P values of multiple comparisons with respect to descriptive statistics, leave-site-out analyses, and assessments of transdiagnostic generalizability. Descriptive univariate analyses were carried out using SPSS version 23 (IBM). Data analysis was completed from April 2017 to January 2018.

We recruited a total of 116 patients in the CHR state and 120 patients with ROD. In addition, we recruited 176 healthy control participants who were matched to the respective patient groups by age, site, and sex (eTable 1 in the Supplement).

The sociodemographic characteristics of patients in CHR states who had impaired social functioning at follow-up did not differ significantly from those of patients with unimpaired outcomes. However, patients in CHR states with impaired role functioning outcomes showed more eductional problems in terms of educational years repeated (patients with impaired outcomes: mean [SD], 0.5 [1.0] years; patients with unimpaired outcomes: mean [SD], 0.1 [0.3] years; P = .01; Table 1). Compared with patients with ROD who were unimpaired at follow-up, patients with ROD who had impaired outcomes had a younger mean (SD) age (patients with impaired social-functioning outcomes: 24.6 [5.6] years; patients with unimpaired social-functioning outcomes: 28.0 [6.2] years; P = .01; patients with impaired role-functioning outcomes: 24.5 [5.5] years; patients with unimpaired role-functioning outcomes: 28 [6.2] years; P = .01) and fewer years of education (patients with impaired social-functioning outcomes: 14.2 [2.9] years; patients with unimpaired social-functioning outcomes: 16.1 [3.0] years; P = .01; patients with impaired role-functioning outcomes: 14.3 [3.0] years; patients with unimpaired role-functioning outcomes: 15.8 [3.1] years; P = .03). Impaired social functioning at follow-up was associated with a lower likelihood of having a partner at baseline in patients with ROD (impaired social functioning: 29 of 65 [44.6%]; unimpaired social-functioning: 40 of 55 [72.7%]; P = .01).

Across study groups, impaired social-functioning outcomes were associated with lower Global Assessment of Functioning disability scores in the month before study inclusion (patients in CHR states: mean [SD] Global Assessment of Functioning disability scores, impaired at follow-up: 52.5 [11.7]; unimpaired at follow-up: 60.2 [14.9]; P = .02; patients with ROD: mean [SD] scores, impaired at follow-up: 52.5 [13.4]; unimpaired at follow-up: 61.4 [14.6]; P = .01). Furthermore, impaired social functioning at follow-up was associated with reduced social functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.0 [1.3]; unimpaired at follow-up: 7.1 [1.1]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.5]; unimpaired at follow-up: 6.9 [1.0]; P < .001), and reduced global role functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 5.7 [1.4]; unimpaired at follow-up: 6.7 [1.2]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.7]; unimpaired at follow-up: 6.8 [1.4]; P = .01).

These differences extended to the social functioning in the past year (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.7 [1.4]; unimpaired at follow-up: 7.8 [0.7]; P < .001; patients with ROD: impaired at follow-up: 6.9 [1.4]; unimpaired at follow-up: 7.7 [0.8]; P < .001) and lifetime social functioning (patients in CHR states: mean [SD] scores, impaired at follow-up: 7.5 [0.8]; unimpaired at follow-up: 8.3 [0.6]; P < .001; patients with ROD: impaired at follow-up: 7.8 [0.9]; unimpaired at follow-up: 8.4 [0.8]; P < .001).

At the level of psychopathology, we observed transdiagnostic baseline differences between groups with impaired and unimpaired outcomes in social functioning in the attenuated negative symptoms domain, particularly an item in the Standardized Interview for Prodromal Symptoms (SIPS) scale specific to expression of emotions (patients in CHR states: mean [SD] scores, impaired at follow-up: 1.3 [1.6]; unimpaired at follow-up: 0.6 [1.2]; P = .03; patients with ROD: mean [SD] scores, impaired at follow-up: 1.1 [1.4]; unimpaired at follow-up: 0.6 [1.0]; P = .03). Further differences were measured in the Positive and Negative Syndrome Scale (PANSS) total scores at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 52.9 [15.2]; unimpaired at follow-up: 46.7 [10.1]; P = .03; patients with ROD: impaired at follow-up: 49.5 [10.3]; unimpaired at follow-up: 44.0 [9.6]; P = .01), and PANSS negative symptoms scores at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 14.6 [6.6]; unimpaired at follow-up: 9.8 [3.6]; P < .001; patients with ROD: impaired at follow-up: 13.6 [5.2]; unimpaired at follow-up: 11.1 [4.0]; P = .02).

Similar but less pronounced effects were observed in the role functioning outcome analyses of patients who were impaired and unimpaired, with the patients in CHR states who were impaired at follow-up being more affected in the social functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.2 [1.4]; unimpaired at follow-up: 6.9 [1.1]; P = .03; patients with ROD: mean [SD] scores, impaired at follow-up: 6.0 [1.5]; unimpaired at follow-up: 6.8 [1.3]; P = .01), role functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 5.6 [1.2]; unimpaired at follow-up: 6.9 [1.3]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.7]; unimpaired at follow-up: 6.8 [1.5]; P = .03). Significantly more severe baseline negative and PANSS total symptoms were found in the patients in CHR states who showed role functioning impairments at follow-up compared with those who were unimpaired (SIPS assessment of ideational richness: mean [SD] scores, impaired at follow-up: 0.7 [1.5]; unimpaired at follow-up: 0.2 [0.5]; P = .01; SIPS assessment of occupational functioning: impaired at follow-up: 3.3 [1.7]; unimpaired at follow-up: 1.4 [1.7]; P < .001; PANSS total score: impaired at follow-up: 52.9 [14.9]; unimpaired at follow-up: 46.4 [10.1]; P = .03; PANSS negative score: impaired at follow-up: 14.1 [6.8]; unimpaired at follow-up: 10.1 [3.6]; P < 001).

Current mood, anxiety, and substance use diagnoses were prevalent at baseline in the CHR group (33 of 51 patients [64.7%]) and ROD group (42 of 49 [85.7%]) but did not differ between outcome-defined samples (eTable 11 in the Supplement). During the follow-up period, symptomatic diagnoses declined, but less so in the groups who were impaired at follow-up: at least 1 DSM-IV diagnosis was present in 27 of 62 patients in CHR states who were impaired at follow-up (44%) and in 25 of 52 patients with ROD who were impaired at follow-up (48%), compared with 4 of 39 patients in CHR states who were unimpaired at follow-up (10%) and 12 of 48 patients with ROD who were unimpaired at follow-up (25%) (eTable 11 in the Supplement). These effects were driven by current depression and were independent of transition to psychosis (which occurred in 8 patients in CHR states and 2 patients with ROD; eTable 12 in the Supplement). Finally, we observed a significant interaction between study sites and social-functioning outcomes in the patients in CHR states (with the sample size per site of patients in CHR states who were impaired at follow-up varying from 22 of 33 [66%] in Munich to 5 of 6 [83%] in Milan, 4 of 15 [27%] in Basel, 10 of 19 [53%] in Cologne, 3 of 12 [25%] in Birmingham, 14 of 19 [74%] in Turku, and 8 of 12 [67%] in Udine, and the sample size per site of patients in CHR states who were unimpaired at follow-up ranging from 11 [33%] in Munich to 1 [17%] in Milan, 11 [73%] in Basel, 9 [47%] in Cologne, 9 [75%] in Birmingham, 5 [26%] in Turku, and 4 [33%] in Udine; P = .03 for groupwise comparison, Table 1). This interaction motivated additional MRI-based validation analyses, as described in the eMethods, eFigure 7, and eTable 7 in the Supplement.

The models evaluating the patients’ social and role functioning at and before study inclusion estimated the social-functioning outcomes of patients in CHR states with a significant leave-site-out balanced accuracy of 76.9% (sensitivity: 69.7%; specificity: 84.0%; P = .002 after false-detection rate adjustment; Table 2). Balanced accuracy of the group with ROD was 66.1%, with a sensitivity of 63.1% and specificity of 69.1% (P = .049; Table 2).

In the patients in CHR states, a model predicting role functioning scores was also significant and performed at balanced accuracy of 67.7% (sensitivity: 60.9%; specificity: 74.5%; P = .02). Clinical models outperformed the sMRI prediction model in their transdiagnostic transferability (Table 3). The features most useful for the model predicting social functioning outcome scores in the patients with ROD were Global Functioning Scale scores at baseline, reduced Global Functioning Scale scores in the year before study inclusion and reduced Global Functioning Scale scores over the lifetime (Figure 1). In the group in CHR states, the most useful features were a reduced social functioning score in the year before study inclusion and a reduced highest global functioning score over the lifetime (Figure 1).

The sMRI-based prediction models determined social-functioning outcomes in the CHR groups with a balanced accuracy of 76.2% (sensitivity: 80.3%, specificity: 72.0%; P = .002; Table 2). The sMRI-based prediction models determined social functioning in the patients with ROD with a balanced accuracy of 65.0%, a sensitivity of 64.6%, and a specificity of 65.5% (P = .04). The performance of models was not influenced by site effects (eTable 7 and eFigure 7 in the Supplement), follow-up duration (eFigure 6 in the Supplement), or baseline functional differences between outcome classes (eFigure 8 in the Supplement). In contrast with the social functioning domain, sMRI data could not be used to accurately estimate role-functioning outcomes (Table 2).

Divergent neuroanatomical patterns (Figure 2) impeded the sMRI model transfer between study groups (Table 3). In the group in CHR states, social functioning impairments at follow-up were associated with (1) reduced baseline GMV in medial prefrontal, cingulate, orbitofrontal, insular, temporal, parietal, and occipital brain regions, and (2) increased cerebellar, dorsomedial, and dorsolateral prefrontal GMVs (Figure 2A). To understand whether this signature represented a pattern of neuroanatomical abnormality, we compared the prognostic samples in CHR states to the data from matched healthy control participants. The reductions in GMV and increments that were associated with impaired social-functioning outcomes differentiated patients in CHR states who had a poor prognosis (defined as a poor outcome predicted by the model, as distinct from observed impairment at follow-up) from healthy control participants (eFigure 8 in the Supplement).

Different from the group in CHR states, the neuroanatomical pattern associated with social-functioning outcomes in patients with ROD included (1) reduced GMV in the hippocampus, amygdala, inferior temporal cortex, thalamus, and dorsal anterior cingulate cortex, and (2) increased GMV in the medial and lateral prefrontal, orbitofrontal, insular, and superior temporal cortices. These GMV increments, but not the temporolimbic reductions, were encountered again when comparing patients with ROD and poor prognoses with healthy control participants using VBM (eFigure 9 in the Supplement). In contrast, patients with ROD and unimpaired social functioning prognosis showed reduced prefrontal GMV and increased temporolimbic GMV compared with healthy control participants.

Combined models predicting social functioning scores estimated the outcomes of patients in CHR states with a balanced accuracy of 82.7% (sensitivity: 83.2%; specificity: 82.0%; P < .001; Table 2). The same models estimated outcomes of patients with ROD with a balanced accuracy of 70.3% (sensitivity: 76.9%; specificity: 63.6%; P = .01). The combined models’ prognostic summary index (PSI) outperformed the sMRI-based models’ PSI by 12.2% in the CHR group (sMRI model: 52.6%; combined model: 64.8%) and 11.5% in the ROD group (sMRI model: 29.9%; combined model: 41.4%); the prognostic summary index outperformed clinical models by 11.9% in the CHR group (clinical model: 52.9%; combined model: 64.8%) and 9.4% in the ROD group (clinical model: 32.0%; combined model: 41.4%). Furthermore, the sequential social functioning prediction analysis showed that, with increasing clinical model uncertainty (decision scores closer to the support vector machines’ decision boundary), the prognostic summary index of the combined model increased to 82.6% in the group in CHR states and was stable (35%-50%) in patients with ROD (eFigures 12 and 13 in the Supplement). Thus, in ambiguous cases, combined models provided a 1.9-fold prognostic gain for patients in the CHR state and a 10.5-fold prognostic gain for patients with ROD compared with a purely clinical prediction model that evaluated the patients' social and role functioning at baseline and before study inclusion.

Finally, expert raters’ global functioning estimates correctly identified the social functioning outcomes of participants in CHR states with a balanced accuracy of 71.8% (sensitivity: 51.5%, specificity: 92.0%). Expert raters accurately identified the social-functioning outcomes for participants with ROD with a balanced accuracy of 59.6% (sensitivity: 26.6%; specificity: 92.6%). The raters’ estimates correctly identified role-functioning outcomes with a balanced accuracy of 70.4% (sensitivity: 49.3%; specificity: 91.5%) for patients in CHR states and a balanced accuracy of 57.9% (sensitivity: 25.0%; specificity: 90.7%) for patients with ROD. Thus, the raters underestimated the risk of impairment in social and role functioning in patients in CHR states and patients with ROD at follow-up (Table 2). Models outperformed raters in all predictive tasks (Table 2 and eTable 9 in the Supplement) and were not influenced by the patients’ age, sex, or ethnicity (eTable 13 in the Supplement).

In the CHR group, a poor social functioning prognosis as provided by the clinical prediction model was associated with an increased prevalence of DSM-IV-TR diagnoses at follow-up: 13 of 49 patients in CHR states (27.1%) with poor prognoses (eTable 10 in the Supplement) had a major depressive disorder at follow-up, compared with 0% in the group with a good prognosis (χ²₁ = 15.8; P < .001). Hence, in these patients, the clinical prediction model projected a major depressive disorder at follow-up with a balanced accuracy of 79.5% (P < .001; eTable 10 in the Supplement). Similarly, 25 of 49 patients in CHR states (51%) who had poor prognoses had at least 1 DSM-IV-TR mood, anxiety, or substance use disorder, compared with 6 of the 52 patients in CHR states (11.5%) who had good prognoses (χ²₁ = 18.5; P < .001; balanced accuracy, 73.2%; P = .03). The clinical model predicting social functioning significantly generalized to the prediction of these diagnostic outcomes (eTable 10 in the Supplement). However, only the sMRI model consistently assigned a poor social functioning prognosis to the patients who transitioned to psychosis (balanced accuracy, 72.7%; P = .01), resulting in a transition risk of 11.9% for those with poor prognoses (8 of 67) vs 0% in those with prognoses of good outcomes (χ²₁ = 6.28; P = .02). In patients with ROD, social functioning predictions did not generalize to diagnostic outcomes.

Persistent social and role functioning deficits drive the personal and socioeconomic burden of psychotic and mood disorders.⁴¹ Yet mental health care lacks the computational tools that could enable the early recognition of these deficits in help-seeking patients.^42,43 Such prognostic tools may catalyze the development of novel biobehavioral therapies for individualized secondary and tertiary prevention.^4,44-47 In this article, we report on what is to our knowledge the first international effort to develop such tools for the prediction of functional outcomes in young patients at risk for psychosis or recurrent depression.⁴⁸ Via thoroughly cross-validated machine-learning methods, we found that social functioning impairments can be correctly predicted in up to 83% of patients in CHR states and 70% of patients with ROD who were recruited from community-based and hospital-based pathways to care across geographically distinct European populations. These results suggest that the individualized quantification of risk for impaired functional recovery is feasible despite site-associated heterogeneity and even without a prestudy calibration of sMRI procedures. Additionally, our inclusive study protocol running in clinical early recognition services facilitated the prognostic algorithms’ derivation and validation in patients representing some of the real-world diversity of CHR states for psychosis, depression, and comorbid psychiatric conditions. In this realistic test bed of model generalizability, we observed that role functioning was less clearly associated with predictors than social functioning when analyzed functional baseline data was used and was even less so when structural neuroimaging data was used. Role functioning may be more strongly determined by concurrent environmental and clinical factors than social deficits, leading to a greater degree of temporal fluctuation, which in turn potentially mediates differential associations with baseline clinical and neuroanatomical predictors (eFigure 4 in the Supplement).^28,49,50 Future research should therefore assess whether the inclusion of environmental and clinical variables may improve the association with outcomes of role functioning deficits in similar help-seeking populations.

Importantly, we observed that combined models integrating clinical and brain structural data outperformed human clinical raters, suggesting that these models could improve the prognostic process beyond the current level. Interestingly, clinical raters overestimated patients’ social functioning improvement, particularly in the ROD group, in which only 27% of patients who ultimately experienced impaired outcomes were correctly identified. An additional sensitivity analysis showed that raters performed better in patients with ROD when impaired social functioning was categorized at lower cutoff values, suggesting that prognostic reasoning is sensitive to more severe functional deficits at follow-up (eTable 14 in the Supplement). In the group in CHR states, however, raters prognosticated best at the original cutoff levels but were less accurate than our machine learning models. This observation suggests that internal heuristics, which are potentially informed by the study groups’ differing clinical profiles, may influence clinicians’ prognostication. Taken together, rater-based prognostication may considerably overestimate vulnerable patients’ capacity to recover from social functioning deficits. Further research is needed to understand the factors governing functional prognostication in psychiatry to design algorithms that optimally mitigate prognostic bias.

The comparisons of models revealed that combined risk stratification generally outperformed unimodal prediction models. Despite the benefits of combined prediction models, however, our clinical models evaluating only 8 functioning variables would provide a cost-effective first-line strategy for estimating the risk of future social disability similar to the psychosis, depression, and suicide risk calculators recently proposed for secondary and outpatient care.^12,21,23,51 In the group with CHR states, we were able to show that our functional risk calculator not only estimated future social impairment but also broader psychiatric morbidity (eTable 10 in the Supplement). These findings were corroborated by the observation that patients in CHR states with an MRI-based prognosis of impaired social functioning experienced unremitting symptoms, occupational disability, and poor quality of life (eFigure 10 in the Supplement). Thus, risk calculators for social impairment may provide accessible tools for a more generalized psychiatric risk screening for individuals in the CHR state. In contrast, the models specific to patients with ROD did not show a comparable prognostic generalizability. This discrepancy may align with recent findings revealing pronounced social cognition deficits in individuals with psychosis vs individuals with depression,⁵² and a close link between psychiatric morbidity, social functioning, and social cognitive processes in psychosis.^53-55

Because MRI is cost intensive, we explored whether sequential predictive modeling could provide a rationale for the targeted use of structural neuroimaging (eFigures 12 and 13 in the Supplement). We observed pronounced benefits of blending sMRI and functional data in patients with increasingly ambiguous clinical decision scores, suggesting that the cost-benefit ratio of sMRI can be maximized by including it at a later, more elaborated stage of the prognostic workflow. Notably, this may include the assessment of transition risk based on the finding that only the sMRI-based model predicting social functioning showed significant prognostic generalizability in this context (eTable 10 in the Supplement). Taken together, these results may provide a first empirical account of prognostic improvements brought about by sequential multimodal risk assessment.³⁰ Future studies should assess the added value of different data combinations, including neurocognition,^18,56electroencephalography,²⁰ sensor-based activity patterns,³² and language patterns,⁵⁷ as well as multiomics information.⁵⁸

We observed striking differences between the neuroanatomical patterns associated with different social-functioning outcomes in the patients in the CHR state and the patients with ROD. In patients in the CHR state, impaired functioning was associated with GMV alterations that represented a deviation from normal brain variation and that mapped to the salience network,⁵⁹ the perisylvian language-associated system, the default-mode network, and the central executive network.⁶⁰ Similar GMV reductions have been previously described in studies comparing the CHR, first-episode psychosis, and relapsing stages of psychosis with healthy control participants.^61,62 Longitudinal data pointed to altered neurodevelopmental trajectories of these brain systems, suggesting a disturbed process of cortical reorganization in different stages of psychosis development.^63-67 Structural dysmaturation affecting these brain systems may influence the capacity to switch between self-referential thinking, salience attribution, and executive functioning,^68,69 thus disrupting complex processes of social cognition and behavior, and finally predisposing to poor functional outcomes of the CHR state.⁵³

In contrast, impaired social functioning in the group with ROD was associated with extended prefrontal, insular, and lateral temporal GMV increments. This pattern also differentiated patients from healthy control participants. Brain volume reductions focused on the medial temporal lobe, the dorsal anterior cingulate cortex, and temporooccipital cortices. While these abnormalities have been associated with earlier disease onset and poor outcomes of patients with major depression,⁷⁰ findings in prefrontal and temporal areas have remained equivocal; for example, a recent VBM meta-analysis of ROD reported increased insular, thalamic, and temporal brain volumes and decreased dorsolateral prefrontal cortical volumes.⁷¹ However, extended insular and prefrontal volume reductions were particularly found in patients with relapsing depression and early disease onset.^72,73 Together with our findings, this may point to dynamic brain volume changes with potentially insufficient prefrontal compensatory processes in patients with ROD who fail to recover.

These largely nonoverlapping neuroanatomical patterns of impaired social functioning outcome explain the low MRI model transferability between the 2 diagnostic groups. This finding challenges the hypothesis of shared brain surrogates underlying transdiagnostic phenotypes, such as the Systems for Social Processes domain proposed by the National Institute of Mental Health’s Research Domain Criteria.⁷⁴ Alternatively, the recently revealed brain-behavioral heterogeneity of depression⁷⁵ may have impeded our algorithms from detecting a salient and thus generalizable neuroanatomical signature of social functioning in the ROD group and/or reduced the generalizability of the CHR-specific model to an umbrella construct of depression. These alternative hypotheses call for more research using subtyping strategies⁷⁶ to test whether the current neurobiological and outcome-associated heterogeneity of mental disorders can be deconvolved into patient strata with distinct prognostic profiles.

One limitation may be the dichotomization of the global functioning scales and the use of classification models for prediction. Regression requires a sufficient representation of the tails in the target scale to learn a predictive pattern explaining the scale’s full range. Because this requirement was not met by our data (eFigure 3 in the Supplement), we preferred classification to regression models. Notably, based on the high correlation between the patients’ ordinal social functioning follow-up scores and the outcome probability estimates of the combined models asssessing social functioning (Table 2), we tested whether post hoc regression models could successfully map these estimates to the global functioning score range. The low mean average errors of these models (eFigure 11 in the Supplement) suggest that continuous targets could be approximated through soft classification.

In summary, we identified generalizable clinical, imaging-based, and combined prediction models of persistent social functioning impairments in young patients at increased risk for psychosis and recurrent depression. To further elucidate the clinical, environmental, and neurobiological factors that facilitate or limit the transferability of the risk calculators presented here, external and prospective validation is needed in ethnically diverse patient populations recruited at sites beyond the European catchment areas of this study. This is the next important step toward quantifying the feasibility and utility of precision psychiatry approaches for the secondary and tertiary prevention of severe mental illnesses.

Corresponding Author: Nikolaos Koutsouleris, MD, Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Nussbaumstr. 7, D-80336 Munich, Germany (nikolaos.koutsouleris@med.uni-muenchen.de).

Accepted for Publication: June 26, 2018.

Author Contributions: Dr Koutsouleris had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Published Online: September 26, 2018. doi:10.1001/jamapsychiatry.2018.2165

Correction: This article was corrected on April 3, 2019. Dr Paolini’s affiliation was listed incorrectly as the Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany. It was corrected to the Department of Radiology in the same institution.

Concept and design: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Kambeitz, Falkai, Reiser, Riecher-Rossler, Upthegrove, Hietala, Salokangas, Meisenzahl, Wood, Brambilla, Borgwardt.

Acquisition, analysis, or interpretation of data: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Rosen, Ruef, Dwyer, Paolini, Chisholm, Kambeitz, Haidl, Schmidt, Gillam, Schultze-Lutter, Riecher-Rossler, Upthegrove, Hietala, Pantelis, Wood, Beque, Brambilla, Borgwardt.

Drafting of the manuscript: Koutsouleris, Rosen, Dwyer, Kambeitz, Gillam, Schultze-Lutter.

Critical revision of the manuscript for important intellectual content: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Rosen, Ruef, Dwyer, Paolini, Chisholm, Kambeitz, Haidl, Schmidt, Schultze-Lutter, Falkai, Reiser, Riecher-Rossler, Upthegrove, Hietala, Salokangas, Pantelis, Meisenzahl, Wood, Beque, Brambilla, Borgwardt.

Statistical analysis: Koutsouleris, Kambeitz, Upthegrove.

Obtained funding: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Riecher-Rossler, Salokangas, Pantelis, Brambilla, Borgwardt, Wood.

Administrative, technical, or material support: Koutsouleris, Rosen, Ruef, Paolini, Chisholm, Haidl, Schmidt, Gillam, Reiser, Upthegrove, Hietala, Meisenzahl, Beque, Brambilla, Borgwardt.

Supervision: Koutsouleris, Ruhrmann, Rosen, Schultze-Lutter, Falkai, Reiser, Riecher-Rossler, Hietala, Wood, Brambilla, Borgwardt.

Conflict of Interest Disclosures: Dr Koutsouleris received honoraria for talks presented at education meetings organized by Otsuka/Lundbeck. Dr Pantelis participated in advisory boards for Janssen-Cilag, AstraZeneca, Lundbeck, and Servier and received honoraria for talks presented at educational meetings organized by AstraZeneca, Janssen-Cilag, Eli Lilly, Pfizer, Lundbeck, and Shire. Dr Upthegrove received honoraria for talks presented at educational meetings organized by Sunovion. No other disclosures were reported.

Group Information: PRONIA consortium members listed here performed the screening, recruitment, rating, examination, and follow-up of the study participants and were involved in implementing the examination protocols of the study, setting up its information technological infrastructure, and organizing the flow and quality control of the data analyzed in this article between the local study sites and the central study database. Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Bavaria, Germany: Linda Betz, BSc, Anne Erkens, Eva Gussmann, BSc, Shalaila Haas, MSc, Alkomiet Hasan, MD, Claudius Hoff, MD, Ifrah Khanyaree, BSc, Aylin Melo, BSc, Susanna Muckenhuber-Sternbauer, MD, Janis Köhler, Ömer Öztürk, MD, Nora Penzel, MSc, David Popovic, MD, Adrian Rangnick, BSc, Sebastian von Saldern, MD, Rachele Sanfelici, MSc, Moritz Spangemacher, Ana Tupac, MSc, Maria Fernanda Urquijo, MSc, Johanna Weiske, MSc, and Antonia Wosgien. University of Cologne, North Rhineland–Westphalia, Germany: Dennis Hedderich, MD, Karsten Blume, Mauro Seves, MSc, Nathalie Kaiser, MSc, Thorsten Lichtenstein, MD, and Christiane Woopen, MD. Psychiatric University Hospital, University of Basel, Basel, Switzerland: Christina Andreou, MD, PhD, Laura Egloff, PhD, Fabienne Harrisberger, PhD, Claudia Lenz, PhD, Letizia Leanza, MSc, Amatya Mackintosh, MSc, Renata Smieskova, PhD, Erich Studerus, PhD, Anna Walter, MD, and Sonja Widmayer, MSc. Institute of Mental Health, University of Birmingham, Birmingham, United Kingdom: Chris Day, BSc, Sian Lowri Griffiths, PhD, Mariam Iqbal, BSc, Mirabel Pelton, MSc, Pavan Mallikarjun, MBBS, DPM, MRCPsych, PhD, Alexandra Stainton, MSci, and Ashleigh Lin, PhD. Department of Psychiatry, University of Turku, Turku, Finland: Alexander Denissoff, MD, Anu Ellilä, RN, Tiina From, MSc, Markus Heinimaa, MD, PhD, Tuula Ilonen, PhD, Päivi Jalo, RN, Heikki Laurikainen, MD, Maarit Lehtinen, RN, Antti Luutonen, BA, Akseli Mäkela, BA, Janina Paju, MSc, Henri Pesonen, PhD, Reetta-Liina Armio (Säilä), MD, Elina Sormunen, MD, Anna Toivonen, MSc, and Otto Turtonen, MD. General Electric Global Research Inc, Munich, Germany: Ana Beatriz Solana, PhD, Manuela Abraham, MBA, Nicolas Hehn, PhD, and Timo Schirmer, PhD. Workgroup of Paolo Brambilla, MD, PhD, University of Milan, Milan, Italy: Department of Neuroscience and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, University of Milan, Milan, Italy: Carlo Altamura, MD, Marika Belleri, PsychD, Francesca Bottinelli, PsychD, Adele Ferro PsychD, PhD, and Marta Re, PhD. Programma2000, Niguarda Hospital, Milan: Emiliano Monzani, MD, Mauro Percudani, MD, and Maurizio Sberna, MD. San Paolo Hospital, Milan: Armando D’Agostino, MD, and Lorenzo Del Fabro, MD. Villa San Benedetto Menni, Albese con Cassano: Giampaolo Perna, MD, Maria Nobile MD, PhD, and Alessandra Alciati, MD. Workgroup of Paolo Brambilla, University of Udine, Udine, Italy: Department of Medical Area, University of Udine: Matteo Balestrieri, MD, Carolina Bonivento, PsychD, PhD, Giuseppe Cabras, PhD, and Franco Fabbro, MD, PhD. IRCCS Scientific Institute “E. Medea”, Polo FVG, Udine: Marco Garzitto, PsychD, PhD and Sara Piccin, PsychD, PhD.

Funding/Support: PRONIA is a Collaborative Project funded by the European Union under the 7th Framework Programme (grant 602152). Dr Pantelis was supported by National Health and Medical Research Council Senior Principal Research Fellowship (grants 628386 and 1105825) and European Union–National Health and Medical Research Council (grant 1075379).

Role of the Funder/Sponsor: The funding organizations were not involved in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; the preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the Recognition and Prevention Program at the Zucker Hillside Hospital in New York, directed by Barbara Cornblatt, PhD, MBA, for providing the Global Functioning: Social and Role scales. We thank Andrea M. Auther, PhD, Associate Director of Recognition and Prevention Program and coauthor of the Global Functioning scales for overseeing the training and implementation of the scales. They were not compensated for their contributions.

Additional Information: For more information on training or use of the Global Functioning: Social and Role scales, contact Dr Auther at aauther@northwell.edu. This work was supported by the National Institute of Mental Health–funded Harmonization of At Risk Multisite Observational Networks for Youth (HARMONY) project (administrative supplement to grant 3U01MH081928-07S1).