Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis | Depressive Disorders | JAMA Psychiatry | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Comparison of Functional Baseline and Combined Model Signatures
Comparison of Functional Baseline and Combined Model Signatures

The predictive value of baseline global functioning scores used by models with significant associations with functional outcomes was measured in terms of the variable selection frequency across all the support-vector machine models generated in the nested leave-site-out cross-validation experiment. A value of 1 indicates that all models had retained the given variable during sequential backward feature elimination. Horizontal bar plots show the variable selection profiles of the clinical models making predictions of social functioning scores (A) and role functioning scores (B) in the group in the clinical high-risk (CHR) state and the model trained on social functioning scores from patients with recent-onset depression (ROD) (C), with orange lines at 0.5, which equals 50% of support-vector machine models’ selected given variable. Reliability profiles of the combined social functioning model, trained in the patients in CHR states (D) and patients with ROD (E); and a reliability profile model trained on role functioning scores from patients with ROD (F), with orange lines at a cross-validation ratio of 2, which indicates 95% confidence in the reliable involvement of given variable in the model's decision rule. MRI indicates magnetic resonance imaging.

Figure 2.  Comparison of Predictive Neuroanatomical Baseline Signatures in Patient Groups, Detected by the Structural Magnetic Resonance Imaging–Based Model
Comparison of Predictive Neuroanatomical Baseline Signatures in Patient Groups, Detected by the Structural Magnetic Resonance Imaging–Based Model

The reliability of predictive voxels in significant models was measured via a cross-validation ratio map with a threshold of ± 2, which corresponded to an α level of .05. Color scales indicate increased vs decreased gray matter volume in individuals in the clinical high-risk state or with recent-onset depression who were impaired on follow-up, compared with patients with no impairment on follow-up. The open-source 3-dimensional rendering software MRIcroGL (McCausland Center for Brain Imaging, University of South Carolina; https://www.nitrc.org/projects/mricrogl/) was used to overlay the cross-validation ratio maps on the Montreal Neurological Institute single-participant template and produce 3-dimensional renderings and axial mosaic slices. The cool color scale indicates increased gray matter volume and the warm color scale reduced gray matter volume in individuals in clinical high-risk states or with recent-onsent depression who were impaired on follow-up, compared with patients with no impairment at follow-up.

Table 1.  Study-Associated, Sociodemographic, Physical, Clinical, and Functional Differences at Baseline in Individuals in Clinical High-Risk States and Individuals With Recent-Onset Depression With Impaired vs Unimpaired Social-Functioning and Role-Functioning Outcomes at Follow-up
Study-Associated, Sociodemographic, Physical, Clinical, and Functional Differences at Baseline in Individuals in Clinical High-Risk States and Individuals With Recent-Onset Depression With Impaired vs Unimpaired Social-Functioning and Role-Functioning Outcomes at Follow-up
Table 2.  Leave-Site-Out Classification Performance of Clinical, Imaging-Based, and Combined Machine-Learning Predictors of Global Functioning Social Scales or Global Functioning Role Scale Outcomes in Individuals in a Clinical High-Risk and Individuals With Recent-Onset Depressiona
Leave-Site-Out Classification Performance of Clinical, Imaging-Based, and Combined Machine-Learning Predictors of Global Functioning Social Scales or Global Functioning Role Scale Outcomes in Individuals in a Clinical High-Risk and Individuals With Recent-Onset Depressiona
Table 3.  Transdiagnostic Generalizability of Clinical, Structional Magnetic Resonance Imaging–Based, and Combined Modelsa
Transdiagnostic Generalizability of Clinical, Structional Magnetic Resonance Imaging–Based, and Combined Modelsa
Supplement.

eMethods.

eResults.

eReferences.

eTable 1. Descriptive analysis of the sociodemographic and functioning data of the HC sam-ples matched for site, age and sex to the CHR and ROD groups.

eTable 2. Characteristics of the recruiting institutions in the PRONIA consortium.

eTable 3. Clinical and neurocognitive examinations performed in the CHR, ROD, recent-onset psychosis (ROP), and HC groups during the 18-month follow-up period of the study.

eTable 4. MR scanner systems and structural MRI sequence parameters used at the respec-tive PRONIA sites.

eTable 5. DGPPN S3 Guidelines for the treatment of first-episode psychosis and schizo-phrenia.

eTable 6. Intra-class correlation analysis of the GF:S / GF:R scores generated by the PRO-NIA raters on the test cases.

eTable 7. Comparison of the performance measures obtained from the leave-site-out valida-tion of the original MRI-based GF:S outcome predictor and the predictor trained on the GMV data of age- and sex-matched healthy controls.

eTable 8. Comparison of the performance measures of the MRI-based GF:S outcome pre-dictors and sMRI-based classifiers predicting the baseline GF:S classes of the same CHR or ROD individuals.

eTable 9. Pairwise comparisons of classifier performance using McNemar’s tests.

eTable 10. Prognostic generalization performance of clinical, imaging-based, and combined models trained to predict social functioning outcomes in the CHR and ROD groups.

eTable 11. Prevalence comparisons of the DSM-IV-TR diagnoses in the CHR and ROD samples characterized by impaired vs. good social and role functioning at baseline (T0) and at the T1 follow-up examination.

eTable 12. List of the SCID-IV diagnoses in CHR and ROD patients who developed a psy-chotic disorder during the follow-up period.

eTable 13. Interactions between classification performance and age, sex, and ethnicity.

eTable 14. Assessment of expert raters’ prediction performance as a function of decreasing categorization thresholds applied to the CHR individuals and ROD patients follow-up GF:S scores.

eFigure 1. Observational study design of PRONIA.

eFigure 2. CONSORT Chart (A) and design of the main machine learning analyses per-formed in the multi-site discovery database of PRONIA (B).

eFigure 3. GF:S (left) and GF:R (right) score distributions of the CHR and ROD study group at the baseline and follow-up examinations.

eFigure 4. Histogram analysis of GF:S (blue) and GF:R (orange) changes over time in the CHR (left) and ROD (right) groups.

eFigure 5. Image quality assessment performed in the T1-weighted images of 412 study participants using the quality assessment functionality of the CAT12 toolbox.

eFigure 6. Correlation analyses were conducted to assess whether the prognostic decision scores generated by the clinical and sMRI classifiers were influenced by the follow-up inter-vals in the CHR and ROD groups.

eFigure 7. The predictive signature underlying the original GF:S outcome prediction model (A) was qualitatively compared to the signature produced by the site effects analysis (B).

eFigure 8. Results of voxel-based analysis of variance between 67 CHR persons with pre-dicted impaired (CHRImpaired) GF:S outcome vs. 49 CHR persons with predicted good (CHRGood) GF:S outcomes (A) and 67 CHR persons with predicted impaired GF:S outcome vs. 116 healthy volunteers matched for site, age, and sex to the CHR group (B).

eFigure 9. Results of voxel-based analysis of variance between 120 healthy volunteers and 60 ROD persons with predicted good (RODGood) social functioning outcome (A), 120 healthy volunteers and 60 ROD patients with predicted impaired (RODImpaired) social functioning out-comes, (B) and the ROD patients with predicted good vs. impaired outcomes at follow-up (C).

eFigure 10. Prognostic generalization of the sMRI-based classifiers across 4 psychometric factors detected by Non-Negative Matrix Factorization (NNMF).

eFigure 11. Scatter plots in the upper panel show associations between observed GF:S scores (GF:Sobs) at follow-up and classification probabilities produced by combined clinical-sMRI based GF:S outcome predictor in the CHR (left) and ROD (right) group.

eFigure 12. Analysis of a sequential clinical-combined outcome predictor algorithm in the CHR group.

eFigure 13. Analysis of a clinical-combined outcome predictor algorithm in the ROD group.

1.
Addington  J, Cornblatt  BA, Cadenhead  KS,  et al.  At clinical high risk for psychosis: outcome for nonconverters.  Am J Psychiatry. 2011;168(8):800-805. doi:10.1176/appi.ajp.2011.10081191PubMedGoogle ScholarCrossref
2.
Salokangas  RKR, Nieman  DH, Heinimaa  M,  et al; EPOS group.  Psychosocial outcome in patients at clinical high risk of psychosis: a prospective follow-up.  Soc Psychiatry Psychiatr Epidemiol. 2013;48(2):303-311. doi:10.1007/s00127-012-0545-2PubMedGoogle ScholarCrossref
3.
Lin  A, Wood  SJ, Nelson  B, Beavan  A, McGorry  P, Yung  AR.  Outcomes of nontransitioned cases in a sample at ultra-high risk for psychosis.  Am J Psychiatry. 2015;172(3):249-258. doi:10.1176/appi.ajp.2014.13030418PubMedGoogle ScholarCrossref
4.
McGorry  P, Keshavan  M, Goldstone  S,  et al.  Biomarkers and clinical staging in psychiatry.  World Psychiatry. 2014;13(3):211-223. doi:10.1002/wps.20144PubMedGoogle ScholarCrossref
5.
Schmidt  SJ, Schultze-Lutter  F, Schimmelmann  BG,  et al.  EPA guidance on the early intervention in clinical high risk states of psychoses.  Eur Psychiatry. 2015;30(3):388-404. doi:10.1016/j.eurpsy.2015.01.013PubMedGoogle ScholarCrossref
6.
Fusar-Poli  P, Borgwardt  S, Bechdolf  A,  et al.  The psychosis high-risk state: a comprehensive state-of-the-art review.  JAMA Psychiatry. 2013;70(1):107-120. doi:10.1001/jamapsychiatry.2013.269PubMedGoogle ScholarCrossref
7.
Carrión  RE, McLaughlin  D, Goldberg  TE,  et al.  Prediction of functional outcome in individuals at clinical high risk for psychosis.  JAMA Psychiatry. 2013;70(11):1133-1142. doi:10.1001/jamapsychiatry.2013.1909PubMedGoogle ScholarCrossref
8.
Simon  AE, Velthorst  E, Nieman  DH, Linszen  D, Umbricht  D, de Haan  L.  Ultra high-risk state for psychosis and non-transition: a systematic review.  Schizophr Res. 2011;132(1):8-17. doi:10.1016/j.schres.2011.07.002PubMedGoogle ScholarCrossref
9.
Michel  C, Ruhrmann  S, Schimmelmann  BG, Klosterkötter  J, Schultze-Lutter  F.  Course of clinical high-risk states for psychosis beyond conversion.  Eur Arch Psychiatry Clin Neurosci. 2018;268(1):39-48. doi:10.1007/s00406-016-0764-8PubMedGoogle ScholarCrossref
10.
Cornblatt  BA, Auther  AM, Niendam  T,  et al.  Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia.  Schizophr Bull. 2007;33(3):688-702. doi:10.1093/schbul/sbm029PubMedGoogle ScholarCrossref
11.
Fusar-Poli  P, Byrne  M, Valmaggia  L,  et al; OASIS Team.  Social dysfunction predicts two years clinical outcome in people at ultra high risk for psychosis.  J Psychiatr Res. 2010;44(5):294-301. doi:10.1016/j.jpsychires.2009.08.016PubMedGoogle ScholarCrossref
12.
Cannon  TD, Yu  C, Addington  J,  et al.  An individualized risk calculator for research in prodromal psychosis.  Am J Psychiatry. 2016;173(10):980-988. doi:10.1176/appi.ajp.2016.15070890PubMedGoogle ScholarCrossref
13.
Salokangas  RK, Heinimaa  M, From  T,  et al; EPOS group.  Short-term functional outcome and premorbid adjustment in clinical high-risk patients: results of the EPOS project.  Eur Psychiatry. 2014;29(6):371-380. doi:10.1016/j.eurpsy.2013.10.003PubMedGoogle ScholarCrossref
14.
Bora  E, Harrison  BJ, Yücel  M, Pantelis  C.  Cognitive impairment in euthymic major depressive disorder: a meta-analysis.  Psychol Med. 2013;43(10):2017-2026. doi:10.1017/S0033291712002085PubMedGoogle ScholarCrossref
15.
Fusar-Poli  P, Nelson  B, Valmaggia  L, Yung  AR, McGuire  PK.  Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis.  Schizophr Bull. 2014;40(1):120-131. doi:10.1093/schbul/sbs136PubMedGoogle ScholarCrossref
16.
Bodatsch  M, Ruhrmann  S, Wagner  M,  et al.  Prediction of psychosis by mismatch negativity.  Biol Psychiatry. 2011;69(10):959-966. doi:10.1016/j.biopsych.2010.09.057PubMedGoogle ScholarCrossref
17.
Koutsouleris  N, Riecher-Rössler  A, Meisenzahl  EM,  et al.  Detecting the psychosis prodrome across high-risk populations using neuroanatomical biomarkers.  Schizophr Bull. 2015;41(2):471-482. doi:10.1093/schbul/sbu078PubMedGoogle ScholarCrossref
18.
Michel  C, Ruhrmann  S, Schimmelmann  BG, Klosterkötter  J, Schultze-Lutter  F.  A stratified model for psychosis prediction in clinical practice.  Schizophr Bull. 2014;40(6):1533-1542. doi:10.1093/schbul/sbu025PubMedGoogle ScholarCrossref
19.
Ruhrmann  S, Schultze-Lutter  F, Salokangas  RKR,  et al.  Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study.  Arch Gen Psychiatry. 2010;67(3):241-251. doi:10.1001/archgenpsychiatry.2009.206PubMedGoogle ScholarCrossref
20.
Nieman  DH, Ruhrmann  S, Dragt  S,  et al.  Psychosis prediction: stratification of risk estimation with information-processing and premorbid functioning variables.  Schizophr Bull. 2014;40(6):1482-1490. doi:10.1093/schbul/sbt145PubMedGoogle ScholarCrossref
21.
Fusar-Poli  P, Rutigliano  G, Stahl  D,  et al.  Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis.  JAMA Psychiatry. 2017;74(5):493-500. doi:10.1001/jamapsychiatry.2017.0284PubMedGoogle ScholarCrossref
22.
Chekroud  AM, Zotti  RJ, Shehzad  Z,  et al.  Cross-trial prediction of treatment outcome in depression: a machine learning approach.  Lancet Psychiatry. 2016;3(3):243-250. doi:10.1016/S2215-0366(15)00471-XPubMedGoogle ScholarCrossref
23.
Kessler  RC, van Loo  HM, Wardenaar  KJ,  et al.  Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports.  Mol Psychiatry. 2016;21(10):1366-1371. doi:10.1038/mp.2015.198PubMedGoogle ScholarCrossref
24.
Koutsouleris  N, Kahn  RS, Chekroud  AM,  et al.  Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach.  Lancet Psychiatry. 2016;3(10):935-946. doi:10.1016/S2215-0366(16)30171-7PubMedGoogle ScholarCrossref
25.
Kambeitz-Ilankovic  L, Meisenzahl  EM, Cabral  C,  et al.  Prediction of outcome in the psychosis prodrome using neuroanatomical pattern classification.  Schizophr Res. 2016;173(3):159-165. doi:10.1016/j.schres.2015.03.005PubMedGoogle ScholarCrossref
26.
de Wit  S, Ziermans  TB, Nieuwenhuis  M,  et al.  Individual prediction of long-term outcome in adolescents at ultra-high risk for psychosis: applying machine learning techniques to brain imaging data.  Hum Brain Mapp. 2017;38(2):704-714. doi:10.1002/hbm.23410PubMedGoogle ScholarCrossref
27.
Velthorst  E, Nieman  DH, Linszen  D,  et al.  Disability in people clinically at high risk of psychosis.  Br J Psychiatry. 2010;197(4):278-284. doi:10.1192/bjp.bp.109.075036PubMedGoogle ScholarCrossref
28.
Cornblatt  BA, Carrión  RE, Addington  J,  et al.  Risk factors for psychosis: impaired social and role functioning.  Schizophr Bull. 2012;38(6):1247-1257. doi:10.1093/schbul/sbr136PubMedGoogle ScholarCrossref
29.
Studerus  E, Ramyead  A, Riecher-Rössler  A.  Prediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting.  Psychol Med. 2017;47(7):1163-1178. doi:10.1017/S0033291716003494PubMedGoogle ScholarCrossref
30.
Schmidt  A, Cappucciati  M, Radua  J,  et al.  Improving prognostic accuracy in subjects at clinical high risk for psychosis: systematic review of predictive models and meta-analytical sequential testing simulation.  Schizophr Bull. 2017;43(2):375-388.PubMedGoogle Scholar
31.
Steyerberg  EW, Moons  KGM, van der Windt  DA,  et al; PROGRESS Group.  Prognosis research strategy (PROGRESS) 3: prognostic model research.  PLoS Med. 2013;10(2):e1001381. doi:10.1371/journal.pmed.1001381PubMedGoogle ScholarCrossref
32.
Hahn  T, Nierenberg  AA, Whitfield-Gabrieli  S.  Predictive analytics in mental health: applications, guidelines, challenges and perspectives.  Mol Psychiatry. 2017;22(1):37-43. doi:10.1038/mp.2016.201PubMedGoogle ScholarCrossref
33.
Chekroud  AM, Koutsouleris  N.  The perilous path from publication to practice.  Mol Psychiatry. 2018;23(1):24-25. doi:10.1038/mp.2017.227PubMedGoogle ScholarCrossref
34.
Fowler  D, Hodgekins  J, French  P,  et al.  Social recovery therapy in combination with early intervention services for enhancement of social recovery in patients with first-episode psychosis (SUPEREDEN3): a single-blind, randomised controlled trial.  Lancet Psychiatry. 2018;5(1):41-50. doi:10.1016/S2215-0366(17)30476-5PubMedGoogle ScholarCrossref
35.
De Silva  MJ, Cooper  S, Li  HL, Lund  C, Patel  V.  Effect of psychosocial interventions on social functioning in depression and schizophrenia: meta-analysis.  Br J Psychiatry. 2013;202(4):253-260. doi:10.1192/bjp.bp.112.118018PubMedGoogle ScholarCrossref
36.
McFarlane  WR, Levin  B, Travis  L,  et al.  Clinical and functional outcomes after 2 years in the early detection and intervention for the prevention of psychosis multisite effectiveness trial.  Schizophr Bull. 2015;41(1):30-43. doi:10.1093/schbul/sbu108PubMedGoogle ScholarCrossref
37.
Seidman  LJ, Nordentoft  M.  New targets for prevention of schizophrenia: is it time for interventions in the premorbid phase?  Schizophr Bull. 2015;41(4):795-800. doi:10.1093/schbul/sbv050PubMedGoogle ScholarCrossref
38.
Wolpert  DH.  Stacked generalization.  Neural Netw. 1992;5:241-259. doi:10.1016/S0893-6080(05)80023-1Google ScholarCrossref
39.
Gaonkar  B, T Shinohara  R, Davatzikos  C; Alzheimers Disease Neuroimaging Initiative.  Interpreting support vector machine models for multivariate group wise analysis in neuroimaging.  Med Image Anal. 2015;24(1):190-204. doi:10.1016/j.media.2015.06.008PubMedGoogle ScholarCrossref
40.
Ding  C, Li  T, Peng  W, Park  H. Orthogonal Nonnegative Matrix T-factorizations for Clustering. In: Proceedings of the 12th Association for Computing Machinery's Special Interest Group on Knowledge Discovery and Data Mining International Conference, Philadelphia, PA; August 20-23, 2006; 126-135.
41.
Gustavsson  A, Svensson  M, Jacobi  F,  et al; CDBE2010Study Group.  Cost of disorders of the brain in Europe 2010  [published correction appears in Eur Neuropsychopharmacol. 2012;22(3):237-8].  Eur Neuropsychopharmacol. 2011;21(10):718-779. doi:10.1016/j.euroneuro.2011.08.008PubMedGoogle ScholarCrossref
42.
Huys  QJ, Maia  TV, Frank  MJ.  Computational psychiatry as a bridge from neuroscience to clinical applications.  Nat Neurosci. 2016;19(3):404-413. doi:10.1038/nn.4238PubMedGoogle ScholarCrossref
43.
Paulus  MP, Huys  QJ, Maia  TV.  A roadmap for the development of applied computational psychiatry.  Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):386-392. doi:10.1016/j.bpsc.2016.05.001PubMedGoogle ScholarCrossref
44.
Abi-Dargham  A, Horga  G.  The search for imaging biomarkers in psychiatric disorders.  Nat Med. 2016;22(11):1248-1255. doi:10.1038/nm.4190PubMedGoogle ScholarCrossref
45.
Eack  SM, Newhill  CE, Keshavan  MS.  Cognitive enhancement therapy improves resting-state functional connectivity in early course schizophrenia.  J Soc Social Work Res. 2016;7(2):211-230. doi:10.1086/686538PubMedGoogle ScholarCrossref
46.
Eack  SM, Hogarty  GE, Cho  RY,  et al.  Neuroprotective effects of cognitive enhancement therapy against gray matter loss in early schizophrenia: results from a 2-year randomized controlled trial.  Arch Gen Psychiatry. 2010;67(7):674-682. doi:10.1001/archgenpsychiatry.2010.63PubMedGoogle ScholarCrossref
47.
Keshavan  MS, Eack  SM, Wojtalik  JA,  et al.  A broad cortical reserve accelerates response to cognitive enhancement therapy in early course schizophrenia.  Schizophr Res. 2011;130(1-3):123-129. doi:10.1016/j.schres.2011.05.001PubMedGoogle ScholarCrossref
48.
Burcusa  SL, Iacono  WG.  Risk for recurrence in depression.  Clin Psychol Rev. 2007;27(8):959-985. doi:10.1016/j.cpr.2007.02.005PubMedGoogle ScholarCrossref
49.
Evert  H, Harvey  C, Trauer  T, Herrman  H.  The relationship between social networks and occupational and self-care functioning in people with psychosis.  Soc Psychiatry Psychiatr Epidemiol. 2003;38(4):180-188. doi:10.1007/s00127-003-0617-4PubMedGoogle ScholarCrossref
50.
Lidwall  U, Bill  S, Palmer  E, Olsson Bohlin  C.  Mental disorder sick leave in Sweden: a population study.  Work. 2018;59(2):259-272. doi:10.3233/WOR-172672PubMedGoogle ScholarCrossref
51.
Kessler  RC, Stein  MB, Petukhova  MV,  et al; Army STARRS Collaborators.  Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS).  Mol Psychiatry. 2017;22(4):544-551. doi:10.1038/mp.2016.110PubMedGoogle ScholarCrossref
52.
Cotter  J, Granger  K, Backx  R, Hobbs  M, Looi  CY, Barnett  JH.  Social cognitive dysfunction as a clinical marker: a systematic review of meta-analyses across 30 clinical conditions.  Neurosci Biobehav Rev. 2018;84:92-99. doi:10.1016/j.neubiorev.2017.11.014PubMedGoogle ScholarCrossref
53.
Glenthøj  LB, Fagerlund  B, Hjorthøj  C,  et al.  Social cognition in patients at ultra-high risk for psychosis: what is the relation to social skills and functioning?  Schizophr Res Cogn. 2016;5:21-27. doi:10.1016/j.scog.2016.06.004PubMedGoogle ScholarCrossref
54.
Simons  CJP, Bartels-Velthuis  AA, Pijnenborg  GHM; Genetic Risk and Outcome of Psychosis (GROUP) Investigators.  Cognitive performance and long-term social functioning in psychotic disorder: a three-year follow-up study.  PLoS One. 2016;11(4):e0151299. doi:10.1371/journal.pone.0151299PubMedGoogle ScholarCrossref
55.
Hua  LL, Wilens  TE, Martelon  M, Wong  P, Wozniak  J, Biederman  J.  Psychosocial functioning, familiality, and psychiatric comorbidity in bipolar youth with and without psychotic features.  J Clin Psychiatry. 2011;72(3):397-405. doi:10.4088/JCP.10m06025yelPubMedGoogle ScholarCrossref
56.
Koutsouleris  N, Davatzikos  C, Bottlender  R,  et al.  Early recognition and disease prediction in the at-risk mental states for psychosis using neurocognitive pattern classification.  Schizophr Bull. 2012;38(6):1200-1215. doi:10.1093/schbul/sbr037PubMedGoogle ScholarCrossref
57.
Bedi  G, Carrillo  F, Cecchi  GA,  et al.  Automated analysis of free speech predicts psychosis onset in high-risk youths.  NPJ Schizophr. 2015;1:15030. doi:10.1038/npjschz.2015.30PubMedGoogle ScholarCrossref
58.
Frank  E, Maier  D, Pajula  J,  et al.  Platform for systems medicine research and diagnostic applications in psychotic disorders-The METSY project.  Eur Psychiatry. 2018;50:40-46. doi:10.1016/j.eurpsy.2017.12.001PubMedGoogle ScholarCrossref
59.
Palaniyappan  L, Liddle  PF.  Does the salience network play a cardinal role in psychosis? an emerging hypothesis of insular dysfunction.  J Psychiatry Neurosci. 2012;37(1):17-27. doi:10.1503/jpn.100176PubMedGoogle ScholarCrossref
60.
Wang  H, Zeng  L-L, Chen  Y, Yin  H, Tan  Q, Hu  D.  Evidence of a dissociation pattern in default mode subnetwork functional connectivity in schizophrenia.  Sci Rep. 2015;5:14655. doi:10.1038/srep14655PubMedGoogle ScholarCrossref
61.
Fusar-Poli  P, Radua  J, McGuire  P, Borgwardt  S.  Neuroanatomical maps of psychosis onset: voxel-wise meta-analysis of antipsychotic-naive VBM studies.  Schizophr Bull. 2012;38(6):1297-1307. doi:10.1093/schbul/sbr134PubMedGoogle ScholarCrossref
62.
Wood  SJ, Yung  AR, McGorry  PD, Pantelis  C.  Neuroimaging and treatment evidence for clinical staging in psychotic disorders: from the at-risk mental state to chronic schizophrenia.  Biol Psychiatry. 2011;70(7):619-625. doi:10.1016/j.biopsych.2011.05.034PubMedGoogle ScholarCrossref
63.
Douaud  G, Mackay  C, Andersson  J,  et al.  Schizophrenia delays and alters maturation of the brain in adolescence.  Brain. 2009;132(pt 9):2437-2448. doi:10.1093/brain/awp126PubMedGoogle ScholarCrossref
64.
Falkai  P, Rossner  MJ, Schulze  TG,  et al.  Kraepelin revisited: schizophrenia from degeneration to failed regeneration.  Mol Psychiatry. 2015;20(6):671-676. doi:10.1038/mp.2015.35PubMedGoogle ScholarCrossref
65.
Bartholomeusz  CF, Cropley  VL, Wannan  C, Di Biase  M, McGorry  PD, Pantelis  C.  Structural neuroimaging across early-stage psychosis: Aberrations in neurobiological trajectories and implications for the staging model.  Aust N Z J Psychiatry. 2017;51(5):455-476. doi:10.1177/0004867416670522PubMedGoogle ScholarCrossref
66.
Torres  US, Duran  FL, Schaufelberger  MS,  et al.  Patterns of regional gray matter loss at different stages of schizophrenia: a multisite, cross-sectional VBM study in first-episode and chronic illness.  Neuroimage Clin. 2016;12:1-15. doi:10.1016/j.nicl.2016.06.002PubMedGoogle ScholarCrossref
67.
Palaniyappan  L.  Progressive cortical reorganisation: A framework for investigating structural changes in schizophrenia.  Neurosci Biobehav Rev. 2017;79:1-13. doi:10.1016/j.neubiorev.2017.04.028PubMedGoogle ScholarCrossref
68.
Ryali  S, Supekar  K, Chen  T,  et al.  Temporal dynamics and developmental maturation of salience, default and central-executive network interactions revealed by variational Bayes hidden Markov modeling.  PLoS Comput Biol. 2016;12(12):e1005138. doi:10.1371/journal.pcbi.1005138PubMedGoogle ScholarCrossref
69.
Sherman  LE, Rudie  JD, Pfeifer  JH, Masten  CL, McNealy  K, Dapretto  M.  Development of the default mode and central executive networks across early adolescence: a longitudinal study.  Dev Cogn Neurosci. 2014;10:148-159. doi:10.1016/j.dcn.2014.08.002PubMedGoogle ScholarCrossref
70.
Schmaal  L, Veltman  DJ, van Erp  TG,  et al.  Subcortical brain alterations in major depressive disorder: findings from the ENIGMA Major Depressive Disorder Working Group.  Mol Psychiatry. 2016;21(6):806-812. doi:10.1038/mp.2015.69PubMedGoogle ScholarCrossref
71.
Wang  W, Zhao  Y, Hu  X,  et al.  Conjoint and dissociated structural and functional abnormalities in first-episode drug-naive patients with major depressive disorder: a multimodal meta-analysis.  Sci Rep. 2017;7(1):10401. doi:10.1038/s41598-017-08944-5PubMedGoogle ScholarCrossref
72.
Schmaal  L, Hibar  DP, Sämann  PG,  et al.  Cortical abnormalities in adults and adolescents with major depression based on brain scans from 20 cohorts worldwide in the ENIGMA Major Depressive Disorder Working Group.  Mol Psychiatry. 2017;22(6):900-909. doi:10.1038/mp.2016.60PubMedGoogle ScholarCrossref
73.
Stratmann  M, Konrad  C, Kugel  H,  et al.  Insular and hippocampal gray matter volume reductions in patients with major depressive disorder.  PLoS One. 2014;9(7):e102692. doi:10.1371/journal.pone.0102692PubMedGoogle ScholarCrossref
74.
Insel  T, Cuthbert  B, Garvey  M,  et al.  Research domain criteria (RDoC): toward a new classification framework for research on mental disorders.  Am J Psychiatry. 2010;167(7):748-751. doi:10.1176/appi.ajp.2010.09091379PubMedGoogle ScholarCrossref
75.
Drysdale  AT, Grosenick  L, Downar  J,  et al.  Resting-state connectivity biomarkers define neurophysiological subtypes of depression.  Nat Med. 2017;23(1):28-38. doi:10.1038/nm.4246PubMedGoogle ScholarCrossref
76.
Dwyer  DB, Cabral  C, Kambeitz-Ilankovic  L,  et al.  Brain subtyping enhances the neuroanatomical discrimination of schizophrenia  [published online February 26, 2018].  Schizophr Bull. doi:10.1093/schbul/sby008PubMedGoogle Scholar
Original Investigation
September 26, 2018

Prediction Models of Functional Outcomes for Individuals in the Clinical High-Risk State for Psychosis or With Recent-Onset Depression: A Multimodal, Multisite Machine Learning Analysis

Author Affiliations
  • 1Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany
  • 2Department of Psychiatry and Psychotherapy, University of Cologne, Cologne, Germany
  • 3Institute of Mental Health, University of Birmingham, Birmingham, United Kingdom
  • 4School of Psychology, University of Birmingham, United Kingdom
  • 5Department of Psychiatry, University Psychiatric Clinic, Psychiatric University Hospital, University of Basel, Basel, Switzerland
  • 6Orygen, the National Centre of Excellence for Youth Mental Health, Melbourne, Australia
  • 7Centre for Youth Mental Health, University of Melbourne, Melbourne, Australia
  • 8Department of Psychiatry and Psychotherapy, Medical Faculty, Heinrich-Heine University, Düsseldorf, Germany
  • 9Department of Radiology, Ludwig-Maximilian-University, Munich, Germany
  • 10Department of Psychiatry, University of Turku, Turku, Finland
  • 11Melbourne Neuropsychiatry Centre, University of Melbourne, Melbourne, Australia
  • 12Melbourne Health, Melbourne, Australia
  • 13Corporate Global Research, GE Corporation, Munich, Germany
  • 14Department of Neurosciences and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, University of Milan, Milan, Italy
JAMA Psychiatry. 2018;75(11):1156-1172. doi:10.1001/jamapsychiatry.2018.2165
Key Points

Question  Can we develop accurate prediction models for future social and occupational disability in individuals in clinical high-risk states of psychosis or with recent-onset depression?

Findings  Machine-learning prediction models trained on functional, neuroimaging, and combined baseline data correctly determined social outcomes at 1 year in up to 83% of patients in clinical high-risk states and 70% of patients with recent-onset depression across geographically distinct populations but could not accurately determine role-functioning outcomes. Models outperformed human prognostication and provided a prognostic proxy for broader psychiatric morbidity in patients in clinical high-risk states for psychosis.

Meaning  If further validated, these predictive models could inform the personalized prevention of functional impairment in patients in clinical high-risk states and patients with recent-onset depression.

Abstract

Importance  Social and occupational impairments contribute to the burden of psychosis and depression. There is a need for risk stratification tools to inform personalized functional-disability preventive strategies for individuals in at-risk and early phases of these illnesses.

Objective  To determine whether predictors associated with social and role functioning can be identified in patients in clinical high-risk (CHR) states for psychosis or with recent-onset depression (ROD) using clinical, imaging-based, and combined machine learning; assess the geographic, transdiagnostic, and prognostic generalizability of machine learning and compare it with human prognostication; and explore sequential prognosis encompassing clinical and combined machine learning.

Design, Setting, and Participants  This multisite naturalistic study followed up patients in CHR states, with ROD, and with recent-onset psychosis, and healthy control participants for 18 months in 7 academic early-recognition services in 5 European countries. Participants were recruited between February 2014 and May 2016, and data were analyzed from April 2017 to January 2018.

Main Outcomes and Measures  Performance and generalizability of prognostic models.

Results  A total of 116 individuals in CHR states (mean [SD] age, 24.0 [5.1] years; 58 [50.0%] female) and 120 patients with ROD (mean [SD] age, 26.1 [6.1] years; 65 [54.2%] female) were followed up for a mean (SD) of 329 (142) days. Machine learning predicted the 1-year social-functioning outcomes with a balanced accuracy of 76.9% of patients in CHR states and 66.2% of patients with ROD using clinical baseline data. Balanced accuracy in models using structural neuroimaging was 76.2% in patients in CHR states and 65.0% in patients with ROD, and in combined models, it was 82.7% for CHR states and 70.3% for ROD. Lower functioning before study entry was a transdiagnostic predictor. Medial prefrontal and temporo-parieto-occipital gray matter volume (GMV) reductions and cerebellar and dorsolateral prefrontal GMV increments had predictive value in the CHR group; reduced mediotemporal and increased prefrontal-perisylvian GMV had predictive value in patients with ROD. Poor prognoses were associated with increased risk of psychotic, depressive, and anxiety disorders at follow-up in patients in the CHR state but not ones with ROD. Machine learning outperformed expert prognostication. Adding neuroimaging machine learning to clinical machine learning provided a 1.9-fold increase of prognostic certainty in uncertain cases of patients in CHR states, and a 10.5-fold increase of prognostic certainty for patients with ROD.

Conclusions and Relevance  Precision medicine tools could augment effective therapeutic strategies aiming at the prevention of social functioning impairments in patients with CHR states or with ROD.

Introduction

Recent research has extended the scope of early recognition and prevention of psychosis beyond disease transition to poor outcomes more broadly.1-5 This is because the clinical high-risk (CHR) state for psychosis, present in 2% to 10% of youth,6 may be linked with nonpsychotic morbidity encompassing mood, anxiety, and substance use disorders.3 Moreover, the CHR state frequently entails persistent neurocognitive and functional deficits, which may cause the affected young persons to increasingly lag behind their peers in a critical phase of personal development.7-9 Prospective research has demonstrated that these deficits, in combination with clinical and sociodemographic risk factors, foreshadow poor clinical outcomes.9-13 Similarly, adolescents and young adults who experience a first major affective episode are not only at risk for relapse but frequently have persistent functional deficits, depressive symptoms, and reduced quality of life, as is often seen in patients in the CHR state.14 These shared impairments6,15 may point to a common neurobiological surrogate predating the disabling outcomes of these conditions. Such a marker of brain pathology could accurately estimate the risk for functional deficits, thus informing risk-adapted preventive interventions in these vulnerable persons.

Previous research suggested that psychosis can be predicted in individual patients in CHR states who are recruited in research contexts12,16-20 and secondary health care settings21 through the use of clinical, neurocognitive, neurophysiological, and magnetic resonance imaging (MRI) data. This precision medicine approach has been further strengthened by machine-learning studies showing that clinical baseline data may be associated with predictors of functional and treatment outcomes in first-episode psychosis and depression across multiple sites.22-24 Proof-of-concept studies have also suggested that global functioning of the CHR state can be individually approximated by MRI-based models.25,26 However, the predictability of functional outcomes should be separately assessed for the social and role functioning domains because these may be differentially linked to symptoms, neurocognitive deficits, and adverse outcomes; for instance, disorganization and processing speed may be associated with predictors of social disability (which in turn is associated with transition to psychosis), while motor disturbances and verbal fluency may be linked with occupational disability.7,12,27,28 Furthermore, predictability should be compared across partly overlapping clinical syndromes, such as the CHR state and major depressive disorder, and benchmarked in large, geographically diverse cohorts of vulnerable persons.29

Although it has been conceptually suggested, the question of whether behavioral and MRI-based data could be efficiently combined within sequential prognostic algorithms to optimize predictive power has yet to be empirically tested.30 The clinical implementation of such algorithms does not only depend on the evidence for their generalizability, but also on the accuracy margin between models and the practices of health care professionals.31 Only if a conservative estimate of this margin indicates that clinical reasoning could be enhanced in terms of precision, time, and costs would computer-aided decision support in clinical settings be justified.31-33 Finally, validated predictors associated with specific outcomes could provide resource allocation tools for existing psychosocial interventions,34-36 and foster biobehavioral mechanistic research leading to new personalized and preventive treatments for social and occupational disability.37

To develop such prognostic signatures, the Personalized Prognostic Tools for Early Psychosis Management (PRONIA; https://www.pronia.eu/) study is collecting multimodal data from healthy control participants and young patients who meet criteria for the CHR state, recent-onset psychosis, or recent-onset depression (ROD). In this first study, we test the geographic generalizability of functional, neuroanatomical, and combined machine-learning models, tasked with predicting the 1-year social and role functioning of patients in the CHR state and patients with ROD recruited in 5 European countries. We estimate the models’ transdiagnostic transferability and their associations with prognostic generalizability across diagnostic and psychometric outcome domains, compare them with the prognostic estimates provided by clinical raters, and explore sequential prognostic algorithms combing clinical and imaging-based models.

Methods

The study methods are detailed in the eMethods in the Supplement. We analyzed patients with CHR or ROD who were recruited using internationally established diagnostic criteria, and for whom baseline MRI and follow-up social and role functioning scores were available between the 3-month and 12-month points of the study (eFigure 1 in the Supplement). Additionally, we matched health control participants individually for site, age, and sex to the participants in the clinical groups (eTable 1 in the Supplement). Recruitment took place at 7 sites in 5 countries: the Departments of Psychiatry of the Ludwig-Maximilian-University in Munich, Bavaria, Germany; University of Cologne in Cologne, North Rhineland–Westphalia, Germany; University of Turku, Turku, Finland; University of Basel, Basel, Switzerland; University of Udine, Udine, Italy; the Institute of Mental Health at University of Birmingham, Birmingham, England; and 4 recruitment hospitals associated with the University of Milan, Milan, Italy (Niguarda, Policlinico, San Paolo, and Villa San Benedetto Menni in Albese con Cassano; eTable 2 in the Supplement). Participants were recruited between February 2014 and May 2016 and followed up using a standarized longitudinal study protocol (eFigure 1 in the Supplement). Based on this protocol, they were examined using a comprehensive clinical, neuropsychological, and neuroimaging protocol (eTables 3 and 4 in the Supplement). Inclusion and exclusion criteria are detailed in the eMethods and eTable 5 of the Supplement. A Consolidated Standards of Reporting Trials flowchart of the patients studied here is provided in eFigure 2 of the Supplement.

All adult participants provided their written informed consent prior to study inclusion. Minor participants (defined at all sites as those younger than 18 years) provided written informed assent and their guardians, written informed consent. The study was registered at the German Clinical Trials Register (DRKS00005042) and approved by the local research ethics committees in each location.

The Global Functioning: Social and Global Functioning: Role scales7,10 were used to define good vs impaired social and role functioning at a threshold of more than 7 points (good) vs 7 or fewer points (impaired), using the participants’ latest examination within the 3-month to 12-month follow-up period. This cutoff demarcates a mild but already persistent or frequent social-functioning or role-functioning impairment. Regular interrater reliability tests were performed to calibrate the Global Functioning scales across study sites (eTable 6 in the Supplement). The distributions of Global Functioning Social and Role scores at baseline and follow-up are shown in eFigure 3 in the Supplement, and their changes over time were analyzed in eFigure 4 in the Supplement.

Our machine-learning software, NeuroMiner (version 0.998; https://www.pronia.eu/neurominer/), was used to train 3 types of models to predict these outcomes (eFigure 2 in the Supplement). The first used the participants’ 8 baseline global functioning social and role scores (including each patient’s highest lifetime score, highest or lowest score in the past year, and current social and role Global Functioning Scale scores). The second model analyzed gray matter volume (GMV) images. The third model combined the former 2 models’ outputs into a single result.38 The models were geographically validated using nested leave-site-out cross-validation. Further validation analyses assessed the influence of image quality (eFigure 5 in the Supplement), follow-up interval variations (eFigure 6 in the Supplement), site-associated variations (eFigure 7 and eTable 7 in the Supplement), and baseline social functioning variations (eTable 8 in the Supplement). Voxel-based morphometric (VBM) analyses compared the patterns of association of predictors in patient groups with the neuroanatomical variation of the samples from the matched healthy control participants (eFigures 8 and 9 in the Supplement). Details of structural MRI (sMRI) sequence parameters are in eTable 4 in the Supplement.

The reliability of MRI-based and clinical predictions feeding into combined models was measured via a cross-validation ratio profile (CVR = mean(w)/standard error(w), where w is the normalized weight vector of the support-vector machine models generated in the study’s nested leave-site-out cross-validation setup. Normalization was performed using the Euclidean norm of w, defined as s=w/||w||2.39

Models were compared with each other and with expert raters’ prognostic performance (eTable 9 in the Supplement). The number of raters is indeterminate, because the health care professionals who rated each patient were not recorded in the central database for security reasons.

Furthermore, we tested model transferability between the 2 clinical groups and assessed their prognostic generalizability to outcomes beyond social functioning, including (1) transition to psychosis, (2) mood, anxiety, and substance-associated DSM-IV-TR diagnoses at the follow-up examination 9 months after study entry (eTable 10 in the Supplement), and (3) multivariate patterns40 of clinical and functional changes between baseline and follow-up (eFigure 10 in the Supplement). We also explored whether the outcome probability estimates provided by the sMRI-based social functioning predictor could be used for the prediction of the ordinal GF scores (eMethods and eFigure 11 in the Supplement). Finally, we assessed prognostic algorithms sequentially combining clinical and sMRI-based models (eFigures 12 and 13 in the Supplement).

Permutation testing was used to assess the models’ statistical significance, which was defined as a P value less than .05 (further details in the eMethods in the Supplement). The false-detection rate was used to correct the P values of multiple comparisons with respect to descriptive statistics, leave-site-out analyses, and assessments of transdiagnostic generalizability. Descriptive univariate analyses were carried out using SPSS version 23 (IBM). Data analysis was completed from April 2017 to January 2018.

Results
Group-Level Sociodemographic and Clinical Differences at Baseline

We recruited a total of 116 patients in the CHR state and 120 patients with ROD. In addition, we recruited 176 healthy control participants who were matched to the respective patient groups by age, site, and sex (eTable 1 in the Supplement).

The sociodemographic characteristics of patients in CHR states who had impaired social functioning at follow-up did not differ significantly from those of patients with unimpaired outcomes. However, patients in CHR states with impaired role functioning outcomes showed more eductional problems in terms of educational years repeated (patients with impaired outcomes: mean [SD], 0.5 [1.0] years; patients with unimpaired outcomes: mean [SD], 0.1 [0.3] years; P = .01; Table 1). Compared with patients with ROD who were unimpaired at follow-up, patients with ROD who had impaired outcomes had a younger mean (SD) age (patients with impaired social-functioning outcomes: 24.6 [5.6] years; patients with unimpaired social-functioning outcomes: 28.0 [6.2] years; P = .01; patients with impaired role-functioning outcomes: 24.5 [5.5] years; patients with unimpaired role-functioning outcomes: 28 [6.2] years; P = .01) and fewer years of education (patients with impaired social-functioning outcomes: 14.2 [2.9] years; patients with unimpaired social-functioning outcomes: 16.1 [3.0] years; P = .01; patients with impaired role-functioning outcomes: 14.3 [3.0] years; patients with unimpaired role-functioning outcomes: 15.8 [3.1] years; P = .03). Impaired social functioning at follow-up was associated with a lower likelihood of having a partner at baseline in patients with ROD (impaired social functioning: 29 of 65 [44.6%]; unimpaired social-functioning: 40 of 55 [72.7%]; P = .01).

Across study groups, impaired social-functioning outcomes were associated with lower Global Assessment of Functioning disability scores in the month before study inclusion (patients in CHR states: mean [SD] Global Assessment of Functioning disability scores, impaired at follow-up: 52.5 [11.7]; unimpaired at follow-up: 60.2 [14.9]; P = .02; patients with ROD: mean [SD] scores, impaired at follow-up: 52.5 [13.4]; unimpaired at follow-up: 61.4 [14.6]; P = .01). Furthermore, impaired social functioning at follow-up was associated with reduced social functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.0 [1.3]; unimpaired at follow-up: 7.1 [1.1]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.5]; unimpaired at follow-up: 6.9 [1.0]; P < .001), and reduced global role functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 5.7 [1.4]; unimpaired at follow-up: 6.7 [1.2]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.7]; unimpaired at follow-up: 6.8 [1.4]; P = .01).

These differences extended to the social functioning in the past year (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.7 [1.4]; unimpaired at follow-up: 7.8 [0.7]; P < .001; patients with ROD: impaired at follow-up: 6.9 [1.4]; unimpaired at follow-up: 7.7 [0.8]; P < .001) and lifetime social functioning (patients in CHR states: mean [SD] scores, impaired at follow-up: 7.5 [0.8]; unimpaired at follow-up: 8.3 [0.6]; P < .001; patients with ROD: impaired at follow-up: 7.8 [0.9]; unimpaired at follow-up: 8.4 [0.8]; P < .001).

At the level of psychopathology, we observed transdiagnostic baseline differences between groups with impaired and unimpaired outcomes in social functioning in the attenuated negative symptoms domain, particularly an item in the Standardized Interview for Prodromal Symptoms (SIPS) scale specific to expression of emotions (patients in CHR states: mean [SD] scores, impaired at follow-up: 1.3 [1.6]; unimpaired at follow-up: 0.6 [1.2]; P = .03; patients with ROD: mean [SD] scores, impaired at follow-up: 1.1 [1.4]; unimpaired at follow-up: 0.6 [1.0]; P = .03). Further differences were measured in the Positive and Negative Syndrome Scale (PANSS) total scores at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 52.9 [15.2]; unimpaired at follow-up: 46.7 [10.1]; P = .03; patients with ROD: impaired at follow-up: 49.5 [10.3]; unimpaired at follow-up: 44.0 [9.6]; P = .01), and PANSS negative symptoms scores at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 14.6 [6.6]; unimpaired at follow-up: 9.8 [3.6]; P < .001; patients with ROD: impaired at follow-up: 13.6 [5.2]; unimpaired at follow-up: 11.1 [4.0]; P = .02).

Similar but less pronounced effects were observed in the role functioning outcome analyses of patients who were impaired and unimpaired, with the patients in CHR states who were impaired at follow-up being more affected in the social functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 6.2 [1.4]; unimpaired at follow-up: 6.9 [1.1]; P = .03; patients with ROD: mean [SD] scores, impaired at follow-up: 6.0 [1.5]; unimpaired at follow-up: 6.8 [1.3]; P = .01), role functioning at baseline (patients in CHR states: mean [SD] scores, impaired at follow-up: 5.6 [1.2]; unimpaired at follow-up: 6.9 [1.3]; P < .001; patients with ROD: mean [SD] scores, impaired at follow-up: 5.9 [1.7]; unimpaired at follow-up: 6.8 [1.5]; P = .03). Significantly more severe baseline negative and PANSS total symptoms were found in the patients in CHR states who showed role functioning impairments at follow-up compared with those who were unimpaired (SIPS assessment of ideational richness: mean [SD] scores, impaired at follow-up: 0.7 [1.5]; unimpaired at follow-up: 0.2 [0.5]; P = .01; SIPS assessment of occupational functioning: impaired at follow-up: 3.3 [1.7]; unimpaired at follow-up: 1.4 [1.7]; P < .001; PANSS total score: impaired at follow-up: 52.9 [14.9]; unimpaired at follow-up: 46.4 [10.1]; P = .03; PANSS negative score: impaired at follow-up: 14.1 [6.8]; unimpaired at follow-up: 10.1 [3.6]; P < 001).

Current mood, anxiety, and substance use diagnoses were prevalent at baseline in the CHR group (33 of 51 patients [64.7%]) and ROD group (42 of 49 [85.7%]) but did not differ between outcome-defined samples (eTable 11 in the Supplement). During the follow-up period, symptomatic diagnoses declined, but less so in the groups who were impaired at follow-up: at least 1 DSM-IV diagnosis was present in 27 of 62 patients in CHR states who were impaired at follow-up (44%) and in 25 of 52 patients with ROD who were impaired at follow-up (48%), compared with 4 of 39 patients in CHR states who were unimpaired at follow-up (10%) and 12 of 48 patients with ROD who were unimpaired at follow-up (25%) (eTable 11 in the Supplement). These effects were driven by current depression and were independent of transition to psychosis (which occurred in 8 patients in CHR states and 2 patients with ROD; eTable 12 in the Supplement). Finally, we observed a significant interaction between study sites and social-functioning outcomes in the patients in CHR states (with the sample size per site of patients in CHR states who were impaired at follow-up varying from 22 of 33 [66%] in Munich to 5 of 6 [83%] in Milan, 4 of 15 [27%] in Basel, 10 of 19 [53%] in Cologne, 3 of 12 [25%] in Birmingham, 14 of 19 [74%] in Turku, and 8 of 12 [67%] in Udine, and the sample size per site of patients in CHR states who were unimpaired at follow-up ranging from 11 [33%] in Munich to 1 [17%] in Milan, 11 [73%] in Basel, 9 [47%] in Cologne, 9 [75%] in Birmingham, 5 [26%] in Turku, and 4 [33%] in Udine; P = .03 for groupwise comparison, Table 1). This interaction motivated additional MRI-based validation analyses, as described in the eMethods, eFigure 7, and eTable 7 in the Supplement.

Machine-Learning Analyses

The models evaluating the patients’ social and role functioning at and before study inclusion estimated the social-functioning outcomes of patients in CHR states with a significant leave-site-out balanced accuracy of 76.9% (sensitivity: 69.7%; specificity: 84.0%; P = .002 after false-detection rate adjustment; Table 2). Balanced accuracy of the group with ROD was 66.1%, with a sensitivity of 63.1% and specificity of 69.1% (P = .049; Table 2).

In the patients in CHR states, a model predicting role functioning scores was also significant and performed at balanced accuracy of 67.7% (sensitivity: 60.9%; specificity: 74.5%; P = .02). Clinical models outperformed the sMRI prediction model in their transdiagnostic transferability (Table 3). The features most useful for the model predicting social functioning outcome scores in the patients with ROD were Global Functioning Scale scores at baseline, reduced Global Functioning Scale scores in the year before study inclusion and reduced Global Functioning Scale scores over the lifetime (Figure 1). In the group in CHR states, the most useful features were a reduced social functioning score in the year before study inclusion and a reduced highest global functioning score over the lifetime (Figure 1).

The sMRI-based prediction models determined social-functioning outcomes in the CHR groups with a balanced accuracy of 76.2% (sensitivity: 80.3%, specificity: 72.0%; P = .002; Table 2). The sMRI-based prediction models determined social functioning in the patients with ROD with a balanced accuracy of 65.0%, a sensitivity of 64.6%, and a specificity of 65.5% (P = .04). The performance of models was not influenced by site effects (eTable 7 and eFigure 7 in the Supplement), follow-up duration (eFigure 6 in the Supplement), or baseline functional differences between outcome classes (eFigure 8 in the Supplement). In contrast with the social functioning domain, sMRI data could not be used to accurately estimate role-functioning outcomes (Table 2).

Divergent neuroanatomical patterns (Figure 2) impeded the sMRI model transfer between study groups (Table 3). In the group in CHR states, social functioning impairments at follow-up were associated with (1) reduced baseline GMV in medial prefrontal, cingulate, orbitofrontal, insular, temporal, parietal, and occipital brain regions, and (2) increased cerebellar, dorsomedial, and dorsolateral prefrontal GMVs (Figure 2A). To understand whether this signature represented a pattern of neuroanatomical abnormality, we compared the prognostic samples in CHR states to the data from matched healthy control participants. The reductions in GMV and increments that were associated with impaired social-functioning outcomes differentiated patients in CHR states who had a poor prognosis (defined as a poor outcome predicted by the model, as distinct from observed impairment at follow-up) from healthy control participants (eFigure 8 in the Supplement).

Different from the group in CHR states, the neuroanatomical pattern associated with social-functioning outcomes in patients with ROD included (1) reduced GMV in the hippocampus, amygdala, inferior temporal cortex, thalamus, and dorsal anterior cingulate cortex, and (2) increased GMV in the medial and lateral prefrontal, orbitofrontal, insular, and superior temporal cortices. These GMV increments, but not the temporolimbic reductions, were encountered again when comparing patients with ROD and poor prognoses with healthy control participants using VBM (eFigure 9 in the Supplement). In contrast, patients with ROD and unimpaired social functioning prognosis showed reduced prefrontal GMV and increased temporolimbic GMV compared with healthy control participants.

Combined models predicting social functioning scores estimated the outcomes of patients in CHR states with a balanced accuracy of 82.7% (sensitivity: 83.2%; specificity: 82.0%; P < .001; Table 2). The same models estimated outcomes of patients with ROD with a balanced accuracy of 70.3% (sensitivity: 76.9%; specificity: 63.6%; P = .01). The combined models’ prognostic summary index (PSI) outperformed the sMRI-based models’ PSI by 12.2% in the CHR group (sMRI model: 52.6%; combined model: 64.8%) and 11.5% in the ROD group (sMRI model: 29.9%; combined model: 41.4%); the prognostic summary index outperformed clinical models by 11.9% in the CHR group (clinical model: 52.9%; combined model: 64.8%) and 9.4% in the ROD group (clinical model: 32.0%; combined model: 41.4%). Furthermore, the sequential social functioning prediction analysis showed that, with increasing clinical model uncertainty (decision scores closer to the support vector machines’ decision boundary), the prognostic summary index of the combined model increased to 82.6% in the group in CHR states and was stable (35%-50%) in patients with ROD (eFigures 12 and 13 in the Supplement). Thus, in ambiguous cases, combined models provided a 1.9-fold prognostic gain for patients in the CHR state and a 10.5-fold prognostic gain for patients with ROD compared with a purely clinical prediction model that evaluated the patients' social and role functioning at baseline and before study inclusion.

Finally, expert raters’ global functioning estimates correctly identified the social functioning outcomes of participants in CHR states with a balanced accuracy of 71.8% (sensitivity: 51.5%, specificity: 92.0%). Expert raters accurately identified the social-functioning outcomes for participants with ROD with a balanced accuracy of 59.6% (sensitivity: 26.6%; specificity: 92.6%). The raters’ estimates correctly identified role-functioning outcomes with a balanced accuracy of 70.4% (sensitivity: 49.3%; specificity: 91.5%) for patients in CHR states and a balanced accuracy of 57.9% (sensitivity: 25.0%; specificity: 90.7%) for patients with ROD. Thus, the raters underestimated the risk of impairment in social and role functioning in patients in CHR states and patients with ROD at follow-up (Table 2). Models outperformed raters in all predictive tasks (Table 2 and eTable 9 in the Supplement) and were not influenced by the patients’ age, sex, or ethnicity (eTable 13 in the Supplement).

Transdiagnostic Prognostic Generalization

In the CHR group, a poor social functioning prognosis as provided by the clinical prediction model was associated with an increased prevalence of DSM-IV-TR diagnoses at follow-up: 13 of 49 patients in CHR states (27.1%) with poor prognoses (eTable 10 in the Supplement) had a major depressive disorder at follow-up, compared with 0% in the group with a good prognosis (χ21 = 15.8; P < .001). Hence, in these patients, the clinical prediction model projected a major depressive disorder at follow-up with a balanced accuracy of 79.5% (P < .001; eTable 10 in the Supplement). Similarly, 25 of 49 patients in CHR states (51%) who had poor prognoses had at least 1 DSM-IV-TR mood, anxiety, or substance use disorder, compared with 6 of the 52 patients in CHR states (11.5%) who had good prognoses (χ21 = 18.5; P < .001; balanced accuracy, 73.2%; P = .03). The clinical model predicting social functioning significantly generalized to the prediction of these diagnostic outcomes (eTable 10 in the Supplement). However, only the sMRI model consistently assigned a poor social functioning prognosis to the patients who transitioned to psychosis (balanced accuracy, 72.7%; P = .01), resulting in a transition risk of 11.9% for those with poor prognoses (8 of 67) vs 0% in those with prognoses of good outcomes (χ21 = 6.28; P = .02). In patients with ROD, social functioning predictions did not generalize to diagnostic outcomes.

Discussion

Persistent social and role functioning deficits drive the personal and socioeconomic burden of psychotic and mood disorders.41 Yet mental health care lacks the computational tools that could enable the early recognition of these deficits in help-seeking patients.42,43 Such prognostic tools may catalyze the development of novel biobehavioral therapies for individualized secondary and tertiary prevention.4,44-47 In this article, we report on what is to our knowledge the first international effort to develop such tools for the prediction of functional outcomes in young patients at risk for psychosis or recurrent depression.48 Via thoroughly cross-validated machine-learning methods, we found that social functioning impairments can be correctly predicted in up to 83% of patients in CHR states and 70% of patients with ROD who were recruited from community-based and hospital-based pathways to care across geographically distinct European populations. These results suggest that the individualized quantification of risk for impaired functional recovery is feasible despite site-associated heterogeneity and even without a prestudy calibration of sMRI procedures. Additionally, our inclusive study protocol running in clinical early recognition services facilitated the prognostic algorithms’ derivation and validation in patients representing some of the real-world diversity of CHR states for psychosis, depression, and comorbid psychiatric conditions. In this realistic test bed of model generalizability, we observed that role functioning was less clearly associated with predictors than social functioning when analyzed functional baseline data was used and was even less so when structural neuroimaging data was used. Role functioning may be more strongly determined by concurrent environmental and clinical factors than social deficits, leading to a greater degree of temporal fluctuation, which in turn potentially mediates differential associations with baseline clinical and neuroanatomical predictors (eFigure 4 in the Supplement).28,49,50 Future research should therefore assess whether the inclusion of environmental and clinical variables may improve the association with outcomes of role functioning deficits in similar help-seeking populations.

Importantly, we observed that combined models integrating clinical and brain structural data outperformed human clinical raters, suggesting that these models could improve the prognostic process beyond the current level. Interestingly, clinical raters overestimated patients’ social functioning improvement, particularly in the ROD group, in which only 27% of patients who ultimately experienced impaired outcomes were correctly identified. An additional sensitivity analysis showed that raters performed better in patients with ROD when impaired social functioning was categorized at lower cutoff values, suggesting that prognostic reasoning is sensitive to more severe functional deficits at follow-up (eTable 14 in the Supplement). In the group in CHR states, however, raters prognosticated best at the original cutoff levels but were less accurate than our machine learning models. This observation suggests that internal heuristics, which are potentially informed by the study groups’ differing clinical profiles, may influence clinicians’ prognostication. Taken together, rater-based prognostication may considerably overestimate vulnerable patients’ capacity to recover from social functioning deficits. Further research is needed to understand the factors governing functional prognostication in psychiatry to design algorithms that optimally mitigate prognostic bias.

The comparisons of models revealed that combined risk stratification generally outperformed unimodal prediction models. Despite the benefits of combined prediction models, however, our clinical models evaluating only 8 functioning variables would provide a cost-effective first-line strategy for estimating the risk of future social disability similar to the psychosis, depression, and suicide risk calculators recently proposed for secondary and outpatient care.12,21,23,51 In the group with CHR states, we were able to show that our functional risk calculator not only estimated future social impairment but also broader psychiatric morbidity (eTable 10 in the Supplement). These findings were corroborated by the observation that patients in CHR states with an MRI-based prognosis of impaired social functioning experienced unremitting symptoms, occupational disability, and poor quality of life (eFigure 10 in the Supplement). Thus, risk calculators for social impairment may provide accessible tools for a more generalized psychiatric risk screening for individuals in the CHR state. In contrast, the models specific to patients with ROD did not show a comparable prognostic generalizability. This discrepancy may align with recent findings revealing pronounced social cognition deficits in individuals with psychosis vs individuals with depression,52 and a close link between psychiatric morbidity, social functioning, and social cognitive processes in psychosis.53-55

Because MRI is cost intensive, we explored whether sequential predictive modeling could provide a rationale for the targeted use of structural neuroimaging (eFigures 12 and 13 in the Supplement). We observed pronounced benefits of blending sMRI and functional data in patients with increasingly ambiguous clinical decision scores, suggesting that the cost-benefit ratio of sMRI can be maximized by including it at a later, more elaborated stage of the prognostic workflow. Notably, this may include the assessment of transition risk based on the finding that only the sMRI-based model predicting social functioning showed significant prognostic generalizability in this context (eTable 10 in the Supplement). Taken together, these results may provide a first empirical account of prognostic improvements brought about by sequential multimodal risk assessment.30 Future studies should assess the added value of different data combinations, including neurocognition,18,56electroencephalography,20 sensor-based activity patterns,32 and language patterns,57 as well as multiomics information.58

We observed striking differences between the neuroanatomical patterns associated with different social-functioning outcomes in the patients in the CHR state and the patients with ROD. In patients in the CHR state, impaired functioning was associated with GMV alterations that represented a deviation from normal brain variation and that mapped to the salience network,59 the perisylvian language-associated system, the default-mode network, and the central executive network.60 Similar GMV reductions have been previously described in studies comparing the CHR, first-episode psychosis, and relapsing stages of psychosis with healthy control participants.61,62 Longitudinal data pointed to altered neurodevelopmental trajectories of these brain systems, suggesting a disturbed process of cortical reorganization in different stages of psychosis development.63-67 Structural dysmaturation affecting these brain systems may influence the capacity to switch between self-referential thinking, salience attribution, and executive functioning,68,69 thus disrupting complex processes of social cognition and behavior, and finally predisposing to poor functional outcomes of the CHR state.53

In contrast, impaired social functioning in the group with ROD was associated with extended prefrontal, insular, and lateral temporal GMV increments. This pattern also differentiated patients from healthy control participants. Brain volume reductions focused on the medial temporal lobe, the dorsal anterior cingulate cortex, and temporooccipital cortices. While these abnormalities have been associated with earlier disease onset and poor outcomes of patients with major depression,70 findings in prefrontal and temporal areas have remained equivocal; for example, a recent VBM meta-analysis of ROD reported increased insular, thalamic, and temporal brain volumes and decreased dorsolateral prefrontal cortical volumes.71 However, extended insular and prefrontal volume reductions were particularly found in patients with relapsing depression and early disease onset.72,73 Together with our findings, this may point to dynamic brain volume changes with potentially insufficient prefrontal compensatory processes in patients with ROD who fail to recover.

These largely nonoverlapping neuroanatomical patterns of impaired social functioning outcome explain the low MRI model transferability between the 2 diagnostic groups. This finding challenges the hypothesis of shared brain surrogates underlying transdiagnostic phenotypes, such as the Systems for Social Processes domain proposed by the National Institute of Mental Health’s Research Domain Criteria.74 Alternatively, the recently revealed brain-behavioral heterogeneity of depression75 may have impeded our algorithms from detecting a salient and thus generalizable neuroanatomical signature of social functioning in the ROD group and/or reduced the generalizability of the CHR-specific model to an umbrella construct of depression. These alternative hypotheses call for more research using subtyping strategies76 to test whether the current neurobiological and outcome-associated heterogeneity of mental disorders can be deconvolved into patient strata with distinct prognostic profiles.

Limitations

One limitation may be the dichotomization of the global functioning scales and the use of classification models for prediction. Regression requires a sufficient representation of the tails in the target scale to learn a predictive pattern explaining the scale’s full range. Because this requirement was not met by our data (eFigure 3 in the Supplement), we preferred classification to regression models. Notably, based on the high correlation between the patients’ ordinal social functioning follow-up scores and the outcome probability estimates of the combined models asssessing social functioning (Table 2), we tested whether post hoc regression models could successfully map these estimates to the global functioning score range. The low mean average errors of these models (eFigure 11 in the Supplement) suggest that continuous targets could be approximated through soft classification.

Conclusions

In summary, we identified generalizable clinical, imaging-based, and combined prediction models of persistent social functioning impairments in young patients at increased risk for psychosis and recurrent depression. To further elucidate the clinical, environmental, and neurobiological factors that facilitate or limit the transferability of the risk calculators presented here, external and prospective validation is needed in ethnically diverse patient populations recruited at sites beyond the European catchment areas of this study. This is the next important step toward quantifying the feasibility and utility of precision psychiatry approaches for the secondary and tertiary prevention of severe mental illnesses.

Back to top
Article Information

Corresponding Author: Nikolaos Koutsouleris, MD, Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Nussbaumstr. 7, D-80336 Munich, Germany (nikolaos.koutsouleris@med.uni-muenchen.de).

Accepted for Publication: June 26, 2018.

Author Contributions: Dr Koutsouleris had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Published Online: September 26, 2018. doi:10.1001/jamapsychiatry.2018.2165

Correction: This article was corrected on April 3, 2019. Dr Paolini’s affiliation was listed incorrectly as the Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Germany. It was corrected to the Department of Radiology in the same institution.

Concept and design: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Kambeitz, Falkai, Reiser, Riecher-Rossler, Upthegrove, Hietala, Salokangas, Meisenzahl, Wood, Brambilla, Borgwardt.

Acquisition, analysis, or interpretation of data: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Rosen, Ruef, Dwyer, Paolini, Chisholm, Kambeitz, Haidl, Schmidt, Gillam, Schultze-Lutter, Riecher-Rossler, Upthegrove, Hietala, Pantelis, Wood, Beque, Brambilla, Borgwardt.

Drafting of the manuscript: Koutsouleris, Rosen, Dwyer, Kambeitz, Gillam, Schultze-Lutter.

Critical revision of the manuscript for important intellectual content: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Rosen, Ruef, Dwyer, Paolini, Chisholm, Kambeitz, Haidl, Schmidt, Schultze-Lutter, Falkai, Reiser, Riecher-Rossler, Upthegrove, Hietala, Salokangas, Pantelis, Meisenzahl, Wood, Beque, Brambilla, Borgwardt.

Statistical analysis: Koutsouleris, Kambeitz, Upthegrove.

Obtained funding: Koutsouleris, Kambeitz-Ilankovic, Ruhrmann, Riecher-Rossler, Salokangas, Pantelis, Brambilla, Borgwardt, Wood.

Administrative, technical, or material support: Koutsouleris, Rosen, Ruef, Paolini, Chisholm, Haidl, Schmidt, Gillam, Reiser, Upthegrove, Hietala, Meisenzahl, Beque, Brambilla, Borgwardt.

Supervision: Koutsouleris, Ruhrmann, Rosen, Schultze-Lutter, Falkai, Reiser, Riecher-Rossler, Hietala, Wood, Brambilla, Borgwardt.

Conflict of Interest Disclosures: Dr Koutsouleris received honoraria for talks presented at education meetings organized by Otsuka/Lundbeck. Dr Pantelis participated in advisory boards for Janssen-Cilag, AstraZeneca, Lundbeck, and Servier and received honoraria for talks presented at educational meetings organized by AstraZeneca, Janssen-Cilag, Eli Lilly, Pfizer, Lundbeck, and Shire. Dr Upthegrove received honoraria for talks presented at educational meetings organized by Sunovion. No other disclosures were reported.

Group Information: PRONIA consortium members listed here performed the screening, recruitment, rating, examination, and follow-up of the study participants and were involved in implementing the examination protocols of the study, setting up its information technological infrastructure, and organizing the flow and quality control of the data analyzed in this article between the local study sites and the central study database. Department of Psychiatry and Psychotherapy, Ludwig-Maximilian-University, Munich, Bavaria, Germany: Linda Betz, BSc, Anne Erkens, Eva Gussmann, BSc, Shalaila Haas, MSc, Alkomiet Hasan, MD, Claudius Hoff, MD, Ifrah Khanyaree, BSc, Aylin Melo, BSc, Susanna Muckenhuber-Sternbauer, MD, Janis Köhler, Ömer Öztürk, MD, Nora Penzel, MSc, David Popovic, MD, Adrian Rangnick, BSc, Sebastian von Saldern, MD, Rachele Sanfelici, MSc, Moritz Spangemacher, Ana Tupac, MSc, Maria Fernanda Urquijo, MSc, Johanna Weiske, MSc, and Antonia Wosgien. University of Cologne, North Rhineland–Westphalia, Germany: Dennis Hedderich, MD, Karsten Blume, Mauro Seves, MSc, Nathalie Kaiser, MSc, Thorsten Lichtenstein, MD, and Christiane Woopen, MD. Psychiatric University Hospital, University of Basel, Basel, Switzerland: Christina Andreou, MD, PhD, Laura Egloff, PhD, Fabienne Harrisberger, PhD, Claudia Lenz, PhD, Letizia Leanza, MSc, Amatya Mackintosh, MSc, Renata Smieskova, PhD, Erich Studerus, PhD, Anna Walter, MD, and Sonja Widmayer, MSc. Institute of Mental Health, University of Birmingham, Birmingham, United Kingdom: Chris Day, BSc, Sian Lowri Griffiths, PhD, Mariam Iqbal, BSc, Mirabel Pelton, MSc, Pavan Mallikarjun, MBBS, DPM, MRCPsych, PhD, Alexandra Stainton, MSci, and Ashleigh Lin, PhD. Department of Psychiatry, University of Turku, Turku, Finland: Alexander Denissoff, MD, Anu Ellilä, RN, Tiina From, MSc, Markus Heinimaa, MD, PhD, Tuula Ilonen, PhD, Päivi Jalo, RN, Heikki Laurikainen, MD, Maarit Lehtinen, RN, Antti Luutonen, BA, Akseli Mäkela, BA, Janina Paju, MSc, Henri Pesonen, PhD, Reetta-Liina Armio (Säilä), MD, Elina Sormunen, MD, Anna Toivonen, MSc, and Otto Turtonen, MD. General Electric Global Research Inc, Munich, Germany: Ana Beatriz Solana, PhD, Manuela Abraham, MBA, Nicolas Hehn, PhD, and Timo Schirmer, PhD. Workgroup of Paolo Brambilla, MD, PhD, University of Milan, Milan, Italy: Department of Neuroscience and Mental Health, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, University of Milan, Milan, Italy: Carlo Altamura, MD, Marika Belleri, PsychD, Francesca Bottinelli, PsychD, Adele Ferro PsychD, PhD, and Marta Re, PhD. Programma2000, Niguarda Hospital, Milan: Emiliano Monzani, MD, Mauro Percudani, MD, and Maurizio Sberna, MD. San Paolo Hospital, Milan: Armando D’Agostino, MD, and Lorenzo Del Fabro, MD. Villa San Benedetto Menni, Albese con Cassano: Giampaolo Perna, MD, Maria Nobile MD, PhD, and Alessandra Alciati, MD. Workgroup of Paolo Brambilla, University of Udine, Udine, Italy: Department of Medical Area, University of Udine: Matteo Balestrieri, MD, Carolina Bonivento, PsychD, PhD, Giuseppe Cabras, PhD, and Franco Fabbro, MD, PhD. IRCCS Scientific Institute “E. Medea”, Polo FVG, Udine: Marco Garzitto, PsychD, PhD and Sara Piccin, PsychD, PhD.

Funding/Support: PRONIA is a Collaborative Project funded by the European Union under the 7th Framework Programme (grant 602152). Dr Pantelis was supported by National Health and Medical Research Council Senior Principal Research Fellowship (grants 628386 and 1105825) and European Union–National Health and Medical Research Council (grant 1075379).

Role of the Funder/Sponsor: The funding organizations were not involved in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; the preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the Recognition and Prevention Program at the Zucker Hillside Hospital in New York, directed by Barbara Cornblatt, PhD, MBA, for providing the Global Functioning: Social and Role scales. We thank Andrea M. Auther, PhD, Associate Director of Recognition and Prevention Program and coauthor of the Global Functioning scales for overseeing the training and implementation of the scales. They were not compensated for their contributions.

Additional Information: For more information on training or use of the Global Functioning: Social and Role scales, contact Dr Auther at aauther@northwell.edu. This work was supported by the National Institute of Mental Health–funded Harmonization of At Risk Multisite Observational Networks for Youth (HARMONY) project (administrative supplement to grant 3U01MH081928-07S1).

References
1.
Addington  J, Cornblatt  BA, Cadenhead  KS,  et al.  At clinical high risk for psychosis: outcome for nonconverters.  Am J Psychiatry. 2011;168(8):800-805. doi:10.1176/appi.ajp.2011.10081191PubMedGoogle ScholarCrossref
2.
Salokangas  RKR, Nieman  DH, Heinimaa  M,  et al; EPOS group.  Psychosocial outcome in patients at clinical high risk of psychosis: a prospective follow-up.  Soc Psychiatry Psychiatr Epidemiol. 2013;48(2):303-311. doi:10.1007/s00127-012-0545-2PubMedGoogle ScholarCrossref
3.
Lin  A, Wood  SJ, Nelson  B, Beavan  A, McGorry  P, Yung  AR.  Outcomes of nontransitioned cases in a sample at ultra-high risk for psychosis.  Am J Psychiatry. 2015;172(3):249-258. doi:10.1176/appi.ajp.2014.13030418PubMedGoogle ScholarCrossref
4.
McGorry  P, Keshavan  M, Goldstone  S,  et al.  Biomarkers and clinical staging in psychiatry.  World Psychiatry. 2014;13(3):211-223. doi:10.1002/wps.20144PubMedGoogle ScholarCrossref
5.
Schmidt  SJ, Schultze-Lutter  F, Schimmelmann  BG,  et al.  EPA guidance on the early intervention in clinical high risk states of psychoses.  Eur Psychiatry. 2015;30(3):388-404. doi:10.1016/j.eurpsy.2015.01.013PubMedGoogle ScholarCrossref
6.
Fusar-Poli  P, Borgwardt  S, Bechdolf  A,  et al.  The psychosis high-risk state: a comprehensive state-of-the-art review.  JAMA Psychiatry. 2013;70(1):107-120. doi:10.1001/jamapsychiatry.2013.269PubMedGoogle ScholarCrossref
7.
Carrión  RE, McLaughlin  D, Goldberg  TE,  et al.  Prediction of functional outcome in individuals at clinical high risk for psychosis.  JAMA Psychiatry. 2013;70(11):1133-1142. doi:10.1001/jamapsychiatry.2013.1909PubMedGoogle ScholarCrossref
8.
Simon  AE, Velthorst  E, Nieman  DH, Linszen  D, Umbricht  D, de Haan  L.  Ultra high-risk state for psychosis and non-transition: a systematic review.  Schizophr Res. 2011;132(1):8-17. doi:10.1016/j.schres.2011.07.002PubMedGoogle ScholarCrossref
9.
Michel  C, Ruhrmann  S, Schimmelmann  BG, Klosterkötter  J, Schultze-Lutter  F.  Course of clinical high-risk states for psychosis beyond conversion.  Eur Arch Psychiatry Clin Neurosci. 2018;268(1):39-48. doi:10.1007/s00406-016-0764-8PubMedGoogle ScholarCrossref
10.
Cornblatt  BA, Auther  AM, Niendam  T,  et al.  Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia.  Schizophr Bull. 2007;33(3):688-702. doi:10.1093/schbul/sbm029PubMedGoogle ScholarCrossref
11.
Fusar-Poli  P, Byrne  M, Valmaggia  L,  et al; OASIS Team.  Social dysfunction predicts two years clinical outcome in people at ultra high risk for psychosis.  J Psychiatr Res. 2010;44(5):294-301. doi:10.1016/j.jpsychires.2009.08.016PubMedGoogle ScholarCrossref
12.
Cannon  TD, Yu  C, Addington  J,  et al.  An individualized risk calculator for research in prodromal psychosis.  Am J Psychiatry. 2016;173(10):980-988. doi:10.1176/appi.ajp.2016.15070890PubMedGoogle ScholarCrossref
13.
Salokangas  RK, Heinimaa  M, From  T,  et al; EPOS group.  Short-term functional outcome and premorbid adjustment in clinical high-risk patients: results of the EPOS project.  Eur Psychiatry. 2014;29(6):371-380. doi:10.1016/j.eurpsy.2013.10.003PubMedGoogle ScholarCrossref
14.
Bora  E, Harrison  BJ, Yücel  M, Pantelis  C.  Cognitive impairment in euthymic major depressive disorder: a meta-analysis.  Psychol Med. 2013;43(10):2017-2026. doi:10.1017/S0033291712002085PubMedGoogle ScholarCrossref
15.
Fusar-Poli  P, Nelson  B, Valmaggia  L, Yung  AR, McGuire  PK.  Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis.  Schizophr Bull. 2014;40(1):120-131. doi:10.1093/schbul/sbs136PubMedGoogle ScholarCrossref
16.
Bodatsch  M, Ruhrmann  S, Wagner  M,  et al.  Prediction of psychosis by mismatch negativity.  Biol Psychiatry. 2011;69(10):959-966. doi:10.1016/j.biopsych.2010.09.057PubMedGoogle ScholarCrossref
17.
Koutsouleris  N, Riecher-Rössler  A, Meisenzahl  EM,  et al.  Detecting the psychosis prodrome across high-risk populations using neuroanatomical biomarkers.  Schizophr Bull. 2015;41(2):471-482. doi:10.1093/schbul/sbu078PubMedGoogle ScholarCrossref
18.
Michel  C, Ruhrmann  S, Schimmelmann  BG, Klosterkötter  J, Schultze-Lutter  F.  A stratified model for psychosis prediction in clinical practice.  Schizophr Bull. 2014;40(6):1533-1542. doi:10.1093/schbul/sbu025PubMedGoogle ScholarCrossref
19.
Ruhrmann  S, Schultze-Lutter  F, Salokangas  RKR,  et al.  Prediction of psychosis in adolescents and young adults at high risk: results from the prospective European prediction of psychosis study.  Arch Gen Psychiatry. 2010;67(3):241-251. doi:10.1001/archgenpsychiatry.2009.206PubMedGoogle ScholarCrossref
20.
Nieman  DH, Ruhrmann  S, Dragt  S,  et al.  Psychosis prediction: stratification of risk estimation with information-processing and premorbid functioning variables.  Schizophr Bull. 2014;40(6):1482-1490. doi:10.1093/schbul/sbt145PubMedGoogle ScholarCrossref
21.
Fusar-Poli  P, Rutigliano  G, Stahl  D,  et al.  Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis.  JAMA Psychiatry. 2017;74(5):493-500. doi:10.1001/jamapsychiatry.2017.0284PubMedGoogle ScholarCrossref
22.
Chekroud  AM, Zotti  RJ, Shehzad  Z,  et al.  Cross-trial prediction of treatment outcome in depression: a machine learning approach.  Lancet Psychiatry. 2016;3(3):243-250. doi:10.1016/S2215-0366(15)00471-XPubMedGoogle ScholarCrossref
23.
Kessler  RC, van Loo  HM, Wardenaar  KJ,  et al.  Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports.  Mol Psychiatry. 2016;21(10):1366-1371. doi:10.1038/mp.2015.198PubMedGoogle ScholarCrossref
24.
Koutsouleris  N, Kahn  RS, Chekroud  AM,  et al.  Multisite prediction of 4-week and 52-week treatment outcomes in patients with first-episode psychosis: a machine learning approach.  Lancet Psychiatry. 2016;3(10):935-946. doi:10.1016/S2215-0366(16)30171-7PubMedGoogle ScholarCrossref
25.
Kambeitz-Ilankovic  L, Meisenzahl  EM, Cabral  C,  et al.  Prediction of outcome in the psychosis prodrome using neuroanatomical pattern classification.  Schizophr Res. 2016;173(3):159-165. doi:10.1016/j.schres.2015.03.005PubMedGoogle ScholarCrossref
26.
de Wit  S, Ziermans  TB, Nieuwenhuis  M,  et al.  Individual prediction of long-term outcome in adolescents at ultra-high risk for psychosis: applying machine learning techniques to brain imaging data.  Hum Brain Mapp. 2017;38(2):704-714. doi:10.1002/hbm.23410PubMedGoogle ScholarCrossref
27.
Velthorst  E, Nieman  DH, Linszen  D,  et al.  Disability in people clinically at high risk of psychosis.  Br J Psychiatry. 2010;197(4):278-284. doi:10.1192/bjp.bp.109.075036PubMedGoogle ScholarCrossref
28.
Cornblatt  BA, Carrión  RE, Addington  J,  et al.  Risk factors for psychosis: impaired social and role functioning.  Schizophr Bull. 2012;38(6):1247-1257. doi:10.1093/schbul/sbr136PubMedGoogle ScholarCrossref
29.
Studerus  E, Ramyead  A, Riecher-Rössler  A.  Prediction of transition to psychosis in patients with a clinical high risk for psychosis: a systematic review of methodology and reporting.  Psychol Med. 2017;47(7):1163-1178. doi:10.1017/S0033291716003494PubMedGoogle ScholarCrossref
30.
Schmidt  A, Cappucciati  M, Radua  J,  et al.  Improving prognostic accuracy in subjects at clinical high risk for psychosis: systematic review of predictive models and meta-analytical sequential testing simulation.  Schizophr Bull. 2017;43(2):375-388.PubMedGoogle Scholar
31.
Steyerberg  EW, Moons  KGM, van der Windt  DA,  et al; PROGRESS Group.  Prognosis research strategy (PROGRESS) 3: prognostic model research.  PLoS Med. 2013;10(2):e1001381. doi:10.1371/journal.pmed.1001381PubMedGoogle ScholarCrossref
32.
Hahn  T, Nierenberg  AA, Whitfield-Gabrieli  S.  Predictive analytics in mental health: applications, guidelines, challenges and perspectives.  Mol Psychiatry. 2017;22(1):37-43. doi:10.1038/mp.2016.201PubMedGoogle ScholarCrossref
33.
Chekroud  AM, Koutsouleris  N.  The perilous path from publication to practice.  Mol Psychiatry. 2018;23(1):24-25. doi:10.1038/mp.2017.227PubMedGoogle ScholarCrossref
34.
Fowler  D, Hodgekins  J, French  P,  et al.  Social recovery therapy in combination with early intervention services for enhancement of social recovery in patients with first-episode psychosis (SUPEREDEN3): a single-blind, randomised controlled trial.  Lancet Psychiatry. 2018;5(1):41-50. doi:10.1016/S2215-0366(17)30476-5PubMedGoogle ScholarCrossref
35.
De Silva  MJ, Cooper  S, Li  HL, Lund  C, Patel  V.  Effect of psychosocial interventions on social functioning in depression and schizophrenia: meta-analysis.  Br J Psychiatry. 2013;202(4):253-260. doi:10.1192/bjp.bp.112.118018PubMedGoogle ScholarCrossref
36.
McFarlane  WR, Levin  B, Travis  L,  et al.  Clinical and functional outcomes after 2 years in the early detection and intervention for the prevention of psychosis multisite effectiveness trial.  Schizophr Bull. 2015;41(1):30-43. doi:10.1093/schbul/sbu108PubMedGoogle ScholarCrossref
37.
Seidman  LJ, Nordentoft  M.  New targets for prevention of schizophrenia: is it time for interventions in the premorbid phase?  Schizophr Bull. 2015;41(4):795-800. doi:10.1093/schbul/sbv050PubMedGoogle ScholarCrossref
38.
Wolpert  DH.  Stacked generalization.  Neural Netw. 1992;5:241-259. doi:10.1016/S0893-6080(05)80023-1Google ScholarCrossref
39.
Gaonkar  B, T Shinohara  R, Davatzikos  C; Alzheimers Disease Neuroimaging Initiative.  Interpreting support vector machine models for multivariate group wise analysis in neuroimaging.  Med Image Anal. 2015;24(1):190-204. doi:10.1016/j.media.2015.06.008PubMedGoogle ScholarCrossref
40.
Ding  C, Li  T, Peng  W, Park  H. Orthogonal Nonnegative Matrix T-factorizations for Clustering. In: Proceedings of the 12th Association for Computing Machinery's Special Interest Group on Knowledge Discovery and Data Mining International Conference, Philadelphia, PA; August 20-23, 2006; 126-135.
41.
Gustavsson  A, Svensson  M, Jacobi  F,  et al; CDBE2010Study Group.  Cost of disorders of the brain in Europe 2010  [published correction appears in Eur Neuropsychopharmacol. 2012;22(3):237-8].  Eur Neuropsychopharmacol. 2011;21(10):718-779. doi:10.1016/j.euroneuro.2011.08.008PubMedGoogle ScholarCrossref
42.
Huys  QJ, Maia  TV, Frank  MJ.  Computational psychiatry as a bridge from neuroscience to clinical applications.  Nat Neurosci. 2016;19(3):404-413. doi:10.1038/nn.4238PubMedGoogle ScholarCrossref
43.
Paulus  MP, Huys  QJ, Maia  TV.  A roadmap for the development of applied computational psychiatry.  Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(5):386-392. doi:10.1016/j.bpsc.2016.05.001PubMedGoogle ScholarCrossref
44.
Abi-Dargham  A, Horga  G.  The search for imaging biomarkers in psychiatric disorders.  Nat Med. 2016;22(11):1248-1255. doi:10.1038/nm.4190PubMedGoogle ScholarCrossref
45.
Eack  SM, Newhill  CE, Keshavan  MS.  Cognitive enhancement therapy improves resting-state functional connectivity in early course schizophrenia.  J Soc Social Work Res. 2016;7(2):211-230. doi:10.1086/686538PubMedGoogle ScholarCrossref
46.
Eack  SM, Hogarty  GE, Cho  RY,  et al.  Neuroprotective effects of cognitive enhancement therapy against gray matter loss in early schizophrenia: results from a 2-year randomized controlled trial.  Arch Gen Psychiatry. 2010;67(7):674-682. doi:10.1001/archgenpsychiatry.2010.63PubMedGoogle ScholarCrossref
47.
Keshavan  MS, Eack  SM, Wojtalik  JA,  et al.  A broad cortical reserve accelerates response to cognitive enhancement therapy in early course schizophrenia.  Schizophr Res. 2011;130(1-3):123-129. doi:10.1016/j.schres.2011.05.001PubMedGoogle ScholarCrossref
48.
Burcusa  SL, Iacono  WG.  Risk for recurrence in depression.  Clin Psychol Rev. 2007;27(8):959-985. doi:10.1016/j.cpr.2007.02.005PubMedGoogle ScholarCrossref
49.
Evert  H, Harvey  C, Trauer  T, Herrman  H.  The relationship between social networks and occupational and self-care functioning in people with psychosis.  Soc Psychiatry Psychiatr Epidemiol. 2003;38(4):180-188. doi:10.1007/s00127-003-0617-4PubMedGoogle ScholarCrossref
50.
Lidwall  U, Bill  S, Palmer  E, Olsson Bohlin  C.  Mental disorder sick leave in Sweden: a population study.  Work. 2018;59(2):259-272. doi:10.3233/WOR-172672PubMedGoogle ScholarCrossref
51.
Kessler  RC, Stein  MB, Petukhova  MV,  et al; Army STARRS Collaborators.  Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS).  Mol Psychiatry. 2017;22(4):544-551. doi:10.1038/mp.2016.110PubMedGoogle ScholarCrossref
52.
Cotter  J, Granger  K, Backx  R, Hobbs  M, Looi  CY, Barnett  JH.  Social cognitive dysfunction as a clinical marker: a systematic review of meta-analyses across 30 clinical conditions.  Neurosci Biobehav Rev. 2018;84:92-99. doi:10.1016/j.neubiorev.2017.11.014PubMedGoogle ScholarCrossref
53.
Glenthøj  LB, Fagerlund  B, Hjorthøj  C,  et al.  Social cognition in patients at ultra-high risk for psychosis: what is the relation to social skills and functioning?  Schizophr Res Cogn. 2016;5:21-27. doi:10.1016/j.scog.2016.06.004PubMedGoogle ScholarCrossref
54.
Simons  CJP, Bartels-Velthuis  AA, Pijnenborg  GHM; Genetic Risk and Outcome of Psychosis (GROUP) Investigators.  Cognitive performance and long-term social functioning in psychotic disorder: a three-year follow-up study.  PLoS One. 2016;11(4):e0151299. doi:10.1371/journal.pone.0151299PubMedGoogle ScholarCrossref
55.
Hua  LL, Wilens  TE, Martelon  M, Wong  P, Wozniak  J, Biederman  J.  Psychosocial functioning, familiality, and psychiatric comorbidity in bipolar youth with and without psychotic features.  J Clin Psychiatry. 2011;72(3):397-405. doi:10.4088/JCP.10m06025yelPubMedGoogle ScholarCrossref
56.
Koutsouleris  N, Davatzikos  C, Bottlender  R,  et al.  Early recognition and disease prediction in the at-risk mental states for psychosis using neurocognitive pattern classification.  Schizophr Bull. 2012;38(6):1200-1215. doi:10.1093/schbul/sbr037PubMedGoogle ScholarCrossref
57.
Bedi  G, Carrillo  F, Cecchi  GA,  et al.  Automated analysis of free speech predicts psychosis onset in high-risk youths.  NPJ Schizophr. 2015;1:15030. doi:10.1038/npjschz.2015.30PubMedGoogle ScholarCrossref
58.
Frank  E, Maier  D, Pajula  J,  et al.  Platform for systems medicine research and diagnostic applications in psychotic disorders-The METSY project.  Eur Psychiatry. 2018;50:40-46. doi:10.1016/j.eurpsy.2017.12.001PubMedGoogle ScholarCrossref
59.
Palaniyappan  L, Liddle  PF.  Does the salience network play a cardinal role in psychosis? an emerging hypothesis of insular dysfunction.  J Psychiatry Neurosci. 2012;37(1):17-27. doi:10.1503/jpn.100176PubMedGoogle ScholarCrossref
60.
Wang  H, Zeng  L-L, Chen  Y, Yin  H, Tan  Q, Hu  D.  Evidence of a dissociation pattern in default mode subnetwork functional connectivity in schizophrenia.  Sci Rep. 2015;5:14655. doi:10.1038/srep14655PubMedGoogle ScholarCrossref
61.
Fusar-Poli  P, Radua  J, McGuire  P, Borgwardt  S.  Neuroanatomical maps of psychosis onset: voxel-wise meta-analysis of antipsychotic-naive VBM studies.  Schizophr Bull. 2012;38(6):1297-1307. doi:10.1093/schbul/sbr134PubMedGoogle ScholarCrossref
62.
Wood  SJ, Yung  AR, McGorry  PD, Pantelis  C.  Neuroimaging and treatment evidence for clinical staging in psychotic disorders: from the at-risk mental state to chronic schizophrenia.  Biol Psychiatry. 2011;70(7):619-625. doi:10.1016/j.biopsych.2011.05.034PubMedGoogle ScholarCrossref
63.
Douaud  G, Mackay  C, Andersson  J,  et al.  Schizophrenia delays and alters maturation of the brain in adolescence.  Brain. 2009;132(pt 9):2437-2448. doi:10.1093/brain/awp126PubMedGoogle ScholarCrossref
64.
Falkai  P, Rossner  MJ, Schulze  TG,  et al.  Kraepelin revisited: schizophrenia from degeneration to failed regeneration.  Mol Psychiatry. 2015;20(6):671-676. doi:10.1038/mp.2015.35PubMedGoogle ScholarCrossref
65.
Bartholomeusz  CF, Cropley  VL, Wannan  C, Di Biase  M, McGorry  PD, Pantelis  C.  Structural neuroimaging across early-stage psychosis: Aberrations in neurobiological trajectories and implications for the staging model.  Aust N Z J Psychiatry. 2017;51(5):455-476. doi:10.1177/0004867416670522PubMedGoogle ScholarCrossref
66.
Torres  US, Duran  FL, Schaufelberger  MS,  et al.  Patterns of regional gray matter loss at different stages of schizophrenia: a multisite, cross-sectional VBM study in first-episode and chronic illness.  Neuroimage Clin. 2016;12:1-15. doi:10.1016/j.nicl.2016.06.002PubMedGoogle ScholarCrossref
67.
Palaniyappan  L.  Progressive cortical reorganisation: A framework for investigating structural changes in schizophrenia.  Neurosci Biobehav Rev. 2017;79:1-13. doi:10.1016/j.neubiorev.2017.04.028PubMedGoogle ScholarCrossref
68.
Ryali  S, Supekar  K, Chen  T,  et al.  Temporal dynamics and developmental maturation of salience, default and central-executive network interactions revealed by variational Bayes hidden Markov modeling.  PLoS Comput Biol. 2016;12(12):e1005138. doi:10.1371/journal.pcbi.1005138PubMedGoogle ScholarCrossref
69.
Sherman  LE, Rudie  JD, Pfeifer  JH, Masten  CL, McNealy  K, Dapretto  M.  Development of the default mode and central executive networks across early adolescence: a longitudinal study.  Dev Cogn Neurosci. 2014;10:148-159. doi:10.1016/j.dcn.2014.08.002PubMedGoogle ScholarCrossref
70.
Schmaal  L, Veltman  DJ, van Erp  TG,  et al.  Subcortical brain alterations in major depressive disorder: findings from the ENIGMA Major Depressive Disorder Working Group.  Mol Psychiatry. 2016;21(6):806-812. doi:10.1038/mp.2015.69PubMedGoogle ScholarCrossref
71.
Wang  W, Zhao  Y, Hu  X,  et al.  Conjoint and dissociated structural and functional abnormalities in first-episode drug-naive patients with major depressive disorder: a multimodal meta-analysis.  Sci Rep. 2017;7(1):10401. doi:10.1038/s41598-017-08944-5PubMedGoogle ScholarCrossref
72.
Schmaal  L, Hibar  DP, Sämann  PG,  et al.  Cortical abnormalities in adults and adolescents with major depression based on brain scans from 20 cohorts worldwide in the ENIGMA Major Depressive Disorder Working Group.  Mol Psychiatry. 2017;22(6):900-909. doi:10.1038/mp.2016.60PubMedGoogle ScholarCrossref
73.
Stratmann  M, Konrad  C, Kugel  H,  et al.  Insular and hippocampal gray matter volume reductions in patients with major depressive disorder.  PLoS One. 2014;9(7):e102692. doi:10.1371/journal.pone.0102692PubMedGoogle ScholarCrossref
74.
Insel  T, Cuthbert  B, Garvey  M,  et al.  Research domain criteria (RDoC): toward a new classification framework for research on mental disorders.  Am J Psychiatry. 2010;167(7):748-751. doi:10.1176/appi.ajp.2010.09091379PubMedGoogle ScholarCrossref
75.
Drysdale  AT, Grosenick  L, Downar  J,  et al.  Resting-state connectivity biomarkers define neurophysiological subtypes of depression.  Nat Med. 2017;23(1):28-38. doi:10.1038/nm.4246PubMedGoogle ScholarCrossref
76.
Dwyer  DB, Cabral  C, Kambeitz-Ilankovic  L,  et al.  Brain subtyping enhances the neuroanatomical discrimination of schizophrenia  [published online February 26, 2018].  Schizophr Bull. doi:10.1093/schbul/sby008PubMedGoogle Scholar
×