[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.205.87.3. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure.
Multivariate Gaussian Process Classifier (GPC) Weight Maps
Multivariate Gaussian Process Classifier (GPC) Weight Maps

The maps display a region’s contribution to overall classification accuracy combining the acquisition (left) and the extinction (right) phases. Only the highest 10% of the weights are displayed. Colors represent the absolute whole-brain GPC weights; numbers, arbitrary units (AU) to indicate the relative contribution of a region to the prediction of treatment response.

Table 1.  
Demographic and Clinical Characteristics of Responders and Nonrespondersa
Demographic and Clinical Characteristics of Responders and Nonrespondersa
Table 2.  
Top 10% Whole-Brain GPC Weights for the Combined Acquisition and Extinction Contrasts
Top 10% Whole-Brain GPC Weights for the Combined Acquisition and Extinction Contrasts
1.
Etkin  A, Wager  TD.  Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. Am J Psychiatry. 2007;164(10):1476-1488.
PubMedArticle
2.
Sylvester  CM, Corbetta  M, Raichle  ME,  et al.  Functional network dysfunction in anxiety and anxiety disorders. Trends Neurosci. 2012;35(9):527-535.
PubMedArticle
3.
Fu  CHY, Costafreda  SG.  Neuroimaging-based biomarkers in psychiatry. Can J Psychiatry. 2013;58(9):499-508.
PubMed
4.
Savitz  JB, Rauch  SL, Drevets  WC.  Clinical application of brain imaging for the diagnosis of mood disorders: the current state of play. Mol Psychiatry. 2013;18(5):528-539.
PubMedArticle
5.
Fu  CHY, Mourao-Miranda  J, Costafreda  SG,  et al.  Pattern classification of sad facial processing: toward the development of neurobiological markers in depression. Biol Psychiatry. 2008;63(7):656-662.
PubMedArticle
6.
Hahn  T, Marquand  AF, Ehlis  AC,  et al.  Integrating neurobiological markers of depression. Arch Gen Psychiatry. 2011;68(4):361-368.
PubMedArticle
7.
Mourão-Miranda  J, Almeida  JR, Hassel  S,  et al.  Pattern recognition analyses of brain activation elicited by happy and neutral faces in unipolar and bipolar depression. Bipolar Disord. 2012;14(4):451-460.
PubMedArticle
8.
Orrù  G, Pettersson-Yeo  W, Marquand  AF, Sartori  G, Mechelli  A.  Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci Biobehav Rev. 2012;36(4):1140-1152.
PubMedArticle
9.
Klöppel  S, Abdulkadir  A, Jack  CR  Jr, Koutsouleris  N, Mourão-Miranda  J, Vemuri  P.  Diagnostic neuroimaging across diseases. Neuroimage. 2012;61(2):457-463.
PubMedArticle
10.
Sundermann  B, Herr  D, Schwindt  W, Pfleiderer  B.  Multivariate classification of blood oxygen level–dependent fMRI data with diagnostic intention: a clinical perspective. AJNR Am J Neuroradiol. 2014;35(5):848-855.
PubMedArticle
11.
Gong  Q, Wu  Q, Scarpazza  C,  et al.  Prognostic prediction of therapeutic response in depression using high-field MR imaging. Neuroimage. 2011;55(4):1497-1503.
PubMedArticle
12.
Costafreda  SG, Chu  C, Ashburner  J, Fu  CHY.  Prognostic and diagnostic potential of the structural neuroanatomy of depression. PLoS One. 2009;4(7):e6353. doi:10.1371/journal.pone.0006353.
PubMedArticle
13.
Costafreda  SG, Khanna  A, Mourao-Miranda  J, Fu  CHY.  Neural correlates of sad faces predict clinical remission to cognitive behavioural therapy in depression. Neuroreport. 2009;20(7):637-641.
PubMedArticle
14.
Doehrmann  O, Ghosh  SS, Polli  FE,  et al.  Predicting treatment response in social anxiety disorder from functional magnetic resonance imaging. JAMA Psychiatry. 2013;70(1):87-97.
PubMedArticle
15.
Wittchen  HU, Jacobi  F, Rehm  J,  et al.  The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21(9):655-679.
PubMedArticle
16.
Nutt  DJ.  The full cost and burden of disorders of the brain in Europe exposed for the first time. Eur Neuropsychopharmacol. 2011;21(10):715-717.
PubMedArticle
17.
McHugh  RK, Smits  JA, Otto  MW.  Empirically supported treatments for panic disorder. Psychiatr Clin North Am. 2009;32(3):593-610.
PubMedArticle
18.
Gloster  AT, Wittchen  HU, Einsle  F,  et al.  Psychological treatment for panic disorder with agoraphobia: a randomized controlled trial to examine the role of therapist-guided exposure in situ in CBT. J Consult Clin Psychol. 2011;79(3):406-420.
PubMedArticle
19.
Sánchez-Meca  J, Rosa-Alcázar  AI, Marín-Martínez  F, Gómez-Conesa  A.  Psychological treatment of panic disorder with or without agoraphobia: a meta-analysis. Clin Psychol Rev. 2010;30(1):37-50.
PubMedArticle
20.
Bouton  ME, Mineka  S, Barlow  DH.  A modern learning theory perspective on the etiology of panic disorder. Psychol Rev. 2001;108(1):4-32.
PubMedArticle
21.
Lissek  S, Powers  AS, McClure  EB,  et al.  Classical fear conditioning in the anxiety disorders: a meta-analysis. Behav Res Ther. 2005;43(11):1391-1424.
PubMedArticle
22.
Lissek  S, Rabin  S, Heller  RE,  et al.  Overgeneralization of conditioned fear as a pathogenic marker of panic disorder. Am J Psychiatry. 2010;167(1):47-55.
PubMedArticle
23.
Michael  T, Blechert  J, Vriends  N, Margraf  J, Wilhelm  FH.  Fear conditioning in panic disorder: enhanced resistance to extinction. J Abnorm Psychol. 2007;116(3):612-617.
PubMedArticle
24.
Tuescher  O, Protopopescu  X, Pan  H,  et al.  Differential activity of subgenual cingulate and brainstem in panic disorder and PTSD. J Anxiety Disord. 2011;25(2):251-257.
PubMedArticle
25.
Kircher  T, Arolt  V, Jansen  A,  et al.  Effect of cognitive-behavioral therapy on neural correlates of fear conditioning in panic disorder. Biol Psychiatry. 2013;73(1):93-101.
PubMedArticle
26.
Lueken  U, Straube  B, Konrad  C,  et al.  Neural substrates of treatment response to cognitive-behavioral therapy in panic disorder with agoraphobia. Am J Psychiatry. 2013;170(11):1345-1355.
PubMedArticle
27.
Gloster  AT, Wittchen  HU, Einsle  F,  et al.  Mechanism of Action in CBT (MAC): methods of a multi-center randomized controlled trial in 369 patients with panic disorder and agoraphobia. Eur Arch Psychiatry Clin Neurosci. 2009;259(suppl 2):S155-S166.
PubMedArticle
28.
Bandelow  B. Panic and Agoraphobia Scale (PAS). Ashland, OH: Hogrefe & Huber Publishers; 1999.
29.
Guy  W. ECDEU Assessment Manual for Psychopharmacology Review. Rockville, MD: US National Institute of Health, Psychopharmacology Research Branch; 1976.
30.
Reiss  S, Peterson  RA, Gursky  DM, McNally  RJ.  Anxiety sensitivity, anxiety frequency and the prediction of fearfulness. Behav Res Ther. 1986;24(1):1-8.
PubMedArticle
31.
Beck  AT, Steer  RA, Brown  GK. Beck Depression Inventory.2nd ed. San Antonio, TX: Psychological Corp; 1996.
32.
Shear  MK, Vander Bilt  J, Rucci  P,  et al.  Reliability and validity of a structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depress Anxiety. 2001;13(4):166-178.
PubMedArticle
33.
Gloster  AT, Hauke  C, Höfler  M,  et al.  Long-term stability of cognitive behavioral therapy effects for panic disorder with agoraphobia: a two-year follow-up study. Behav Res Ther. 2013;51(12):830-839.
PubMedArticle
34.
Reinhardt  I, Jansen  A, Kellermann  T,  et al.  Neural correlates of aversive conditioning. Eur Arch Psychiatry Clin Neurosci. 2010;260(6):443-453.
PubMedArticle
35.
Carter  RM, Bowling  DL, Reeck  C, Huettel  SA.  A distinct role of the temporal-parietal junction in predicting socially guided decisions. Science. 2012;337(6090):109-111.
PubMedArticle
36.
Marquand  A, Howard  M, Brammer  M, Chu  C, Coen  S, Mourão-Miranda  J.  Quantitative prediction of subjective pain intensity from whole-brain fMRI data using gaussian processes. Neuroimage. 2010;49(3):2178-2189.
PubMedArticle
37.
Rasmussen  CE, Williams  CKI. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press; 2006.
38.
Hahn  T, Marquand  AF, Plichta  MM,  et al.  A novel approach to probabilistic biomarker-based classification using functional near-infrared spectroscopy. Hum Brain Mapp. 2013;34(5):1102-1114.
PubMedArticle
39.
Shen  H, Wang  L, Liu  Y, Hu  D.  Discriminative analysis of resting-state functional connectivity patterns of schizophrenia using low dimensional embedding of fMRI. Neuroimage. 2010;49(4):3110-3121.
PubMedArticle
40.
Costafreda  SG, Fu  CHY, Picchioni  M,  et al.  Pattern of neural responses to verbal fluency shows diagnostic specificity for schizophrenia and bipolar disorder. BMC Psychiatry. 2011;11(18):18. doi:10.1186/1471-244X-11-18.
PubMedArticle
41.
Marquand  AF, Mourão-Miranda  J, Brammer  MJ, Cleare  AJ, Fu  CHY.  Neuroanatomy of verbal working memory as a diagnostic biomarker for depression. Neuroreport. 2008;19(15):1507-1511.
PubMedArticle
42.
Sehlmeyer  C, Schöning  S, Zwitserlood  P,  et al.  Human fear conditioning and extinction in neuroimaging: a systematic review. PLoS One. 2009;4(6):e5865. doi:10.1371/journal.pone.0005865.
PubMedArticle
43.
Lueken  U, Straube  B, Reinhardt  I,  et al.  Altered top-down and bottom-up processing of fear conditioning in panic disorder with agoraphobia. Psychol Med. 2014;44(2):381-394.
PubMedArticle
44.
Mantovani  A, Aly  M, Dagan  Y, Allart  A, Lisanby  SH.  Randomized sham controlled trial of repetitive transcranial magnetic stimulation to the dorsolateral prefrontal cortex for the treatment of panic disorder with comorbid major depression. J Affect Disord. 2013;144(1-2):153-159.
PubMedArticle
45.
Otto  MW, Tolin  DF, Simon  NM,  et al.  Efficacy of d-cycloserine for enhancing response to cognitive-behavior therapy for panic disorder. Biol Psychiatry. 2010;67(4):365-370.
PubMedArticle
46.
Siegmund  A, Golfels  F, Finck  C,  et al.  d-Cycloserine does not improve but might slightly speed up the outcome of in-vivo exposure therapy in patients with severe agoraphobia and panic disorder in a randomized double blind clinical trial. J Psychiatr Res. 2011;45(8):1042-1047.
PubMedArticle
47.
Reif  A, Richter  J, Straube  B,  et al.  MAOA and mechanisms of panic disorder revisited. Mol Psychiatry. 2014;19(1):122-128.
PubMedArticle
Original Investigation
January 2015

Predicting Treatment Response to Cognitive Behavioral Therapy in Panic Disorder With Agoraphobia by Integrating Local Neural Information

Author Affiliations
  • 1Department of Cognitive Psychology II, Goethe University Frankfurt am Main, Frankfurt am Main, Germany
  • 2Department of Psychiatry and Psychotherapy, Philipps University of Marburg, Marburg, Germany
  • 3Institute of Clinical Psychology and Psychotherapy, Department of Psychology, Technische Universität Dresden, Dresden, Germany
  • 4Neuroimaging Center, Department of Psychology, Technische Universität Dresden, Dresden, Germany
  • 5Department of Psychiatry and Psychotherapy, Campus Charité Mitte, Charité–University Medicine Berlin, Berlin, Germany
  • 6Department of Clinical Radiology, University Hospital Münster, Münster, Germany
  • 7Department of Psychiatry, Psychosomatics, and Psychotherapy, University Hospital of Würzburg, Würzburg, Germany
  • 8Department of Psychiatry and Psychotherapy, University Hospital Münster, Münster, Germany
JAMA Psychiatry. 2015;72(1):68-74. doi:10.1001/jamapsychiatry.2014.1741
Abstract

Importance  Although neuroimaging research has made substantial progress in identifying the large-scale neural substrate of anxiety disorders, its value for clinical application lags behind expectations. Machine-learning approaches have predictive potential for individual-patient prognostic purposes and might thus aid translational efforts in psychiatric research.

Objective  To predict treatment response to cognitive behavioral therapy (CBT) on an individual-patient level based on functional magnetic resonance imaging data in patients with panic disorder with agoraphobia (PD/AG).

Design, Setting, and Participants  We included 49 patients free of medication for at least 4 weeks and with a primary diagnosis of PD/AG in a longitudinal study performed at 8 clinical research institutes and outpatient centers across Germany. The functional magnetic resonance imaging study was conducted between July 2007 and March 2010.

Interventions  Twelve CBT sessions conducted 2 times a week focusing on behavioral exposure.

Main Outcomes and Measures  Treatment response was defined as exceeding a 50% reduction in Hamilton Anxiety Rating Scale scores. Blood oxygenation level–dependent signal was measured during a differential fear-conditioning task. Regional and whole-brain gaussian process classifiers using a nested leave-one-out cross-validation were used to predict the treatment response from data acquired before CBT.

Results  Although no single brain region was predictive of treatment response, integrating regional classifiers based on data from the acquisition and the extinction phases of the fear-conditioning task for the whole brain yielded good predictive performance (accuracy, 82%; sensitivity, 92%; specificity, 72%; P < .001). Data from the acquisition phase enabled 73% correct individual-patient classifications (sensitivity, 80%; specificity, 67%; P < .001), whereas data from the extinction phase led to an accuracy of 74% (sensitivity, 64%; specificity, 83%; P < .001). Conservative reanalyses under consideration of potential confounders yielded nominally lower but comparable accuracy rates (acquisition phase, 70%; extinction phase, 71%; combined, 79%).

Conclusions and Relevance  Predicting treatment response to CBT based on functional neuroimaging data in PD/AG is possible with high accuracy on an individual-patient level. This novel machine-learning approach brings personalized medicine within reach, directly supporting clinical decisions for the selection of treatment options, thus helping to improve response rates.

Introduction

Although neuroimaging research has made substantial progress in identifying the neural substrate of anxiety disorders,1,2 its value for clinical translational research lags behind expectations. One reason is that most studies are based on conventional group analyses. In contrast, a useful biomarker has to provide sufficient sensitivity and specificity to predict a given patient’s status on the individual level.3,4 Machine-learning algorithms, such as Support Vector Machines or gaussian process classifiers (GPC), have shown predictive potential for single-subject diagnostic purposes.58

Although results of such proof-of-concept studies appear highly encouraging, predictive approaches need to facilitate clinical decisions. To date, however, most studies that use machine learning merely predict a diagnostic label (a review and notable exceptions appear in Klöppel et al9 and Sundermann et al10). Among such relevant clinical decisions, the prediction of treatment outcome is of utmost importance because identifying individuals who will not benefit from a therapeutic intervention enables early treatment modification and may offer better outcomes for otherwise nonresponsive patients.

Evidence from the field of mood disorders suggests that neuroimaging biomarkers may aid in classifying disease status6 and predicting response to pharmacological treatment11 or cognitive behavioral therapy (CBT).12,13 However, despite the large number of neuroimaging studies on anxiety disorders, analyses on response markers in this patient group are surprisingly scarce. Application of regression methods to neuroimaging data from social anxiety disorder14 has resulted in improved rates of explained variance compared with clinical outcome parameters. These improved rates have been reported when adding neuroimaging data (12% vs 41%) that were tested in a cross-validation procedure using machine-learning algorithms.

As one of the most disabling anxiety disorders, panic disorder (PD) is associated with high individual and societal burdens.15,16 Although CBT has been proven efficient and is recommended as a first-line treatment for PD,17 response rates are far from satisfying. Recent data from the national research initiative PANIC-NET originating from the multicenter trial Mechanism of Action in CBT on patients with PD with agoraphobia (PD/AG) who were treated with exposure-based CBT indicated positive response rates in approximately half of the sample.18 Comparable effect sizes have been reported for other randomized clinical trials as shown in a meta-analysis by Sánchez-Meca et al.19 Identifying patients with a higher risk for nonresponse before treatment could allow for individualized treatment decisions and thus improve response rates.

Fear conditioning is a core process for the development and maintenance of PD/AG.20 The nature of the learning deficit remains poorly described, but studies point toward the relevance of discriminatory learning, overgeneralization of fear, and attenuated extinction learning.2123 On a neural level, brain systems mediating threat in response to stimuli that signal safety have been reported as a pathophysiological correlate of PD/AG.24 In line with this correlate, results in a subset of PD/AG patients from the previous study18,25,26 indicate that, before treatment, nonresponse was characterized by enhanced safety-signal processing in brain systems associated with the processing of threat (eg, pregenual anterior cingulate cortex, right hippocampus, and right amygdala) and altered anterior cingulate cortex–amygdala coupling.26

Although these findings are informative about phenotype characteristics related to treatment response, they are not applicable to the individual patient. Herein we aim to predict individual responses to CBT in the previously described PD/AG sample,25 thereby providing information of high clinical relevance based on functional magnetic resonance imaging (fMRI) data. We developed a novel multivariate pattern classification approach with regional patterns of neural responses while allowing for the integration of local predictions in the whole brain, thus considering local and whole-brain neural information to facilitate the prediction of treatment response in individual patients.

Methods
Multicenter Mechanism of Action in CBT Study

This work is part of the German multicenter trial Mechanism of Action in CBT.27 After a complete description of the study protocol, written informed consent was obtained from the participants, and the protocol was approved by the local ethics committees in each fMRI center according to the Declaration of Helsinki. The randomized clinical trial (isrctn.org identifier: ISRCTN80046034) was approved by the ethics committee of the Medical Faculty of the Technische Universität Dresden (agreement EK 164082006). The neuroimaging components were approved by the ethics committee of the Medical Faculty of the RWTH Aachen University, Aachen (agreement EK 073/07) and at all local sites. The experimental pharmacology study was approved by the ethics committee of the state of Berlin (EudraCT 2006-00-4860-29).

Within the German psychotherapy research network PANIC-NET, a multicenter randomized clinical trial of CBT for PD/AG patients was conducted in 8 centers across Germany. Three hundred sixty-nine patients free of medication for at least 4 weeks who met DSM-IV-TR criteria for PD/AG were treated with a manualized treatment protocol or assigned to a waiting list that consisted of 12 sessions of CBT 2 times a week focusing on behavioral exposure in situ. Two procedural variants of CBT (identical in content and dose) were compared that differed only with regard to therapist guidance during exposure sessions. In the therapist-guided condition, the therapist accompanied the patient during exposure, whereas patients were instructed by the therapist but performed the exposure on their own in the nonguided condition. Patients were randomly assigned to 1 of the 2 CBT arms; both groups exhibited significant symptom reduction after CBT18; therefore, they were combined in the fMRI responder analyses26 (Table 1). Response was defined as a reduction in Hamilton Anxiety Rating Scale scores32 exceeding 50% from baseline to posttreatment assessment.26 Details on the study design and immediate and long-term treatment effects have been reported previously18,33 (eAppendix in the Supplement). Four of the 8 centers participated in an fMRI add-on study, yielding 49 quality-controlled fMRI data sets at baseline. The fMRI study was conducted between July 2007 and March 2010. Patient characteristics are described in Table 1 (a detailed description of measures of quality control and the fMRI patient flowchart appears elsewhere25).

fMRI Assessment

The task, fMRI data acquisition, and preprocessing pathways have been described in detail.25,26,34 We applied a differential fear-conditioning task. Colored geometric stimuli served as conditioned stimuli (CS) (presentation time, 2000 milliseconds, with a variable intertrial interval of 4785-7250 milliseconds). An aversive auditory tone (white noise, 100 milliseconds) was used as the unconditioned stimulus (US) that was pseudorandomly paired with a CS (CS+) during the acquisition phase, whereas the unpaired CS (CS−) was never followed by the US (counterbalanced among patients; reinforcement rate, 50%). During acquisition, only those trials in which no US was delivered (CS+ unpaired) were analyzed. Image acquisition and analysis pathways are described in the eAppendix in the Supplement. For the GPC analyses, we used contrast maps from individual patients reflecting the main contrasts of interest in a differential fear-conditioning task (acquisition and extinction phases).

Gaussian Process Classification
Regional GPCs

Using the contrast images for the acquisition and the extinction phases, whole-brain data from 55 regions drawn from the Harvard-Oxford Brain Atlas as described in Carter et al35 were extracted for each patient. These data were analyzed as described by Marquand et al36 using GPCs37 (details are given in the eAppendix in the Supplement). We predicted a patient’s probability to be a responder independently for each region based on all voxels within the respective region using leave-one-out cross-validation.

We evaluated the performance of the 55 regional classifiers by converting the predictive probabilities to categorical predictions. We applied a threshold that categorized a patient as a responder or a nonresponder if his or her probability of response was larger than 0.5 or smaller than 0.5, respectively. Accuracies were calculated as the ratio of correct predictions to the number of patients for each GPC. Because the numbers of responders and nonresponders were not the same, classification accuracies were calibrated.38 To establish whether these regional accuracies are statistically significant, we ran each classifier 1000 times with randomly permuted labels and counted the number of permutations that achieved higher accuracy than the one observed with the true labels. We calculated the P value by dividing this number by 1000. We corrected for multiple comparisons (for 55 regions) using the false discovery rate.

Integration of Regional Predictive Probabilities

We applied a second classifier with the same specifications used for the local classifiers to integrate the predictive classification probabilities obtained from the local leave-one-out classifiers. Those probabilities were used as predictors for the algorithm in order to determine the classification of a participant. To calculate the overall prediction accuracy of this approach, a nested leave-one-out procedure was implemented (details are given in the eAppendix in the Supplement). This procedure ensures complete independence of the training and the test data set.

The significance of the whole-brain classifier’s prediction accuracy was tested in analogy to the regional classifiers. To compare the best local classifier with the whole-brain classifier, we used a permutation-based procedure. We first calculated the difference between the best local classifier and the whole-brain classifier. Then, we approximated the null distribution of this difference by computing the difference between the 2 respective classifiers based on 1000 accuracies under permutation for the best local classifier and the whole-brain classifier. Finally, we deemed 2 accuracies significantly different if less than 5% of the absolute accuracy differences under permutation were larger than the difference obtained with the true labels.

Regional and Multivariate Mapping

To determine those brain regions that contributed most to classification, we derived weight maps from the GPC models as described in Marquand et al36 and calculated the means for all cross-validation folds. Although this procedure provides a multivariate estimate of the contribution of each region to classifier performance, one should be aware that the maps describe a nonlinear multivariate pattern. Importance scores for each of the regions should be interpreted in the context of the entire multivariate pattern. Against this background, we also present a more readily interpretable, univariate mapping procedure in which we computed classification accuracies for each of the 55 regions separately as described above. A region is shown if the accuracy estimate for this region exceeded chance level (P < .05). As described above, we corrected for multiple comparisons (for 55 regions) using the false discovery rate.

Results
Regional GPCs

Independent regional GPCs from each of the 55 brain regions revealed significant accuracies for conditioned responses during acquisition (CS+ unpaired > CS−) for the inferior frontal gyrus (pars triangularis), which was greater than chance level (accuracy, 69%; P = .006) but did not survive multiple comparison correction. For the contrast reflecting conditioned responses during extinction (CS+ > CS−), 2 regions were identified: the anterior division of the middle temporal gyrus (accuracy, 65%; P = .03) and the postcentral gyrus (accuracy, 67%; P = .03). Again, none of these displayed accuracy greater than chance after correction for multiple comparisons.

Integration of Regional Predictive Probabilities

Integrating the descriptive probabilities from all regional classifiers for the acquisition and the extinction phases of the fear-conditioning task yielded high predictive performance (accuracy, 82%; sensitivity, 92%; specificity, 72%; P < .001). This finding constituted an improvement in accuracy of 13% (P = .04) compared with the single best of all local classifiers. Data from the acquisition phase alone enabled 73% correct individual-patient classifications (sensitivity, 80%; specificity, 67%; P < .001). For the contrast extinction (CS+ > CS−), integrating the descriptive probabilities from all regional classifiers led to an accuracy of 74% (sensitivity, 64%; specificity, 83%; P < .001).

When we considered the patterns learned by the whole-brain classifier for both contrasts of interest, integrating regional predictions using a GPC algorithm substantially boosted classification accuracy by integrating GPC-predictive probabilities (Figure and Table 2). These regions did not have the highest single GPC accuracies but represent the most informative regions in a multivariate, whole-brain framework for both contrasts combined.

We also conducted a tougher test by taking into account a nonsignificant trend in age differences and a second assessment just before the start of the therapy (Panic and Agoraphobia Scale values28). Because the acquisition of Panic and Agoraphobia Scale values thus technically took place when the treatment had already begun, this analysis might be too conservative. Combining both data sources as described above yielded an accuracy of 79% (sensitivity, 76%; specificity, 82%; P < .001), with an accuracy of 70% (sensitivity, 64%; specificity, 76%; P < .001) for the acquisition phase and 71% (sensitivity, 74%; specificity, 68%; P < .001) for the extinction phase. Although nominally lower than the accuracies reported in the first analyses, the accuracies obtained when taking into account age and Panic and Agoraphobia Scale values did not significantly differ from those obtained in the first analyses (P > .40).

Discussion

Although considerable improvements in understanding the neural substrates of anxiety disorders have been achieved, the utility of this knowledge regarding clinical application is limited. As a proof-of-concept study in PD/AG, this report demonstrates that fMRI data acquired before CBT allow for response prediction on an individual-patient level. When we integrated regional predictive probabilities from whole-brain fMRI data, patients could be classified correctly as responders or nonresponders with an accuracy of 82%. Controlling for potential confounders still yielded an accuracy of 79%.

Machine learning has been successfully applied to classify disease state or to predict the onset of disease in neurological or psychiatric patient groups.8 For psychiatric disorders, classification accuracies based on fMRI data have been reported in the range of 84%39 to 92%40 for schizophrenia or from 67% to 86% in major depression.5,6,13,41 Compared with the relatively large number of studies on disease classification, surprisingly few exist on prognostic markers, almost all of which focus on major depression. With a single exception,13 all of these studies5,11,12 investigated response to pharmacotherapy. In the only available study on treatment response to CBT, Costafreda et al13 observed a sensitivity of 71% and a specificity of 86%, leading to an accuracy of 78% for predicting response in a small sample of 16 patients with major depression. In a similar vein, Doehrmann et al14 demonstrated the potential of fMRI data in predicting treatment response to CBT in social anxiety disorder. Present findings with an accuracy rate of 82% are comparable to the highest obtained in this field. After controlling for potential confounders, the accuracy rate was still in the range of previous reports. Because the Panic and Agoraphobia Scale test was taken within a second assessment just before CBT onset and not after study inclusion, these differences may not reflect true baseline differences, rendering the reanalysis possibly too conservative.

Because our study used a different functional paradigm and aimed at predicting treatment response in patients with PD/AG and not major depression, comparison of regional contributions to overall prediction is of limited utility. Costafreda et al13 applied an emotional face perception task in patients with major depression, whereas the present study investigated neural responses during fear conditioning as a pathomechanism in PD/AG.20 Fear conditioning has been reported to activate a widespread network consisting of subcortical structures, such as the thalamus, amygdala, and hippocampus, but also the anterior cingulate cortex, prefrontal and orbitofrontal cortices, and temporal regions.42 In previous reports on this sample,25,43 PD/AG patients exhibited increased inferior frontal gyrus activity during differential conditioning, which possibly indicates aberrant cognitive processing or behavioral inhibition that was sensitive to treatment. The regions contributing to the predictive performance in the present analysis (combined whole-brain acquisition and extinction data) partly overlap with fear network circuits associated with fear conditioning, including the orbitofrontal cortices and the inferior frontal gyrus.

From a health economics viewpoint, identifying potential nonresponders before an intervention is highly desirable. Patients who are not likely to respond could be provided with additional treatment options. Experimental augmentation strategies, such as repetitive transcranial magnetic stimulation,44 or cognitive enhancers, such as d-cycloserine,45,46 are undergoing evaluation for anxiety disorders that include PD. A prognostic marker with high sensitivity would be desirable to stratify patients to available add-on therapies and therefore speed up treatment time and increase response rates. In that sense, a sensitivity of 92% represents a near-maximum detection rate. Patients predicted not to benefit might respond favorably from intensified psychotherapy or neurotherapy/pharmacotherapy, which has yet to be established. Nevertheless, such approaches might aid to govern treatment regimens in the sense of personalized medicine approaches. Future research should apply prognostic markers in an independent sample and evaluate the increment in treatment success and cost-effectiveness. We acknowledge that neuroimaging is not yet a routine diagnostic tool in psychiatry. Thus, health economic analyses are needed to balance the costs of additional diagnostic procedures against the potential benefits of improved outcomes.

The present results are based on a medication-free patient sample treated solely with CBT, which limits their generalization to other populations with PD/AG or anxiety disorders and to alternative (eg, pharmacological) treatment. To conform to previous reports on this sample,18,26,33 we dichotomized treatment response according to the primary outcome. However, treatment effects are continuous, and information might get lost using dichotomization. Not having true baseline fMRI (eg, resting state) data precludes us from interpreting the specificity of findings for the selected contrasts. The discriminatory pattern could represent state, task-dependent effects as a function of response (eg, altered fear conditioning and safety-signal processing26) or trait effects. Although we used a nested leave-one-out cross-validation procedure and showed good generalization performance on a comparatively large sample of patients in this study, investigations on heterogeneous real-life samples are needed to determine whether our model will perform equally well in other patient samples. However, including 4 separate sites with 4 different scanners represents a strength suggesting a certain degree of generalizability. Results do not inform about potential mechanisms underlying processes of change. In that sense, this approach is complementary to group comparisons that help identify phenotype characteristics.25,26,43,47

Conclusions

We applied a novel machine-learning approach to investigate the potential of fMRI data for CBT response prediction on the individual level in a large sample of PD/AG patients. Findings showed an accuracy of 82% with a high sensitivity of 92%. The present work can be understood as a proof-of-concept study showing the potential utility of our approach for individual-patient response prediction. This multivariate, individual-patient classification approach substantially facilitates current translational efforts aimed at personalizing treatment and supporting clinical decisions when selecting among treatment options, thus helping to improve response rates.

Back to top
Article Information

Submitted for Publication: January 31, 2014; final revision received July 17, 2014; accepted July 21, 2014.

Corresponding Author: Ulrike Lueken, PhD, Institute of Clinical Psychology and Psychotherapy, Department of Psychology, Technische Universität Dresden, Chemnitzer Straße 46, D-01187 Dresden, Germany (ulrike.lueken@tu-dresden.de).

Published Online: November 19, 2014. doi:10.1001/jamapsychiatry.2014.1741.

Author Contributions: Drs Hahn and Lueken had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Kircher, Wittchen, Konrad, Ströhle, Reif, Arolt, Lueken.

Acquisition, analysis, or interpretation of data: Hahn, Straube, Wittchen, Konrad, Ströhle, Wittmann, Pfleiderer, Arolt, Lueken.

Drafting of the manuscript: Hahn, Kircher, Arolt, Lueken.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hahn, Wittchen, Lueken.

Obtained funding: Kircher, Wittchen, Konrad, Ströhle, Arolt.

Administrative, technical, or material support: Hahn, Kircher, Straube, Wittchen, Konrad, Ströhle, Wittmann, Reif, Arolt.

Study supervision: Kircher, Straube, Wittchen, Konrad, Ströhle, Pfleiderer, Lueken.

Conflict of Interest Disclosures: Dr Kircher received fees for educational programs from Astra-Zeneca, Bristol-Myers Squibb, Eli Lilly and Company, Janssen-Cilag, Lundbeck, Pfizer, and Servier; received travel support/sponsorship for congresses from Servier; received speaker honoraria from Janssen-Cilag; and received research grants from Lundbeck and Pfizer. Dr Wittchen has served as a general consultant (not related to the product) for Essex Pharma, Organon, Pfizer, and Servier and has received grant funding for his institution from Essex Pharma, Lundbeck, Novartis, Pfizer, sanofi-aventis, Servier, and Whyet. Dr Konrad received fees for an educational program from Esparma GmbH/Aristo Pharma GmbH, Lilly Deutschland GmbH, MagVenture GmbH, and Servier Deutschland GmbH. Dr Ströhle received research funding from the European Commission (FP6), the German Federal Ministry of Education and Research, and Lundbeck and speaker honoraria from AstraZeneca, Boehringer Ingelheim, Eli Lilly and Company, Lundbeck, Pfizer, UCB, and Wyeth. Dr Reif participated in 2 noninterventional trials sponsored by AstraZeneca. Dr Arolt is member of advisory boards and/or gave presentations for AstraZeneca, Eli Lilly and Company, Janssen-Organon, Lundbeck, Pfizer, Servier, and Wyeth and received research grants from AstraZeneca, Lundbeck, and Servier. Dr Ströhle has received educational grants given by the Stifterverband für die Deutsche Wissenschaft, the Berlin Brandenburgische Akademie der Wissenschaften, the Boehringer Ingelheim Fonds, and the Eli Lilly International Foundation. No other disclosures were reported.

Funding/Support: This work is part of the German multicenter trial Mechanism of Action in CBT (MAC), which is supported by projects 01GV0615 and 01GV0611 (neuroimaging study) by the German Federal Ministry of Education and Research (BMBF) as part of the BMBF Psychotherapy Research Funding Initiative.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Previous Presentation: This paper was presented as a poster at the 20th Annual Meeting of the Organization for Human Brain Mapping; June 9, 2014; Hamburg, Germany. The paper was presented at the 27th European Colleague in Neuropsychopharmacology Congress; October 19, 2014; Berlin, Germany.

Additional Information: The following principal investigators (PIs) participated in the MAC trial (areas of responsibility): V. Arolt (overall MAC program coordination), Münster, Germany; H.-U. Wittchen (PI for the randomized clinical trial [RCT] and manual 4 development), Dresden, Germany; A. Hamm (PI for psychophysiology), Greifswald, Germany; A. L. Gerlach (PI for psychophysiology and panic subtypes), Münster; A. Ströhle (PI for experimental pharmacology), Berlin, Germany; T. Kircher (PI for functional neuroimaging), Marburg, Germany; and J. Deckert (PI for genetics), Würzburg, Germany. Additional site directors in the RCT component of the program are G. W. Alpers, Würzburg; T. Fydrich and L. Fehm, Berlin-Adlershof, Germany; and T. Lang, Bremen, Germany. The following staff members participated by site: C. Melzig, J. Richter, S. Richter, and M. von Rad, Greifswald (coordinating site for psychophysiology); H. Bruhn, A. Siegmund, M. Stoy, and A. Wittmann, Berlin-Charité, Germany (coordinating center for experimental pharmacology); I. Schulz, Berlin-Adlershof; A. Behnken, K. Domschke, A. Ewert, C. Konrad, B. Pfleiderer, C. Uhlmann, and P. Zwanzger, Münster (overall MAC program coordination, genetics and functional neuroimaging); J. Eidecker, S. Koller, F. Rist, and A. Vossbeck-Elsebusch, Münster (coordinating site for psychophysiology and subtyping); B. Drüke, S. Eskens, T. Forkmann, S. Gauggel, S. Gruber, A. Jansen, T. Kellermann, I. Reinhardt, and N. Vercamer-Fabri, Marburg/Aachen, Germany (coordinating center for functional neuroimaging); F. Einsle, C. Froehlich, A. T. Gloster, C. Hauke, S. Heinze, M. Hoefler, U. Lueken, P. Neudeck, S. Preiß, and D. Westphal, Dresden (coordinating site for data collection, analysis, and the RCT); A. Reif and C. Gagel, Würzburg Psychiatry Department (coordinating center for genetics); J. Duerner, H. Eisenbarth, A. B. M. Gerdes, H. Krebs, P. Pauli, S. Schad, and N. Steinhäuser, Würzburg Psychology Department; V. Bamann, S. Helbig-Lang, A. Kordt, P. Ley, F. Petermann, and E.-M. Schroeder, Bremen. Additional support was provided by X. Graehlert and M. Käppler, KKS Dresden (coordinating center for clinical studies).

References
1.
Etkin  A, Wager  TD.  Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. Am J Psychiatry. 2007;164(10):1476-1488.
PubMedArticle
2.
Sylvester  CM, Corbetta  M, Raichle  ME,  et al.  Functional network dysfunction in anxiety and anxiety disorders. Trends Neurosci. 2012;35(9):527-535.
PubMedArticle
3.
Fu  CHY, Costafreda  SG.  Neuroimaging-based biomarkers in psychiatry. Can J Psychiatry. 2013;58(9):499-508.
PubMed
4.
Savitz  JB, Rauch  SL, Drevets  WC.  Clinical application of brain imaging for the diagnosis of mood disorders: the current state of play. Mol Psychiatry. 2013;18(5):528-539.
PubMedArticle
5.
Fu  CHY, Mourao-Miranda  J, Costafreda  SG,  et al.  Pattern classification of sad facial processing: toward the development of neurobiological markers in depression. Biol Psychiatry. 2008;63(7):656-662.
PubMedArticle
6.
Hahn  T, Marquand  AF, Ehlis  AC,  et al.  Integrating neurobiological markers of depression. Arch Gen Psychiatry. 2011;68(4):361-368.
PubMedArticle
7.
Mourão-Miranda  J, Almeida  JR, Hassel  S,  et al.  Pattern recognition analyses of brain activation elicited by happy and neutral faces in unipolar and bipolar depression. Bipolar Disord. 2012;14(4):451-460.
PubMedArticle
8.
Orrù  G, Pettersson-Yeo  W, Marquand  AF, Sartori  G, Mechelli  A.  Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci Biobehav Rev. 2012;36(4):1140-1152.
PubMedArticle
9.
Klöppel  S, Abdulkadir  A, Jack  CR  Jr, Koutsouleris  N, Mourão-Miranda  J, Vemuri  P.  Diagnostic neuroimaging across diseases. Neuroimage. 2012;61(2):457-463.
PubMedArticle
10.
Sundermann  B, Herr  D, Schwindt  W, Pfleiderer  B.  Multivariate classification of blood oxygen level–dependent fMRI data with diagnostic intention: a clinical perspective. AJNR Am J Neuroradiol. 2014;35(5):848-855.
PubMedArticle
11.
Gong  Q, Wu  Q, Scarpazza  C,  et al.  Prognostic prediction of therapeutic response in depression using high-field MR imaging. Neuroimage. 2011;55(4):1497-1503.
PubMedArticle
12.
Costafreda  SG, Chu  C, Ashburner  J, Fu  CHY.  Prognostic and diagnostic potential of the structural neuroanatomy of depression. PLoS One. 2009;4(7):e6353. doi:10.1371/journal.pone.0006353.
PubMedArticle
13.
Costafreda  SG, Khanna  A, Mourao-Miranda  J, Fu  CHY.  Neural correlates of sad faces predict clinical remission to cognitive behavioural therapy in depression. Neuroreport. 2009;20(7):637-641.
PubMedArticle
14.
Doehrmann  O, Ghosh  SS, Polli  FE,  et al.  Predicting treatment response in social anxiety disorder from functional magnetic resonance imaging. JAMA Psychiatry. 2013;70(1):87-97.
PubMedArticle
15.
Wittchen  HU, Jacobi  F, Rehm  J,  et al.  The size and burden of mental disorders and other disorders of the brain in Europe 2010. Eur Neuropsychopharmacol. 2011;21(9):655-679.
PubMedArticle
16.
Nutt  DJ.  The full cost and burden of disorders of the brain in Europe exposed for the first time. Eur Neuropsychopharmacol. 2011;21(10):715-717.
PubMedArticle
17.
McHugh  RK, Smits  JA, Otto  MW.  Empirically supported treatments for panic disorder. Psychiatr Clin North Am. 2009;32(3):593-610.
PubMedArticle
18.
Gloster  AT, Wittchen  HU, Einsle  F,  et al.  Psychological treatment for panic disorder with agoraphobia: a randomized controlled trial to examine the role of therapist-guided exposure in situ in CBT. J Consult Clin Psychol. 2011;79(3):406-420.
PubMedArticle
19.
Sánchez-Meca  J, Rosa-Alcázar  AI, Marín-Martínez  F, Gómez-Conesa  A.  Psychological treatment of panic disorder with or without agoraphobia: a meta-analysis. Clin Psychol Rev. 2010;30(1):37-50.
PubMedArticle
20.
Bouton  ME, Mineka  S, Barlow  DH.  A modern learning theory perspective on the etiology of panic disorder. Psychol Rev. 2001;108(1):4-32.
PubMedArticle
21.
Lissek  S, Powers  AS, McClure  EB,  et al.  Classical fear conditioning in the anxiety disorders: a meta-analysis. Behav Res Ther. 2005;43(11):1391-1424.
PubMedArticle
22.
Lissek  S, Rabin  S, Heller  RE,  et al.  Overgeneralization of conditioned fear as a pathogenic marker of panic disorder. Am J Psychiatry. 2010;167(1):47-55.
PubMedArticle
23.
Michael  T, Blechert  J, Vriends  N, Margraf  J, Wilhelm  FH.  Fear conditioning in panic disorder: enhanced resistance to extinction. J Abnorm Psychol. 2007;116(3):612-617.
PubMedArticle
24.
Tuescher  O, Protopopescu  X, Pan  H,  et al.  Differential activity of subgenual cingulate and brainstem in panic disorder and PTSD. J Anxiety Disord. 2011;25(2):251-257.
PubMedArticle
25.
Kircher  T, Arolt  V, Jansen  A,  et al.  Effect of cognitive-behavioral therapy on neural correlates of fear conditioning in panic disorder. Biol Psychiatry. 2013;73(1):93-101.
PubMedArticle
26.
Lueken  U, Straube  B, Konrad  C,  et al.  Neural substrates of treatment response to cognitive-behavioral therapy in panic disorder with agoraphobia. Am J Psychiatry. 2013;170(11):1345-1355.
PubMedArticle
27.
Gloster  AT, Wittchen  HU, Einsle  F,  et al.  Mechanism of Action in CBT (MAC): methods of a multi-center randomized controlled trial in 369 patients with panic disorder and agoraphobia. Eur Arch Psychiatry Clin Neurosci. 2009;259(suppl 2):S155-S166.
PubMedArticle
28.
Bandelow  B. Panic and Agoraphobia Scale (PAS). Ashland, OH: Hogrefe & Huber Publishers; 1999.
29.
Guy  W. ECDEU Assessment Manual for Psychopharmacology Review. Rockville, MD: US National Institute of Health, Psychopharmacology Research Branch; 1976.
30.
Reiss  S, Peterson  RA, Gursky  DM, McNally  RJ.  Anxiety sensitivity, anxiety frequency and the prediction of fearfulness. Behav Res Ther. 1986;24(1):1-8.
PubMedArticle
31.
Beck  AT, Steer  RA, Brown  GK. Beck Depression Inventory.2nd ed. San Antonio, TX: Psychological Corp; 1996.
32.
Shear  MK, Vander Bilt  J, Rucci  P,  et al.  Reliability and validity of a structured interview guide for the Hamilton Anxiety Rating Scale (SIGH-A). Depress Anxiety. 2001;13(4):166-178.
PubMedArticle
33.
Gloster  AT, Hauke  C, Höfler  M,  et al.  Long-term stability of cognitive behavioral therapy effects for panic disorder with agoraphobia: a two-year follow-up study. Behav Res Ther. 2013;51(12):830-839.
PubMedArticle
34.
Reinhardt  I, Jansen  A, Kellermann  T,  et al.  Neural correlates of aversive conditioning. Eur Arch Psychiatry Clin Neurosci. 2010;260(6):443-453.
PubMedArticle
35.
Carter  RM, Bowling  DL, Reeck  C, Huettel  SA.  A distinct role of the temporal-parietal junction in predicting socially guided decisions. Science. 2012;337(6090):109-111.
PubMedArticle
36.
Marquand  A, Howard  M, Brammer  M, Chu  C, Coen  S, Mourão-Miranda  J.  Quantitative prediction of subjective pain intensity from whole-brain fMRI data using gaussian processes. Neuroimage. 2010;49(3):2178-2189.
PubMedArticle
37.
Rasmussen  CE, Williams  CKI. Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press; 2006.
38.
Hahn  T, Marquand  AF, Plichta  MM,  et al.  A novel approach to probabilistic biomarker-based classification using functional near-infrared spectroscopy. Hum Brain Mapp. 2013;34(5):1102-1114.
PubMedArticle
39.
Shen  H, Wang  L, Liu  Y, Hu  D.  Discriminative analysis of resting-state functional connectivity patterns of schizophrenia using low dimensional embedding of fMRI. Neuroimage. 2010;49(4):3110-3121.
PubMedArticle
40.
Costafreda  SG, Fu  CHY, Picchioni  M,  et al.  Pattern of neural responses to verbal fluency shows diagnostic specificity for schizophrenia and bipolar disorder. BMC Psychiatry. 2011;11(18):18. doi:10.1186/1471-244X-11-18.
PubMedArticle
41.
Marquand  AF, Mourão-Miranda  J, Brammer  MJ, Cleare  AJ, Fu  CHY.  Neuroanatomy of verbal working memory as a diagnostic biomarker for depression. Neuroreport. 2008;19(15):1507-1511.
PubMedArticle
42.
Sehlmeyer  C, Schöning  S, Zwitserlood  P,  et al.  Human fear conditioning and extinction in neuroimaging: a systematic review. PLoS One. 2009;4(6):e5865. doi:10.1371/journal.pone.0005865.
PubMedArticle
43.
Lueken  U, Straube  B, Reinhardt  I,  et al.  Altered top-down and bottom-up processing of fear conditioning in panic disorder with agoraphobia. Psychol Med. 2014;44(2):381-394.
PubMedArticle
44.
Mantovani  A, Aly  M, Dagan  Y, Allart  A, Lisanby  SH.  Randomized sham controlled trial of repetitive transcranial magnetic stimulation to the dorsolateral prefrontal cortex for the treatment of panic disorder with comorbid major depression. J Affect Disord. 2013;144(1-2):153-159.
PubMedArticle
45.
Otto  MW, Tolin  DF, Simon  NM,  et al.  Efficacy of d-cycloserine for enhancing response to cognitive-behavior therapy for panic disorder. Biol Psychiatry. 2010;67(4):365-370.
PubMedArticle
46.
Siegmund  A, Golfels  F, Finck  C,  et al.  d-Cycloserine does not improve but might slightly speed up the outcome of in-vivo exposure therapy in patients with severe agoraphobia and panic disorder in a randomized double blind clinical trial. J Psychiatr Res. 2011;45(8):1042-1047.
PubMedArticle
47.
Reif  A, Richter  J, Straube  B,  et al.  MAOA and mechanisms of panic disorder revisited. Mol Psychiatry. 2014;19(1):122-128.
PubMedArticle
×