A, Within-group analyses. In controls, high expected reward (shown in blue) positively modulated the blood oxygenation level–dependent signal in the ventromedial prefrontal cortex (vmPFC; Brodmann areas [BAs] 10, 32, and 24), the posterior insula (area 1), and other paralimbic structures (the midcingulate cortex [BA 24] and precuneus [BAs 7 and 31; area 2]). A large cortical network responded to high ambiguity/low value (shown in red). This network included the lateral prefrontal areas of the ventrolateral PFC (vlPFC; lateral BA 10 and BAs 13, 44, and 45; operculoinsular cortex [area 3]) and the dorsolateral PFC (dlPFC; BA 9 and lateral BA 8) that we expected to be activated a priori and the lateral parietal cortex (lat parietal; BAs 7 and 40) and dorsomedial PFC (dmPFC; medial BA 8; shown in red). Although the maps were qualitatively similar in nonsuicidal depressed participants, suicide attempters did not display paralimbic responses to high expected reward. Area 4 indicates associative striatum and thalamus. P < .005, uncorrected for illustration. B, A history of suicide attempts was related to a weaker response to expected reward in the pericallosal vmPFC (pregenual cingulate [BA 32, 24, and 25] on the posterior periphery of the reward-modulated vmPFC region of interest). P < .05, corrected, controlling for depression group status. The x and z values indicate Montreal Neurological Institute coordinates.
A, Paralimbic functional network masks were independently derived in controls. B, Depressed participants (nonsuicidal and suicide attempters) with weaker paralimbic responses to expected reward were more likely to ignore negative feedback after a reversal, making multiple perseverative errors. C, Weak paralimbic responses were related to bets against the odds on the Cambridge Gamble Task (CGT). D, Weak paralimbic responses were related to poor attempt planning in suicide attempters. E, Weak paralimbic responses were related to higher scores on the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory. F, Weak paralimbic responses were related to higher scores on the Nonplanning subscale of the Barratt Impulsiveness Scale.
We compared the paralimbic (blue) vs neocortical task (red) networks in healthy controls (n = 20). The onset of the time window corresponds to reward delivery on the preceding trial. Paralimbic responses to high expected value peaked earlier than responses to the low expected value in the lateralized frontoparietal network, operculoinsular cortex, ventrolateral prefrontal cortex (vlPFC), and dorsolateral prefrontal cortex (dlPFC), ostensibly aligned with the next stimulus presentation. This alignment suggests that paralimbic structures responded to the reward from the preceding trial. Indeed, the timing of paralimbic response is unlikely to coincide with actual choice because the peak to response would have to be less than 4 seconds. ANOVA indicates analysis of variance; vmPFC, ventromedial prefrontal cortex.
A, Within-group analysis in controls (n = 20). Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus. Area 1 indicates the putamen; area 2, inferior parietal lobule; area 3, operculoinsular cortex; area 4, thalamus; and dmPFC, dorsomedial prefrontal cortex. P < .05, corrected. B, As a group, depressed participants (nonsuicidal and suicide attempters) displayed weaker modulation by positive prediction error in the right thalamus, the bilateral superior temporal gyrus (Brodmann area [BA] 22 and 39), the bilateral operculoinsular cortex (BA 13, 45, and 46), the bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area/cingulate. P < .05, corrected, controlling for a history of suicide attempts.
eFigure 1. Probabilistic reversal learning task
eFigure 2. Estimated hemodynamic response function (HRF) for the switch vs stay contrast in healthy controls (n = 20)
eFigure 3. Behavioral performance, perseverative errors, and impulsivity by group
Customize your JAMA Network experience by selecting one or more topics from the list below.
Dombrovski AY, Szanto K, Clark L, Reynolds CF, Siegle GJ. Reward Signals, Attempted Suicide, and Impulsivity in Late-Life Depression. JAMA Psychiatry. 2013;70(10):1020–1030. doi:10.1001/jamapsychiatry.2013.75
Copyright 2013 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Suicide can be viewed as an escape from unendurable punishment at the cost of any future rewards. Could faulty estimation of these outcomes predispose to suicidal behavior? In behavioral studies, many of those who have attempted suicide misestimate expected rewards on gambling and probabilistic learning tasks.
To describe the neural circuit abnormalities that underlie disadvantageous choices in people at risk for suicide and to relate these abnormalities to impulsivity, which is one of the components of vulnerability to suicide.
Case-control functional magnetic resonance imaging study of reward learning using a reinforcement learning model.
University hospital and outpatient clinic.
Fifty-three participants 60 years or older, including 15 depressed patients who had attempted suicide, 18 depressed patients who had never attempted suicide (depressed control subjects), and 20 psychiatrically healthy controls.
Main Outcomes and Measures
Components of the cortical blood oxygenation level–dependent response tracking expected and unpredicted rewards.
Depressed elderly participants displayed 2 distinct disruptions of control over reward-guided behavior. First, impulsivity and a history of suicide attempts (particularly poorly planned ones) were associated with a weakened expected reward signal in the paralimbic cortex, which in turn predicted the behavioral insensitivity to contingency change. Second, depression was associated with disrupted corticostriatothalamic encoding of unpredicted rewards, which in turn predicted the behavioral oversensitivity to punishment. These results were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular illness, and incipient dementia.
Conclusions and Relevance
Altered paralimbic reward signals and impulsivity and/or carelessness may facilitate unplanned suicidal acts. This pattern, also seen in gambling and cocaine use, may reflect a primary deficit in the paralimbic cortex or in its mesolimbic input. The overreactivity to punishment in depression may be caused in part by a disruption of appetitive learning in the corticostriatothalamic circuits.
What causes a person to commit suicide is one of the central questions in psychiatry. We know that stressors precipitate suicidal behavior, particularly in the pathological states of depression, psychosis, intoxication, and pain. However, because only a minority of people facing these circumstances attempt suicide, the individual diathesis must play an important role. This individual diathesis is still not well understood. However, impulsive-aggressive traits,1-6 cognitive deficits,7-13 and persisting hopelessness14-16 have emerged as important dimensions. Suicidal behavior is heterogeneous, and even less is known about the many pathways that lead to it. Suicidal acts follow a decision, and we argue that a propensity to make bad decisions may be part of one of the pathways to suicidal behavior. This view is supported by at least 3 converging lines of evidence. The first line is the association of suicidal behavior with problem gambling17 and drug addiction,18 behaviors defined by disadvantageous choice. In the second line, several although not all studies find that significant subgroups of individuals with a history of suicide attempts lose on gambling tasks, fail to learn from the experience of rewards and punishments,11,19-22 and make short-sighted choices.23 In the third line, suicidal behavior has been linked to disruptions of signaling pathways in the basal ganglia,24-30 disruptions that severely and somewhat selectively impair decision-making processes.31-35 Akin to the analysis of suicide as an escape from self by Baumeister36 and the entrapment theory of Williams et al,37 one can view suicidal behavior as instrumental, aimed at achieving the subjectively preferred outcome in a desperate situation. Our interpretation of the evidence reviewed above is that at least some people who engage in suicidal behavior tend to misestimate future outcomes and see suicide as unrealistically attractive relative to other options. This perspective is supported by our behavioral findings of disrupted instrumental learning in older depressed people who attempt suicide (suicide attempters).38,39 To extend this evidence, we examined whether older depressed suicide attempters demonstrate disruptions in brain signals involved in reward prediction. We also considered whether impulsivity, a key dimension of the suicidal diathesis, exerts effects on these signals.
We focused on older adults because the rate of suicide is high in the elderly, and we believe that the age-dependent decline in decision competence40,41 puts this group at special risk. We view learning from rewards and punishments in a changing, ambiguous environment as a model of behavioral adaptation to change and uncertainty that accompanies a suicidal crisis. Consistent with this idea, a previous study38 observed a complete breakdown of reward learning in older depressed suicide attempters, with some suicide attempters overreacting to punishments and others displaying insensitivity to a contingency change. Older depressed suicide attempters also failed to grasp the changing contingency in an environment with no misleading feedback, the Wisconsin Card Sorting Task.39 This failure to grasp a changing contingency resembles the behavior of animals and humans with lesions of the ventromedial and ventrolateral prefrontal cortex (vmPFC and vlPFC).42-44 To test whether the activity of these areas was disrupted during reward learning in older depressed suicide attempters, we conducted a functional magnetic resonance imaging (fMRI) study with participants facing uncertain and changing rewards and punishments on a probabilistic reversal learning task. To obtain more accurate and interpretable predictions of neural activity, we used a cognitive modeling approach, reinforcement learning, to estimate 2 key reward signals postulated by formal learning theory. The first signal is expected reward, and the second is the discrepancy between the experienced outcome and the prior expectation, or the prediction error.
We were further interested in how these neural signals are affected by impulsivity, a tendency strongly linked to suicidal behavior. We hypothesized that a history of suicide attempts and impulsivity would be associated with weaker expected reward signals in the vmPFC on the basis of studies mapping value representations to the vmPFC45-50 and positron emission tomography findings of vmPFC alterations in suicide attempters51,52 and individuals with maladaptive impulsive behaviors.53
Overreactivity to negative feedback is typical of depression54-58 and was very prominent among acutely depressed older suicide attempters in our behavioral study.38 On the probabilistic reversal learning task, depressed patients tend to switch response after a misleading punishment,58 as if forgetting the preceding history of rewards. According to learning theory,59,60 such responses have to result from a disproportionately big effect of negative prediction errors on choice. This effect could be caused by incorrect encoding of negative or preceding positive prediction errors or improper maintenance of expected reward representations. This reasoning led us to focus on the neural representation of prediction errors as a correlate of behavioral overreactivity to punishments in depression. A prior study61 has found altered error-related activity in the vlPFC of depressed individuals who overreacted to negative feedback during probabilistic reversal learning, although the study did not estimate prediction error signals. Studies using other reward-learning paradigms have found blunted prediction errors in the cingulate gyrus, striatum, and midbrain of depressed patients.62,63 However, these abnormalities were not linked to any particular behavioral tendency.
From March 28, 2008, through November 15, 2011, we recruited 53 participants 60 years or older, including 31 with major depression (15 suicide attempters and 18 with no lifetime history of suicidal behavior or ideation; we hereinafter refer to these 31 participants as depressed) and 20 psychiatrically healthy control subjects. Major depression was diagnosed by the Structured Clinical Interview for DSM-IV Axis I Disorders64,65 (SCID). Unlike participants in the previous behavioral study,38 all of whom had an acute depressive episode, participants in this study had different levels of depressive symptoms, from severe to partial remission (Table). To exclude individuals with clinical dementia and to ensure that participants could engage in the task, all were required to have a score of at least 24 on the Mini-Mental State Examination.66 Our reported sample excludes elderly individuals with sensory disorders that precluded cognitive testing, limited English, mental retardation, delirium, neurologic disorders, bipolar disorder, schizophrenia, schizoaffective disorder, exposure to electroconvulsive therapy in the previous 6 months, and circumstances precluding an fMRI assessment (eg, claustrophobia, metal implants [n = 7]). Participant flow is described in more detail in the material available on the authors’ website (https://db.tt/PHaUDMEs).
All participants provided written informed consent. The University of Pittsburgh institutional review board approved the study.
Suicide attempters had engaged in a self-injurious act with an intent to die; 9 of 15 made their first suicide attempt after 50 years of age and 6 of them after 60 years of age. Fourteen of the 15 were recruited as inpatients, all of whom were admitted out of concern about suicide risk. These participants displayed a high level of suicidal intent (mean [SD] Suicide Intent Scale score, 18.1 [4.6]) during their attempts and severe suicidal ideation during the index episode (Scale for Suicide Ideation score, 23.9 [6.9]); 7 of 15 made repeated attempts. A history of suicide attempts was verified by a study psychiatrist (A.Y.D. or K.S.) based on the interview, medical records, and information from the treatment team and from family or friends. We excluded participants with significant discrepancies between these sources. None of the suicide attempters had experienced head injuries directly related to the attempt; however, we assessed potential anoxic-ischemic or toxic brain injury based on the Beck Lethality Scale,67 medical records, and the clinical interview. A study psychiatrist (A.Y.D.) identified any attempts with a score greater than 4 on the Beck Lethality Scale and any history of systemic hypotension of longer than 5 minutes, asphyxia, or neurotoxic ingestion in 3 of 15 participants. For 1 additional participant, we could not rule out brain injury during past attempts. Thus, we excluded these 4 participants in sensitivity analyses.
Nonsuicidal depressed elderly were included in the study to detect an association between decision making and suicidal behavior beyond the cognitive effects of depression. These participants had no current or lifetime history of suicide attempts or suicidal ideation as established by clinical interview, review of medical records, SCID, and the Scale for Suicidal Ideation (lifetime). One of the 18 nondepressed participants was recruited as an inpatient. Participants were excluded from this group if they had a current passive death wish or a history of indirect self-destructive behaviors.
Nondepressed controls were included as the benchmark group. They had no lifetime history of any psychiatric disorder as determined by SCID.
Clinical and cognitive assessments used to characterize the study groups and the 3-T MRI scanning parameters are described in detail on the authors’ website.
On each of the 300 trials of this instrumental learning task, participants chose between 2 pictures (eFigure 1 in the Supplement). One picture had a higher probability of reward when chosen (varied, .80 ≤ p ≤ .87), and the other, its additive inverse (.13 ≤ [1 − p] ≤ .20). In other words, even while choosing the right picture, participants received occasional misleading (probabilistic) negative feedback. Every 25 trials, the contingency (the good picture) changed without warning (reversal). On this task, one needs to trade off staying with the previously reinforced stimulus despite occasional misleading (probabilistic) feedback and switching when a true reversal occurs. The tendency to stay too long after reversal while ignoring negative feedback leads to perseverative errors. Conversely, the tendency to switch after a single misleading punishment results in probabilistic switch errors, previously linked to depression.58 We used the number of trials with misleading feedback during which the participant resisted this switch as a measure of resilience to it. Spontaneous or win/switch errors occurred when the subject switched away from a stimulus after being rewarded for choosing it. Because spontaneous errors are infrequent, their count is strongly and positively skewed with many values of 0. We also recorded response times, using their variation as an index of whether subjects were consistently on task or distracted at times. To ensure that even the most impaired participants are engaged in the task during scanning, we performed prior behavioral training with 4 reversals, and participants were no longer naive to the change in contingency. Thus, we were not expecting a complete breakdown of contingency learning in suicide attempters, unlike the earlier behavioral study with an unexpected reversal.38
To estimate abstract reward signals from participants’ reinforcement history and behavior, we adapted the Rescorla-Wagner reinforcement learning model, as in the earlier behavioral study.38 On this task, expected reward or expected value tracks the certainty of reward for the best available action. Here, the model’s estimate of expected value is high when the choice of one stimulus is almost certain to be rewarded. Meanwhile, a low expected value means ambiguity about which stimulus is likely to be rewarded. Prediction error is the discrepancy between the prior expectation of reward and the reward actually received on this trial. Prediction errors drive the updates of expected value after feedback. A reward for choosing a stimulus with a low prior expected value produces a positive prediction error and increases expected value. The omission of reward for a stimulus with a previously high expected value produces a negative prediction error and decreases expected value.
We were interested in how neural indices of reward learning are affected by impulsivity for 2 reasons. First, strong evidence implicates impulsivity in the suicidal diathesis.5 Second, impulsivity involves a failure of higher-order control, which could plausibly undermine reward learning in an uncertain, changing environment. Assessments of impulsivity included the Impulsive/Careless Style subscale of the Social Problem-Solving Inventory,68,69 the Nonplanning and Attention/Cognitive subscales of the Barratt Impulsiveness Scale,70 and bets against the odds on the Cambridge Gamble Task (CGT),21 all found to be related to suicidal behavior.
Bets against the odds on the CGT, observed in older suicide attempters, reflect a neglect of information about outcome probability,21 possibly owing to a gambler’s fallacy or belief in a lucky streak. On each trial of the CGT, the participant is presented with an array of 10 boxes at the top of the screen, with each colored red or blue. The ratio of red to blue boxes varies from 1:9 to 9:1, in a pseudorandom order. The participant is instructed that the computer has hidden a token in 1 of the boxes and is told to register a guess as to whether the token is hidden under a red or a blue box by selecting 1 of the 2 corresponding colored panels. Given that the proportions of majority vs minority bets tend to follow a nonlinear, beta distribution, we dichotomized all participants into those who always bet on the majority color (good choices) vs those who sometimes bet on the minority color (bad choices). Of the 33 depressed participants included in the analysis, 2 were missing Social Problem-Solving Inventory data, 6 were missing Barratt Impulsiveness Scale data, and 9 were missing CGT data.
Given the age-related variance in the hemodynamic response function (HRF), an empirical HRF was estimated in the control group using the (unbiased) switch/stay contrast in the vlPFC,71 as detailed on the authors’ website and in the Supplement (eFigure 2). This empirical HRF was then convolved with the expected value, positive prediction error, and negative prediction error signals from the reinforcement learning model for each subject. Voxelwise blood oxygenation level–dependent (BOLD) signal was regressed on these estimates in single-variable single-subjects analyses using an AFNI program (3dDeconvolve).72 Group differences were estimated by regressing the beta weights against 2 factors—depression and history of suicide attempts—across participants using another AFNI program (3dRegAna). To understand the variation in the HRF across regions, we also obtained estimates of the impulse response function for expected value signals using the 3dDeconvolve program with no assumed HRF and extracted the time series of resulting beta weights for each of the nondepressed controls.
To control type I error, we thresholded voxelwise tests at P < .005 and cluster thresholded them using Monte Carlo simulations based on the spatial autocorrelation of derived maps using the AFNI programs 3dFWHMx and 3dAlphaSim72,73 (67 voxels yielding P < .05, corrected). To estimate effects within the anatomical vmPFC mask from the Talairach atlas, 8 voxels at P < .005 controlled type I error at P < .05 with small-volume correction.
To test the relationship between neural responses to expected value and prediction error on one hand and behavioral, clinical, and self-reported measures of impulsivity on the other, we conducted 2 independent analyses described in detail on the authors’ website. In the first, we mapped the networks responsive to expected value and prediction error in healthy controls (n = 20). In the second, we estimated the responses of these networks in the independent group of depressed participants with and without suicide attempts (n = 33) and correlated them with the measures of interest. In these correlational analyses, we chose to examine summary activations across networks responsive to expected value and prediction error instead of activations in single regions of interest to reduce dimensionality and control type I error. This choice was dictated by our initial analyses showing that responses to expected value and prediction error were distributed across large networks and were strongly intercorrelated across regions within each network (Tables 1 and 2 on the authors’ website). Although some regions were less active in suicide attempters and in depressed participants in general regardless of the history of suicide attempts as discussed in the Results section, we found no regions that were more active in the 2 depressed groups than in the controls. Thus, by applying the masks from the controls to the depressed participants, we did not leave out any ostensibly relevant regions.
In an exploratory analysis (described on the authors’ website along with the sensitivity analyses), we examined whether paralimbic structures encoded expected reward when the choices were being made vs when the reward/punishment was received after the choice. Given the low temporal resolution of fMRI, we tested this by comparing the timing of paralimbic response with that of the ostensibly choice-related response in the cortical task networks sensitive to ambiguity.
In within-group analyses, controls demonstrated a positively modulated BOLD signal in the vmPFC (Brodmann areas [BAs] 10, 32, and 24) and other paralimbic structures, including the midcingulate cortex (BA 24), precuneus/posterior cingulate (BAs 7 and 31), and posterior insula for high expected reward (Figure 1A [in blue] and Table 1 on the authors’ website). In the whole-brain group comparison, as hypothesized, the pregenual cingulate subregion of the vmPFC (BA 32, 24, and 25 on the posterior periphery of the reward-modulated vmPFC region of interest) tracked the reinforcement history less reliably in suicide attempters than in the comparison groups, controlling for depression group status (P < .05, corrected; Figure 1B).
Depressed participants whose paralimbic structures were less responsive to expected reward were more likely to ignore negative feedback after reversal (r33 = 0.47 [P = .006]; after excluding 1 outlier, r32 = 0.64 [P < .001]). However, their learning was otherwise relatively preserved, with no statistically reliable relationship between weak paralimbic modulation and inability to track the contingency, probabilistic, or spontaneous switches (r33 ≤ 0.24 [P ≥ .28]) (Figure 2A-C).
Confirming our hypothesis in depressed participants, weak paralimbic response to high expected reward was related to nonplanning impulsivity (r27 = −0.43 [P = .03]; after excluding 1 outlier, r26 = −0.50 [P = .01]) and bad choices on the CGT (ρ27 = 0.50 [P = .01]). The correlation with impulsive/careless problem-solving style was not significant (r31 = −0.20 [P = .29]) but became more apparent after excluding one outlier (r30 = −0.43 [P = .02]). We found no relationship with delay discounting (r23 = −0.10 [P = .64]). Among suicide attempters, weak paralimbic modulation was related to poor attempt planning (ρ15 = −0.58 [P = .03]) and not significantly to low attempt lethality (ρ15 = −0.48 [P = .07]) (Figure 2D-F and Supplement [eFigure 3]).
Did paralimbic structures represent the expected reward signal when the actual choice was being made? Our analysis of BOLD waveforms seems to suggest otherwise. Although the opercularinsular, dorsomedial PFC (dmPFC), and lateral-parietal responses to ambiguity peaked at a time point consistent with choice (Figure 3, in red), responses to high expected reward in the vmPFC and precuneus peaked earlier (Figure 3, in blue; repeated-measures analysis of variance, F99,228 = 9.3 [P < .001]). This peak fell approximately between feedback and the next choice. Thus, paralimbic structures appeared to broadly integrate reinforcement history rather than compare the value of presented options or select between actions.
Unexpected rewards (positive prediction error) modulated the same networks as ambiguity, plus the midcingulate cortex, striatum, and thalamus (Figure 4A and Table 2 on the authors’ website). The behavioral overreactivity of depressed individuals to negative feedback was related to blunted representation of positive prediction errors in the cingulo-opercular and frontoparietal networks (r33 = −0.40 [P = .02]). As a group, depressed participants displayed weaker modulation by positive prediction error in the right thalamus, bilateral superior temporal gyrus (BA 22 and 39), bilateral operculoinsular cortex (BA 13, 45, and 46), bilateral postcentral gyrus (BA 40), and the bilateral supplementary motor area (P < .05, corrected; Figure 4B), and these effects were not related to a history of suicide attempts.
Group differences in expected reward and prediction error representations were robust to the effects of possible brain damage from suicide attempts, depressive severity, co-occurring substance use and anxiety disorders, antidepressant and anticholinergic exposure, lifetime exposure to electroconvulsive therapy, vascular pathological features, incipient dementia, sex, and model fits (see material on the authors’ website). Only depressive severity was a significant covariate, explaining additional variance in expected reward signal in the pericallosal cingulate cortex.
Our study of reward learning in late-life depression uncovered 2 neural patterns associated with disrupted control over reward-guided behavior. The first pattern, manifesting in a blunted modulation of paralimbic structures by expected reward, is associated with impulsivity and attempted suicide. The second pattern, associated with depression, is expressed in a weak thalamocorticostriatal response to unpredicted rewards and in maladaptive overreactivity to punishments. We discuss them in that order.
In a dynamic environment, paralimbic structures guide approach behavior by tracking the value of available options.45,46 This function is sometimes related to the hedonic aspect of pursuing rewards (liking).45,74,75 In the impulsive depressed elderly in our study and in depressed suicide attempters, the paralimbic structures failed to track the reinforcement history accurately. This failure was seen mostly in patients who had made unplanned suicide attempts, again linking this anomaly to impulsive behavior. Participants displaying such a failure had the tendency to perseverate in approach behavior even when it no longer paid off, suggesting that they did not update the value of the option that was no longer rewarded. This interpretation is consistent with disrupted tracking of expected value by the paralimbic structures, which was associated with betting against the odds on a gambling task, a behavior previously observed in older suicide attempters.21 In both cases, impulsive individuals appear to be ignoring key information—odds (CGT) or reinforcement history (reversal learning)—and experiencing losses as a result. Our findings of the relationship among paralimbic structures’ expected reward signals, trait impulsivity, and bad choices on the CGT converge with those of Cox and colleagues,76 who found that vmPFC-insula resting-state functional connectivity was related to disadvantageous risk seeking. Our observations bear a curious resemblance to the behavioral and neural patterns seen in long-term cocaine users77,78 and in impulsive pathological gamblers79 on the reversal learning task. A similar pattern of perseveration has been produced by long-term cocaine administration in monkeys80 and by its withdrawal in rats.81 Whether these populations experience a primary disruption in the paralimbic cortex itself or in its mesolimbic input remains unclear.82,83 We found little support for the primary cortical deficit account from the literature concerning lesions in monkeys. Although lesions of the medial occipital-frontal cortex induce perseverative responding on a pavlovian reversal learning task,84 their effects on instrumental behavior are more complex and not limited to perseveration.85 Meanwhile, the fact that such perseveration can be induced and ameliorated86 by monoaminergic manipulation points toward a source in the ascending reward pathways.
Depressed individuals overreact to negative feedback, making error after error.54-58 Abnormal response to errors in prefrontal areas, such as the vlPFC and the dmPFC, has been suggested to underlie this tendency.61 Extending that finding, we show that this behavior is also associated with blunted response of the cingulo-opercular and frontoparietal networks to positive prediction errors. Notably, this blunting was seen on a different set of trials: those with surprising positive feedback rather than misleading negative feedback. This effect may be best explained in terms of competition between 2 controllers of behavior. A coherent neurobiological account of such competition is offered by the rodent model of controllable vs uncontrollable stress used by Amat et al.87 In the controllable condition, once the animal encodes the action/reinforcement contingency, resulting cortical safety signals inhibit the primitive brainstem and limbic stress response. Conversely, weaker encoding of unpredicted rewards in depressed patients may fail to suppress a primitive response to noncontingent punishment, resembling the state of learned helplessness. Depressed individuals may thus experience punishments as uncontrollable, shifting toward a primitive, subcortical lose-switch policy instead of strategic choice based on reinforcement history. A plausible computational account of depressive overreactivity to punishments is that a primitive, automatic pavlovian controller responding to an aversive stimulus may override the more sophisticated controllers of instrumental learning. Pavlovian influences on instrumental behavior have been invoked to explain a range of disadvantageous approach-avoidance responses88 and appear to depend on limbic structures, such as the basolateral amygdala89 and the ventral striatum.90
Besides the cross-sectional, case-control design and a relatively small sample, several other design weaknesses limit our findings. Imaging assessments are too demanding for our sickest patients. Therefore, being unable to include these patients, we probably lack the opportunity to observe all the neural correlates of suicidal behavior. Because some of our patients were in partial remission, our findings may not apply to the most acutely depressed individuals. With respect to external validity, one can be more confident in the mechanistic findings linking impulsivity and overreactivity to punishment to specific neural aberrations than in group differences as such. Furthermore, expected reward is confounded with (lower) ambiguity and risk in reversal learning, limiting the interpretation of our findings. In addition, because depression status was confounded with antidepressant exposure, we cannot conclusively separate the two, although sensitivity analyses suggest that our results are robust. Also, our correlational study cannot disentangle the exact structure of relationships among the altered paralimbic reward signals, behaviors on reversal learning and gambling tasks, and self-reported impulsivity. Finally, the delta-rule model of associative learning explains only the most basic aspects of human brain activity during reversal learning (additional discussion can be found on the authors’ website). More sophisticated models49,91,92 might provide additional insight into anomalous reward learning in impulsive depressed elderly.
In summary, we found that in older adults, unplanned suicide attempts and impulsivity were associated with disrupted paralimbic tracking of expected reward. Depression was associated with disrupted corticostriatothalamic encoding of unexpected rewards.
Corresponding Author: Alexandre Y. Dombrovski, MD, Department of Psychiatry, University of Pittsburgh, 3811 O’Hara St, Pittsburgh, PA 15213 (firstname.lastname@example.org)
Submitted for Publication: July 25, 2012; final revision received November 7, 2012; accepted December 18, 2012.
Published Online: August 7, 2013. doi:10.1001/jamapsychiatry.2013.75.
Author Contributions: Dr Dombrovski had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Dombrovski, Clark, Reynolds, Siegle.
Acquisition of data: Dombrovski, Szanto, Reynolds, Siegle.
Analysis and interpretation of data: Dombrovski, Clark, Siegle.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: Dombrovski, Clark, Siegle.
Statistical analysis: Dombrovski, Siegle.
Obtained funding: Dombrovski, Clark.
Administrative, technical, and material support: Dombrovski, Reynolds.
Study supervision: Dombrovski, Szanto, Reynolds, Siegle.
Funding/Support: The study was supported by grants K23MH086620, K23MH070471, and R01MH085651 from the National Institute of Mental Health and by the American Foundation for Suicide Prevention.
Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.
Previous Presentation: Some of the results reported herein were previously presented at the 50th Annual Meeting of the American College of Neuropsychopharmacology; December 7, 2011; Waikoloa, Hawaii.
Additional Contributions: Michael Hallquist, PhD, and Henry Chase, PhD, provided critical comments on the manuscript. Mandy Collier, BS, contributed to data collection and processing. Swathi Gujral, BS, Natalie Truty, BS, and Cori Shollenberger, BS, assisted with recruitment and assessments. Jan Kalkus, BS, provided help with graphics. Howard Aizenstein, MD, PhD, contributed to the structural imaging protocol.
Correction: This article was corrected on July 3, 2014, to update the URL to the authors’ website.