Role of the Medial Prefrontal Cortex in Impaired Decision Making in Juvenile Attention-Deficit/Hyperactivity Disorder

Importance  Attention-deficit/hyperactivity disorder (ADHD) has been associated with deficient decision making and learning. Models of ADHD have suggested that these deficits could be caused by impaired reward prediction errors (RPEs). Reward prediction errors are signals that indicate violations of expectations and are known to be encoded by the dopaminergic system. However, the precise learning and decision-making deficits and their neurobiological correlates in ADHD are not well known.

Objective  To determine the impaired decision-making and learning mechanisms in juvenile ADHD using advanced computational models, as well as the related neural RPE processes using multimodal neuroimaging.

Design, Setting, and Participants  Twenty adolescents with ADHD and 20 healthy adolescents serving as controls (aged 12-16 years) were examined using a probabilistic reversal learning task while simultaneous functional magnetic resonance imaging and electroencephalogram were recorded.

Main Outcomes and Measures  Learning and decision making were investigated by contrasting a hierarchical Bayesian model with an advanced reinforcement learning model and by comparing the model parameters. The neural correlates of RPEs were studied in functional magnetic resonance imaging and electroencephalogram.

Results  Adolescents with ADHD showed more simplistic learning as reflected by the reinforcement learning model (exceedance probability, P_x = .92) and had increased exploratory behavior compared with healthy controls (mean [SD] decision steepness parameter β: ADHD, 4.83 [2.97]; controls, 6.04 [2.53]; P = .02). The functional magnetic resonance imaging analysis revealed impaired RPE processing in the medial prefrontal cortex during cue as well as during outcome presentation (P < .05, family-wise error correction). The outcome-related impairment in the medial prefrontal cortex could be attributed to deficient processing at 200 to 400 milliseconds after feedback presentation as reflected by reduced feedback-related negativity (ADHD, 0.61 [3.90] μV; controls, −1.68 [2.52] μV; P = .04).

Conclusions and Relevance  The combination of computational modeling of behavior and multimodal neuroimaging revealed that impaired decision making and learning mechanisms in adolescents with ADHD are driven by impaired RPE processing in the medial prefrontal cortex. This novel, combined approach furthers the understanding of the pathomechanisms in ADHD and may advance treatment strategies.

Attention-deficit/hyperactivity disorder (ADHD) has been associated with deficits in decision making and learning.¹ These skills are guided by the dopaminergic system,² which is impaired in ADHD.^3-5 However, little is known about the cortical mechanisms and processes that cause these deficits.¹ Several influential ADHD models^6-8 suggest that these decision-making and learning impairments are caused by impaired processing of what are termed reward prediction errors (RPEs).

Reward prediction errors have been discovered to reflect neural signals that drive learning and decision making.^2,9,10 Reward prediction errors signal violations of expectations and can be estimated by using computational reinforcement learning models.¹¹ It is now widely accepted that RPE signals are encoded by the phasic firing rate of dopaminergic neurons in the mesencephalon.¹² Reward prediction errors occur at 2 points during a decision-making trial: at cue and at outcome presentation. At cue presentation, RPEs (RPE_cue) reflect the expected value of a selected stimulus. At outcome, the RPE (RPE_outcome) is the difference between the reward received and the expected value of the selected stimulus.¹³ These RPE signals are projected from the dopaminergic midbrain to several prefrontal and striatal areas that are also crucially involved in decision making, such as the ventral striatum and the medial prefrontal cortex (mPFC).^8,14,15Neuroimaging studies^16-19 have consistently identified these regions as being impaired in ADHD. Additionally, studies on feedback-related negativity (FRN),^20-22 an electroencephalogram (EEG) component reflecting RPE processing in the mPFC, have suggested that RPE processing may be impaired as early as 200 to 400 milliseconds after outcome presentation in ADHD.^23-26

Although several lines of evidence suggest RPE impairments in ADHD, no study has investigated the neural substrates of RPE processing by means of computational modeling of learning and decision making in juvenile ADHD. Additionally, it remains unknown how these RPE impairments may relate to deficient learning mechanisms. Computational simulations of ADHD behavior²⁷ have suggested that individuals with ADHD make more exploratory decisions or may have a reduced learning rate, but this has not been examined in patients.

In this study, we applied the novel methods of computational psychiatry.²⁸ Computational psychiatry uses biologically plausible models, such as the aforementioned RPE-based reinforcement learning models,¹¹ to understand the mechanisms that underlie disturbed learning and decision making and overcome the limitations of purely descriptive measures, such as error rates. We examined the neural correlates of RPE processing. To overcome the poor temporal resolution of functional magnetic resonance imaging (fMRI) and the weak spatial resolution of EEG,²⁹ we used a simultaneous EEG-fMRI approach that exploits the advantages of both modalities without relying on spatial or other constraints of separate analyses.^30-32

The study was approved by the ethics committee of the Canton of Zurich, Switzerland, and all participants and their parents gave written informed consent. The participants each received a voucher for local stores for their participation.

Forty adolescents aged 12 to 16 years participated in this study (Table 1). Twenty individuals with ADHD were recruited from our outpatient clinics. Twenty healthy adolescents were recruited from local schools to serve as controls. All participants underwent a semistructured clinical interview (Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version, German version).^37,38 All participants with ADHD fulfilled the diagnosis of a combined inattention and hyperactivity-impulsivity subtype (DSM-IV code 314.01), corresponding to the 314.01 combined presentation according to DSM-5. Exclusion criteria were severe psychiatric disorders, such as schizophrenia, major depression, obsessive-compulsive disorder, pervasive developmental disorders, Tourette syndrome, substance abuse, primary mood or anxiety disorder (assessed using the Schedule for Affective Disorders and Schizophrenia for School-Age Children–Present and Lifetime Version), and autism spectrum disorders (assessed using the Social Communication Questionnaire³⁹). At the time of our study, only 1 participant with ADHD met the diagnostic criteria for comorbid conduct disorder, and none had oppositional defiant disorder. The controls were matched for age, sex, handedness, and IQ. Medicated patients with ADHD had to suspend their medication for at least 48 hours before testing. Because of excessive movement during scanning (>1 voxel maximal scan-to-scan movement), we had to exclude 1 participant with ADHD.

The participants played a probabilistic reversal learning task (Figure 1A and eMethods in the Supplement).^15,40 The participants had to learn the stimulus with a higher outcome probability on a trial-and-error basis to gain as much money as possible. The reward probabilities changed occasionally, and the participants had to adjust accordingly.

To infer learning, we compared 2 learning models and 2 decision models. As a standard learning model, we used an advanced Rescorla-Wagner model with an anticorrelated valuation system.¹⁵ This model has been shown^15,40 to be highly successful at inferring learning in probabilistic reversal learning tasks. We compared this model with a flexible Bayesian learning model, the hierarchical Gaussian filter model (HGF).⁴¹

In essence, the 2 learning models differ in their flexibility of learning. The advanced Rescorla-Wagner model has a fixed learning rate across the whole experiment, which means that the values of the stimuli are constantly updated, irrespective of any environmental or other change. The HGF, in contrast, has a flexible learning rate that adapts to changes in the volatility of the environment and according to beliefs about the value of the objects. This assumes a more precise and fine-grained learning process and has been shown^42,43 to be superior to reinforcement learning models. These findings also imply that healthy individuals learn in a more sophisticated manner than is assumed by the more simplistic Rescorla-Wagner learning model.

To ensure that all participants understood the task and performed above chance level, we additionally compared the best-fitting model for each person with a model that assumes performance at chance level. One participant with ADHD had to be excluded from further analysis because the chance model outperformed the other models. A more detailed description of the models and their update equations is provided in eMethods in the Supplement.

To determine the model that fitted behavior optimally, we performed Bayesian model selection for groups⁴⁴ across all participants and for each group independently. To further investigate learning and decision-making impairments, we compared the parameter estimates of the model that performed best across all subjects using Mann-Whitney tests.

Simultaneous EEG-fMRI was recorded (Achieva 3.0T scanner; Philips) using an MR-compatible EEG system (BrainAmp MR Plus; BrainProducts). Preprocessing and analysis of the fMRI were performed using SPM8 (http://www.fil.ion.ucl.ac.uk/spm/). Data obtained with EEG were preprocessed and analyzed using BrainVision Analyzer, version 2.0.2, and EEGLAB toolbox.⁴⁵ To study the neural differences in RPE processing between the groups as captured by fMRI, we entered the model-derived RPE values for every trial into the first-level analysis as 2 separate parametric modulators at the times of cue and outcome presentations. The first regressor corresponded to the RPE_cue and was therefore entered during cue presentation, whereas the second regressor modulated RPE_outcome and was thus entered during outcome presentation. To study the group differences of RPE_cue and RPE_outcome, we used independent-sample t tests and a multiple comparison correction threshold of P < .05 cluster-extent family-wise error corrected (voxel-height threshold, P < .001). As ADHD diagnoses imply, individuals with ADHD also display increased motor activity. Because the scan-to-scan motion differed marginally between our groups (ADHD mean [SD]: 0.10 mm [0.03]; range, 0.05-0.15 mm; and controls: 0.08 mm [0.03]; range, 0.05-0.20 mm; t₃₆, −2.0; P = .052), and because we wanted to ensure that our findings were not biased by movement artifacts, we also decided to analyze reduced groups excluding the 6 adolescents with the highest mean scan-to-scan movements (5 ADHD and 1 control). The reduced groups no longer differed significantly in motion (P > .10), and we subsequently discuss only the findings that were consistent across both analyses.

In the EEG, the FRN was analyzed as the difference between the most negative peak between 200 and 425 milliseconds after feedback and the preceding positive peak between 150 and 300 milliseconds (eFigure 1 in the Supplement).⁴⁶ These peaks were determined for each condition (reward and punishment) and participant separately. The FRN was then computed as the difference between punishments and rewards.

To localize the FRN, we used an EEG-informed fMRI approach^30,32,40 and entered the single-trial amplitudes as parametric modulators during feedback presentation into the first-level fMRI analysis. A detailed description of the preprocessing and data analysis is provided in the eMethods in the Supplement.

Mean reaction times, reaction time variability, and the number of misses did not differ between the groups (eTable 1 in the Supplement). However, participants with ADHD earned marginally less than controls (ADHD, 10.30 [11.70] CHF; controls, 15.60 [5.65] CHF; t₃₈= 1.82; P = .08).

Using Bayesian model selection for groups,⁴⁴ we found that the HGF performed best across all subjects (P_x = .70; P_x is the exceedance probability, ie, the probability that this particular model performs better than any other model included in the comparison) (eTable 2 in the Supplement and Figure 1B). The HGF also performed best for the controls (P_x = .98). For ADHD, however, the anticorrelated Rescorla-Wagner model clearly outperformed the HGF (P_x = .92).

The model parameter comparison of the best-performing model across all participants (HGF) revealed that those with ADHD showed a significantly less steep decision function (β: ADHD, 4.83 [2.97]; controls, 6.04 [2.53]; U = 109; z = −2.276; P = .02) (Figure 1C). We found no significant differences between the groups for the subject-specific volatility estimate (ω: ADHD, −1.70 [1.60]; controls, −1.26 [0.40]; U = 187; z = −0.08; P = .95) or the meta-volatility parameter (ϑ: ADHD, 0.0025 [0.0001]; controls, 0.0025 [0.0001]; U = 166; z = −0.674; P = .51).

During cue presentation (RPE_cue), participants with ADHD were found to process RPEs significantly differently in the mPFC (Table 2 and Figure 2A), both in the analysis containing all subjects and in the reduced groups.

During outcome presentation (RPE_outcome), RPE processing consistently elicited differential activations in the mPFC, both in the analysis containing all participants and in the reduced groups (Table 2 and Figure 2B). Additional group differences in the complete sample (Table 2) did not remain significant in the reduced groups.

The amplitudes for rewards and punishments were found to be largest at electrode Fz (eResults in the Supplement). The FRN at this electrode was significantly larger in the control group than that for the participants with ADHD (controls, −1.68 [2.52] μV; ADHD, 0.61 [3.90] μV; t₃₆ = −2.17; P = .04) (Figure 3A). Further analyses revealed that the controls showed a significant FRN (t₁₉ = −2.98; P = .008), whereas the participants with ADHD did not (t₁₇ = 0.66; P = .52).

To determine the generator of the FRN, we entered the single-trial amplitudes of the FRN as a parametric modulator in the fMRI design matrix. For the healthy controls, we localized the FRN to a cluster in the mPFC (Montreal Neurological Institute: x = −11, y = 56, z = 24; k = 582; z = 3.61) (Figure 3B). For the adolescents with ADHD, we did not find any significant activation. Strikingly, the source of the FRN in the controls overlapped with the region that also shows a significant difference between the groups in the RPE_outcome contrast.

In this study, we provided insights into the dysfunctional decision-making and learning mechanisms in adolescent ADHD using advanced learning models in combination with simultaneously recorded EEG and fMRI data.

By using different computational models of learning, we found that the behavior of healthy controls was better explained by the more-flexible Bayesian HGF model, whereas the simpler Rescorla-Wagner model was better suited for the participants with ADHD. The 2 models differ mainly in their flexibility. The Rescorla-Wagner model has a fixed learning rate, which entails that RPEs always have the same effect on learning, and the HGF has a more flexible learning rate that builds on environmental volatility and the participants’ current beliefs about the value of the objects. This diverging model selection result does not imply that the groups use strongly diverging learning mechanisms or diverging cognitive strategies. Rather, it suggests that adolescents with ADHD do not profit from the increased flexibility of the HGF and that they are not sensitive to subtle changes in reward contingencies, such as changes in environmental volatility or their current beliefs.

Comparison of the model parameters revealed that adolescents with ADHD have a less steep decision parameter β. This means that these participants differ in the exploration-exploitation dimension.^47,48 Participants with ADHD seem to exploit the best option less frequently according to their inferred beliefs, but to behave in a more exploratory way and examine the alternative option more often. This finding fits nicely with previous computational simulations,²⁷ which suggested that this decision steepness can cause ADHD-like behavior. In decision making during uncertainty, exploratory behavior is crucial to success because it facilitates the detection of changes in reward contingencies.^47,48 However, the fact that the healthy controls earned marginally more implies that the exploratory behavior of the participants with ADHD was too high for optimal task performance and that they were not able to adequately adjust their exploratory behavior.

Our analysis further revealed that ADHD cannot be characterized by an altered learning rate per se, because the higher-order volatility parameters (ϑ, ω) do not differ. This is in line with a previous study that did not find any learning rate impairments in ADHD.⁴⁹ Our finding also indicates that the differences in the model selection are not primarily caused by the volatility estimate, but rather by the current belief about the value. This finding also confirms that the participants with ADHD learned the reward contingencies properly and that the increased exploratory behavior found in the present study does not simply reflect randomness in behavior.

To understand the neural mechanisms that are responsible for the changes in the decision-making and learning processes, we examined RPE processing during cue and outcome presentations between the groups. Critically, we found activation differences during both phases in adjoining regions in the mPFC. This finding fits neatly with our behavioral finding of an altered decision steepness in ADHD, because we found the mPFC to be part of a network that is correlated with the decision-steepness parameter β in our participants (eResults, eTable 3, and eFigure 2C-D in the Supplement). Moreover, the mPFC is well known for processing prediction errors^50,51 and guiding value comparison and response selection,^52-54 and has been suggested to be a locus of malfunctioning decision making in ADHD.⁸ Although the findings in previous studies^55,56 on reversal learning tasks in ADHD were not consistent regarding mPFC impairment, overall, this region has frequently been associated with neural alterations in ADHD during rest⁵⁷ and cognitive tasks.¹⁶ Our findings indicate that deficient RPE processing in the mPFC may cause the suboptimal choice selection that is reflected by their more exploratory behavior.

The regions in the mPFC that we found to be impaired in ADHD are adjacent to the core regions known to process RPEs (Figure 2A and B). This suggests that individuals with ADHD may not process RPEs differently in the RPE core regions. Rather, it seems as if RPEs are processed in a less-extended area. This is also in line with our behavioral findings that learning in ADHD is not completely impaired; rather, there are more subtle differences, as reflected by the lowered decision steepness.

To better understand the temporal characteristics of RPE processing, we analyzed the FRN using an EEG-fMRI integration approach. We found that participants with ADHD did not have a significant FRN in contrast to the healthy controls, who showed a clear FRN. We successfully localized the FRN to the mPFC in healthy controls. Remarkably, the source of the FRN overlapped with the RPE_outcome impairment in the participants with ADHD. This may explain why we did not find a significant FRN and were not able to localize the component in the ADHD group. It also suggests that the impairment in the mPFC in ADHD reflects an early cognitive deficiency that occurs less than 400 milliseconds after feedback. Research^23-25 so far has investigated the FRN in ADHD with mixed results. Our study not only adds additional evidence for FRN attenuation but also clarifies its role and the neural origin.

Given that previous studies^16,17,19 on ADHD have often focused on the ventral striatum, a region also known to process RPEs,^58-60 we performed a supplemental analysis of a cluster in the subgenual anterior cingulate cortex and ventral striatum, which was found to be active in our RPE analysis (eResults in the Supplement). We found no RPE-related difference between the groups during cue presentation, but significantly deficient RPE processing occurred in the ADHD group during the outcome (eFigure 3 in the Supplement). A connectivity analysis revealed that the connectivity between this region of interest and the mPFC_outcome cluster is significantly lowered in ADHD. This finding indicates that both regions belong to a single frontostriatal loop, which has impaired connectivity in ADHD.

Attention-deficit/hyperactivity disorder has been discussed in the context of developmental delays.^61-66 Although our study cannot answer whether our findings reflect a developmental delay or an age-independent impairment, it is interesting that studies on healthy RPE development have found that the ventral striatum displays characteristic developmental trajectories^19,67,68 and that exploratory behavior decreases with age.⁶⁷

A limitation of the present study is that most of our ADHD sample received methylphenidate. We interrupted the medication for the experiment and therefore ensured that our findings were not biased by acute medication effects. However, we cannot exclude the possibility that our findings were influenced by some long-term effects of the medication. We also decided to investigate individuals who had received medication because we think that untreated ADHD may represent a possibly less severely affected ADHD subgroup rather than a representative sample of the ADHD population.

Taken together, the results of the behavioral modeling, fMRI, and EEG data suggest that adolescents with ADHD have specific learning and decision-making deficits. Individuals with ADHD cannot be characterized by an impaired learning rate per se, in contrast to what has been suggested by theoretical models.^6,27 Rather, they show a less fine-grained decision process and explore more frequently. These impairments are most likely caused by impaired RPE processing in the mPFC, a well-known integrative hub in decision making and learning.

By using a computational psychiatric approach²⁸ in combination with multimodal imaging, this study provides novel insights into impaired decision-making mechanisms and RPE deficits in adolescents with ADHD. Our findings further the understanding of potential pathomechanisms underlying impaired decision making and learning. Given that therapeutic interventions focus strongly on reinforcement modification, our findings could also inform interventional strategies for cognitive behavioral therapy (eg, working toward less-exploratory behavior). Moreover, our neural findings reinforce interventions in ADHD that focus on the mPFC, such as tomographic neurofeedback,⁶³ but may also encourage the use of extended neurofeedback methods, such as FRN-based training or real-time fMRI neurofeedback in the mPFC.

Submitted for Publication: January 22, 2014; final revision received March 28, 2014; accepted May 6, 2014.

Corresponding Author: Tobias U. Hauser, PhD, University Clinics for Child and Adolescent Psychiatry, University of Zurich, Neumünsterallee 9, 8032 Zürich, Switzerland (tobias.hauser@kjpd.uzh.ch).

Published Online: August 20, 2014. doi:10.1001/jamapsychiatry.2014.1093.

Author Contributions: Drs Walitza and Brem contributed equally to the study. Drs Walitza and Brem had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Hauser, Brandeis, Walitza, Brem.

Acquisition, analysis, or interpretation of data: Hauser, Iannaccone, Ball, Mathys, Brandeis, Brem.

Drafting of the manuscript: Hauser, Brem.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Hauser.

Obtained funding: Brandeis, Walitza, Brem.

Administrative, technical, or material support: Iannaccone, Ball, Mathys, Brandeis, Walitza, Brem.

Study supervision: Brandeis, Walitza, Brem.

Conflict of Interest Disclosures: Dr Hauser is supported by the Swiss National Science Foundation. Dr Walitza has received speakers’ honoraria from Eli Lilly, Janssen-Cilag, and AstraZeneca in the past 5 years. No other disclosures were reported.

Funding/Support: This study was supported by the Swiss National Science Foundation (grant 320030_130237) and the Hartmann Müller Foundation (grant 1460).

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: Niklas Brons, MD, Renate Drechsler, PhD, Carolin Knie, MSc, Julia Frey, MSc (University Clinics for Child and Adolescent Psychiatry, University of Zurich), and Philipp Stämpfli, PhD (Center of the Psychiatric University Hospital and the Department of Child and Adolescent Psychiatry, University of Zurich), provided technical assistance and helped to recruit the participants. No financial compensation was provided.