In studies with known measures, symbols show the mean, and error bars, the 95% confidence interval (CI). Studies with nonstatistically significant unreported effects are distinguished by the presence of a shaded bar around the mean showing the interval containing 95% of the imputed effect sizes; the central symbol shows the mean, and the error bars, the 95% CI. Size of the data marker corresponds to the relative weight assigned in the pooled analysis. FEP indicates first episode of psychosis; FGA, first-generation antipsychotic; HR, individuals at high risk for schizophrenia; and SGA, second-generation antipsychotic. The location of the ventral striatum is indicated for reference.
eMethods. Description and validation of the MetaNSUE method
eFigure 1. PRISMA flow chart
eFigure 2. Forest plots of the ventral striatum response to reward feedback in psychosis
eFigure 3. Forest plots of the ventral striatum response to reward reward prediction error in psychosis
eTable 1. MOOSE checklist
eTable 2. fMRI ventral striatum findings of studies addressing anticipation of monetary loss, aversive feedback and aversive prediction error
eTable 3. fMRI ventral striatum findings of studies addressing other salience related contrasts
eTable 4. Quality assessment of the included studies and the rating of the studies
Customize your JAMA Network experience by selecting one or more topics from the list below.
Radua J, Schmidt A, Borgwardt S, et al. Ventral Striatal Activation During Reward Processing in Psychosis: A Neurofunctional Meta-Analysis. JAMA Psychiatry. 2015;72(12):1243–1251. doi:10.1001/jamapsychiatry.2015.2196
Abnormal reward processing is suggested to underlie the formation of psychotic symptoms, likely driven by elevated ventral striatal (VS) dopamine levels. Functional magnetic resonance imaging studies reveal alterations of VS activity during reward processing in patients with chronic psychosis and first episode of psychosis, as well as individuals at high risk for psychosis, but findings are inconclusive, conflicting, and difficult to subject to meta-analysis without introducing bias because several studies reported that findings were not statistically significant but did not report statistics.
To assess the differences between patients with schizophrenia spectrum disorders and healthy controls in VS activation during reward processing.
Web of Knowledge database (incorporating Web of Science and MEDLINE) until July 2015, including references of eligible articles and reviews.
Functional magnetic resonance imaging studies comparing VS activity during monetary reward processing between patients with schizophrenia spectrum disorders or clinical or genetic high-risk state for psychosis and healthy controls.
Data Extraction and Synthesis
Statistics and thresholds related to the main outcome measures and potential moderators were independently retrieved by 2 investigators. Effect sizes were analyzed using MetaNSUE, a random-effects method that enables the unbiased inclusion of nonstatistically significant unreported effects.
Main Outcomes and Measures
Effect size of the group differences in VS activity, and correlation between VS activity and negative and positive symptom scores in patients.
The meta-analysis included 23 studies (917 patients) for reward anticipation, 9 studies (358 patients) for reward feedback, and 8 studies (314 patients) for reward prediction error. We found significant bilateral VS hypoactivation during reward anticipation (23 studies, n = 917) in patients compared with healthy controls (left/right Cohen d, −0.50/−0.70; P < .001). Left VS abnormality was more severe in patients with high scores of negative symptoms during reward anticipation (r = −0.41; P < .001). Patients also showed hypoactivation during reward feedback (left/right d, −0.57/−0.56; P < .001). Simulations showed that exclusion of studies with nonstatistically significant unreported effects was associated with a strong bias (d bias = 0.22), whereas estimations using MetaNSUE were unbiased even when statistics were seldom reported (d bias < 0.001).
Conclusions and Relevance
This meta-analysis provides evidence that patients with psychosis demonstrate VS hypoactivation during reward anticipation. The assessment of VS prediction errors seems to be promising, but more studies are needed to draw valid conclusions.
Studies from more than half a century ago had already reported that patients with schizophrenia often describe how irrelevant stimuli capture their attention and are assigned unduly high significance.1,2 This phenomenon was suggested to result from abnormal stimulus-reinforcement formations induced by chaotic dopaminergic firing in mesolimbic reward pathways, which could consequently produce psychotic symptoms.3 The phenomenon was later on framed in terms of “aberrant salience,” a phenomenological concept proposing that psychosis may arise from an inappropriate assignment of salience to contextually irrelevant external events and internal mental states.4,5 In this context, salience refers to the motivational aspect of stimuli, which catch attention because of their association with primary reinforcement,6 and is mediated by dopamine in the ventral striatum (VS).6
Recent proposals link aberrant salience to abnormal prediction error processing, driving the formation of reinforcement learning abnormalities and possibly psychotic symptoms.7,8 Prediction errors, the discrepancy between the actual inputs and the prediction about it, are mediated via midbrain dopamine neurons and their targets in the VS.9,10 A contemporary animal model proposed that psychosis may develop as a result of an impaired inhibitory functioning of the medial temporal lobe, leading to elevated VS dopamine level.11 It was suggested that consequently the number of dopamine neurons participating in providing prediction error signals may be upregulated in schizophrenia, contributing to abnormal salience processing.11 Whereas diverse neural measures have been interpreted to reflect aberrant salience attribution in psychosis studies12 up to now, most studies—as meta-analyzed in this article—focused on the processing of reward-indicating cues. In this context, a blunted response between reward-indicating and neutral cues was taken as a measure of aberrant salience.
Positron emission tomography and single-photon emission computed tomography studies have consistently shown that striatal dopamine synthesis capacity is increased in psychosis.13,14 Functional magnetic resonance imaging (fMRI) studies during monetary reward anticipation have also detected altered VS activation in patients experiencing their first episode of psychosis, patients with chronic psychosis, and individuals at clinical or genetic high risk (HR) for psychosis. However, these fMRI findings seem inconclusive because some studies found reduced VS activation15-25 whereas others reported no abnormalities.26-37 Similar conflicts are evident for fMRI reward prediction error studies, with reduced VS activation being found in some studies38-43 and VS hyperactivation26 or no VS differences in others.44 Additionally, the clinical relevance of VS alterations in psychosis is unclear. For example, most of the studies investigated the relation between VS activation and negative symptoms, with some studies revealing significant correlations15,17,23,28 and others not.30,32,40 These inconsistencies may be due to small and heterogeneous patient samples, varying tasks and imaging analyses, and/or confounding effects of antipsychotic medication. Unfortunately, these studies are difficult to subject to meta-analysis because several studies state that findings were not statistically significant without reporting statistics, and attempts to retrieve unpublished data by contacting authors have been shown to be mostly (~80%) unsuccessful.45
To address these inconsistencies, we present here, to our knowledge, the first systematic review and neurofunctional meta-analysis of VS activation during reward processing in patients at HR, those experiencing their first episode of psychosis, and those with chronic psychosis. In particular, we focused on reward anticipation and feedback, as well as reward prediction error. We investigated the robustness of the meta-analytic findings and the effects of potential clinical and methodological moderators. In addition, we present an innovative methodological approach to include all studies even if they only report that they did not find any statistically significant effect (but not the specific effect size or related statistics).
Two investigators (A.S., P.F.-P.) conducted an independent systematic 2-step literature search to identify relevant articles. First, the Web of Knowledge database (incorporating Web of Science and MEDLINE) was searched to detect abstracts in English published through July 2015 (keywords: “psychosis,” “schizophrenia,” “high-risk psychosis,” “salience,” “fMRI,” “ventral striatum,” “reward,” “prediction error”). Second, Scopus was used to detect citations of previous systematic reviews and to perform manual searches of the reference lists of the retrieved articles. Identified articles were then screened according to the selection criteria. The Meta-analysis of Observational Studies in Epidemiology (MOOSE) checklist46 was adopted (eTable 1 in the Supplement).
Studies were eligible for inclusion if they (1) were original articles written in English; (2) compared patients with ICD-10 Classification of Mental and Behavioral Disorders47 and/or DSM-548 diagnosis of schizophrenia spectrum disorders or clinical49 or genetic50 HR state for psychosis with healthy controls; and (3) investigated fMRI VS responses explicitly during monetary reward processing, in particular contrasts addressing reward anticipation, feedback of reward, or reward prediction error (Table). If available, multiple contrasts from the same study were included. Studies were excluded if they (1) only included anticipation of monetary loss, aversive feedback, and aversive predictive error or other salience-related contrasts (eTables 2 and 3 in the Supplement) or (2) used overlapping data sets16,21 (we included the original and larger data sets15,17,18 instead of pooled results21).
As detailed in the Statistical Analysis section, we did not exclude studies that did not report effect sizes; note that exclusion of studies with statistically nonsignificant differences would bias the meta-analysis toward those studies with large differences. Literature search was summarized according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (eFigure 1 in the Supplement).51
Data extraction and quality assessment were independently performed by 2 investigators (A.S., P.F.-P.). The following variables were recorded from each article: reference, sample sizes, age, sex, illness stage, antipsychotic treatments, type of analysis (“region of interest” [ROI] vs whole-brain), statistical thresholds, VS group differences (eg, t values), publication year, clinical and methodological items objectively used to calculate quality scores (eTable 4 in the Supplement), and correlations between VS activity and negative and/or positive symptoms.
Notably, whole-brain studies did not report the mean VS effect but the peak VS effect. This latter effect is larger than the mean VS effect, and its use would thus bias the meta-analysis. To address this issue, whole-brain coordinates and t values of the maxima were introduced into anisotropic effect-size signed differential mapping52,53 in order to recreate the image of effect size based on the correlations between adjacent voxels, and the mean effect of all voxels with any probability of being located in the nucleus accumbens according to the Harvard-Oxford atlas54 was then extracted. To overcome the potential downward bias associated to this estimation,53 maps were scaled to make differences with ROI studies minimal. Also, analyses were repeated after only including the latter.
As noted, some studies with nonsignificant group differences did not report any statistics (eg, t, F, or P values) that could be converted into effect sizes. Exclusion of these findings (“nonstatistically significant unreported effects” [NSUEs]) would bias the meta-analysis—taken to the extreme, including only the 3 studies reporting differences but excluding the 1000 studies not detecting any difference (as detailed in the eMethods section in the Supplement, we also empirically test this bias with simulations). Inclusion of studies with NSUEs by assuming them to have a null effect size would be a more conservative option but still not free from bias.53
To correctly include studies with NSUEs into meta-analyses, we developed a new method, called MetaNSUE, based on multiple imputations algorithms.55 First, the MetaNSUE calculates the bounds of nonstatistical significance of each study with NSUEs (ie, the unreported t value or r coefficient must be within these bounds) and converts them to unbiased effect sizes (Cohen d or z) using a standard formula.53 Second, an estimation of the parameters for subsequent imputations is conducted by maximizing the likelihood that the reported effect sizes have those values, as well as the likelihood that the unreported effect sizes are within those bounds. Relevantly, these parameters include the between-study heterogeneity (ie, random differences between studies beyond those due to sampling) and potential covariates used to individually predict the expected effect size of each study (with or without NSUEs). Third, multiple imputations of the unreported effect sizes are randomly created according to the expected value (within- and between-study), variance, and statistical significance bounds of each study with NSUEs. This step is needed to create realistic “noisy” imputations because imputing the unreported effect sizes using their expected value would mean assuming that within-study variability and between-study heterogeneity are null. Finally, a standard meta-analysis is separately conducted for each set of imputed effect sizes using restricted maximum-likelihood random-effect models,56 and results from these meta-analyses are pooled using a standard formula for multiple imputations.57
Empirical validation using simulations showed that estimations were strongly biased when conducting a standard meta-analysis without NSUEs (d bias = 0.22), moderately negatively biased when converting NSUEs to zeros (d bias = −0.07), and nearly unbiased when using MetaNSUE (d bias < 0.001, even when statistics were seldom reported). See eMethods in the Supplement for details of the procedure and the validation.
A separate meta-analysis was conducted for group differences in left and right VS activation. Robustness of these differences was assessed by studying the between-study heterogeneity, estimating the potential reporting bias (metaregression by standard error to detect whether results from small imprecise studies might have been reported only if they were significant58-60), and conducting jackknife analyses (ie, iteratively repeating the meta-analysis with all studies but 1 in order to detect whether results may be driven by a single study). Differences between left and right group differences in activation were also investigated, along with subgroups (medication-free, patients, HR individuals, ROI studies) and metaregressions by mean age, percentage of males, percentage of medicated patients, publication year, and quality score. Meta-analysis of the correlations between psychotic symptoms and VS activation was also conducted. Finally, we performed an exploratory analysis treating all 3 domains (anticipation/feedback and prediction error) together.
On the basis of the results of the empirical validation, moderator variables were included in the maximum-likelihood step. All findings with P < .05 are reported as trends, but given the multiple tests conducted (3 different reward processing domains and 2 brain sides), only those at P ≤ .05/6 = .008 were considered statistically significant.
This meta-analysis included 23 studies (n = 917) (see Table), with reported group differences in 7 (left) and 9 (right) studies, and NSUEs in the remaining studies. Patients showed significant hypoactivation in both left and right VS (d left/right = −0.50/−0.70; P < .001 in both cases) (Figure). No residual heterogeneity or potential reporting bias was observed (P = .12-.92). Jackknife analyses showed similar hypoactivation when any single study was discarded (left d range, −0.45 to −0.58; right d range, −0.66 to −0.77; P < .001 in all cases) or when the meta-analysis was restricted to studies in medication-free patients or to studies using ROIs (left/right d = −0.45/−0.72 and −0.59/−0.65, respectively; P < .001 in all cases). No differences were detected between left and right VS (P = .18). No effects of age, sex, antipsychotic medication use, publication year, or quality score were observed (P = .14 to >.99). Similar hypoactivation was found in patients and HR individuals (left/right d = −0.55/−0.67 vs −0.44/−0.78; P = .53/.57).
Eleven studies analyzed the correlation between VS activation and negative symptoms,15,17,23,24,27-30,32,33 and we were able to retrieve the correlation coefficient in 5 (left) and 2 (right) studies, with the remaining studies reporting NSUEs. Left hypoactivation was more pronounced in patients with higher negative symptoms (r = −0.41; P < .001), an effect that was not evident at the right VS (P = .86). No residual heterogeneity or potential reporting bias was observed (P = .35-.96). Jackknife analyses (only conducted for left VS) showed similar correlations when any single study was discarded (r range, −0.36 to −0.45; P < .003 in all cases).
Six studies had investigated the relationship with positive symptoms20,23,24,27,32,33 and we were able to retrieve the correlation coefficient in 2 (left) and 3 (right) studies, with the remaining studies reporting NSUEs. No relationship between VS activation and positive symptoms could be detected (P = .47-.79), although this result should be taken with caution because only 6 studies could be included and there was residual heterogeneity among them (left/right: I2 = 63%/72%; P = .03/.003, probably due to studies reporting opposite findings).
This meta-analysis included 9 studies (n = 358) (Table), with reported group differences in 1 study, and NSUEs in the remaining studies. Patients showed significant hypoactivation in both left and right VS ( left/right d= −0.57/−0.56; P < .001) (eFigure 2 in the Supplement), and no residual heterogeneity or potential reporting bias was observed (P = .89-.93). One may wonder that a single study makes the meta-analysis statistically significant. However, it must be noted that the empirical validation showed that MetaNSUE’s false-positive rate is not increased when only 1 study reports significant differences. That said, jackknife analyses showed similar hypoactivation when any study with NSUEs was discarded (left d range, −0.59 to−0.64; right d range, −0.57 to −0.62; P < .001 in all cases) but no differences if the only study detecting group differences23 was discarded (P = .95-.96), indicating a lower replicability of this finding, which should thus be taken with caution. No differences were detected between left and right VS (P = .72). No meta-regression analyses were conducted because of the high probability that the study detecting differences behaved as a leverage point. No relationship with negative or positive symptoms could be observed (eResults in the Supplement).
This meta-analysis included 8 studies (n = 314) (Table), with reported between-group differences in 4 (left) and 6 (right) studies, and NSUEs in the remaining studies. Patients showed hypoactivation in both left and right VS (left/right d = −0.28/−0.53) (eFigure 3 in the Supplement), although not statistically significant at the left and only at trend level (.008 < P < .05) at the right side (left/right P = .37/.01). This lack of significance should be taken with caution because only 8 studies were included, and there was residual high heterogeneity (which decreases the precision and thus the statistical significance of the estimates) among them (I2 = 83% and 64%; P < .001 and .02). No potential reporting bias was observed (P = .12-.76). Jackknife analyses (only conducted for right VS) showed relatively similar hypoactivation when any single study was discarded (d range, −0.39 to −0.62; all P < .06). Differences between left and right VS were not statistically significant (P = .53). No metaregression analyses were conducted because of the paucity of studies. No relationship with negative or positive symptoms could be observed (eResults in the Supplement).
Results of this exploratory analysis are explained in the eResults in the Supplement.
Our meta-analysis revealed that psychosis was robustly associated with VS hypoactivation during reward anticipation, including 23 studies with 917 participants, whereas no residual heterogeneity, potential reporting bias, or jackknife abnormalities have been detected. Meta-analyses of reward feedback and prediction error also showed VS hypoactivation but should be carefully considered, because findings were driven by a single study in the first (although validation showed an excellent control of the false-positive rate) and were not statistically significant in the second (probably due to between-study heterogeneity). No differences were observed between left and right VS in any of the reward processes. Our meta-analysis further showed that left VS activation during reward anticipation was negatively correlated with negative symptoms in patients.
The robust finding of VS hypoactivation during reward anticipation may support that psychosis is characterized by impaired learning of stimulus-reinforcement associations.7,61 However, studies of reward anticipation did not investigate the learning process directly but rather the neural response to reward-indicating cues learned before scanning, and thus our finding may reflect a more general blunting of VS responsivity rather than a specific deficit during reinforcement learning. The attribution of incentive salience to rewarding cues has been proposed to be mediated by phasic dopamine increase in the striatum and contributes to the wanting of reward.6 A recent meta-analysis has shown that striatal presynaptic dopaminergic markers are consistently altered in psychosis.13,62 In chronic psychosis,13,63 and already in clinical HR subjects,64 striatal dopamine levels are elevated in the absence of incoming stimuli, which may steer the assignment of salience to normally irrelevant stimuli whose presence happens to temporally coincide with dopamine release.4,65 Moreover, a chaotic stress-associated striatal dopamine release5,7 may also impede a phasic dopamine release in response to contextually relevant (eg, reward indicating) cues, leading to decreased differentiation between the responses to relevant and irrelevant stimuli.65 In this framework, the VS hypoactivation during reward anticipation can be taken as a measure of altered salience processing because it reflects a blunted response toward reward-indicating compared with neutral cues. However, general caution is recommended in relating dopamine release to neural activation because they are often not measured in the same region (elevated dopamine levels have been found in the associative striatum63,64 and different VS definitions are used across studies) and also because fMRI signals have limited statistical relationships to neurochemical markers.66 For example, sophisticated attempts to relate VS fMRI signals during reward anticipation with measures of dopamine release induced by monetary incentive delay task have revealed mixed results.67,68 The Discussion section should be read with this caveat in mind.
We found that patients also showed a trend for right VS hypoactivation during reward prediction error processing. However, the robustness of this meta-analysis was limited by a relevant between-study heterogeneity, which is probably due to the different reinforcement learning algorithm,38,40,42 and the diverse psychological task designs and contrasts used across studies.26,39,41,43 Previous studies in healthy participants combining fMRI and positron-emission tomography measures showed that the right VS prediction error signal is negatively associated with VS presynaptic dopamine level.69,70 We could thus speculate that the elevated striatal dopamine level in psychosis14 might be associated with the right VS hypoactivation during prediction error processing that we found. In line with this argumentation, a behavioral measure of aberrant salience attribution (derived from the salience attribution task), which is heightened in patients with schizophrenia,71 was found to be positively correlated with striatal dopamine synthesis capacity and negatively correlated with fMRI striatal prediction errors signal in health controls.72
Current antipsychotic drug use was not found to moderate the reduced VS activation during reward anticipation. However, most studies included patients treated with both typical and atypical antipsychotics, and it has been proposed that the effect of several atypical drugs may result from a dopamine-mediated attenuation of aberrant salience processing,4,65 whereas typical but not atypical drugs have been shown to reduce the VS response to reward-indicating stimuli.15 In line with this finding, a recent meta-analysis in psychosis did not find a significant modulation of antipsychotics on striatal dopamine synthesis capacity.13,62
We found that left VS hypoactivation during reward anticipation was more pronounced in patients with higher negative psychotic symptoms in accordance with evidence from a previous meta-analysis.73 Our correlation findings thus suggest that VS hypoactivation may impair the positive and motivational effect of rewarding events and in turn promote negative symptoms.5 This corresponds with a first pilot study, which suggested that decreased left VS activation is inversely correlated with the severity of negative symptoms in antipsychotic-free patients.16 The authors suggested that high striatal dopamine turnover may increase the “noise” in the reward system, thus interfering with the neuronal processing of reward-predicting cues by phasic dopamine release.16 This, in turn, may lead to negative symptoms, which group in 2 factors, one involving diminished expression of affect and alogia and the second involving avolition including anhedonia and asociality.74 Our findings may also have some translational effects, given that negative psychotic symptoms are refractory to all available treatments.75 However, this conclusion requires further research given that there is also evidence showing that dopamine function in the VS was inversely correlated with negative symptom severity.63 Furthermore, the relation between VS activation during reward anticipation and positive symptoms requires further investigation because only 6 studies were available and there was residual heterogeneity among them.
This study has some limitations. Findings of VS hypoactivation during reward feedback and prediction error, as well as the correlation with positive symptoms, are less robust than the hypoactivation during reward anticipation or the correlation with negative symptoms and should be taken with caution until more studies are available; the first finding might also reflect that VS activation is not crucially involved during the feedback phase in the monetary incentive delay task.76 We could not explore whether group differences may be due to negative symptoms. Region of interest delimitation was slightly heterogeneous across studies, resulting in different VS definitions. Similarly, recreation of images from peak t values may not be free from downward bias,53 although maps were scaled and analyses were repeated after only including the latter. To contain consistency in fMRI contrasts, we focused on rewarding stimuli without considering anticipation of monetary loss, aversive feedback, and aversive predictive error or other salience-related contrasts due to the high heterogeneity of these contrasts; however, a preliminary meta-analysis during loss anticipation revealed bilateral VS hypoactivation in psychotic patients, suggesting that VS hypoactivation in psychosis may be associated with a general deficit in salience processing (eTable 2 in the Supplement). More studies are needed to support this finding. The present analysis was restricted to the VS given its relevance during reward processing and psychosis.12 However, the VS does not function in isolation during reward processing and future studies should consider this process from a network perspective.77 A whole-brain analysis will be of interest when more whole-brain studies are available. Finally, the lack of a significant effect in the moderator variable for disease stage does not necessarily mean that no differences exist between the groups. More studies for each group, particularly for the HR group, are needed to draw robust inferences.
This meta-analysis demonstrates reduced VS activation during reward anticipation in psychosis, which supports altered processing of salient reward-indicating stimuli. We further showed that the VS dysfunction during reward prediction is correlated with negative symptoms. More studies are needed to assess whether the abnormality also affects reward feedback and prediction error.
Submitted for Publication: June 2, 2015; final revision received August 10, 2015; accepted September 19, 2015.
Corresponding Author: Joaquim Radua, MD, BStat, PhD, Department of Psychosis Studies, Institute of Psychology, Psychiatry and Neuroscience, PO 63, 16 De Crespigny Park, London SE5 8AF, England (firstname.lastname@example.org).
Published Online: November 11, 2015. doi:10.1001/jamapsychiatry.2015.2196.
Author Contributions: Drs Radua and Fusar-Poli had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Radua and Schmidt served as co–first authors, each with equal contribution to the manuscript.
Study concept and design: Radua, Schmidt, Heinz, Fusar-Poli.
Acquisition, analysis, or interpretation of data: Radua, Schmidt, Borgwardt, Schlagenhauf, McGuire.
Drafting of the manuscript: Radua, Schmidt, Fusar-Poli.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Radua.
Obtained funding: Schmidt, Heinz, McGuire.
Administrative, technical, or material support: Schmidt, Heinz, McGuire.
Study supervision: Schmidt, McGuire, Fusar-Poli.
Conflict of Interest Disclosures: None reported.
Funding/Support: Dr Radua received grant support from Instituto de Salud Carlos III–Subdirección General de Evaluación and the European Regional Development Fund (personal grant Miguel Servet CP14/00041 and project PI14/00292 integrated into the National Plan for research, development and innovation). Dr Schmidt was supported by the Swiss National Science Foundation (SNSF; No. 155184). Dr Fusar-Poli was supported in part by a 2014 National Alliance for Research on Schizophrenia and Depression (NARSAD) Young Investigator Award.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Information: MetaNSUE software is freely available at http://www.metansue.com for other researchers.