Each column represents a different profile in the multitrajectory model and is defined by the trajectory of irritability and depressive/anxious mood at 6, 7, 8, 10, and 12 years of age. Boxes represent observed value, and lines represent the fitted regression slopes. Fit indices of the model include log-likelihood, −18246.25; Bayesian information criterion, −18323.91; entropy, 0.730 (ie, quality of classification; adequate if >0.70); and mean odds of correct classification, 19.2 (ie, the model classifies the participants 19.2 times better than classification by chance; adequate if >5.0).
eTable 1. Factor Loadings of the Exploratory Factor Analysis of the Oppositionality Items by Year (Promax Rotation)
eTable 2. Independent Predictors of Profile Membership
eTable 3. Sensitivity Analyses
Customize your JAMA Network experience by selecting one or more topics from the list below.
Orri M, Galera C, Turecki G, et al. Association of Childhood Irritability and Depressive/Anxious Mood Profiles With Adolescent Suicidal Ideation and Attempts. JAMA Psychiatry. 2018;75(5):465–473. doi:10.1001/jamapsychiatry.2018.0174
Do children showing different profiles of irritability and depressive /anxious mood have different risk for suicide during adolescence?
In this population-based cohort study, 1430 children presenting with high irritability and high depressive/anxious mood symptoms during childhood (aged 6-12 years) were 2 times more likely to think about suicide or make a suicide attempt in adolescence (aged 13-17 years) compared with children showing depressive/anxious mood only or low irritability and low depressive/anxious mood.
Childhood irritability should be considered when assessing adolescent suicidal risk, especially among those presenting with symptoms of high depressive/anxious mood.
Suicidal ideation and suicide attempt (suicidality) are common in adolescence and a public health concern. Childhood depression is a key risk factor for later suicidality and often co-occurs with irritability. No study to date has examined the joint association of depressive mood and irritability during childhood with later suicidality.
To investigate the association of childhood irritability and depressive/anxious mood profiles with adolescent suicidality.
Design, Setting, and Participants
This population-based cohort study included 1430 participants in the Québec Longitudinal Study of Child Development. Participants underwent assessment yearly or bi-yearly (5 months to 17 years). Data were collected from March 16, 1998, through July 17, 2015.
Profiles defined by the joint developmental trajectories of irritability and depressive/anxious mood at 6 to 12 years of age.
Main Outcomes and Measures
Self-reported past-year suicidality (ie, serious suicidal ideation or suicide attempt) at 13, 15, and 17 years of age. Irritability and depressive/anxious mood were assessed using teacher report 5 times from 6 to 12 years of age.
The study included 1430 participants (676 boys [47.3%] and 754 girls [52.7%]) followed up to 17 years of age. Group-based multitrajectory modeling identified the following profiles: combined no irritability and low depressive/anxious mood with low irritability and low depressive/anxious mood (831 [58.1%]; reference group), moderate irritability and low depressive/anxious mood (353 [24.7%]), high depressive/anxious mood only (94 [6.6%]), and high irritability and depressive/anxious mood (152 [10.6%]). Children with high irritability and high depressive/anxious mood reported higher rates of suicidality (25 of 152 [16.4%]) compared with the group with the lowest symptom levels (91 of 831 [11.0%]). In logistic regression analyses, the high irritability and depressive/anxious mood profile (odds ratio [OR], 2.22; 95% CI, 1.32-3.74; number needed to be exposed [NNE], 18) was associated with suicidality. To a lesser extent, the moderate irritability and low depressive/anxious mood profile was also associated with suicidality (OR, 1.51; 95% CI, 1.02-2.25; NNE = 48). The high depressive/anxious mood only profile was not associated with later suicidality (OR, 0.96; 95% CI, 0.47-1.95; NNE = −320). The high irritability and depressive/anxious mood profile was associated with a higher suicidal risk compared with the depressive/anxious mood only profile (OR, 2.28; 95% CI, 1.02-5.15). Girls with the high irritability and high depressive/anxious mood profile had higher risk for suicidality (OR, 3.07; 95% CI, 1.54-6.12; NNE = 5).
Conclusions and Relevance
Children with high irritability and depressive/anxious mood and, to a lesser extent, with moderate irritability only had a higher suicidal risk during adolescence compared with children with low symptom levels. Early manifestation of chronic irritability during childhood, especially when combined with depressive/anxious mood, may be associated with an elevated risk for adolescent suicidality. The putatively causal role of irritability should be investigated.
Suicidal ideation and suicide attempt in adolescence constitute a major public health concern and are linked to long-term adjustment problems.1,2 Individuals with depression are more at risk of thinking about suicide, attempting suicide, and killing themselves.3-5 Therefore, it is important to recognize and prevent early manifestations of depression that are associated with later suicidal behaviors.
Irritability and depressive mood are core symptoms of depression. However, during childhood, irritability is more frequently observed as a core symptom of depression than in adulthood.6,7 Irritability is reported by one-third of depressed children6 and is reflected in increased proneness to anger. According to the DSM-5, irritability manifests itself clinically with frequent temper outbursts typically occurring in response to frustration and can be verbal or behavioral (ie, aggression against others, self, or objects).8 Irritability can be observed as a behavior during the interactions between children and their parents, teachers, and peers.9 Chronic severe irritability is the main feature of disruptive mood dysregulation disorder (affecting 3% of children in the general population),10 which is a new DSM-5 diagnosis that was introduced to solve the controversy surrounding the diagnosis of childhood bipolar disorder.11
Despite the importance of childhood irritability in the characterization of mood disorders and depression, only 2 longitudinal population-based studies have investigated its predictive association with suicidality.12,13 Both studies suggested that irritability in adolescence increased suicidal risk in adulthood independently from depression. However, neither study examined the joint contribution of irritability and depression because they did not distinguish between individuals presenting with 1 or both dimensions. Thus, evidence exists on the independent effect of depression and irritability but is lacking on their joint effect on suicidal risk. Moreover, irritability was measured in adolescence. Childhood assessment of irritability is important from a developmental perspective, because childhood is the most relevant period in the manifestation of irritability.9 Finally, both studies assessed suicidality in adulthood. Suicidality usually emerges and peaks during adolescence2,14; therefore, studies focusing on adolescence provide information on the first manifestations of suicidality.
Using data from a large birth cohort prospectively followed up for 17 years, we aimed to (1) identify longitudinal profiles of irritable and depressive and/or anxious mood during the course of childhood and (2) examine their associations with suicidality (ie, suicidal ideation and suicide attempt) during adolescence. To capture the overlap between the development of irritability and depressive/anxious mood in children, we estimated the joint developmental trajectories of these symptoms. This approach allowed us to identify distinct profiles of individuals who may show depressive/anxious or irritable mood or both types of symptoms. Sex differences were explored, because there are important sexual differences in suicidality.15
Participants were drawn from the Québec Longitudinal Study of Child Development (QLSCD), a representative sample of 2120 infants born in the Canadian province of Québec in 1997 and 1998 who were followed up to 17 years of age. The original sample was selected from the Québec Birth Registry using a stratified procedure based on living area and birth rate. Families were included if the pregnancy lasted 24 to 42 weeks and the mother could speak French or English.16 Data were collected yearly during childhood and biyearly during adolescence by the Québec Statistics Institute.17 We used information on irritability and depressive/anxious mood assessed by teachers from 6 to 12 years of age and subsequent self-reported suicidality at 13, 15, and 17 years of age, resulting in a sample of 1430 participants. Those participants were broadly representative of the original sample but differed in terms of child sex, socioeconomic status,18 and child verbal IQ.19 No differences were found in birth weight, child temperament,20 family functioning,21 hostile-reactive parenting, maternal depressive symptoms,22 parental antisociality,23 and parental age at child birth (Table 1). The QLSCD protocol was approved by the Québec Statistics Institute and the St Justine Hospital Research Center ethics committees. Written informed consent was obtained from all participants.
School teachers rated children on the Behavior Questionnaire at 6, 7, 8, 10, and 12 years of age. The Behavior Questionnaire was created for the Canadian National Longitudinal Study of Children and Youth24 and incorporates items from the Child Behavior Checklist,25 the Ontario Child Health Study Scales,26 and the Preschool Behavior Questionnaire.27 Items were rated using a 3-point Likert scale according to the frequency of the behavior in the past 6 months (0 indicates never; 1, sometimes; and 2, often). At each point, the child’s behavior was assessed by a different teacher.
Depressive/anxious mood was assessed with the following 9 items (α range, .84-.86): seemed to be unhappy or sad; was not as happy as other children; had no energy, was feeling tired; had trouble enjoying himself or herself; cried a lot; was too fearful or anxious; was worried; was nervous, high-strung, or tense; and was incapable of making decisions. At all points, the mean of the items was calculated to obtain the depressive/anxious mood score (range, 0-10, with higher scores indicating high depressive/anxious mood).
Irritability was assessed with the following 4 items (α range, .85-.91): had temper tantrums or hot temper, reacted in an aggressive manner when teased, reacted in an aggressive manner when contradicted, and reacted in an aggressive manner when something was taken away from him or her. At all points, the irritability score was obtained by summing the first item (temper tantrum) with the mean of the other 3 items, because they evaluated the same behavior (reacting in an aggressive manner) in 3 different situations (range, 0-10, with higher scores indicating high irritability) (eTable 1 in the Supplement).
Serious suicidal ideation and suicide attempt were assessed at ages 13, 15, and 17 years.28,29 Adolescents were asked, “In the past 12 months, did you ever seriously think of attempting suicide?” and if so, “How many times did you attempt suicide?” dichotomized as no (0) or yes (1). The variables lifetime suicidality (ie, reporting ≥1 serious suicide ideation or attempt at 13, 15, or 17 years), lifetime suicidal ideation (ie, reporting ≥1 serious suicidal ideation at 13, 15, or 17 years but no attempt), and lifetime suicide attempt (ie, reporting ≥1 suicide attempt at 13, 15, or 17 years) were derived. In our sample, 172 participants (12.0%) reported suicidal ideation or suicide attempt, 90 (6.3%) reported serious suicidal ideation, and 82 (5.7%) reported suicide attempt (Table 2 ).
We jointly estimated developmental trajectories of irritability and depressive/anxious mood from 6 to 12 years of age using multitrajectory modeling.30 This new application of group-based trajectory modeling31 allows the joint modeling of the trajectories of multiple outcomes using semiparametric mixture models. The result of this analysis allowed the identification of different profiles defined by the joint development of irritability and depressive/anxious mood across childhood. Parameters were estimated using maximum likelihood estimation through a Newton-Raphson optimization algorithm and censored-normal models. The selection of the best model in terms of the number of groups and polynomial order of the trajectories was based on the Bayesian information criterion. Each participant was assigned to the group having the highest posterior probability.
We estimated the proportion of adolescents reporting suicidal ideation and/or suicide attempt for each profile. We computed 2 statistics to estimate the risk. First, we computed the number needed to be exposed (NNE), that is, the mean number of individuals needed to be observed in a given profile (compared with the profile exhibiting the lowest symptoms) to determine an additional suicidal outcome (NNE = 1/[nonexposed event rate – exposed event rate]).32,33 Second, we conducted logistic regressions to estimate the odds of showing suicidal ideation and/or a suicide attempt for each of the mood profiles compared with the profile exhibiting the lowest level of symptoms. We provided the following estimates: (1) unadjusted, (2) adjusted for sex, and (3) adjusted for sex and socioeconomic status. We tested a sex × profile interaction. However, because this analysis is underpowered, we additionally conducted exploratory analyses stratified by sex because of the important sexual differences in suicidality.15P values were calculated using the unpaired t test or the Mann-Whitney test for continuous variables and the χ2 test for categorical variables. Statistical analysis was performed using Stata software (version 14; StataCorp).
The study included 1430 children who were followed up to 17 years of age; 676 (47.3%) were boys, and 754 (52.7%) were girls. The best model identified the following 5 profiles (Figure): no irritability and low depressive/anxious mood (465 [32.5%]), (2) low irritability and low depressive/anxious mood (366 [25.6%]), (3) moderate irritability and low depressive/anxious mood (353 [24.7%]), (4) moderate declining irritability and high depressive/anxious mood (hereafter high depressive/anxious mood only) (94 [6.6%]), and (5) high irritability and depressive/anxious mood (152 [10.6%]). Profiles 1 and 2 were combined and used as the reference group (hereafter low irritability and low depressive/anxious mood) (831 [58.1%]). Individual and family characteristics of each profile are presented in eTable 2 in the Supplement. In brief, children in the high irritability and depressive/anxious mood profile were more likely to be male (relative risk ratio [RRR], 0.22; 95% CI, 0.16-0.32), to be raised by a depressive (RRR, 1.16; 95% CI, 1.02-1.33) and/or hostile-reactive (RRR, 1.20; 95% CI, 1.04-1.37) mother, and to be from a socioeconomically advantaged family (RRR, 0.64; 95% CI, 0.51-0.80).
Table 2 shows the proportion of adolescents presenting with suicidal ideation/suicide attempt in each profile. Suicidality varied from 10 of 94 adolescents (10.6%) with high depressive/anxious mood only to 25 of 152 (16.4%) with high irritability and high depressive/anxious mood. The NNE for this profile was 18, suggesting that if 18 children with high irritability and high depressive/anxious mood were sampled, we would observe 1 more case of suicidality compared with a group of 18 children sampled from the low irritability and depressive/anxious mood profile.
The sex × profile interaction was nonsignificant, although girls had higher rates of suicidality and were disproportionally represented in some profiles. Exploratory analyses by sex suggested that the suicidal risk in the high irritability and high depressive/anxious mood profile was clinically more important for girls (NNE = 5) than for boys (NNE = 20). Among children in the moderate irritability profile and low depressive/anxious mood profile, 46 of 353 (13.0%) reported suicidality (NNE = 48). Although the proportion of girls and boys reporting suicidality was different within this profile (26 of 144 girls [18.1%] and 20 of 209 boys [9.6%]), a similar number of girls and boys needed to be exposed to observe 1 additional case of suicidality (NNE = 28 and NNE = 22, respectively). In analyses based on the whole sample and stratified by sex, the proportion of suicidality in the high depressive/anxious mood only profile (10 of 94 [10.6%]) was similar to the proportion observed in the low irritability and depressive/anxious mood profile (91 of 831 [11.3%]; NNE>50).
Table 3 shows odds ratios (ORs) and 95% CIs for moderate irritability and low depressive/anxious mood, high depressive/anxious mood only, and high irritability and high depressive/anxious mood profiles compared with low irritability and low depressive/anxious mood profile. Consistently with NNE, we found that children with high irritability and high depressive/anxious mood had twice the odds of showing suicidality (OR, 2.22; 95% CI, 1.32-3.74) after adjusting for sex and socioeconomic status. In addition, children with moderate irritability and low depressive/anxious mood had 1.5 times the odds of showing suicidality (OR, 1.51; 95% CI, 1.02-2.25). The high depressive/anxious mood only profile had the same odds of suicidality (OR, 0.96; 95% CI, 0.46-1.97) as the reference group.
When the 2 outcomes of suicidal ideation and suicide attempt were examined separately in exploratory analyses (Table 4), findings were consistent with those obtained for combined outcomes. The high irritability and high depressive/anxious mood profile was associated with suicidal ideation (OR, 2.07; 95% CI, 1.05-4.10) and suicide attempt (OR, 2.03; 95% CI, 1.00-4.15). For the moderate irritability and low depressive/anxious mood profile, the associations with suicidal ideation (OR, 1.44; 95% CI, 0.85-2.43) and suicide attempt (OR, 1.53; 95% CI, 0.88-2.64) were comparable to those obtained for suicidality.
The risk of suicidality was higher in the high irritability and high depressive/anxious mood profile (OR, 2.28; 95% CI, 1.02-5.15) compared with the high depressive/anxious mood only profile. This risk was also higher for suicide attempt (OR, 3.11; 95% CI, 0.93-10.38) but not for suicidal ideation, for which no difference between the high irritability and high depressive/anxious mood and the high depressive/anxious mood only profiles was found (OR, 1.58; 95% CI, 0.57-4.36).
Consistent with the NNE analyses, the OR analyses stratified by sex showed different patterns for boys and girls. The moderate irritability and low depressive/anxious mood profile was more strongly associated with suicidality in boys (OR, 2.05; 95% CI, 1.02-4.10) than girls (OR, 1.30; 95% CI, 0.80-2.13), whereas the high irritability and depressive/anxious mood profile was more strongly associated with suicidality in girls (OR, 3.07; 95% CI, 1.54-6.12) than boys (OR, 2.13; 95% CI, 0.95-4.78) (Table 3). Results were robust to missing data and attrition in sensitivity analyses (eTable 3 in the Supplement).
This population-based study is the first, to our knowledge, to examine the joint association of irritability and depressive/anxious mood assessed repeatedly across childhood with suicidality during adolescence. We revealed that children presenting with high irritability and high depressive/anxious mood were 2 times more likely to report serious suicidal ideation and/or to attempt suicide during adolescence compared with those presenting with neither irritability nor depressive/anxious mood. Consistent results were obtained for suicidal ideation and suicide attempt analyzed as separate outcomes. Exploratory analyses by sex indicated that this association was more important for girls than boys, as indicated by the need to prevent the exposure among 5 girls to avoid 1 case of suicidality (NNE = 5 vs NNE = 20 in boys). However, the sex × profile interaction was nonsignificant, probably owing to low statistical power to detect this interaction. The magnitude of these associations was in line with those reported in previous population-based studies on the association of internalizing behaviors, previous suicide attempts, and childhood adversity with suicide.2,4,13 Children presenting with moderate irritability and low depressive/anxious mood had slightly higher suicidal risk during adolescence than those with low symptoms. The corresponding NNE of 48 suggests that the clinical importance of this association is modest. Children presenting with depressive/anxious mood (but no irritability) showed a similar risk of later suicidality compared with those presenting without depressive/anxious mood or irritability. In addition, children showing irritability and depressive/anxious mood had 2 times greater risk of showing suicidality during their adolescence compared with children showing depressive/anxious mood only.
This study used an innovative, person-centered approach that described the joint course of irritability and depressive/anxious mood from 6 to 12 years of age. Previous studies using trajectory modeling only investigated the development of depression/anxiety or irritability34-37 or described depression phenotypes (depression only vs irritable depression) based on the cross-sectional presence of 1 or both symptoms.6,38 Instead, our multitrajectory model captured the joint development of irritability and depressive/anxious mood throughout childhood. Such approaches are substantially different because profiles identified herein account for the correlation of irritability and depressive/anxious mood within the same participant and over time. We found rank stability in the developmental profiles of irritability and depressive/anxious mood; that is, children with the highest level of symptoms at 6 years of age also exhibited the highest levels of symptoms at 12 years of age. To date, stability of these phenotypes has only been shown in 1 previous study among clinically depressed individuals.6 Therefore, our findings widen previous evidence by showing that irritability and depressive/anxious mood profiles are stable at the population level (ie, considering subsyndromic level of irritability and depressive/anxious mood) and detectable during middle childhood.
Another contribution of this study is the identification of a group of children (approximately 25%) with moderate and stable levels of irritability (but low depressive/anxious mood) who are at elevated risk of suicidality during adolescence. Although previous studies reported associations between irritability during childhood and adolescence and later depression, anxiety,6,7,39-42 and suicidality,12,13 we found that even moderate levels of irritability may contribute to suicidal risk. The absence of depressive or anxious symptoms in those children may result in low levels of treatment seeking. Such results indicate that children presenting with only irritability symptoms may benefit from an assessment for suicidal behaviors. The clinical relevance of this observation should take into account the high NNE (NNE = 22 for boys and NNE = 28 for girls). Thus, although screening for suicide may be indicated in the presence of irritability even in the absence of depression or anxiety, additional studies are needed to quantify this finding in clinical samples before clinical recommendation can be formulated.
The comparison of our findings with those of previous studies is limited by the lack of longitudinal population-based studies examining associations of childhood irritability and depression/anxiety with suicidality. However, our findings are consistent with those from the STAR*D (Sequenced Treatment Alternatives to Relieve Depression) study,43 the National Comorbidity Survey Replication,38 and the National Institute of Mental Health Collaborative Depression Study,44 which reported more severe symptoms (eg, comorbid disruptive disorders, poorer impulse control) along with a history of suicide attempt43 and suicidal ideation38,43 among adult participants with irritable depression compared with those with nonirritable depression. Our findings are also in accordance with previous studies showing that individuals with an increased suicidal risk experience depressive and aggressive symptoms3 and that childhood symptoms related to externalizing problems are more predictive of suicidal behaviors than childhood internalizing symptoms.5,45
This study was conducted using a large representative cohort of children followed up from 5 months to 17 years of age, as well as innovative joint trajectory modeling techniques and behavioral assessments performed by 5 different teachers interacting daily with children and observing behaviors in a social setting (school). Despite these strengths, this study has limitations. First, as in other longitudinal population-based studies,35,39,41,46 the scales used to assess childhood symptoms are not clinical instruments assessing specific psychiatric diagnoses. They instead assess behaviors and emotions along a continuum, with satisfactory psychometric proprieties and good construct validity.7 However, because our assessment of childhood symptoms is based on teachers only, depressive/anxious mood might have been underrated compared with irritability, because internalizing symptoms may be more difficult to observe in a school setting than externalized symptoms.47 Second, owing to sample attrition (eg, emigration, loss to follow-up, and refusal), our analyses were conducted on 1430 of 2120 individuals (67.5%) of the initial sample (without sampling weights). Included and excluded participants were broadly comparable, except for sex, socioeconomic status, and IQ. To minimize attrition biases, analyses were repeated using sample weights accounting for the probability of being missing at follow-up. Results with and without weights were similar (eTable 3 in the Supplement), suggesting that bias due to attrition is rather minimal. Third, because we assessed past 12-month suicidality biyearly, participants reporting suicidal ideation or suicide attempt between data collections may have been incorrectly classified in the not-at-risk group. This potential misclassification may underestimate the size of the associations.
In this population study, we described, to our knowledge for the first time, the joint development of irritability and depressive/anxious mood, showing stable profiles across middle-to-late childhood. Manifestations of irritability during childhood are associated with a significant risk for suicidal behaviors during adolescence. This risk was especially high when high levels of irritability were accompanied by high levels of depressive/anxious mood and particularly for girls. The value of assessing irritability as part of the suicide risk assessment should be investigated in future population-based and clinical studies. Because our findings are exploratory, additional studies are needed to test the putatively causal role of irritability on suicidality. For instance, randomized intervention aiming at reducing childhood irritability, especially when accompanied by high depressive/anxious mood, should examine the effect of childhood irritability on future suicidal symptoms.
Accepted for Publication: January 21, 2018.
Corresponding Author: Marie-Claude Geoffroy, PhD, McGill Group for Suicide Studies, Douglas Mental Health University Institute, Department of Psychiatry, McGill University, Frank B. Common Pavilion, 6875 LaSalle Blvd, Montreal, QC H4H 1R3, Canada (email@example.com).
Published Online: March 28, 2018. doi:10.1001/jamapsychiatry.2018.0174
Author Contributions: Drs Côté and Geoffroy shared senior authorship. Dr Orri had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Orri, Côté, Geoffroy.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Orri, Galera, Tremblay, Côté, Geoffroy.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Orri, Galera, Geoffroy.
Obtained funding: Boivin, Tremblay, Côté.
Administrative, technical, or material support: Boivin, Tremblay, Côté, Geoffroy.
Study supervision: Turecki, Tremblay, Côté, Geoffroy.
Conflict of Interest Disclosures: Dr Orri reports receiving a postdoctoral research fellowship from Canadian Institutes of Health Research (CIHR). Dr Turecki reports holding a Canada Research Chair (Tier 1) funded by CIHR and a National Alliance for Research on Schizophrenia and Depression Distinguished Investigator Award. Dr Boivin reports holding a Canada Research Chair in Child Development funded by CIHR. Drs Côté and Geoffroy report being research fellows of the Fonds de Recherche du Québec. No other disclosures were reported.
Funding/Support: The larger Québec Longitudinal Study of Child Development was supported by the Québec Government’s Ministry of Health, Ministry of Education, and Ministry of Family Affairs, The Lucie and André Chagnon Foundation, the Robert-Sauvé Research Institute of Health and Security at Work, and the Québec Statistics Institute. Additional funding was received by the Fonds de Recherche du Québec–Santé, the Fonds de Recherche du Québec–Societé et Culture, Canada’s Social Science and Humanities Research Council, the CIHR, and the St-Justine Research Centre.
Role of the Funder/Sponsor: Québec Statistics Institute collected data. The sponsors had no role in the design and conduct of the study; management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: