Clinical Risk Score for Persistent Postconcussion Symptoms Among Children With Acute Concussion in the ED

Importance  Approximately one-third of children experiencing acute concussion experience ongoing somatic, cognitive, and psychological or behavioral symptoms, referred to as persistent postconcussion symptoms (PPCS). However, validated and pragmatic tools enabling clinicians to identify patients at risk for PPCS do not exist.

Objective  To derive and validate a clinical risk score for PPCS among children presenting to the emergency department.

Design, Setting, and Participants  Prospective, multicenter cohort study (Predicting and Preventing Postconcussive Problems in Pediatrics [5P]) enrolled young patients (aged 5-<18 years) who presented within 48 hours of an acute head injury at 1 of 9 pediatric emergency departments within the Pediatric Emergency Research Canada (PERC) network from August 2013 through September 2014 (derivation cohort) and from October 2014 through June 2015 (validation cohort). Participants completed follow-up 28 days after the injury.

Exposures  All eligible patients had concussions consistent with the Zurich consensus diagnostic criteria.

Main Outcomes and Measures  The primary outcome was PPCS risk score at 28 days, which was defined as 3 or more new or worsening symptoms using the patient-reported Postconcussion Symptom Inventory compared with recalled state of being prior to the injury.

Results  In total, 3063 patients (median age, 12.0 years [interquartile range, 9.2-14.6 years]; 1205 [39.3%] girls) were enrolled (n = 2006 in the derivation cohort; n = 1057 in the validation cohort) and 2584 of whom (n = 1701 [85%] in the derivation cohort; n = 883 [84%] in the validation cohort) completed follow-up at 28 days after the injury. Persistent postconcussion symptoms were present in 801 patients (31.0%) (n = 510 [30.0%] in the derivation cohort and n = 291 [33.0%] in the validation cohort). The 12-point PPCS risk score model for the derivation cohort included the variables of female sex, age of 13 years or older, physician-diagnosed migraine history, prior concussion with symptoms lasting longer than 1 week, headache, sensitivity to noise, fatigue, answering questions slowly, and 4 or more errors on the Balance Error Scoring System tandem stance. The area under the curve was 0.71 (95% CI, 0.69-0.74) for the derivation cohort and 0.68 (95% CI, 0.65-0.72) for the validation cohort.

Conclusions and Relevance  A clinical risk score developed among children presenting to the emergency department with concussion and head injury within the previous 48 hours had modest discrimination to stratify PPCS risk at 28 days. Before this score is adopted in clinical practice, further research is needed for external validation, assessment of accuracy in an office setting, and determination of clinical utility.

Concussion is a serious public health epidemic.^1,2 Rates have doubled during the last decade³ with an estimated 750 000 pediatric acute concussion visits to emergency departments (EDs) occurring annually in the United States.^1,4 Although many children experience symptom resolution within 2 weeks, approximately 33% experience ongoing somatic, cognitive, psychological, behavioral symptoms, or a combination of these symptoms.^5,6 Symptoms persisting beyond 28 days are referred to as persistent postconcussion symptoms (PPCS)⁷ and can have serious adverse effects, resulting in school absenteeism, impaired academic performance, depressed mood, loss of social activities, and lower quality of life.⁸

Validated and pragmatic tools to identify children at high risk of developing PPCS do not exist.⁹ Adolescent age, female sex, and physician-diagnosed history of migraine have been associated with PPCS in children^5,10; however, prior studies have had significant limitations. Retrospective studies are limited by poor data quality, missing data, minimal use of validated symptom scoring scales, and lack of standardized acute evaluation.^5,6,9-11

Additional limitations include small sample sizes,^6,12 recruitment beyond the acute injury period,^13,14 and inconsistent definition and measurement of PPCS.⁹ Studies including elite adolescent athletes and adults dominate the literature, limiting applicability to subsets of children. The Institute of Medicine and the National Research Council emphasized the need for a large, prospective study to quantify PPCS risk in children and youth and to establish “objective, sensitive, and specific metrics and markers of concussion diagnosis, prognosis, and recovery in youth.”¹⁵

The Predicting and Preventing Postconcussive Problems in Pediatrics (5P) study was designed to derive and validate a clinical risk score to stratify PPCS risk occurring after acute concussion in children and youth using readily available clinical features.

The 5P was a prospective, multicenter cohort study.¹⁶ Participants were recruited from 9 pediatric emergency departments within the Pediatric Emergency Research Canada (PERC) network. Enrollment occurred from August 2013 through September 2014 (derivation cohort) and from October 2014 through June 2015 (validation cohort) (Figure 1). The study complied with the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement¹⁷ and was approved by the ethics committees of the PERC participating institutions. Written consent and assent was obtained from all participants and their parents or guardians as appropriate. The trial protocol appears in Supplement 1.

Quiz Ref IDEligible patients were aged 5 years through younger than 18 years, presented to a participating ED with a head injury within the preceding 48 hours, and met concussion diagnostic criteria consistent with the fourth Zurich consensus statement.¹⁸ Concussion was defined as a complex pathophysiological process caused by a direct blow to the head, face, neck, or elsewhere on the body with an impulsive force transmitted to the head (which may or may not have involved loss of consciousness), resulting in a brain injury with 1 or more symptoms in 1 or more of the following clinical domains: somatic, cognitive, emotional or behavioral, or sleep (eTable 1 in Supplement 2).¹⁸ Patients were excluded for (1) a Glasgow Coma Scale score of 13 or less, (2) a structural abnormality on neuroimaging (if performed), (3) a neurosurgical intervention, (4) intubation or intensive care unit admission, (5) multisystem injury requiring hospitalization, (6) procedural sedation, (7) severe preexisting neurological developmental delay resulting in communication difficulties, (8) intoxication, (9) absence of trauma as primary event, (10) previously enrolled in this same study, (11) insurmountable language barrier, or (12) the inability to follow-up by telephone or email.

Procedures were identical for the derivation and validation phases of the study. Prior to study initiation, participating site ED physicians and research staff were trained on data collection methods using standardized training sessions during site visits by the principal investigator and the national coordinator. Trained research assistants completed standardized assessments of all patients as described in the published protocol.¹⁶ Data were collected and managed using research electronic data capture.¹⁹

Patients and parents provided information on demographics, history, and injury characteristics using the Acute Concussion Evaluation inventory.²⁰ Patients and parents quantified state of being prior to the injury and current symptoms using the Postconcussion Symptom Inventory (eFigure 1 in Supplement 2).^21,22 Cognition, physical examination, and balance were assessed using the third edition of the Child-Sport Concussion Assessment Tool.²³ At enrollment, concussion; developmental, neurological, and psychiatric history; therapies received during the ED visit; discharge instructions; and treating physician prognostication of PPCS risk with predicted symptom duration were prospectively collected. Blinded, independent second raters in a convenience subset of 10% of patients duplicated data collection to assess reliability.²⁴

Participants completed electronic follow-up surveys at 7, 14, and 28 days after the injury, including the patient-reported Postconcussion Symptom Inventory¹⁶; electronic capture was not expected to affect reporting.²⁵ Patients opting for web-based follow-up received email reminders 24 hours following each survey deadline; research assistants telephoned nonresponders and those opting for telephone follow-up up to 5 times to complete measures orally.

Quiz Ref IDThe primary outcome measure, PPCS, was defined in keeping with the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) definition of postconcussion syndrome, which requires persistence beyond 4 weeks of at least 3 symptoms compared with state of being prior to the injury.²⁶ In the study, an individual symptom was defined as a positive difference between the patient-reported current minus the perceived preinjury symptom rating; both were completed 28 days after the injury.²⁶

Physician performance on prediction of PPCS was measured and compared with PPCS risk score performance. A risk assessment tool should outperform clinician accuracy to be relevant.²⁴ Treating physicians completed standardized surveys, which included the following question: “How likely is this patient to develop persistent symptoms beyond 1 month?” (response options: 0%-10%, 11%-20%, 21%-30%, 31%-50%, 51%-70%, 71%-90%, and 91%-100%).

Forty-six variables were selected a priori for assessment based on a national planning meeting, recent systematic reviews, previous studies, and clinical experience.¹⁶ Factors occurring after the ED assessment (eg, compliance with recommendations regarding rest or exertion) were omitted because this would reduce the face validity of a predictive score. At a subsequent consensus meeting, the total parent-reported Postconcussion Symptom Inventory score obtained during the ED visit was separated into its 20 individual components and each was analyzed as independent candidate variable scores.

Based on the pilot study, it was estimated that 25% of participants would experience PPCS when applying ICD-10 criteria.⁹ Including 10 events per each candidate predictor variable,¹⁷ 345 cases of PPCS would be required after screening for acceptable interrater agreement, assuming a dropout rate of 25% for the a priori selected variables.²⁷ To obtain 345 cases of PPCS, 1380 patients with new concussion had to be enrolled. Factoring a loss to follow-up rate of 23% based on pilot data,²⁸ the final derivation cohort sample size required was 1792 patients. To capture potential seasonal variability in PPCS rates, a 1-year enrollment period was required.

Based on a survey of PERC members,²⁹ 90% sensitivity was targeted to predict PPCS. To validate PPCS risk score with clinically acceptable confidence bounds (95% CI, 85%-95%), 200 patients with PPCS were required in a separate validation cohort. Assuming a rate of PPCS of 25%, 800 patients with new concussion had to be enrolled. With a loss to follow-up rate of 15% based on the derivation phase, the required validation sample size was 920 patients.

Descriptive statistics were used to summarize baseline characteristics. The differences between children with and without PPCS were assessed using the χ² test or the Fisher exact test as appropriate. Emphasizing clinical relevance and face validity, predictors with continuous outcomes were categorized or dichotomized. Interrater agreement was assessed for all candidate variables using the κ statistic; those variables with acceptable reliability (κ ≥0.6) remained eligible for the multivariable analysis.²⁴ Missing data were handled via list-wise deletion.

All reliable variables associated with PPCS (P < .20) were entered into a multivariable model using forward stepwise binary logistic regression analysis (P = .05 included but P = .10 removed). Variables in the regression model were assessed for co-linearity using the variance inflation factor.

The risk score was evaluated as a diagnostic test calculating sensitivity, specificity, and positive and negative likelihood ratios. The final model was validated internally using bootstrap resampling.³⁰ A risk score for the final multivariable model was derived using the model by Sullivan et al,³¹ in which points were assigned to each predictor variable with point totals corresponding to risk estimate. High- and low-risk cut points for the PPCS risk score were determined by consensus at a team meeting following the derivation phase.

Temporal validation was performed using a separate independent cohort in the same institutions from which the derivation data were collected (ie, no data from the validation cohort were used to derive the risk score, and no data from the derivation cohort were used to validate). Validation performance was evaluated with correlated receiver operating characteristic analysis and test characteristics. Score calibration was assessed using the Hosmer-Lemeshow test and graphically using a calibration plot.³²

Physicians’ prediction was analyzed by logistic regression to predict PPCS. The accuracy of the validated risk stratification score was compared with that of physicians’ predictions using the receiver operating characteristic analysis by DeLong et al.³³

Statistical analyses were performed using SPSS Statistics versions 21 and 23 (SPSS Inc) and R version 3.0.2 (R Foundation for Statistical Computing). Two-sided P values of less than .05 were considered statistically significant.

There was complete assessment of the primary outcome of PPCS for 1701 of 2006 participants (84.8%) in the derivation cohort and 883 of 1057 participants (83.5%) in the validation cohort (Figure 1). The median age for both cohorts was 12.0 years (interquartile range, 9.2-14.6 years). The baseline patient characteristics appear in Table 1. Details about the injuries sustained and the types of medications used appear in Table 2.

The characteristics of patients with missing primary outcome data appear in eTable 2 in Supplement 2. The type of treatments provided in the ED appear in eTable 3.

There were 510 participants (30.0%) who met the criteria of having PPCS in the derivation cohort. Forty-seven potential predictor variables were associated with PPCS in the bivariable analysis (Table 3, Table 4, and eTable 4 in Supplement 2). There were 294 patients (15%) who had blinded duplicate assessments (research assistant only: n = 145 [7%]; physician only: n = 92 [5%]; both research assistant and physician: n = 57 [3%]). Excellent overall interrater agreement was demonstrated (median κ = 0.97 [κ interquartile range, 0.75-0.99]).

The final multivariable model included (1) age, (2) sex, (3) prior concussion with symptom duration of longer than 1 week, (4) physician-diagnosed migraine history, (5) headache, (6) sensitivity to noise, (7) fatigue, (8) answering questions slowly, and (9) abnormal tandem stance (Table 5). The area under the curve (AUC) was 0.71 (95% CI, 0.69 to 0.74; eFigure 2 in Supplement 2). All variables had a variance inflation factor of less than 2.5, indicating a lack of multicollinearity between predictors.

Bootstrapping analysis (ie, resampling the model 1000 times) revealed a mean overoptimism value of 0.01 (95% CI, –0.02 to 0.03) and a corrected AUC of 0.70. In the final derivation model, 94.3% (1604/1701) of the participants with primary outcome data had complete data on all 9 predictor variables included in the multivariable model. The PPCS risk score derived from the multivariable model (score range, 0 to 12) linearly corresponded to risk estimate. Three cutoff points were selected to stratify PPCS risk (low risk: ≤3 points; medium risk: 4-8 points; and high risk: ≥9 points; Table 6).

There were 291 patients (33.0%) who met the criteria of having PPCS. The AUC for the model was 0.68 (95% CI, 0.65-0.72). For patients not at low risk (≤3 points), the sensitivity was 93.5% (95% CI, 90.0%-95.8%), specificity was 18.1% (95% CI, 15.2%-21.4%), and the negative likelihood ratio was 0.36 (95% CI, 0.23-0.58); the negative predictive value was 84.9% (95% CI, 77.6%-90.1%) and the positive predictive value was 35.9% (95% CI, 32.6%-39.5%).

For high-risk patients (≥9 points), the specificity was 93.4% (95% CI, 91.1%-95.1%), sensitivity was 20.3% (95% CI, 16.1%-25.3%), and the positive likelihood ratio was 3.00 (95% CI, 2.06-4.37); the negative predictive value was 70.4% (95% CI, 67.1%-73.5%) and the positive predictive value was 59.6% (95% CI, 50.3%-69.3%).

Validation test characteristics for all point values appear in eTable 5 in Supplement 2. The posttest probabilities for the 3 risk strata (low, medium, and high) appear in eTable 6. The Hosmer-Lemeshow test indicated goodness of fit for the model (P = .50). The calibration plot of observed frequency compared with the predicted probability of PPCS showed an intercept of 0.07 and a slope of 0.90, suggesting acceptable calibration (eFigure 3).

The data for physicians’ prediction at time of emergency department visit for probability of PPCS at 28 days appear in Table 7. A model with 9 variables from the risk score as well as physicians’ prediction had an AUC of 0.68 (95% CI, 0.63-0.73), whereas physicians’ prediction alone had an AUC of 0.55 (95% CI, 0.50-0.59; Figure 2). Thus, in the validation cohort, the addition of the derived prediction model to the physicians’ judgment alone resulted in an incremental C statistic improvement of 0.13 (95% CI, 0.07-0.20; P < .001).

Quiz Ref IDA PPCS clinical risk score derived in a large, diverse cohort of children presenting to the ED with concussion within 48 hours of head injury was significantly better than physician judgment in predicting future PPCS, although the discrimination of the risk score model was modest (AUC of 0.71). The PPCS risk score incorporates 9 clinical variables containing information from demographics, history, initial symptoms, cognitive complaints, and physical examination. Evaluation in an independent validation cohort demonstrated good test characteristic retention.

Even though prior research found an association between prolonged recovery and total postinjury symptom burden score (22 items using a 7-point scale),³⁴ such a complex scale is a barrier to adoption by acute care clinicians.²⁴ We instead analyzed individual symptoms, resulting in a final model that includes 4 early symptoms and signs in the PPCS risk score. Several final model variables have been associated with PPCS, including headache, answering questions slowly, and sensitivity to noise.^9,11,35 Female sex and older age are associated with prolonged recovery in children and adults.^9,35

Although the clinical utility of the PPCS risk score will need to be assessed in an externally validated implementation study prior to adoption into routine practice, the risk stratification score has the potential to individualize concussion care through optimal symptom management and appropriate follow-up.^9,16 Therefore, future research needs to determine if the moderate test characteristics of the PPCS risk score allow for clinicians to confidently provide reassurance, alter management plans, or both. Future clinical benefits might include identifying high-risk individuals for further screening, prioritization for specialized concussion evaluations, and initiation of emerging treatments to prevent PPCS.³⁶

Strengths of this study include standardized assessment of predictor and outcome variables using validated scales in a cohort with acute concussions (exclusion of presentations >48 hours after injury). Moreover, a large, cross-country, multisite validation cohort confirmed good predictive performance of the risk score model used in the derivation cohort. Inclusion of participants from a wide age range and spectrum of injuries and those with behavioral, learning, and psychological problems enhances generalizability.

There were minor baseline differences between the derivation and validation cohorts (eg, age group, learning disabilities, depression, and type of sports played). These differences increase the generalizability of our model. Therefore, we believe our findings are applicable to the assessment of children seeking acute care following a concussive event.

Quiz Ref IDDespite collective agreement across guidelines that initial management should include physical and cognitive rest followed by graduated return to normal activities, wide practice variation exists.^29,37 The lack of evidence for initial management of concussion (including protocols regarding timing of returning to activity) is a crucial issue in the field of pediatric concussion, and results of this study should be applied in urgently needed future comparative clinical trials.³⁸ The PPCS risk score will benefit concussion care research by providing a tool for the targeted selection of patients in greatest need of intervention.⁹ Selection of pediatric patients at high risk for PPCS may optimize research recruitment by offering more efficient and cost-effective enrollment strategies, or may be used to stratify participants in clinical trials according to PPCS risk.

Because no objective criterion standards for concussion or PPCS diagnoses exist (ie, no readily available biomarkers or imaging modalities),³⁹ the PPCS risk prediction score may be less precise than prediction studies for other diseases. Nonetheless, the outcome measures used in this study generated the best-available evidence through the use of validated tools. In addition, the definitions we used aligned with current concussion guidelines and ICD-10 standards.^18,26,40 The PPCS risk model demonstrated only modest ability to discriminate patients who will and will not have PPCS, resulting in erroneous categorization. Test characteristics could be further refined through inclusion of biomarkers, genetic data, or advanced neuroimaging techniques. The pragmatic, generalizable PPCS risk model does not require expensive and painful testing, and could therefore be used to triage initial management.

There are several limitations to this study. Selection bias may limit generalizability. The patients with concussion in our derivation and validation cohorts may have higher PPCS rates and different risk characteristics than those patients with less severe injuries who may have not sought pediatric ED care. Nonetheless, the study included a heterogeneous population recruited through the use of a large number of study sites with great geographical variation.

Even with inclusion of concussions sustained by a variety of mechanisms, some of which may have involved higher forces than those generally seen in sports (eg, motor vehicle collisions), we observed similar rates of PPCS and loss of consciousness as the rates in the sideline assessment and outpatient literature.^41-43 Because it is possible that the PPCS risk score may not perform as well in different populations, validation should occur in other clinical settings, such as non–tertiary care EDs, primary care, and sideline assessments.

In addition, the sample was limited to participants without observable lesions on imaging; therefore, it may not be representative of a more complicated spectrum of mild traumatic brain injury. However, because the presence of an intracranial lesion on standard imaging no longer meets the current concussion definition,^18,40 outcome prediction in this population is beyond the study’s scope. Future research should seek to determine the performance of this PPCS risk assessment tool in a nuanced population of patients with mild traumatic brain injury.

Although other injuries might have contributed to ongoing symptoms, patients with multisystem injuries requiring hospitalization were excluded. An additional limitation is that measures of socioeconomic status or family functioning were not included, which are variables that could have an effect on follow-up care and resources thus affecting symptom burden with alteration of PPCS risk.

Even though the study had missing data, it was limited in scope. Baseline characteristics were similar between those with and without the primary outcome in both the derivation and validation cohorts, and most enrolled participants had no missing data.

Given the wide age range, potential variation exists in respondent type (parent vs patient); however the Postconcussion Symptom Inventory has good parent and self-report correlation.²² In addition, without a control group, we cannot definitively attribute ongoing symptoms to the acute injury. However, literature examining PPCS incidence in patients with head injury compared with controls (eg, orthopedic injury) has yielded similar rates in the head injury group as in our cohort.⁹

Quiz Ref IDA clinical risk score developed among children presenting to the ED with concussion and head injury within the previous 48 hours had modest discrimination to stratify PPCS risk at 28 days. Before this score is adopted in clinical practice, further research is needed for external validation, assessment of accuracy in an office setting, and determination of clinical utility.

Corresponding Author: Roger Zemek, MD, Children’s Hospital of Eastern Ontario Research Institute, 401 Smyth Rd, Room R-139, Ottawa, ON K1H 8L1, Canada (rzemek@cheo.on.ca).

Correction: This article was corrected online June 21, 2016, to fix incorrect wording in the Results section of the text and data transcription errors in Tables 3 and 4 that occurred when these Tables were split.

Author Contributions: Drs Zemek and Barrowman had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Zemek, Barrowman, Freedman, Gravel, Gagnon, McGahern, Sangha, Boutis, Craig, Burns, Farion, Mikrogianakis, Barlow, Giola, Meehan, Kamil, Hoshizaki, Anderson, Klassen, Richer, DeMatteo, Osmond.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Zemek, Barrowman, Aglipay, Burns, Farion, Mikrogianakis, Grool, Brooks, Yeates, Keightley, Richer, Osmond.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Zemek, Barrowman, Aglipay, Yeates, Keightley.

Obtained funding: Zemek, Freedman, Gravel, Gagnon, Boutis, Farion, Barlow, Richer, Osmond.

Administrative, technical, or material support: Zemek, Gravel, Gagnon, McGahern, Sangha, Boutis, Craig, Farion, Mikrogianakis, Barlow, Giola, Meehan, Beauchamp, Kamil, Grool, Anderson, Brooks, Vassilyadi.

Study supervision: Zemek, Freedman, Gagnon, McGahern, Sangha, Boutis, Craig, Dubrovsky, Hoshizaki, Anderson, Klassen, Keightley, Osmond.

Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Gioia reported being the author of the Postconcussion Symptom Inventory (PCSI); the PCSI is freely available and he receives no financial benefit for its use. Dr Meehan reported receiving royalties from ABC-Clio publishing for Kids, Sports, and Concussion: A Guide for Coaches and Parents and from Wolters-Kluwer for being an author for UpToDate; being under contract with ABC-Clio publishing and Springer International for future books; and receiving research funding from the National Football League Players Association and the National Hockey League Alumni Association through the Corey C. Griffin Pro-Am Tournament. No other disclosures were reported.

Funding/Support: This study was supported by operating grant 126197 from the Canadian Institutes of Health Research; grant TM1:#127047 from the Canadian Institutes of Health Research–Ontario Neurotrauma Foundation Mild Traumatic Brain Injury Team; and planning grant MRP: #119829 from the Canadian Institutes of Health Research. Dr Freedman is supported by the Alberta Children’s Hospital Foundation Professorship in Child Health and Wellness.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Group Information: Pediatric Emergency Research Canada Concussion (PERC) Team members: Jialing Xie, MD, and Jennifer Chatfield, MN (Alberta Children’s Hospital, Calgary, Canada); Nadia Dow, BA (Stollery Children’s Hospital, Edmonton, Alberta, Canada); Rena Papadimitropoulos, MSc, and Tracey Levesque, RN (Winnipeg Children’s Hospital, Winnipeg, Manitoba, Canada); Cindy Langford, RN (Children’s Hospital of Western Ontario, London, Canada); Tinh Trung Tran, NRP (Hospital for Sick Children, Toronto, Ontario, Canada); Candice McGahern, BA (Children’s Hospital of Eastern Ontario, Ottawa, Canada); Vanessa DiGirolamo, RT, and Joanna Mazza, BSc (Montreal Children’s Hospital, Montreal, Quebec, Canada); Maryse Lagacé, RN, and Ramona Cook, RN (Centre Hospitalier Universitaire Sainte Justine, Montreal, Quebec, Canada); and Eleanor Fitzpatrick, MN, Jessica MacIntyre, MSc, and Jill Moore, MSc (IWK Children’s Hospital, Halifax, Nova Scotia, Canada).

Previous Presentation: The derivation cohort results were presented at the Pediatric Academic Societies Annual Meeting; April 26, 2015; San Diego, California.

Additional Contributions: We thank the parents and children who participated in this study. We appreciate the collaboration of the treating physicians and the research assistants at each site. We are grateful to Pediatric Emergency Research Canada Network for making this study possible.