[Skip to Content]
[Skip to Content Landing]
Original Investigation
May 13, 2019

Comparison of Machine Learning Optimal Classification Trees With the Pediatric Emergency Care Applied Research Network Head Trauma Decision Rules

Author Affiliations
  • 1Operations Research Center, Massachusetts Institute of Technology, Cambridge
  • 2Departments of Emergency Medicine and Pediatrics, Alpert Medical School, Brown University, Providence, Rhode Island
  • 3Center for Evidence Synthesis in Health, Brown University School of Public Health, Providence, Rhode Island
JAMA Pediatr. 2019;173(7):648-656. doi:10.1001/jamapediatrics.2019.1068
Key Points

Question  Can machine learning improve the Pediatric Emergency Care Applied Research Network (PECARN) rules’ predictive accuracy to identify children at very low, intermediate, and high risk of clinically important traumatic brain injury?

Findings  In this cohort study of 42 412 children with head trauma, reanalysis of data from the PECARN group empirically suggests that novel machine-learning (optimal classification tree)–based rules perform as well as or better than the PECARN rules in identifying more children at very low risk of clinically important traumatic brain injury without missing more patients with clinically important traumatic brain injury.

Meaning  If implemented in the electronic health record, the new rules may help reduce the number of unnecessary computed tomographic imaging scans, without missing more patients with clinically important traumatic brain injury than the PECARN rules.

Abstract

Importance  Computed tomographic (CT) scanning is the standard for the rapid diagnosis of intracranial injury, but it is costly and exposes patients to ionizing radiation. The Pediatric Emergency Care Applied Research Network (PECARN) rules for identifying children with minor head trauma who are at very low risk of clinically important traumatic brain injury (ciTBI) are widely used to triage CT imaging.

Objective  To examine whether optimal classification trees (OCTs), which are novel machine-learning classifiers, improve on PECARN rules’ predictive accuracy.

Design, Setting, and Participants  A secondary analysis of prospective, publicly available data on emergency department visits for head trauma used by the PECARN group to develop their tool was conducted to derive OCT-based prediction rules for ciTBI in a development cohort and compare their predictive performance vs the PECARN rules in a validation cohort among children who were younger than 2 years and 2 years or older. Data on 42 412 children with head trauma and without severely altered mental status who were examined between June 1, 2004, and September 30, 2006, were gathered from 25 emergency departments in North America participating in PECARN. Data analysis was conducted from September 15, 2016, to December 18, 2018.

Main Outcomes and Measures  The outcome was ciTBI, with predictive performance measured by estimating the sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio for the OCT and the PECARN rules. The OCT and PECARN rules’ performance was compared by estimating ratios for each measure.

Results  Of the 42 412 children (15 996 [37.7%] girls) included in the analysis, 10 718 were younger than 2 years (25.3%; mean [SD] age, 11.6 [0.6] months) and 31 694 were 2 years or older (74.7%; age, 9.1 [4.9] years). Compared with PECARN rules, OCTs misclassified 0 vs 1 child with ciTBI in the younger and 10 vs 9 children with ciTBI in the older cohort, and correctly identified more children with very low risk of ciTBI in the younger (7605 vs 5701) and older (20 594 vs 18 134) cohorts. In the validation cohorts, compared with the PECARN rules, the OCTs had statistically significantly better specificity (in the younger cohort: 69.3%; 95% CI, 67.4%-71.2% vs 52.8%; 95% CI, 50.8%-54.9%; in the older cohort: 65.6%; 95% CI, 64.5%-66.8% vs 57.6%; 95% CI, 56.4%-58.8%), positive predictive value (odds ratios, 1.54; 95% CI, 1.36-1.74 and 1.23; 95% CI, 1.17-1.30, in younger and older children, respectively), and positive likelihood ratio (risk ratios, 1.54; 95% CI, 1.36-1.74 and 1.23; 95% CI, 1.17-1.30, in younger and older children, respectively). There were no statistically significant differences in the sensitivity, negative predictive value, and negative likelihood ratio between the 2 sets of rules.

Conclusions and Relevance  If implemented, OCTs may help reduce the number of unnecessary CT scans, without missing more patients with ciTBI than the PECARN rules.

×