AHT indicates abusive head trauma; BIBIS, Biomarkers for Infant Brain Injury Score; and ICH, intracranial hemorrhage.
The area under the curve was 0.906 (95% CI, 0.893-0.919). Sensitivity and specific for prediction of abusive head trauma was 95.8% (95%, CI, 94.4-97.0) and 54.9% (95% CI, 50.9-58.9), at a cutoff of 0.182. The dark line represents the ROC for the entire training set. The gray lines represent the ROC developed for each of the 20-fold cross-validations.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Berger RP, Pak BJ, Kolesnikova MD, et al. Derivation and Validation of a Serum Biomarker Panel to Identify Infants With Acute Intracranial Hemorrhage. JAMA Pediatr. 2017;171(6):e170429. doi:10.1001/jamapediatrics.2017.0429
Is it possible to develop a panel of serum biomarkers that can be used to identify infants who are at increased risk of having an acute intracranial hemorrhage and/or abusive head trauma?
This study used a retrospective cohort of 99 patients followed by a prospective cohort of 599 patients to derive and validate the Biomarkers of Infant Brain Injury Score. The Biomarkers of Infant Brain Injury Score predicted intracranial hemorrhage with a higher sensitivity than clinical judgement, the current criterion standard.
The Biomarkers of Infant Brain Injury Score may be useful in clinical practice to identify infants who should undergo neuroimaging to identify abusive head trauma.
Abusive head trauma is the leading cause of death from physical abuse. Missing the diagnosis of abusive head trauma, particularly in its mild form, is common and contributes to increased morbidity and mortality. Serum biomarkers may have potential as quantitative point-of-care screening tools to alert physicians to the possibility of intracranial hemorrhage.
To identify and validate a set of biomarkers that could be the basis of a multivariable model to identify intracranial hemorrhage in well-appearing infants using the Ziplex System.
Design, Setting, and Participants
Binary logistic regression was used to develop a multivariable model incorporating 3 serum biomarkers (matrix metallopeptidase-9, neuron-specific enolase, and vascular cellular adhesion molecule-1) and 1 clinical variable (total hemoglobin). The model was then prospectively validated. Multiplex biomarker measurements were performed using Flow-Thru microarray technology on the Ziplex System, which has potential as a point-of-care system. The model was tested at 3 pediatric emergency departments in level I pediatric trauma centers (Children’s Hospital of Pittsburgh of University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania; Primary Children’s Hospital, Salt Lake City, Utah; and Lurie Children’s Hospital, Chicago, Illinois) among well-appearing infants who presented for care owing to symptoms that placed them at increased risk of abusive head trauma. The study took place from November 2006 to April 2014 at Children’s Hospital of Pittsburgh, June 2010 to August 2013 at Primary Children’s Hospital, and January 2011 to August 2013 at Lurie Children’s Hospital.
Main Outcomes and Measures
A mathematical model that can predict acute intracranial hemorrhage in infants at increased risk of abusive head trauma.
The multivariable model, Biomarkers for Infant Brain Injury Score, was applied prospectively to 599 patients. The mean (SD) age was 4.7 (3.1) months. Fifty-two percent were boys, 78% were white, and 8% were Hispanic. At a cutoff of 0.182, the model was 89.3% sensitive (95% CI, 87.7-90.4) and 48.0% specific (95% CI, 47.3-48.9) for acute intracranial hemorrhage. Positive and negative predictive values were 21.3% and 95.6%, respectively. The model was neither sensitive nor specific for atraumatic brain abnormalities, isolated skull fractures, or chronic intracranial hemorrhage.
Conclusion and Relevance
The Biomarkers for Infant Brain Injury Score, a multivariable model using 3 serum biomarker concentrations and serum hemoglobin, can identify infants with acute intracranial hemorrhage. Accurate and timely identification of intracranial hemorrhage in infants without a history of trauma in whom trauma may not be part of the differential diagnosis has the potential to decrease morbidity and mortality from abusive head trauma.
Abusive head trauma (AHT) is the leading cause of death from traumatic brain injury in infants1-3 and the leading cause of death from physical abuse in the United States.4 Proper diagnosis of mild AHT (Glasgow Coma Scale score, 13-15) is difficult because caretakers often provide inaccurate histories,5 infants present with nonspecific symptoms such as vomiting, and physical examination is often normal.6-8 Unfortunately, misdiagnosis is common and can have catastrophic medical consequences.8-10 In a landmark study of children diagnosed as having AHT,9 31% (54 of 173) were previously evaluated by a physician for symptoms that, in hindsight, were missed opportunities to diagnose AHT. Fifteen years later, a multicenter study demonstrated a similar rate of missed diagnoses,11 suggesting that early, accurate diagnosis of AHT is still challenging.
The frequency with which AHT is misdiagnosed and the resulting morbidity and mortality indicate the need for a screening test to help physicians identify infants in whom neuroimaging should be considered. The screening test would not be used to diagnose AHT per se. If neuroimaging demonstrated acute intracranial hemorrhage (ICH), additional evaluation for AHT would be indicated.12
During the past decade, we have reported on the possible use of serum biomarkers as screening tools for ICH in infants at risk of missed AHT.13-18 A single study evaluating the use of multiple biomarkers simultaneously suggests that combining biomarkers improves sensitivity for ICH detection.16 To our knowledge, no studies have evaluated multiple biomarkers on a clinically relevant device, an important step in bringing brain biomarkers to clinical practice.19,20
The Ziplex System (Axela, Inc) is a fully automated multiplex platform based on proprietary Flow-Thru microarrays. The Ziplex System can measure multiple biomarkers simultaneously and rapidly using 2 µL of serum. The objective of this study was to identify and validate a set of biomarkers that could be the basis of a multivariable model to identify ICH in infants using the Ziplex System.
There were 3 study groups: the retrospective derivation, the prospective validation, and the supplemental cohort of infants with rare intracranial abnormalities.
To derive the biomarker-based formula to discriminate infants with and without ICH, serum was used from patients from an institutional review board–approved serum databank at the Safar Center for Resuscitation Research at the University of Pittsburgh. Sera samples from patients in the databank were included in the derivation if patients were aged 30 to 364 days, well-appearing, afebrile (temperature < 38.3°C), and presented to Children’s Hospital of Pittsburgh of the University of Pittsburgh Medical Center with nonspecific symptoms including vomiting or fussiness, which can be associated with an increased risk of AHT.8,9 Patients in the retrospective cohort were enrolled from November 9, 2000, to June 13, 2013.
A multicenter prospective validation of the model developed with the derivation cohort was then performed. The study, which will be referred to as the parent study, was approved by the institutional review boards at Children’s Hospital of Pittsburgh of the University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, Primary Children’s Hospital, Salt Lake City, Utah, and Ann and Robert H. Lurie Children’s Hospital, Chicago, Illinois. Enrollment at Children’s Hospital of started on October 1, 2006, at Primary Children’s Hospital on June 1, 2010, and at Ann and Robert H. Lurie Children’s Hospital on January 1, 2011. Written consent was obtained except in cases of suspected abuse, when a waiver of informed consent was approved.
Inclusion criteria are in Table 1. Enrollment was not consecutive and based on investigator availability. Detailed information about patient classification and data collection has been published previously.25
While the overarching goal of the biomarker panel is to identify infants with acute ICH owing to AHT, infants with atraumatic intracranial abnormalities, ICH not owing to AHT, chronic ICH, or isolated skull fractures can present with similar symptoms as infants with AHT. Therefore, it is critical to assess any biomarker panel in infants with these abnormalities. Because these abnormalities are rarer than AHT-related ICH,26,27 we expected an inadequate number of these patients in the prospective validation. As a result, serum from infants with these abnormalities was selected from the databank described in the previous section and included with prospective validation samples.
Blood was collected as soon as possible after presentation in the emergency department, processed by the hospital laboratory, and deidentified. Serum was stored at −70°C and shipped to Axela Inc for biomarker analysis. Total hemoglobin concentration was measured at the time of enrollment in the hospital laboratory.
As part of the parent study, patients were classified as cases or controls based on neuroimaging used in a previously published paradigm.25 Among cases, brain abnormalities were classified based on etiology (abuse vs not abuse) and type of abnormality (acute ICH not underlying skull fracture, acute ICH underlying skull fracture, chronic ICH, atraumatic abnormality, or isolated skull fracture) (Figure 1).
Computed tomography and magnetic resonance imaging scans were interpreted for clinical care and by a study neuroradiologist. If infants had any acute ICH, they were classified in the acute ICH group. Chronic subdural hemorrhage was confirmed with brain magnetic resonance imaging, which is standard of care at all 3 sites.
Abusive head trauma was defined as acute ICH and the assessment of probable or definite abuse by each site’s hospital-based Child Protection Team. Using a Child Protection Team assessment is a common way to define AHT in clinical research.1,9,15 The Child Protection Team did not have access to biomarker data.
Based on data from prior publications,14-16,28-34 brain-specific (neuron-specific enolase [NSE]) and nonbrain-specific (matrix metallopeptidase-9 [MMP-9] and vascular cellular adhesion molecule-1 [VCAM-1]) markers were included in the model. Three clinical variables (total hemoglobin, age, and sex) were evaluated for inclusion in the model owing to data suggesting an association with ICH and AHT.1,25,35
Capture and biotinylated reporter antibodies for MMP-9 and VCAM-1 and recombinant protein were obtained from R&D Systems. Antibodies for NSE and recombinant NSE were obtained from International Point of Care. EZ-Link Sulfo-NHS-Biotin for biotinylation of NSE reporter antibodies and streptavidin Poly-HRP were obtained from ThermoFisher Scientific. All other materials and reagents were obtained from Axela Inc.
Capture antibodies were spotted in quadruplicate onto epoxy-activated chips using a pin microarray printer.
Multiplex immunoassays were performed on the Ziplex System using 2-µL serum and biotinylated secondary antibodies. Following incubation with streptavidin-HRP, biomarkers were detected by chemiluminescence. Biomarker concentrations were taken as the mean of replicate sample analyses.
Eight-point calibration curves were generated for each biomarker spanning the clinically relevant concentrations. Five-parameter logistic regression was used for fitting MMP-9 and VCAM-1 calibration curves; linear regression was used for NSE. The serum concentration of each biomarker was interpolated from its respective calibration curve.
Owing to the presence of endogenous NSE in red blood cells, NSE concentrations in serum can be artificially elevated by hemolysis. Therefore, NSE concentrations were adjusted to account for hemolysis using a previously described method.36 Samples with a hemocue of at least 0.5 g/dL were excluded (to convert to grams per liter, multiply by 10). The technician performing all assays was blinded to clinical data.
A logistic regression model was fit to the derivation data to identify a prediction model for acute intracranial hemorrhage as a function of biomarkers and covariates. The data for the 4 continuous-value markers were centered on the median values of the control samples in the derivation set that serve as the baseline for all other samples. A cutoff was chosen so that the sensitivity for the samples in the derivation set was 95%. The model including this cutoff was used for resampling cross validation of the derivation set and for evaluation of the validation set. Binary logistic regression was evaluated with a cross-validation procedure in which the samples in the derivation or validation were divided randomly (resampling without replacement) within 20 folds of approximately equal numbers of resampled derivation and test sets with stratification according to whether samples were cases or controls. The mean and standard deviation area under the curve for the receiver operating characteristic curves and the specificity and sensitivity and negative and positive predictive for the validation sets from the 20 folds were calculated. Multiple combination of markers were evaluated (Table 2). A Support Vector Machine (SVM) model was also evaluated using the R SVM function with linear kernel and cost = 0.08 (R Programming). A P value of less than .05 was considered significant. All P values were 2-sided.
Descriptive statistics were performed using IBM SPSS, version 23.0 (IBM Corp). Statistical modeling was performed with R, version 3.3.2 of the R statistical programming language37 with the packages ROCR,38 e7071,39 and openxlsx (R Programming).40 R was used with the RStudio Integrated Development Environment.41
Ninety-nine patients, 48 with ICH and 51 without ICH, were in the derivation cohort. All 48 patients with ICH had AHT. Serum hemoglobin, MMP-9, adjusted NSE, and VCAM-1, but not age or sex, were significantly associated with ICH in the derivation cohort. The model containing serum hemoglobin plus MMP-9, adjusted NSE, and VCAM-1 provided the greatest predictive value (Table 2).
The receiver operating characteristic curve of the binary logistic regression model trained with markers is in Figure 2. The area under the curve was 0.906 (95% CI, 0.893-0.919). Sensitivity and specificity for prediction of AHT was 95.8% (95% CI, 94.4-97.0) and 54.9% (95% CI, 50.9-58.9) at a cutoff of 0.182. The SVM models provided very similar results, with substantial overlap of the confidence intervals from the cross validation (area under the curve, 0.907; 95% CI, 0.894-0.919; with a sensitivity and specificity of 95.8%; 95% CI, 94.3-97.4; and 52.9%; 95% CI, 47.6-53.0, respectively). Because of the similarity between the models, the SVM model was not pursued further.
The formula that was developed to classify patients as cases or controls was:
−2.442 + 0.000430 × [MMP9 − Median MMP-9] + 0.1058 × [Adjusted NSE − Median Adjusted NSE] − 1.306 × [Hemoglobin − Median Hemoglobin] −0.004165 × [VCAM1 − MedianVCAM1]
where “median” represents the median value for the marker among the control infants in the derivation cohort. The coefficients represent the change per unit increase in the contribution of each of the markers to the calculation of the log odds of a patient being a case. Matrix metallopeptidase-9, NSE, and VCAM-1 are measured in nanograms per milliliter, and total serum hemoglobin is measured in grams per deciliter. This formula will be referred to as Biomarker of Infant Brain Injury Score (BIBIS).
There were 1040 patients enrolled in the parent study; 54% (561 of 1040) had BIBIS calculated. Of the 479 patients in the parent study who could not have BIBIS calculated, the most common reason was that serum was not available for biomarker analysis. Many patients in the parent study had no blood collected for clinical care, and parents did not consent for a blood draw specifically for research. Eighteen patients were excluded owing to a hemocue greater than 0.5 g/dL.36 There was no difference in the mean (SD) age (4.7 [3.0] vs 4.7 [3.1] months, P = .51), proportion of boys (52% vs 53%, P = .80), or the likelihood of being a case patient (22.0% vs. 19.4%, P = .40) in patients who did and did not have blood available for analysis. A total of 599 patients (561 from the parent study and 38 with other intracranial abnormalities) had BIBIS calculated (Figure 1).
The binary logistic regression model was initially evaluated with a validation set that included the 440 control infants and the 36 patients with AHT using the cutoff developed in the derivation set. This was done to provide a direct comparison between the derivation and validation sets. The sensitivity and specificity of the cutoff calculated in the derivation set and applied in this cohort was 86.4% (95% CI, 84.1-88.7) and 48.9% (95% CI, 47.9-49.8), respectively.
The binary logistic regression model was then evaluated with a validation set that included the 440 control infants and the 71 patients with acute ICH of any etiology. The sensitivity and specificity of the cutoff in this cohort was 89.34% (95% CI, 87.7-90.4) and 48.0% (95% CI, 47.3-48.9), respectively. The positive and negative predictive values were 21.3% and 95.6%, respectively. There was substantial overlap with the model characteristics in patients with AHT compared with those with acute ICH of any etiology, consistent with the modeling predicting acute ICH rather than AHT.
The model was unable to identify abnormalities other than acute ICH. The sensitivity and specificity in the cohort of patients with atraumatic abnormalities (n = 17), chronic ICH (n = 8), and isolated skull fracture (n = 34) were 52% and 49%, respectively.
To our knowledge, this is the first study to use a combination of serum biomarkers and a clinical variable to derive and validate a screening tool that predicts acute ICH in infants at increased risk of AHT. The ability to identify infants who would benefit from neuroimaging has the potential to improve early recognition of AHT and thereby decrease morbidity and mortality. The consistency of the retrospective single-site derivation and the prospective multisite validation as well as the consistency of the binary logistic regression and SVM models supports the robustness of this approach. The fact that the sensitivity and specificity for acute ICH owing to AHT and acute ICH from other etiologies is the same is consistent with the fact that BIBIS does not identify AHT but rather acute ICH. Maximizing sensitivity rather than accuracy was done because missing AHT has more serious implications than performing neuroimaging in patients without ICH. This is particularly true if rapid magnetic resonance imaging rather than head computed tomography is the diagnostic test performed in patients with an abnormal BIBIS.42,43
The decision not to recalculate the intercept of the BIBIS model in the validation cohort to account for differences in baseline characteristics of the validation and derivation cohorts was intentional. While there was clearly a difference between the 2 cohorts given the lower sensitivity of the model in the validation cohort, and this difference could have been adjusted for by changing the intercept, we chose not to do this because of the implication for clinical practice. To use BIBIS for prediction in real time for a single patient, BIBIS must be calculated in a single patient using a predefined formula with a known regression slope and intercept. The intercept cannot be recalculated in each patient.
When developing a new screening tool, comparison with the preexisting gold standard is important. Because clinical judgement is the criterion standard for determining which infants with nonspecific symptoms should undergo neuroimaging, it is the most relevant comparison. Two studies9,11,25 suggest that clinical judgment has a sensitivity of approximately 70%; our data demonstrate that BIBIS is more sensitive.
As with all screening tests, BIBIS is meant to supplement rather than replace clinical judgment in well-appearing infants in whom brain injury may not be in the initial differential diagnosis. It should not be used in infants who need emergent neuroimaging and/or to justify not obtaining neuroimaging when there are other clinical indicators. It is also not designed to diagnose AHT; the equivalent sensitivity of BIBIS for prediction of acute ICH owing to AHT and acute ICH from other etiologies is consistent with this.
The lack of predictive value of BIBIS for children with atraumatic injuries, chronic ICH, and skull fractures is not surprising. The biomarkers in BIBIS would not be expected to be increased in chronic ICH because of their short biologic half-lives. Studies suggest that some serum biomarkers may increase weeks to months after ICH44,45; evaluation of these markers in addition to BIBIS could be an intriguing area of future research. The lack of sensitivity for atraumatic injury and isolated skull fractures is consistent with the lack of ICH in these cases.
The selection of biomarkers occurred prior to the derivation and was based on the literature and our own experience. Despite its recognized limitations, NSE has been a primary brain-specific biomarker evaluated for more than a decade.14,15,28 Matrix metallopeptidase-9 and VCAM-1 were identified as potential markers in a study in which 45 potential biomarkers were evaluated using multiplex bead technology.16 Adult studies also demonstrate increased serum MMP-9 after traumatic brain injury29-31 and implicate it as a marker of hemorrhagic transformation and injury to the blood-brain barrier after stroke.32 While MMP-9 and NSE are increased after acute ICH, VCAM-1 is decreased. This is in contrast to cerebrospinal fluid concentrations of VCAM-1, which are increased after pediatric traumatic brain injury.33 The physiologic basis for the decreased serum VCAM-1 is unknown, but a 2015 publication suggests that endogenous growth factors down-regulate VCAM-1 in brain microcirculation.34
The ability to measure these biomarkers simultaneously on the Ziplex device is an important step toward clinical usefulness and suggests that a point-of-care test may be feasible using this type of device. Additional testing and evaluation is required.
Of the 1040 patients in the parent study, only a subset had BIBIS calculated. As discussed in the Results section, this was primarily related to the lack of available serum. It was not surprising that parents were often unwilling to have blood drawn in their infant for research purposes. We do not believe this would be a limitation if BIBIS were part of routine clinical care because there would then be no need for written research consent.
While patients were not enrolled daily, enrollment was consecutive when an investigator was available at each site. There is no reason to believe that infants presenting with nonspecific symptoms at 2 am are different than infants presenting at 4 pm.
Finally, only infants were included because this is the age group at highest risk for AHT. Approximately 25% of children with AHT are older than 1 year of age.46 Prior to bringing BIBIS to clinical use, it will be important to assess whether it could be used in high-risk toddlers.
We have derived and prospectively validated a brain injury score, BIBIS, that uses 3 serum biomarkers plus serum hemoglobin to predict which infants are most likely to have acute ICH and would benefit from neuroimaging. Future studies will focus on combining BIBIS with the Pittsburgh Infant Brain Injury Score (PIBIS)25 to determine whether a combined score is more accurate than either score alone. Future research must also focus on bringing brain biomarker measurement from the bench to the bedside as it relates to commercialization of the Ziplex device. Finally, while our focus is development of an accurate screening tool, for BIBIS to be successful in clinical practice, physicians will need to know when to use it. Developing and evaluating strategies to help physicians do this is an important future step.
Corresponding Author: Rachel Padres Berger, MD, MPH, Division of Child Advocacy, Department of Pediatrics, Children’s Hospital of Pittsburgh of UPMC, 4401 Penn Ave, Pittsburgh, PA 15224 (firstname.lastname@example.org).
Accepted for Publication: February 16, 2017.
Published Online: April 10, 2017. doi:10.1001/jamapediatrics.2017.0429
Author Contributions: Dr Berger had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Berger, Saladino, Herman, Pierce, Smith, Kochanek.
Acquisition, analysis, or interpretation of data: Berger, Pak, Kolesnikova, Fromkin, Saladino, Herman, Pierce, Englert.
Drafting of the manuscript: Berger, Pak, Kolesnikova, Smith.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Pak, Englert, Smith.
Obtained funding: Berger, Pierce, Smith, Kochanek.
Administrative, technical, or material support: Fromkin, Saladino, Herman.
Supervision: Pak, Saladino, Pierce, Smith.
Conflict of Interest Disclosures: Drs Berger, Herman, and Pierce have all received subpoenas to provide expert opinions in child abuse cases. They have not been paid for their testimony.
Funding/Support: The study was funded by grants R01HD055986 and NIHK23HD43843 from the National Institutes of Health. The data in this article are the basis for patent 62/334,345 filed on May 10, 2016, Compositions and Methods for Identifying Subjects at risk for Traumatic Brain Injury.
Role of the Funder/Sponsor: The funding source was involved in the design and conduct of the study but had no role in the collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: We thank Pamela Rubin, RN (Children’s Hospital of Pittsburgh of UPMC), and Janet Flores, BS (Ann and Robert H. Lurie Children’s Hospital), for data collection; Kammy Jacobsen, BS (Primary Children’s Hospital, Salt Lake City, Utah), for data collection; Lauren McCullaugh, MPH (Hofstra North Shore Long Island School of Medicine), for data analysis and interpretation; and Tania Martinez Fundora (Axela, Inc) for sample analysis. The National Institutes of Health grant that sponsored the research provided salary support for Mss Rubin, Flores, and Jacobsen.