A, The dotted line indicates a disease risk score threshold of −3.79, determined by the point on the definite bacterial vs definite viral ROC curve that maximized sensitivity and specificity. This was used to calculate the quoted sensitivity and specificity. Boxes indicate the interquartile ranges and the median (bold line); whiskers represent 1 or less times the interquartile range. B, Data are as reported in the study by Mahajan et al.2 Data points indicate the corresponding thresholds.
Myrsini Kaforou, Jethro A. Herberg, Victoria J. Wright, Lachlan J. M. Coin, Michael Levin. Diagnosis of Bacterial Infection Using a 2-Transcript Host RNA Signature in Febrile Infants 60 Days or Younger. JAMA. 2017;317(15):1577–1578. doi:10.1001/jama.2017.1365
Distinguishing children with potentially life-threatening bacterial infections from febrile children with viral infections remains a major challenge. Herberg and colleagues,1 in a preliminary, cross-sectional study of 370 febrile children (aged <17 years) in Europe and the United States, reported that children with bacterial infection may be characterized by the difference in blood RNA expression values of 2 genes. In a recent study, Mahajan and colleagues2 reported a 66-transcript blood RNA signature that distinguished bacterial from viral infection in 279 febrile infants younger than 60 days. Young infants are at high risk of bacterial infection; diagnosis is difficult and prompt treatment important. To provide further validation of the 2-gene signature and to evaluate its performance in the infant population, we applied the signature to the RNA expression data of Mahajan et al.
Institutional review board approval was obtained from all 22 sites in the Pediatric Emergency Care Applied Research Network included in the study by Mahajan et al. Parents or guardians provided written informed consent. The RNA expression data of Mahajan et al were downloaded from the Gene Expression Omnibus database accession GSE64456,3 obtained from a convenience sample of febrile neonates and infants less than 60 days old who were recruited from 22 US emergency departments between 2008 and 2010. Eighty-nine infants had culture-positive bacterial infections (32 with bacteremia and 57 with urinary tract infection) and 190 had negative bacterial cultures (111 with proven viral infection, 38 not tested for viruses, and 41 with negative test results). After logarithmic transformation, robust spline normalization and quality control using the lumi Bioconductor package in R (R Foundation), version 3.3.1, the expression values for the 2-signature transcripts IFI44L (RefSeq ID: NM_006820.1) and FAM89A (RefSeq ID: NM_198552.1) were extracted for each patient. These values were combined into a single disease risk score (DRS), as described in the study by Kaforou and colleagues,4 by subtracting the intensity of the IFI44L transcript from the intensity of the FAM89A transcript. We evaluated the predictive accuracy of the DRS first in patients with microbiologically confirmed diagnoses, and then in patients without definite clinical diagnosis. Using the pROC package in R,5 we calculated the area under the characteristic curve (AUC), sensitivity, specificity, and their 95% CIs under the negative binomial distribution.
When the 2-transcript DRS signature was applied to infants with definite bacterial diagnoses (n = 89) and proven viral infections (n = 111), sensitivity was 88.8% (95% CI, 80.3%-94.5%), specificity was 93.7% (95% CI, 87.4%-97.4%) and AUC was 95.7% (95% CI, 92.6%-98.3%) (Figure). The signature distinguished patients with a definite bacterial diagnosis from those with negative results for viruses (n = 41) or no viral tests performed (n = 38) with specificities of 48.8% (95% CI, 32.8%-64.9%) and 78.9% (95% CI, 62.7%-90.5%), whereas the sensitivities remained unchanged. The AUC was 80.5% (95% CI, 72.4%-87.5%) for those with negative results and 90.9% (95% CI, 84.8%-95.9%) for those with no viral test.
The studies by Herberg et al and Mahajan et al reported sensitivities of 100% (95% CI, 85%-100%) and 87% (95% CI, 73%-95%), respectively, and specificities of 96.4% (95% CI, 89.3%-100%) and 89% (95% CI, 81%-93%), respectively, for the discrimination of bacterial from viral and nonbacterial infections. In this study, the 2-transcript RNA signature, which was originally identified and validated in children with a mean age of 19 months, also had high sensitivity and specificity in the specific population of infants younger than 60 days.
The performance of the DRS signature in the groups with uncertain clinical diagnoses should be regarded as exploratory. As bacteria can only be definitively identified as the cause of infection when cultured from normally sterile sites, performance metrics of any new biomarker can only be reliably evaluated in patients with confirmed viral or bacterial infection. However, children with features of bacterial infection but negative cultures are commonly encountered in clinical practice and interpretation of DRS results remains inconclusive. Other limitations include the small number of patients from a limited number of clinical settings.
The 2-gene DRS has the potential to translate into a simple bedside diagnostic test, but further studies in larger cohorts of infants and children representing the broad spectrum of febrile children are needed.
Corresponding Author: Michael Levin, PhD, FRCP, Section of Pediatrics, Division of Infectious Diseases, Department of Medicine, Imperial College London, Norfolk Place, London, W2 1PG (email@example.com).
Author Contributions: Drs Kaforou and Levin had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: All authors.
Acquisition, analysis, or interpretation of data: Kaforou, Herberg, Wright, Levin.
Drafting of the manuscript: Kaforou, Herberg, Wright, Levin.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Kaforou, Herberg.
Obtained funding: Wright, Levin.
Administrative, technical, or material support: Herberg, Levin.
Supervision: Coin, Levin.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported. Drs Kaforou, Herberg, Wright, and Levin report a pending patent for a 2-gene signature. Dr Coin reports a pending patent to Imperial Innovations for a disease risk score.
Funding/Support: This work was supported by grant DMPED P26077 from the Imperial College Comprehensive Biomedical Research Centre; grant RSRO 54990 from the Imperial College-Wellcome Trust Antimicrobial Research Collaborative (Dr Kaforou); grant EC-GA 279185 from the European Union's Seventh Framework Program (Drs Herberg and Coin); grant V1401 from Great Ormond St Hospital Charity (Dr Wright); and a grant from the National Institute for Health Research (Dr Levin).
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.