Assessment of C-Reactive Protein Diagnostic Test Accuracy for Late-Onset Infection in Newborn Infants: A Systematic Review and Meta-analysis

Key Points

Question  Is serum C-reactive protein level sufficiently accurate to aid the diagnosis of late-onset infection in newborn infants?

Findings  In this systematic review and meta-analysis of 22 cohort studies (2255 infants) comparing the diagnostic test accuracy of serum C-reactive protein with microbiological culture, median specificity was 0.74 and pooled sensitivity was 0.62. Assuming a prevalence rate of 40% in a cohort of 1000 infants, serum C-reactive protein would miss 152 cases of infection and wrongly diagnose 156 cases.

Meaning  The findings suggest that serum C-reactive protein level is not sufficiently accurate to aid diagnosis or to inform treatment decisions in infants with suspected late-onset infection.

Importance  Rapid and accurate diagnosis of late-onset infection in newborn infants could inform treatment decisions and avoid unnecessary administration of antibiotics.

Objective  To compare the accuracy of serum C-reactive protein (CRP) with that of microbiological blood culture for diagnosing late-onset infection in newborns.

Data Sources  MEDLINE (1946-2019), Embase (1946-2019), and Science Citation Index (1900-2019) databases were searched for references (any language). The MeSH search terms included were “exp infant, newborn/” or “premature birth/” plus free text synonyms; and “C-reactive protein/” plus free text synonyms; and “exp sepsis/” or “exp bacterial infections/” plus free text synonyms. The proceedings from relevant conferences and references of identified papers were scrutinized. Authors were contacted to request missing data.

Study Selection  Cohort and cross-sectional studies were included that compared the accuracy of serum CRP levels with microbiological culture results to diagnose late-onset (>72 hours after birth) infection in newborns (any gestational age) hospitalized after birth. Two reviewers assessed study eligibility. Among 10 394 records, 148 studies were assessed as full texts.

Data Extraction and Synthesis  The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guideline extension for Diagnostic Test Accuracy (DTA) reviews was followed. Two reviewers assessed the method quality of each study using guidance from the Cochrane Screening and Diagnostic Test Methods Group (adapted from the Quality Assessment of Diagnostic Accuracy Studies 2).

Main Outcomes and Measures  The primary meta-analysis outcome was diagnostic test accuracy of serum CRP level taken at initial investigation of an infant with suspected late-onset infection. The median specificity (proportion of true-negative results) and calculated pooled sensitivity (proportion of true-positive results) were determined by generating hierarchical summary receiver characteristic operating curves.

Results  In total, 22 studies with 2255 infants were included (sample size range, 11-590 infants). Participants in most studies were preterm (<37 weeks) or very low-birth weight (<1500 g) infants. Two studies additionally enrolled infants born at term. Most studies (16) used a prespecified CRP level cutoff for a “positive” index test (5-10 mg/L), and most studies (17) used the culture of a pathogenic microorganism from blood as the reference standard. Risk of bias was low with independent assessment of index and reference tests. At median specificity (0.74), pooled sensitivity was 0.62 (95% CI, 0.50-0.72). Adding serum CRP level to the assessment of an infant with a 40% pretest probability of late-onset infection (the median for the included studies) generated posttest probabilities of 26% for a negative test result and 61% for a positive test result.

Conclusions and Relevance  The findings suggest that determination of serum CRP level at initial evaluation of an infant with suspected late-onset infection is unlikely to aid early diagnosis or to select infants to undergo further investigation or treatment with antimicrobial therapy or other interventions.

Late-onset infection (occurring >72 hours after birth) is one of the most common serious complications associated with intensive care for newborn infants.¹ Preterm infants, particularly very preterm infants, with late-onset infection have a higher risk of mortality, morbidity, and need for intensive care and prolonged hospitalization than newborn infants without infection.² Late-onset infection is associated with adverse neurodevelopmental outcomes, including cerebral palsy and visual, hearing, and cognitive impairments.^3,4

Clinical signs of infection in neonates can be nonspecific, and the diagnosis of late-onset infection in newborn infants can be delayed if these signs are missed. Microbiological culture of a potentially pathogenic organism from a blood sample takes 24 to 48 hours to complete. Delayed treatment of late-onset infection may increase the risk of morbidity and mortality in newborn infants. However, empirical treatment of all infants with suspected infection will result in the administration of unnecessary courses of antibiotics.⁵ Such widespread use, particularly of broad-spectrum antibiotics, is associated with accelerated selective pressure and the emergence of drug resistance through mechanisms such as extended-spectrum β-lactamase production.^6-8 In addition, the exposure to antibiotics in early life carries a risk of adversely altering the developing microbiome,⁹ which could be detrimental, especially to unwell or preterm infants in whom concerns regarding gut health already exist.^10,11

To avoid the unnecessary stress on the already compromised organs, such as in the gastrointestinal tract of preterm infants, and the contribution to the wider problem of antimicrobial resistance, several biomarkers have been proposed as tests to support the diagnosis of late-onset infection in newborn infants. Used in conjunction with blood culture, biomarkers have the potential to indicate whether infection is more or less likely in infants in whom it is suspected.^12,13 The most commonly used biomarker is the serum level of C-reactive protein (CRP), an acute-phase reactant synthesized by hepatocytes in response to inflammatory cytokines generated by white blood cells reacting to microbial pyrogens.¹⁴ Serum CRP might be a useful biomarker for late-onset infection in newborn infants if it can be shown to have acceptable levels of accuracy. Currently, in the absence of robust evidence to inform guideline or protocol development, the role of serum CRP in diagnostic algorithms for late-onset infection varies greatly.^5,15 Most studies examining the accuracy of CRP and other biomarkers of late-onset infection have been conducted in single centers and, therefore, are limited by the small sample size. Previously, we conducted and reported a systematic review and meta-analysis of data from 20 studies on CRP and diagnosis of late-onset infection in newborn infants.¹⁶ Herein, we report an updated systematic review and meta-analysis of CRP diagnostic test accuracy with data from 2 additional studies^17,18 to identify, quality appraise, and synthesize the available evidence to inform policy and practice as well as future research.

We searched MEDLINE (1946-2019), Embase (1946-2019), and Science Citation Index (1900-2019) databases for references published in any language. The MeSH search terms included were “exp infant, newborn/” or “premature birth/” plus free text synonyms; and “C-reactive protein/” plus free text synonyms; and “exp sepsis/” or “exp bacterial infections/” plus free text synonyms. We examined the reference lists of all studies identified as potentially relevant and searched the abstracts from the annual meetings of the Pediatric Academic Societies (1993-2018), the European Society for Pediatric Research (1995-2018), the UK Royal College of Paediatrics and Child Health (2000-2018), and the Perinatal Society of Australia and New Zealand (2000-2018). Studies reported only as abstracts were eligible if sufficient information was available from the report or from contact with the authors to fulfill the inclusion criteria. The detailed search strategy can be found in eAppendix 1 in the Supplement. This systematic review and meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline extension for Diagnostic Test Accuracy (DTA) reviews. This study was excluded from internal review board review and the requirement for informed patient consent based on institutional policies for systematic reviews and meta-analyses.

Three reviewers (J.V.E.B., N.M., and J.C.) independently screened titles and abstracts and retrieved full-text publications for potentially relevant references. All records were screened by 2 of those 3. Disagreements were resolved by a third author (W.M.) if necessary. We included cohort and cross-sectional studies of hospitalized newborn infants of any gestational age with clinically suspected late-onset infection (including bacteremia, fungemia, meningitis, osteomyelitis, septic arthritis, and peritonitis) for whom diagnostic test accuracy data for serum CRP levels were reported. Studies that included infants younger than 72 hours after birth (with suspected early-onset infection) were only eligible for inclusion if data for late-onset infection could be extracted separately. We emailed the authors of studies published after 2004 to request unpublished data and clarification of the study method as needed. We excluded case-control studies because that design does not allow for valid assessment of diagnostic test accuracy in this clinical context. We excluded studies in which the reference standard incorporated the index test, that is, infection was defined as a positive microbiological culture test result and an increased serum CRP level. We did not include studies in which the participants were infants cared for at home or in another community setting and who then presented to a health care facility with possible infection.

Two of 3 reviewers (J.V.E.B., N.M., or J.C.) assessed the method quality of each included study following guidance from the Cochrane Screening and Diagnostic Test Methods Group, which is adapted from the Quality Assessment of Diagnostic Accuracy Studies 2 tool.¹⁹ One author (J.V.E.B., N.M., or J.C.) extracted study characteristics, participant details, and diagnostic data to enable derivation of the number of true-positive results, false-positive results, false-negative results, and true-negative results from the included reports. A second author (J.V.E.B., N.M., or J.C.) checked data extraction. Disagreements were discussed and resolved with input from a third reviewer (W.M.) as needed. Only the index test conducted at the same time as the reference standard was used. We created forest plots with 95% CIs for sensitivity and specificity for each study using RevMan 5 (Review Manager 5) software, version 5.3.²⁰

We calculated estimates of sensitivity at fixed values of specificity (median and lower and upper quartiles reported in the included studies) on the summary receiver operating characteristic (ROC) curves. Because the reported cutoff level for a positive CRP test differed among the studies, we fitted a hierarchical summary ROC model that assumed accuracy and thresholds varied among studies.²¹ Analyses were conducted using the NLMIXED procedure with the SAS system for Windows, version 9.4 (SAS Institute Inc). We also conducted bivariate meta-analyses for studies with a CRP level threshold of 5 to 10 mg/L to provide summary estimates of sensitivity and specificity at that threshold (to convert CRP levels to nanomoles per liter, multiply by 9.524).

To facilitate interpretation of diagnostic accuracy estimates, we presented the number of cases missed and the number of cases wrongly diagnosed in a hypothetical cohort of 1000 neonates suspected of late-onset sepsis based on sensitivity and median specificity estimates. We calculated these values when the expected prevalence rate was 20%, 40%, or 60% for late-onset sepsis.

We assessed heterogeneity by examining forest plots of sensitivity and specificity across studies for variability of study estimates and overlap of 95% CIs. We conducted metaregression analyses to explore the association of serum CRP cutoff levels (categorical covariate: standard threshold 5-10 mg/L vs any other threshold) and reporting a predefined threshold on heterogeneity (categorical covariate: reporting predefined threshold vs not reporting predefined threshold).

We planned to assess gestational age at birth, type of pathogens, and subtype of late-onset infection in metaregression analyses to evaluate the association of these participant level characteristics with the diagnostic accuracy of serum CRP. If sufficient data were available, we planned to explore whether study method quality was associated with the results in sensitivity analyses by removing studies considered at higher risk of bias across key domains (selection, verification).

We assessed publication bias using a funnel plot and the Deeks test.²² This systematic review of diagnostic test accuracy was registered prospectively (PROSPERO CRD42016045585). Protocol changes are given in eAppendix 2 in the Supplement.

Among 10 394 records, 148 studies were assessed as full texts. Of those, 22 studies reported in 22 separate publications were included in this systematic review,^17,18,23-42 including 2 studies^17,18 not included in our previously published meta-analysis.¹⁶ Sample sizes ranged from 11 to 590 infants (total 2255 infants). See Figure 1, PRISMA flow diagram, for details of the study selection process. Most studies were carried out in high-income countries in Europe,^23-30 Asia,^31-33 North America,^17,34,35 South America,³⁶ or Australasia.^37,38 Five studies were conducted in low- and middle-income countries.^18,39-42 All but one³⁴ of the studies were single-center investigations. Studies were published between 1990 and 2018, with most studies (17 of 22) published since 2000. Three studies were cohorts assembled retrospectively.^17,25,26 The remaining 19 studies included prospectively observed cohorts.

The characteristics of the included studies and the study level diagnostic data are summarized in the Table. The method quality of the included studies was good, and the risk of bias was low (Figure 2). In 17 studies, participants were predominantly preterm (or very low-birth-weight) infants. Two studies also included terms infants.^25,38 Three studies did not report the gestational age of the included infants, but it is likely that most participants were preterm or low-birth-weight infants.^18,24,41 Of those studies that reported the gestational age at birth of participants, the mean or median gestational age was younger than 30 weeks in 10 studies, 30 to 32 weeks in 6 studies, and older than 32 weeks in 3 studies (not reported in 3 studies). Birth weight data were not reliably reported in the included studies. Age of the infants at the time of enrollment was also reported poorly and appeared to occur either within 48 to 72 hours after birth or at a later time within the neonatal period (28 days).

Sixteen studies used a prespecified serum CRP level to determine the threshold level (cutoff) for a positive test. Those thresholds ranged from 1 to 12 mg/L, with most studies (16) using a cutoff CRP level between 5 and 10 mg/L. None of the studies reported sensitivity and specificity at multiple thresholds. Six studies determined the CRP threshold level retrospectively by modeling the area under the ROC curve.^{24,26,30,34,36,38} These studies determined thresholds ranging from 2.2 to 111 mg/L.

At the reported median specificity (0.74), sensitivity was 0.62 (95% CI, 0.50-0.72); at the reported lower quartile specificity (0.61), sensitivity was 0.76 (95% CI, 0.66-0.83); at the reported upper quartile specificity (0.84), sensitivity was 0.45 (95% CI, 0.34-0.57). See Figure 3 for the summary ROC curve and Figure 4 for forest plots. As is common with meta-analyses of diagnostic accuracy studies, there was substantial heterogeneity among the studies, with sensitivity and specificity estimates varying widely among them.

We used these data for sensitivity (0.62) and reported median specificity (0.74) to estimate posttest probabilities after a “positive” or “negative” CRP test result for a range of pretest probabilities in infants being evaluated for possible late-onset infection and receiving a CRP test (eTable in the Supplement).

The prevalence rates of late-onset infection in the included studies ranged from 20% to 82% (median, 40%; interquartile range, 27%-61%). We applied the diagnostic test accuracy estimates for sensitivity (0.62) and median specificity (0.74) from our meta-analysis to a hypothetical cohort of 1000 neonates with a prevalence rate of infection of 20% (resulting in a median of 76 cases of infection being missed and 208 cases being wrongly diagnosed as having infection), 40% (152 cases of infection would be missed, and 156 would be wrongly diagnosed as having infection), or 60% (228 cases would be missed, and 104 would be wrongly diagnosed as having infection). Adding serum CRP level to the assessment of an infant with a 40% pretest probability of late onset infection generated posttest probabilities of 26% for a negative test result and 61% for a positive test result.

Bivariate meta-analysis of studies using a threshold of 5 to 10 mg/L found similar estimates of sensitivity (0.61; 95% CI, 0.49-0.72) and specificity (0.73; 95% CI, 0.64-0.80) as the main analyses, including all thresholds. See eFigure 1 in the Supplement for the summary ROC curve.

When covariates with thresholds above or below 5 to 10 mg/L were added, likelihood ratio tests found no statistically significant difference in goodness of fit for any of these models compared with those without covariates.

Six studies did not report using a predefined threshold. There were no statistically significant differences in goodness of fit between any of these models, including a covariate for predefined threshold compared with models without covariates.

One study was an outlier and reported using a cutoff CRP level of 111 mg/L.³⁶ Removing the study did not change effect estimates; at median specificity (0.73), sensitivity was 0.63 (95% CI, 0.52-0.73). We conducted a sensitivity analysis to explore the diagnostic performance of CRP in studies that included only preterm infants. That analysis is presented in eFigure 2 in the Supplement. Published reports of the included studies did not provide sufficient detail to enable us to conduct any of the planned metaregression analyses.

Visual assessment of the funnel plot did not identify important asymmetry and the Deeks test was not statistically significant (eFigure 3 in the Supplement). However, this finding does not rule out publication bias because the Deeks test lacks power, particularly in the presence of high heterogeneity.

This systematic review and meta-analysis suggests that the serum CRP level at initial evaluation of an infant with suspected late-onset infection is unlikely to aid diagnosis or help triage infants for further investigation or treatment. Using the pooled estimate of sensitivity (0.62) at median specificity (0.74) and assuming a 40% prevalence rate of true infection (the median for the included studies) in a hypothetical cohort of 1000 newborn infants, assessing serum CRP level alone would miss 152 cases of infection (false-negative results) and wrongly diagnose 156 cases (false-positive results). For an individual infant with a 40% pretest probability of late-onset infection, adding the serum CRP level to the assessment would generate a posttest probability of 26% for a negative test (does not “rule out” infection) and a posttest probability of 61% for a positive test (does not “rule in” infection).

These findings are similar to those in a systematic review and meta-analysis examining the accuracy of serum CRP levels for diagnosing serious infection in children (aged 1 month to 18 years) with febrile illness.⁴³ The possible explanations for the suboptimal diagnostic accuracy include the potential for false-positive results if CRP levels are increased by triggers, such as inflammation due to extravasation, cholestasis, or gastrointestinal pathology.⁴⁴ Conversely, serum CRP levels may not increase, or increase only slowly, in some infants with infection, particularly very preterm infants with coagulase-negative staphylococcal bacteremia.^13,45

We used methods to reduce reviewer error and bias, including independent and duplicate study selection as well as double-checking data extraction and risk of bias assessments. We assessed the included studies to be at low risk of bias but some use of the “unclear” category was unavoidable owing to missing detail in study reports. To ensure our analyses drew on the most complete evidence base possible, we complemented our comprehensive literature searches with a pragmatic but thorough approach to contacting study authors. This enabled us to use unpublished data and clarify important method questions for several of the included studies. We excluded case-control studies because this design is unlikely to allow for valid assessment of diagnostic test accuracy in this clinical context. We also excluded studies at high risk of incorporation bias (those in which the serum CRP level was part of the reference standard) because these studies overestimate test performance. The only “high” risk of bias was identified in a retrospective study in which the result of the reference standard was known before the index test was performed (although the laboratory test result was not likely to have been affected by this knowledge).

Heterogeneity in the estimates of sensitivity and specificity was evident on forest plot inspection, a well-recognized feature in reviews of diagnostic test accuracy.^46,47 Although variations in case definition may have contributed to differences in the rates of confirmed infection in studied cohorts, it was likely that between-study differences in thresholds for investigating suspected infection were also important factors. Other factors might have been associated with the observed heterogeneity in findings, such as gestational age, blood sampling techniques, and methods used to determine the serum CRP level. However, the included studies provided insufficient details to explore the potential association of these factors.

The serum CRP cutoff level for a “positive” test used in the included studies was typically between 5 and 10 mg/L, consistent with current use in clinical practice. In most studies, the threshold was predefined, and the estimates of test performance were based on this cutoff for a positive test. Six studies did not predefine a cutoff for positivity. Five of these studies calculated levels between 2.2 and 18 mg/L (with 2 studies finding optimal cutoffs between 5 and 10 mg/L). One study was an outlier with an optimal cutoff of 111 mg/L.³⁶ Neither the published study nor any unpublished data we received from the authors explained this unexpectedly high threshold. However, in a sensitivity analysis conducted without this study, there was no change in the estimates of sensitivity at the median, upper, or lower quartiles of specificity reported in the included studies.

The reference standard was microbiologically confirmed late-onset infection (more than 72 hours after birth) including bacteremia, fungemia, meningitis, osteomyelitis, septic arthritis, and peritonitis. There are concerns about how fully this reference standard defines all infants who truly have late-onset infection. Microbiological cultures may not detect cases of bacteremia or fungemia if an insufficient volume of the infant's blood is incubated. Conversely, microbiological cultures may also generate false-positive results if blood sampling techniques allow entry of contaminating microorganisms (typically coagulase-negative staphylococci from the infant’s skin).⁴⁸ Insufficient data were available to undertake a subgroup analysis of infection with coagulase-negative staphylococci vs other bacteria to explore whether test accuracy was associated with the likelihood of identified microorganisms representing true bloodstream infections. Any such analysis, however, may be confounded by between-species differences in the capacity of microorganisms to trigger inflammatory cascades and in the generation of CRP.

We accepted the primary study authors’ definitions of late-onset infection based on the infant’s age at assessment. In the studies that provided this information (13 of 22), definitions ranged from 48 hours (3 studies) to 6 days after birth. The 3 studies that used the earlier onset definition contributed 158 of 2255 infants to our meta-analyses. Although using this earlier onset deviated from our proposed definition of more than 72 hours after birth, we adopted this broader definition to maximize available data and to reflect the variation in definitions of late onset that exists in clinical practice.⁴⁹

Most included studies have assessed the accuracy of elevated serum CRP levels for diagnosing late-onset infection in preterm infants in neonatal units of high- or middle-income countries. Although these data are likely, therefore, to be applicable to preterm infants cared for in modern neonatal units in high- and (some) middle-income countries, the present review findings are less likely to be generalized to resource-limited settings in low- or middle-income countries in which the epidemiology, microbiology, pathogenesis, treatment options, and outcomes for late-onset infection in newborn infants differ.^50,51

Our findings are specific to the accuracy of the serum CRP level in determining whether infection is less or more likely among infants in whom there is a suspicion based on clinical signs (ie, for diagnosing infection). The present review does not address whether serial monitoring of the serum CRP level may be useful in screening well neonates for infection before it is suspected clinically or in assessing the response to treatment, including helping to decide whether to stop antibiotics or rule out infection when CRP levels fail to increase.⁵² C-reactive protein level is frequently used in these contexts in current clinical practice; but as detection of an increased serum CRP level (index test) would be used to trigger application of the reference test, it is not possible to measure diagnostic accuracy. Rather, intervention studies should address this separate question and evaluate the diagnostic utility of using the serum CRP level to inform clinical decisions in these contexts.

Similarly, in clinical practice, CRP may not be the only biomarker that is measured in newborn infants with suspected sepsis. Our review investigated only the diagnostic test accuracy of CRP alone, not of CRP when used in conjunction with other tests.

The timely diagnosis of late-onset infection on the basis of clinical features and signs in newborn infants, particularly very preterm infants, remains challenging.⁵³ Given the poor performance of serum CRP levels in this context, research efforts might focus on other serum biomarkers, such as procalcitonin, that are increased more quickly in response to infection or inflammation.¹³ Newer methods using molecular markers to identify pathogenic microorganisms (such as real-time polymerase chain reaction or microarray techniques) are worthy of further research. Those new techniques can provide results more quickly than standard microbiological culture (6-8 hours vs 24-36 hours), and evidence of their diagnostic accuracy is accumulating.¹⁵

Quick and accurate diagnosis of late-onset infection in newborn infants remains an important goal in clinical practice worldwide. Results from our systematic review and meta-analysis that included data from 2255 infants (22 studies) do not support the use of serum CRP level in this context. Although quick, the use of serum CRP level does not appear to be sufficiently accurate to aid in a diagnosis that is based on clinical features or to support treatment decisions with the aim of avoiding unnecessary administration of antibiotics.

Accepted for Publication: September 25, 2019.

Published Online: February 3, 2020. doi:10.1001/jamapediatrics.2019.5669

Correction: This article was corrected on May 4, 2020, to fix typographical errors in the text and add an omitted reference to a previously published meta-analysis.

Corresponding Author: Jennifer Valeska Elli Brown, MSc, Centre for Reviews and Dissemination, University of York, York YO10 5DD, United Kingdom (jennifer.brown@york.ac.uk).

Author Contributions: Ms Brown and Dr McGuire had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Brown, Meader, McGuire.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Brown, Cleminson, McGuire.

Critical revision of the manuscript for important intellectual content: Brown, Meader, Wright, McGuire.

Statistical analysis: Meader, Cleminson, McGuire.

Obtained funding: McGuire.

Administrative, technical, or material support: Wright, Cleminson, McGuire.

Supervision: McGuire.

Conflict of Interest Disclosures: Dr McGuire reported receiving grants from National Institute for Health Research (United Kingdom) during the conduct of the study. No other disclosures were reported.

Funding/Support: This study was funded by Cochrane Programme grant 16/114/03 from the National Institute for Health Research.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The opinions, results, and conclusions reported in this article are those of the authors and are independent from the funding source.

Additional Contributions: We thank the authors of the primary studies who supplied unpublished information or data and the National Institute for Health Research Complex Reviews Support Unit for helpful method input and guidance.