[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.
In-Hospital Mortality Rate Based on the Maximum Pediatric Sequential Organ Failure Assessment (pSOFA) Score
In-Hospital Mortality Rate Based on the Maximum Pediatric Sequential Organ Failure Assessment (pSOFA) Score

Maximum pSOFA score was the highest daily pSOFA score achieved by day 28 after pediatric intensive care unit admission, discharge, or death (whichever came first). Error bars represent 95% CIs.

Table 1.  
Pediatric Sequential Organ Failure Assessment Score
Pediatric Sequential Organ Failure Assessment Score
Table 2.  
Demographic and Clinical Characteristics of Survivors and Nonsurvivors
Demographic and Clinical Characteristics of Survivors and Nonsurvivors
Table 3.  
Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality
Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality
Table 4.  
Assessment of the Sepsis-3 Definitions in Critically Ill Children With Confirmed or Suspected Infection
Assessment of the Sepsis-3 Definitions in Critically Ill Children With Confirmed or Suspected Infection
1.
Singer  M, Deutschman  CS, Seymour  CW,  et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).  JAMA. 2016;315(8):801-810.PubMedGoogle ScholarCrossref
2.
Vincent  JL, Moreno  R, Takala  J,  et al; Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine.  The SOFA (Sepsis-Related Organ Failure Assessment) score to describe organ dysfunction/failure.  Intensive Care Med. 1996;22(7):707-710.PubMedGoogle ScholarCrossref
3.
Leteurtre  S, Martinot  A, Duhamel  A,  et al.  Validation of the Paediatric Logistic Organ Dysfunction (PELOD) score: prospective, observational, multicentre study.  Lancet. 2003;362(9379):192-197.PubMedGoogle ScholarCrossref
4.
Leteurtre  S, Duhamel  A, Salleron  J, Grandbastien  B, Lacroix  J, Leclerc  F; Groupe Francophone de Réanimation et d’Urgences Pédiatriques (GFRUP).  PELOD-2: an update of the Pediatric Logistic Organ Dysfunction score.  Crit Care Med. 2013;41(7):1761-1773.PubMedGoogle ScholarCrossref
5.
Graciano  AL, Balko  JA, Rahn  DS, Ahmad  N, Giroir  BP.  The Pediatric Multiple Organ Dysfunction Score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children.  Crit Care Med. 2005;33(7):1484-1491.PubMedGoogle ScholarCrossref
6.
Shime  N, Kageyama  K, Ashida  H, Tanaka  Y.  Application of modified sequential organ failure assessment score in children after cardiac surgery.  J Cardiothorac Vasc Anesth. 2001;15(4):463-468.PubMedGoogle ScholarCrossref
7.
Jhang  WK, Kim  YA, Ha  EJ,  et al.  Extrarenal sequential organ failure assessment score as an outcome predictor of critically ill children on continuous renal replacement therapy.  Pediatr Nephrol. 2014;29(6):1089-1095.PubMedGoogle ScholarCrossref
8.
Sanchez-Pinto  LN, Khemani  RG.  Development of a prediction model of early acute kidney injury in critically ill children using electronic health record data.  Pediatr Crit Care Med. 2016;17(6):508-515.PubMedGoogle ScholarCrossref
9.
Hassinger  AB, Garimella  S, Wrotniak  BH, Freudenheim  JL.  The current state of the diagnosis and management of acute kidney injury by pediatric critical care physicians.  Pediatr Crit Care Med. 2016;17(8):e362-e370.PubMedGoogle ScholarCrossref
10.
Sanchez-Pinto  LN, Goldstein  SL, Schneider  JB, Khemani  RG.  Association between progression and improvement of acute kidney injury and mortality in critically ill children.  Pediatr Crit Care Med. 2015;16(8):703-710.PubMedGoogle ScholarCrossref
11.
Khemani  RG, Smith  LS, Zimmerman  JJ, Erickson  S; Pediatric Acute Lung Injury Consensus Conference Group.  Pediatric acute respiratory distress syndrome: definition, incidence, and epidemiology: proceedings from the Pediatric Acute Lung Injury Consensus Conference.  Pediatr Crit Care Med. 2015;16(5)(suppl 1):S23-S40.PubMedGoogle ScholarCrossref
12.
Khemani  RG, Thomas  NJ, Venkatachalam  V,  et al; Pediatric Acute Lung Injury and Sepsis Network Investigators (PALISI).  Comparison of Spo2 to Pao2 based markers of lung disease severity for children with acute lung injury.  Crit Care Med. 2012;40(4):1309-1316.PubMedGoogle ScholarCrossref
13.
Leteurtre  S, Dupré  M, Dorkenoo  A, Lampin  ME, Leclerc  F.  Assessment of the pediatric index of mortality 2 with the Pao2/Fio2 ratio derived from the Spo2/Fio2 ratio: a prospective pilot study in a French pediatric intensive care unit.  Pediatr Crit Care Med. 2011;12(4):e184-e186.PubMedGoogle ScholarCrossref
14.
Shime  N, Kawasaki  T, Nakagawa  S.  Proposal of a new pediatric sequential organ failure assessment score for possible validation.  Pediatr Crit Care Med. 2017;18(1):98-99.PubMedGoogle ScholarCrossref
15.
Schwartz  GJ, Brion  LP, Spitzer  A.  The use of plasma creatinine concentration for estimating glomerular filtration rate in infants, children, and adolescents.  Pediatr Clin North Am. 1987;34(3):571-590.PubMedGoogle ScholarCrossref
16.
Reilly  PL, Simpson  DA, Sprod  R, Thomas  L.  Assessing the conscious level in infants and young children: a paediatric version of the Glasgow Coma Scale.  Childs Nerv Syst. 1988;4(1):30-33.PubMedGoogle Scholar
17.
Pollack  MM, Patel  KM, Ruttimann  UE.  PRISM III: an updated Pediatric Risk of Mortality score.  Crit Care Med. 1996;24(5):743-752.PubMedGoogle ScholarCrossref
18.
Seymour  CW, Liu  VX, Iwashyna  TJ,  et al.  Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).  JAMA. 2016;315(8):762-774.PubMedGoogle ScholarCrossref
19.
DeLong  ER, DeLong  DM, Clarke-Pearson  DL.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.  Biometrics. 1988;44(3):837-845.PubMedGoogle ScholarCrossref
20.
Pencina  MJ, D’Agostino  RB  Sr, D’Agostino  RB  Jr, Vasan  RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.  Stat Med. 2008;27(2):157-172; discussion 207-212.PubMedGoogle ScholarCrossref
21.
Fluss  R, Faraggi  D, Reiser  B.  Estimation of the Youden Index and its associated cutoff point.  Biom J. 2005;47(4):458-472.PubMedGoogle ScholarCrossref
22.
Moons  KG, Altman  DG, Reitsma  JB,  et al.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.  Ann Intern Med. 2015;162(1):W1-W73.PubMedGoogle ScholarCrossref
23.
Hotchkiss  RS, Monneret  G, Payen  D.  Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy.  Nat Rev Immunol. 2013;13(12):862-874.PubMedGoogle ScholarCrossref
24.
Wong  HR, Cvijanovich  NZ, Anas  N,  et al.  Developing a clinically feasible personalized medicine approach to pediatric septic shock.  Am J Respir Crit Care Med. 2015;191(3):309-315.PubMedGoogle ScholarCrossref
25.
Ferreira  FL, Bota  DP, Bross  A, Mélot  C, Vincent  J-L.  Serial evaluation of the SOFA score to predict outcome in critically ill patients.  JAMA. 2001;286(14):1754-1758.PubMedGoogle ScholarCrossref
26.
Weiss  SL, Fitzgerald  JC, Maffei  FA,  et al; SPROUT Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators Network.  Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study.  Crit Care. 2015;19(1):325.PubMedGoogle ScholarCrossref
27.
Maziarz  M, Heagerty  P, Cai  T, Zheng  Y.  On longitudinal prediction with time-to-event outcome: comparison of modeling options.  Biometrics. 2017;73(1):83-93.PubMedGoogle ScholarCrossref
28.
Liu  D, Albert  PS.  Combination of longitudinal biomarkers in predicting binary events.  Biostatistics. 2014;15(4):706-718.PubMedGoogle ScholarCrossref
Original Investigation
Caring for the Critically Ill Patient
October 2, 2017

Adaptation and Validation of a Pediatric Sequential Organ Failure Assessment Score and Evaluation of the Sepsis-3 Definitions in Critically Ill Children

Author Affiliations
  • 1Section of Critical Care, Department of Pediatrics, The University of Chicago, Chicago, Illinois
  • 2Currently with Division of Critical Care Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, Chicago Illinois
JAMA Pediatr. 2017;171(10):e172352. doi:10.1001/jamapediatrics.2017.2352
Key Points

Questions  Is a pediatric version of the Sequential Organ Failure Assessment score valid, and can it be used to evaluate the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) in critically ill children?

Findings  In this large cohort study of 8711 pediatric intensive care unit encounters, the pediatric Sequential Organ Failure Assessment score demonstrated excellent discrimination for in-hospital mortality, and the Sepsis-3 definitions identified a cohort of patients with high mortality and microbiological characteristics associated with severe sepsis in prior studies.

Meaning  The evaluation of the Sepsis-3 definitions in children using the pediatric Sequential Organ Failure Assessment score shows promising results.

Abstract

Importance  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) uses the Sequential Organ Failure Assessment (SOFA) score to grade organ dysfunction in adult patients with suspected infection. However, the SOFA score is not adjusted for age and therefore not suitable for children.

Objectives  To adapt and validate a pediatric version of the SOFA score (pSOFA) in critically ill children and to evaluate the Sepsis-3 definitions in patients with confirmed or suspected infection.

Design, Setting, and Participants  This retrospective observational cohort study included all critically ill children 21 years or younger admitted to a 20-bed, multidisciplinary, tertiary pediatric intensive care unit between January 1, 2009 and August 1, 2016. Data on these children were obtained from an electronic health record database. The pSOFA score was developed by adapting the original SOFA score with age-adjusted cutoffs for the cardiovascular and renal systems and by expanding the respiratory criteria to include noninvasive surrogates of lung injury. Daily pSOFA scores were calculated from admission until day 28 of hospitalization, discharge, or death (whichever came first). Three additional pediatric organ dysfunction scores were calculated for comparison.

Exposures  Organ dysfunction measured by the pSOFA score, and sepsis and septic shock according to the Sepsis-3 definitions.

Main Outcomes and Measures  The primary outcome was in-hospital mortality. The daily pSOFA scores and additional pediatric organ dysfunction scores were compared. Performance was evaluated using the area under the curve. The pSOFA score was then used to assess the Sepsis-3 definitions in the subgroup of children with confirmed or suspected infection.

Results  In all, 6303 patients with 8711 encounters met inclusion criteria. Each encounter was treated independently. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. In-hospital mortality was 2.6%. The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an area under the curve of 0.94 (95% CI, 0.92-0.95). The pSOFA score had a similar or better performance than other pediatric organ dysfunction scores. According to the Sepsis-3 definitions, 1231 patients (14.1%) were classified as having sepsis and had a mortality rate of 12.1%, and 347 (4.0%) had septic shock and a mortality rate of 32.3%. Patients with sepsis were more likely to die than patients with confirmed or suspected infection but no sepsis (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.

Conclusions and Relevance  The pSOFA score was adapted and validated with age-adjusted variables in critically ill children. Using the pSOFA score, the Sepsis-3 definitions were assessed in children with confirmed or suspected infection. This study is the first assessment, to date, of the Sepsis-3 definitions in critically ill children. Use of these definitions in children is feasible and shows promising results.

Introduction

The Sequential Organ Failure Assessment (SOFA) score was selected as the scoring system to quantify organ dysfunction in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).1 The Sepsis-3 Task Force validated the SOFA score in adult patients with suspected infection and found the SOFA system to be either comparable or superior to other scoring systems at discriminating in-hospital mortality.1 The Sepsis-3 definitions are expected to be widely adopted and, by extension, the use of SOFA score in patients with confirmed or suspected infection. One of the major limitations of the SOFA score is that it was developed for adult patients and contains measures that vary significantly with age, which makes it unsuitable for children.2 The Sepsis-3 Task Force recognized this problem and identified it as an area for further development.1

Several pediatric organ dysfunction scores take into account the age dependency of their variables, including the Pediatric Logistic Organ Dysfunction (PELOD) score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3-5 Use of any of these scores as a measure of organ dysfunction in infected children could be considered for adapting the Sepsis-3 definitions to pediatric patients, but the range, scale, and coverage of these scores are significantly different from those of the SOFA score, which makes their concurrent use problematic. Fundamentally, having different definitions of sepsis for patients above or below the pediatric-adult threshold has no known physiologic justification and should therefore be avoided.

Prior studies have attempted to adapt the SOFA score to pediatric patients, mostly focusing on the cardiovascular subscore.6,7 However, none have taken into account the age-related variability of the renal subscore criteria despite the increasingly recognized detrimental effect of kidney dysfunction in younger patients.8-10 In addition, the respiratory subscore criteria—based on the ratio of Pao2 to the fraction of inspired oxygen (Fio2)—have not been modified in previous adaptations of the SOFA score even though the decreased use of arterial blood gases in children is a known limitation.11-13 Fortunately, the cardiovascular and renal components of the SOFA score were evaluated and adapted to pediatric patients by the PELOD-2 score investigators, and the ratio of peripheral oxygen saturation (Spo2) to Fio2 has been validated as an alternative to the Pao2:Fio2 ratio in children.4,12

In this study, we sought to adapt and validate a SOFA score for critically ill pediatric patients (pSOFA) using age-adjusted criteria. In addition, we sought to assess the Sepsis-3 definitions for sepsis and septic shock in the subgroup of critically ill children with confirmed or suspected infection using the pSOFA score.

Methods
Patients and Data Collection

We performed a single-center, retrospective cohort study of critically ill children presenting to a multidisciplinary, tertiary pediatric intensive care unit (PICU). This 20-bed PICU serves a mixed population of medical, surgical, and trauma patients. We included all patients 21 years or younger on admission and who received care in the PICU between January 1, 2009, and August 1, 2016. Each hospitalization with a PICU admission was treated independently. Data were extracted from an electronic health record database (Epic; Epic Systems Corporation). The institutional review board of The University of Chicago approved this study and waived patient informed consent because of the observational nature of the study.

Development of the pSOFA Score

The pSOFA score was developed by adapting the original SOFA score through 2 approaches. First, the age-dependent cardiovascular and renal variables of the original SOFA score were modified using validated cutoffs from the PELOD-2 scoring system.4 Second, the respiratory subscore was expanded to include the Spo2:Fio2 ratio as an alternative surrogate of lung injury (Table 1).12

Cardiovascular Subscore

The age-adjusted mean arterial pressure cutoffs for the first score of the PELOD-2 cardiovascular criteria were used to assign a score of 1 in the pSOFA subscore. Scores 2 to 4 were kept identical to the original SOFA criteria.

Renal Subscore

The age-adjusted serum creatinine level cutoffs for the first score of the PELOD-2 renal criteria were used to assign a score of 1 in the pSOFA renal subscore. Scores 2 to 4 were modified by increasing the cutoff values for each score by the same factor as the original SOFA criteria, similar to the approach proposed by other authors.14 Exceptions to this approach were the cutoff values for the first age group (<1 month) owing to the renal physiologic differences of neonates. For this neonatal age group, the cutoff value increase for each score was done by the same amount as the infant group (1-12 months) given the similarity in the glomerular filtration rate in both age groups.15

Respiratory Subscore

The original Pao2:Fio2 ratio cutoffs were kept identical to the original score, but the Spo2:Fio2 ratio was used as an alternative surrogate of lung injury. The adaptation proposed by Khemani and colleagues12 was used to define the Spo2:Fio2 ratio cutoffs.

Coagulation, Hepatic, and Neurologic Subscores

The original coagulation and hepatic criteria, based on platelet count and bilirubin level, were kept identical to the original score. The Glasgow Coma Scale criteria for the neurologic subscore were also kept identical to the original score, but the pediatric version of the scale was used.16

The calculation of the pSOFA score was performed in the same way as the calculation of the original SOFA score.2 The worst variable in each 24-hour period was used to assign a subscore for each system (ranging from 0-4 points). The sum of the 6 subscores in each 24-hour period resulted in a daily pSOFA score (ranging from 0-24 points; higher scores indicate a worse outcome). If a variable was not measured in a 24-hour period, it was considered to be normal, which is consistent with the original criteria.

To compare the performance of the adapted variables in pSOFA with those in the original SOFA score, we calculated the maximum individual subscores for the cardiovascular, renal, and respiratory components using both the original and the adapted criteria.

Comparison of pSOFA With Other Organ Dysfunction Scores

We compared the performance of the pSOFA score with 3 other pediatric organ dysfunction scores—the PELOD score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3-5 We calculated the daily score of the 4 scoring systems for each 24-hour period from PICU admission until day 28 of hospitalization, discharge, or death, whichever came first. The maximum and mean scores for each scoring system were used to compare the scores and evaluate the clinical validity of pSOFA. To evaluate the clinical utility of pSOFA on admission, we compared it with the Pediatric Risk of Mortality (PRISM) III score, a marker of severity of illness on admission, using information from the first 24 hours.17 To further evaluate the clinical validity and utility of pSOFA in comparison to the other scoring systems, the maximum and mean scores were also calculated at 4 landmarked times—days 2, 4, 7, and 14 after PICU admission. Only patients still alive and hospitalized on landmarked days were used for calculations.

To evaluate the effect of patients who had more than 1 hospitalization with a PICU admission, a sensitivity analysis using only the first PICU admission for each patient was performed.

Sepsis-3 Definitions

We assessed the Sepsis-3 definitions in the subgroup of patients with confirmed or suspected infection by using previously published criteria.18 Patients with suspected infection were defined as those who had negative microbiological study results and initiation of treatment with nonprophylactic antibiotics, antifungals, or antiviral medications within 24 hours.18 Coagulase-negative Staphylococcus was only considered infectious if it was isolated in 2 or more cultures from sterile sites.

Patients with sepsis were defined as those with confirmed or suspected infection who had an acute rise in the pSOFA score of 2 points or more from up to 48 hours before the infection to 24 hours after the infection and who received antimicrobial therapy in the PICU.18Infection time was defined as the time when the first microbiological study or antimicrobial therapy was ordered by a physician, whichever came first. If the patients were not known to have previous organ dysfunction, the preinfection pSOFA score was assumed to be zero. Septic shock was defined as patients with sepsis who required a vasoactive infusion and had a maximum serum lactate level greater than 2 mmol/L (18 mg/dL).1 The microbiological etiology and infection source of patients with a confirmed or suspected infection were analyzed and compared between those who met sepsis criteria and those who did not.

Statistical Analysis

The primary outcome was in-hospital mortality. Data were analyzed using Stata, version 14 (StataCorp LLC), and R, version 3.2.2 (R Foundation for Statistical Computing). Categorical variables were compared using the χ2 test, and continuous variables were compared using the Mann-Whitney test. A 2-sided P < .05 was considered statistically significant.

The performance of the scores to discriminate in-hospital mortality was evaluated using the area under the curve (AUC). Comparison between scores was performed using the DeLong method19 to compare AUCs and the Integrated Discrimination Improvement Index20 to evaluate the reclassification of predicted probabilities between survivors and nonsurvivors. Youden J statistic21 was used to evaluate the optimal cutoff of the pSOFA score to discriminate mortality.

Reporting of this validation study was performed using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.22

Results

In all, 6303 patients with 8711 encounters met inclusion criteria. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. In-hospital mortality was 2.6%. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. The demographic and clinical characteristics of survivors and nonsurvivors are shown in Table 2. (Note that we treated each of the 8711 encounters independently given that the major risk factors for the outcome [ie, organ dysfunctions] and the outcome [ie, survival or nonsurvival at the end of the hospitalization, or in-hospital mortality] are contained within each encounter.)

Performance of pSOFA

Nonsurvivors had a significantly higher median (IQR) maximum pSOFA score than survivors (13 [10-16] vs 2 [1-5]; P < .001; Table 2). The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an AUC of 0.94 (95% CI, 0.92-0.95). The in-hospital mortality rate for patients according to their maximum pSOFA score are shown in the Figure. The performance of the maximum pSOFA score to discriminate in-hospital mortality remained stable across sex, age groups, and admission types (eTable 1 in the Supplement). The optimal pSOFA cutoff to discriminate mortality was a score higher than 8 points.

The performance of the maximum pSOFA cardiovascular, renal, and respiratory subscores at discriminating in-hospital mortality were better than the non–age-adjusted maximum SOFA cardiovascular (AUC, 0.87 vs 0.86; P < .001), renal (AUC, 0.76 vs 0.61; P < .001), and respiratory (AUC, 0.87 vs 0.75; P < .001) subscores. The reclassification of estimated probabilities based on the Integrated Discrimination Improvement Index for the respiratory and renal subscores was significant (total ≥2.6%; P < .001) but negligible for the cardiovascular subscore (eTable 2 in the Supplement).

Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM III

The performance of the maximum pSOFA score at discriminating in-hospital mortality (AUC, 0.94; 95% CI, 0.92-0.95) was similar to the performance of PELOD and PELOD-2 (AUC, 0.93 vs 0.94; 95% CI, 0.91-0.95 vs 0.92-0.95; P > .20) and better than the Pediatric Multiple Organ Dysfunction Score (AUC, 0.91; 95% CI, 0.88-0.93; P = .001). The performance of pSOFA on the day of admission at discriminating in-hospital mortality (AUC, 0.88; 95% CI, 0.86-0.91) was better than the other organ dysfunction scores (P ≤ .02) and similar to PRISM III (AUC, 0.88; 95% CI, 0.86-0.91; P = .94) (Table 3). On the landmarked day analyses, pSOFA performed slightly better on days 2 and 4 and performed similarly to the other scores on days 7 and 14 (Table 3). The Integrated Discrimination Improvement Index showed small differences in the reclassification of estimated probabilities (eTable 3 in the Supplement).

A sensitivity analysis using only the first PICU admission for each patient showed results similar to the complete cohort (eTable 4 in the Supplement).

Assessment of the Sepsis-3 Definitions in Patients With Confirmed or Suspected Infection

Of 8711 patient encounters, 4217 (48.4%) had a confirmed or suspected infection in the PICU. The maximum pSOFA score had excellent discrimination of in-hospital mortality in this subgroup of patients (AUC, 0.92; 95% CI, 0.91- 0.94). When we used the adapted Sepsis-3 definitions with the pSOFA score, 1231 patients (14.1% of the PICU population) met sepsis criteria and had a mortality rate of 12.1%. Of those, 347 patients (4.0% of the PICU population) also met septic shock criteria and had a mortality rate of 32.3%. Among patients with confirmed or suspected infection, being diagnosed with sepsis significantly increased the likelihood of dying in the hospital (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.

The demographic, clinical, and microbiological characteristics of patients with a confirmed or suspected infection with or without sepsis are shown in Table 4.

Discussion

We adapted and validated an age-adjusted version of the SOFA score for pediatric patients (pSOFA). The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population, which was comparable to or better than the performance of other common pediatric organ dysfunction scores. We then used the pSOFA score to perform the first assessment of Sepsis-3 in critically ill children.

Several motivations were behind the adaptation of the adult SOFA score and the Sepsis-3 definitions to children. Having a harmonized definition of sepsis across age groups while recognizing the importance of the age-based variation of its measures can have many benefits, including better design of clinical trials, improved accuracy of reported outcomes, and better translation of the research and clinical strategies in the management of sepsis. Furthermore, as we focus our efforts on uncovering the pathobiological basis of the different subtypes of sepsis,23,24 we should avoid being limited by differing definitions of the syndrome across artificial constructs, such as the pediatric-adult age threshold. In our study, we found that the optimal pSOFA score cutoff to discriminate mortality was identical to the cutoff found by Ferreira and colleagues25 in adult critically ill patients (ie, a SOFA score >8 points). This observation requires further validation; however, when measured with a comparable scoring system—and despite the differences in baseline mortality—the degree of organ dysfunction in adults and children seems to discriminate outcomes in a similar way.

Another motivation for the adaptation of Sepsis-3 to children is that the current definitions of sepsis in pediatrics are not without problems. Weiss and colleagues26 recently demonstrated a lack of agreement between the 2005 International Pediatric Sepsis Consensus Conference criteria and the treating physician’s diagnosis of severe sepsis in the Sepsis Prevalence, Outcomes, and Therapies (SPROUT) study. This discrepancy implies that the results from research using the 2005 International Pediatric Sepsis Consensus Conference criteria could lack generalizability to almost a third of PICU patients with sepsis. One of the goals of Sepsis-3 is to harmonize the definitions of sepsis and septic shock using readily available objective clinical data,18 and its adaptation to children may help balance the existing diagnostic discrepancies in pediatric patients.

The results of our assessment of the Sepsis-3 definitions in critically ill children with confirmed or suspected infection are encouraging. The Sepsis-3 definitions identified a group of patients among those with confirmed or suspected infection who were 18 times more likely to die in the hospital. Furthermore, patients with sepsis in our cohort had characteristics similar to the critically ill children who were diagnosed with severe sepsis by either the 2005 International Pediatric Sepsis Consensus Conference criteria or physician diagnosis in the SPROUT study.26 Patients with the “broader” diagnosis of severe sepsis in the SPROUT cohort had similar incidence of gram-positive (27% vs 28% in our population) and gram-negative (26% vs 28%) bacterial infections as well as viral (22% vs 24%) and fungal (12% vs 8%) infections. The infection sources of both cohorts also showed similarities in the respiratory tract (41% vs 39%), bloodstream (19% vs 20%), and genitourinary tract (4% vs 4%) infection sites. Furthermore, these similarities were all significantly different from the microbiological characteristics of patients with confirmed or suspected infection but no sepsis. This finding suggests that the Sepsis-3 definitions could help bridge the current diagnostic discrepancies in identifying children with severe sepsis. (Of note, the label severe sepsis was eliminated by the Sepsis-3 definitions in favor of sepsis, which is more commonly used in patient care.)1 One major difference with our cohort is that the incidence of severe sepsis in the SPROUT study was lower (10% vs 14%), and those patients had higher mortality (23% vs 12%).26 This difference suggests that our adaptation of the Sepsis-3 definitions with the pSOFA score likely captures a larger cohort of patients with infection and milder organ dysfunction. However, this group might still be a valuable population to consider in the context of sepsis, especially given the shared microbiological characteristics with the SPROUT cohort and a significant increase in the likelihood of dying when compared with infected patients with no sepsis. The mortality difference between our PICU population and the SPROUT cohort may also be attributable to the differences in the patient populations seen in PICUs outside the United States.26 This difference emphasizes the need for further validation of the Sepsis-3 definitions in critically ill children in other settings and populations.

Limitations

There are several limitations to our study. First, our results were generated using retrospective data from a single center. Validating pSOFA in a larger, multicenter sample of critically ill children is necessary to assess the generalizability of the score. The adaptation of the Sepsis-3 definitions to children with confirmed or suspected infection using the pSOFA score or similar approaches must be validated as well in other settings and populations. Furthermore, the utility of the Sepsis-3 definitions for identifying children with sepsis for enrollment in clinical trials, reporting outcomes, and providing clinical care must be carefully examined by the pediatric community. Second, we did not evaluate the performance of the pSOFA as a longitudinal biomarker. Studies of the longitudinal performance of pSOFA would be helpful in further assessing its clinical utility.27,28 We present what we believe to be promising results for the pSOFA score and the adapted Sepsis-3 definitions in a single cohort of critically ill children, but further research is warranted.

Conclusions

We adapted and validated the pSOFA score, an age-adjusted pediatric version of the adult SOFA score, and used it to assess the Sepsis-3 definitions in critically ill children. The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population and in the subgroup of patients with suspected or confirmed infection. Our assessment of the Sepsis-3 definitions in children showed promising results, but further validation in children in different settings and populations is warranted.

Back to top
Article Information

Corresponding Author: L. Nelson Sanchez-Pinto, MD, MBI, Division of Critical Care Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, 225 E Chicago Ave, Chicago, IL 60611 (lsanchezpinto@luriechildrens.org).

Accepted for Publication: June 8, 2017.

Published Online: August 7, 2017. doi:10.1001/jamapediatrics.2017.2352

Author Contributions: Drs Sanchez-Pinto and Matics had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: All authors.

Administrative, technical, or material support: Sanchez-Pinto.

Study supervision: Sanchez-Pinto.

Conflict of Interest Disclosures: None reported.

Additional Contributions: The Center for Research Informatics at the University of Chicago provided the raw electronic health record data for this study.

References
1.
Singer  M, Deutschman  CS, Seymour  CW,  et al.  The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).  JAMA. 2016;315(8):801-810.PubMedGoogle ScholarCrossref
2.
Vincent  JL, Moreno  R, Takala  J,  et al; Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine.  The SOFA (Sepsis-Related Organ Failure Assessment) score to describe organ dysfunction/failure.  Intensive Care Med. 1996;22(7):707-710.PubMedGoogle ScholarCrossref
3.
Leteurtre  S, Martinot  A, Duhamel  A,  et al.  Validation of the Paediatric Logistic Organ Dysfunction (PELOD) score: prospective, observational, multicentre study.  Lancet. 2003;362(9379):192-197.PubMedGoogle ScholarCrossref
4.
Leteurtre  S, Duhamel  A, Salleron  J, Grandbastien  B, Lacroix  J, Leclerc  F; Groupe Francophone de Réanimation et d’Urgences Pédiatriques (GFRUP).  PELOD-2: an update of the Pediatric Logistic Organ Dysfunction score.  Crit Care Med. 2013;41(7):1761-1773.PubMedGoogle ScholarCrossref
5.
Graciano  AL, Balko  JA, Rahn  DS, Ahmad  N, Giroir  BP.  The Pediatric Multiple Organ Dysfunction Score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children.  Crit Care Med. 2005;33(7):1484-1491.PubMedGoogle ScholarCrossref
6.
Shime  N, Kageyama  K, Ashida  H, Tanaka  Y.  Application of modified sequential organ failure assessment score in children after cardiac surgery.  J Cardiothorac Vasc Anesth. 2001;15(4):463-468.PubMedGoogle ScholarCrossref
7.
Jhang  WK, Kim  YA, Ha  EJ,  et al.  Extrarenal sequential organ failure assessment score as an outcome predictor of critically ill children on continuous renal replacement therapy.  Pediatr Nephrol. 2014;29(6):1089-1095.PubMedGoogle ScholarCrossref
8.
Sanchez-Pinto  LN, Khemani  RG.  Development of a prediction model of early acute kidney injury in critically ill children using electronic health record data.  Pediatr Crit Care Med. 2016;17(6):508-515.PubMedGoogle ScholarCrossref
9.
Hassinger  AB, Garimella  S, Wrotniak  BH, Freudenheim  JL.  The current state of the diagnosis and management of acute kidney injury by pediatric critical care physicians.  Pediatr Crit Care Med. 2016;17(8):e362-e370.PubMedGoogle ScholarCrossref
10.
Sanchez-Pinto  LN, Goldstein  SL, Schneider  JB, Khemani  RG.  Association between progression and improvement of acute kidney injury and mortality in critically ill children.  Pediatr Crit Care Med. 2015;16(8):703-710.PubMedGoogle ScholarCrossref
11.
Khemani  RG, Smith  LS, Zimmerman  JJ, Erickson  S; Pediatric Acute Lung Injury Consensus Conference Group.  Pediatric acute respiratory distress syndrome: definition, incidence, and epidemiology: proceedings from the Pediatric Acute Lung Injury Consensus Conference.  Pediatr Crit Care Med. 2015;16(5)(suppl 1):S23-S40.PubMedGoogle ScholarCrossref
12.
Khemani  RG, Thomas  NJ, Venkatachalam  V,  et al; Pediatric Acute Lung Injury and Sepsis Network Investigators (PALISI).  Comparison of Spo2 to Pao2 based markers of lung disease severity for children with acute lung injury.  Crit Care Med. 2012;40(4):1309-1316.PubMedGoogle ScholarCrossref
13.
Leteurtre  S, Dupré  M, Dorkenoo  A, Lampin  ME, Leclerc  F.  Assessment of the pediatric index of mortality 2 with the Pao2/Fio2 ratio derived from the Spo2/Fio2 ratio: a prospective pilot study in a French pediatric intensive care unit.  Pediatr Crit Care Med. 2011;12(4):e184-e186.PubMedGoogle ScholarCrossref
14.
Shime  N, Kawasaki  T, Nakagawa  S.  Proposal of a new pediatric sequential organ failure assessment score for possible validation.  Pediatr Crit Care Med. 2017;18(1):98-99.PubMedGoogle ScholarCrossref
15.
Schwartz  GJ, Brion  LP, Spitzer  A.  The use of plasma creatinine concentration for estimating glomerular filtration rate in infants, children, and adolescents.  Pediatr Clin North Am. 1987;34(3):571-590.PubMedGoogle ScholarCrossref
16.
Reilly  PL, Simpson  DA, Sprod  R, Thomas  L.  Assessing the conscious level in infants and young children: a paediatric version of the Glasgow Coma Scale.  Childs Nerv Syst. 1988;4(1):30-33.PubMedGoogle Scholar
17.
Pollack  MM, Patel  KM, Ruttimann  UE.  PRISM III: an updated Pediatric Risk of Mortality score.  Crit Care Med. 1996;24(5):743-752.PubMedGoogle ScholarCrossref
18.
Seymour  CW, Liu  VX, Iwashyna  TJ,  et al.  Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).  JAMA. 2016;315(8):762-774.PubMedGoogle ScholarCrossref
19.
DeLong  ER, DeLong  DM, Clarke-Pearson  DL.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.  Biometrics. 1988;44(3):837-845.PubMedGoogle ScholarCrossref
20.
Pencina  MJ, D’Agostino  RB  Sr, D’Agostino  RB  Jr, Vasan  RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.  Stat Med. 2008;27(2):157-172; discussion 207-212.PubMedGoogle ScholarCrossref
21.
Fluss  R, Faraggi  D, Reiser  B.  Estimation of the Youden Index and its associated cutoff point.  Biom J. 2005;47(4):458-472.PubMedGoogle ScholarCrossref
22.
Moons  KG, Altman  DG, Reitsma  JB,  et al.  Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration.  Ann Intern Med. 2015;162(1):W1-W73.PubMedGoogle ScholarCrossref
23.
Hotchkiss  RS, Monneret  G, Payen  D.  Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy.  Nat Rev Immunol. 2013;13(12):862-874.PubMedGoogle ScholarCrossref
24.
Wong  HR, Cvijanovich  NZ, Anas  N,  et al.  Developing a clinically feasible personalized medicine approach to pediatric septic shock.  Am J Respir Crit Care Med. 2015;191(3):309-315.PubMedGoogle ScholarCrossref
25.
Ferreira  FL, Bota  DP, Bross  A, Mélot  C, Vincent  J-L.  Serial evaluation of the SOFA score to predict outcome in critically ill patients.  JAMA. 2001;286(14):1754-1758.PubMedGoogle ScholarCrossref
26.
Weiss  SL, Fitzgerald  JC, Maffei  FA,  et al; SPROUT Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators Network.  Discordant identification of pediatric severe sepsis by research and clinical definitions in the SPROUT international point prevalence study.  Crit Care. 2015;19(1):325.PubMedGoogle ScholarCrossref
27.
Maziarz  M, Heagerty  P, Cai  T, Zheng  Y.  On longitudinal prediction with time-to-event outcome: comparison of modeling options.  Biometrics. 2017;73(1):83-93.PubMedGoogle ScholarCrossref
28.
Liu  D, Albert  PS.  Combination of longitudinal biomarkers in predicting binary events.  Biostatistics. 2014;15(4):706-718.PubMedGoogle ScholarCrossref
×