Maximum pSOFA score was the highest daily pSOFA score achieved by day 28 after pediatric intensive care unit admission, discharge, or death (whichever came first). Error bars represent 95% CIs.
eTable 1. Performance of the Maximum pSOFA Score at Discriminating In-Hospital Mortality Across Risk Groups
eTable 2. Integrated Discrimination Index (IDI) for the Respiratory, Cardiovascular and Renal Subscores of SOFA vs pSOFA at Discriminating In-Hospital Mortality
eTable 3. Integrated Discrimination Index (IDI) for pSOFA Against Other Scores at Discriminating In-Hospital Mortality
eTable 4. Sensitivity Analysis: Comparison of pSOFA With Other Pediatric Organ Dysfunction Scores and PRISM-III at Discriminating In-Hospital Mortality for First PICU Admission Only
Customize your JAMA Network experience by selecting one or more topics from the list below.
Matics TJ, Sanchez-Pinto LN. Adaptation and Validation of a Pediatric Sequential Organ Failure Assessment Score and Evaluation of the Sepsis-3 Definitions in Critically Ill Children. JAMA Pediatr. 2017;171(10):e172352. doi:10.1001/jamapediatrics.2017.2352
Is a pediatric version of the Sequential Organ Failure Assessment score valid, and can it be used to evaluate the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) in critically ill children?
In this large cohort study of 8711 pediatric intensive care unit encounters, the pediatric Sequential Organ Failure Assessment score demonstrated excellent discrimination for in-hospital mortality, and the Sepsis-3 definitions identified a cohort of patients with high mortality and microbiological characteristics associated with severe sepsis in prior studies.
The evaluation of the Sepsis-3 definitions in children using the pediatric Sequential Organ Failure Assessment score shows promising results.
The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) uses the Sequential Organ Failure Assessment (SOFA) score to grade organ dysfunction in adult patients with suspected infection. However, the SOFA score is not adjusted for age and therefore not suitable for children.
To adapt and validate a pediatric version of the SOFA score (pSOFA) in critically ill children and to evaluate the Sepsis-3 definitions in patients with confirmed or suspected infection.
Design, Setting, and Participants
This retrospective observational cohort study included all critically ill children 21 years or younger admitted to a 20-bed, multidisciplinary, tertiary pediatric intensive care unit between January 1, 2009 and August 1, 2016. Data on these children were obtained from an electronic health record database. The pSOFA score was developed by adapting the original SOFA score with age-adjusted cutoffs for the cardiovascular and renal systems and by expanding the respiratory criteria to include noninvasive surrogates of lung injury. Daily pSOFA scores were calculated from admission until day 28 of hospitalization, discharge, or death (whichever came first). Three additional pediatric organ dysfunction scores were calculated for comparison.
Organ dysfunction measured by the pSOFA score, and sepsis and septic shock according to the Sepsis-3 definitions.
Main Outcomes and Measures
The primary outcome was in-hospital mortality. The daily pSOFA scores and additional pediatric organ dysfunction scores were compared. Performance was evaluated using the area under the curve. The pSOFA score was then used to assess the Sepsis-3 definitions in the subgroup of children with confirmed or suspected infection.
In all, 6303 patients with 8711 encounters met inclusion criteria. Each encounter was treated independently. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. In-hospital mortality was 2.6%. The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an area under the curve of 0.94 (95% CI, 0.92-0.95). The pSOFA score had a similar or better performance than other pediatric organ dysfunction scores. According to the Sepsis-3 definitions, 1231 patients (14.1%) were classified as having sepsis and had a mortality rate of 12.1%, and 347 (4.0%) had septic shock and a mortality rate of 32.3%. Patients with sepsis were more likely to die than patients with confirmed or suspected infection but no sepsis (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.
Conclusions and Relevance
The pSOFA score was adapted and validated with age-adjusted variables in critically ill children. Using the pSOFA score, the Sepsis-3 definitions were assessed in children with confirmed or suspected infection. This study is the first assessment, to date, of the Sepsis-3 definitions in critically ill children. Use of these definitions in children is feasible and shows promising results.
The Sequential Organ Failure Assessment (SOFA) score was selected as the scoring system to quantify organ dysfunction in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).1 The Sepsis-3 Task Force validated the SOFA score in adult patients with suspected infection and found the SOFA system to be either comparable or superior to other scoring systems at discriminating in-hospital mortality.1 The Sepsis-3 definitions are expected to be widely adopted and, by extension, the use of SOFA score in patients with confirmed or suspected infection. One of the major limitations of the SOFA score is that it was developed for adult patients and contains measures that vary significantly with age, which makes it unsuitable for children.2 The Sepsis-3 Task Force recognized this problem and identified it as an area for further development.1
Several pediatric organ dysfunction scores take into account the age dependency of their variables, including the Pediatric Logistic Organ Dysfunction (PELOD) score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3-5 Use of any of these scores as a measure of organ dysfunction in infected children could be considered for adapting the Sepsis-3 definitions to pediatric patients, but the range, scale, and coverage of these scores are significantly different from those of the SOFA score, which makes their concurrent use problematic. Fundamentally, having different definitions of sepsis for patients above or below the pediatric-adult threshold has no known physiologic justification and should therefore be avoided.
Prior studies have attempted to adapt the SOFA score to pediatric patients, mostly focusing on the cardiovascular subscore.6,7 However, none have taken into account the age-related variability of the renal subscore criteria despite the increasingly recognized detrimental effect of kidney dysfunction in younger patients.8-10 In addition, the respiratory subscore criteria—based on the ratio of Pao2 to the fraction of inspired oxygen (Fio2)—have not been modified in previous adaptations of the SOFA score even though the decreased use of arterial blood gases in children is a known limitation.11-13 Fortunately, the cardiovascular and renal components of the SOFA score were evaluated and adapted to pediatric patients by the PELOD-2 score investigators, and the ratio of peripheral oxygen saturation (Spo2) to Fio2 has been validated as an alternative to the Pao2:Fio2 ratio in children.4,12
In this study, we sought to adapt and validate a SOFA score for critically ill pediatric patients (pSOFA) using age-adjusted criteria. In addition, we sought to assess the Sepsis-3 definitions for sepsis and septic shock in the subgroup of critically ill children with confirmed or suspected infection using the pSOFA score.
We performed a single-center, retrospective cohort study of critically ill children presenting to a multidisciplinary, tertiary pediatric intensive care unit (PICU). This 20-bed PICU serves a mixed population of medical, surgical, and trauma patients. We included all patients 21 years or younger on admission and who received care in the PICU between January 1, 2009, and August 1, 2016. Each hospitalization with a PICU admission was treated independently. Data were extracted from an electronic health record database (Epic; Epic Systems Corporation). The institutional review board of The University of Chicago approved this study and waived patient informed consent because of the observational nature of the study.
The pSOFA score was developed by adapting the original SOFA score through 2 approaches. First, the age-dependent cardiovascular and renal variables of the original SOFA score were modified using validated cutoffs from the PELOD-2 scoring system.4 Second, the respiratory subscore was expanded to include the Spo2:Fio2 ratio as an alternative surrogate of lung injury (Table 1).12
The age-adjusted mean arterial pressure cutoffs for the first score of the PELOD-2 cardiovascular criteria were used to assign a score of 1 in the pSOFA subscore. Scores 2 to 4 were kept identical to the original SOFA criteria.
The age-adjusted serum creatinine level cutoffs for the first score of the PELOD-2 renal criteria were used to assign a score of 1 in the pSOFA renal subscore. Scores 2 to 4 were modified by increasing the cutoff values for each score by the same factor as the original SOFA criteria, similar to the approach proposed by other authors.14 Exceptions to this approach were the cutoff values for the first age group (<1 month) owing to the renal physiologic differences of neonates. For this neonatal age group, the cutoff value increase for each score was done by the same amount as the infant group (1-12 months) given the similarity in the glomerular filtration rate in both age groups.15
The original Pao2:Fio2 ratio cutoffs were kept identical to the original score, but the Spo2:Fio2 ratio was used as an alternative surrogate of lung injury. The adaptation proposed by Khemani and colleagues12 was used to define the Spo2:Fio2 ratio cutoffs.
The original coagulation and hepatic criteria, based on platelet count and bilirubin level, were kept identical to the original score. The Glasgow Coma Scale criteria for the neurologic subscore were also kept identical to the original score, but the pediatric version of the scale was used.16
The calculation of the pSOFA score was performed in the same way as the calculation of the original SOFA score.2 The worst variable in each 24-hour period was used to assign a subscore for each system (ranging from 0-4 points). The sum of the 6 subscores in each 24-hour period resulted in a daily pSOFA score (ranging from 0-24 points; higher scores indicate a worse outcome). If a variable was not measured in a 24-hour period, it was considered to be normal, which is consistent with the original criteria.
To compare the performance of the adapted variables in pSOFA with those in the original SOFA score, we calculated the maximum individual subscores for the cardiovascular, renal, and respiratory components using both the original and the adapted criteria.
We compared the performance of the pSOFA score with 3 other pediatric organ dysfunction scores—the PELOD score, the updated PELOD-2 score, and the Pediatric Multiple Organ Dysfunction Score.3-5 We calculated the daily score of the 4 scoring systems for each 24-hour period from PICU admission until day 28 of hospitalization, discharge, or death, whichever came first. The maximum and mean scores for each scoring system were used to compare the scores and evaluate the clinical validity of pSOFA. To evaluate the clinical utility of pSOFA on admission, we compared it with the Pediatric Risk of Mortality (PRISM) III score, a marker of severity of illness on admission, using information from the first 24 hours.17 To further evaluate the clinical validity and utility of pSOFA in comparison to the other scoring systems, the maximum and mean scores were also calculated at 4 landmarked times—days 2, 4, 7, and 14 after PICU admission. Only patients still alive and hospitalized on landmarked days were used for calculations.
To evaluate the effect of patients who had more than 1 hospitalization with a PICU admission, a sensitivity analysis using only the first PICU admission for each patient was performed.
We assessed the Sepsis-3 definitions in the subgroup of patients with confirmed or suspected infection by using previously published criteria.18 Patients with suspected infection were defined as those who had negative microbiological study results and initiation of treatment with nonprophylactic antibiotics, antifungals, or antiviral medications within 24 hours.18 Coagulase-negative Staphylococcus was only considered infectious if it was isolated in 2 or more cultures from sterile sites.
Patients with sepsis were defined as those with confirmed or suspected infection who had an acute rise in the pSOFA score of 2 points or more from up to 48 hours before the infection to 24 hours after the infection and who received antimicrobial therapy in the PICU.18Infection time was defined as the time when the first microbiological study or antimicrobial therapy was ordered by a physician, whichever came first. If the patients were not known to have previous organ dysfunction, the preinfection pSOFA score was assumed to be zero. Septic shock was defined as patients with sepsis who required a vasoactive infusion and had a maximum serum lactate level greater than 2 mmol/L (18 mg/dL).1 The microbiological etiology and infection source of patients with a confirmed or suspected infection were analyzed and compared between those who met sepsis criteria and those who did not.
The primary outcome was in-hospital mortality. Data were analyzed using Stata, version 14 (StataCorp LLC), and R, version 3.2.2 (R Foundation for Statistical Computing). Categorical variables were compared using the χ2 test, and continuous variables were compared using the Mann-Whitney test. A 2-sided P < .05 was considered statistically significant.
The performance of the scores to discriminate in-hospital mortality was evaluated using the area under the curve (AUC). Comparison between scores was performed using the DeLong method19 to compare AUCs and the Integrated Discrimination Improvement Index20 to evaluate the reclassification of predicted probabilities between survivors and nonsurvivors. Youden J statistic21 was used to evaluate the optimal cutoff of the pSOFA score to discriminate mortality.
Reporting of this validation study was performed using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines.22
In all, 6303 patients with 8711 encounters met inclusion criteria. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. In-hospital mortality was 2.6%. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. The demographic and clinical characteristics of survivors and nonsurvivors are shown in Table 2. (Note that we treated each of the 8711 encounters independently given that the major risk factors for the outcome [ie, organ dysfunctions] and the outcome [ie, survival or nonsurvival at the end of the hospitalization, or in-hospital mortality] are contained within each encounter.)
Nonsurvivors had a significantly higher median (IQR) maximum pSOFA score than survivors (13 [10-16] vs 2 [1-5]; P < .001; Table 2). The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an AUC of 0.94 (95% CI, 0.92-0.95). The in-hospital mortality rate for patients according to their maximum pSOFA score are shown in the Figure. The performance of the maximum pSOFA score to discriminate in-hospital mortality remained stable across sex, age groups, and admission types (eTable 1 in the Supplement). The optimal pSOFA cutoff to discriminate mortality was a score higher than 8 points.
The performance of the maximum pSOFA cardiovascular, renal, and respiratory subscores at discriminating in-hospital mortality were better than the non–age-adjusted maximum SOFA cardiovascular (AUC, 0.87 vs 0.86; P < .001), renal (AUC, 0.76 vs 0.61; P < .001), and respiratory (AUC, 0.87 vs 0.75; P < .001) subscores. The reclassification of estimated probabilities based on the Integrated Discrimination Improvement Index for the respiratory and renal subscores was significant (total ≥2.6%; P < .001) but negligible for the cardiovascular subscore (eTable 2 in the Supplement).
The performance of the maximum pSOFA score at discriminating in-hospital mortality (AUC, 0.94; 95% CI, 0.92-0.95) was similar to the performance of PELOD and PELOD-2 (AUC, 0.93 vs 0.94; 95% CI, 0.91-0.95 vs 0.92-0.95; P > .20) and better than the Pediatric Multiple Organ Dysfunction Score (AUC, 0.91; 95% CI, 0.88-0.93; P = .001). The performance of pSOFA on the day of admission at discriminating in-hospital mortality (AUC, 0.88; 95% CI, 0.86-0.91) was better than the other organ dysfunction scores (P ≤ .02) and similar to PRISM III (AUC, 0.88; 95% CI, 0.86-0.91; P = .94) (Table 3). On the landmarked day analyses, pSOFA performed slightly better on days 2 and 4 and performed similarly to the other scores on days 7 and 14 (Table 3). The Integrated Discrimination Improvement Index showed small differences in the reclassification of estimated probabilities (eTable 3 in the Supplement).
A sensitivity analysis using only the first PICU admission for each patient showed results similar to the complete cohort (eTable 4 in the Supplement).
Of 8711 patient encounters, 4217 (48.4%) had a confirmed or suspected infection in the PICU. The maximum pSOFA score had excellent discrimination of in-hospital mortality in this subgroup of patients (AUC, 0.92; 95% CI, 0.91- 0.94). When we used the adapted Sepsis-3 definitions with the pSOFA score, 1231 patients (14.1% of the PICU population) met sepsis criteria and had a mortality rate of 12.1%. Of those, 347 patients (4.0% of the PICU population) also met septic shock criteria and had a mortality rate of 32.3%. Among patients with confirmed or suspected infection, being diagnosed with sepsis significantly increased the likelihood of dying in the hospital (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.
The demographic, clinical, and microbiological characteristics of patients with a confirmed or suspected infection with or without sepsis are shown in Table 4.
We adapted and validated an age-adjusted version of the SOFA score for pediatric patients (pSOFA). The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population, which was comparable to or better than the performance of other common pediatric organ dysfunction scores. We then used the pSOFA score to perform the first assessment of Sepsis-3 in critically ill children.
Several motivations were behind the adaptation of the adult SOFA score and the Sepsis-3 definitions to children. Having a harmonized definition of sepsis across age groups while recognizing the importance of the age-based variation of its measures can have many benefits, including better design of clinical trials, improved accuracy of reported outcomes, and better translation of the research and clinical strategies in the management of sepsis. Furthermore, as we focus our efforts on uncovering the pathobiological basis of the different subtypes of sepsis,23,24 we should avoid being limited by differing definitions of the syndrome across artificial constructs, such as the pediatric-adult age threshold. In our study, we found that the optimal pSOFA score cutoff to discriminate mortality was identical to the cutoff found by Ferreira and colleagues25 in adult critically ill patients (ie, a SOFA score >8 points). This observation requires further validation; however, when measured with a comparable scoring system—and despite the differences in baseline mortality—the degree of organ dysfunction in adults and children seems to discriminate outcomes in a similar way.
Another motivation for the adaptation of Sepsis-3 to children is that the current definitions of sepsis in pediatrics are not without problems. Weiss and colleagues26 recently demonstrated a lack of agreement between the 2005 International Pediatric Sepsis Consensus Conference criteria and the treating physician’s diagnosis of severe sepsis in the Sepsis Prevalence, Outcomes, and Therapies (SPROUT) study. This discrepancy implies that the results from research using the 2005 International Pediatric Sepsis Consensus Conference criteria could lack generalizability to almost a third of PICU patients with sepsis. One of the goals of Sepsis-3 is to harmonize the definitions of sepsis and septic shock using readily available objective clinical data,18 and its adaptation to children may help balance the existing diagnostic discrepancies in pediatric patients.
The results of our assessment of the Sepsis-3 definitions in critically ill children with confirmed or suspected infection are encouraging. The Sepsis-3 definitions identified a group of patients among those with confirmed or suspected infection who were 18 times more likely to die in the hospital. Furthermore, patients with sepsis in our cohort had characteristics similar to the critically ill children who were diagnosed with severe sepsis by either the 2005 International Pediatric Sepsis Consensus Conference criteria or physician diagnosis in the SPROUT study.26 Patients with the “broader” diagnosis of severe sepsis in the SPROUT cohort had similar incidence of gram-positive (27% vs 28% in our population) and gram-negative (26% vs 28%) bacterial infections as well as viral (22% vs 24%) and fungal (12% vs 8%) infections. The infection sources of both cohorts also showed similarities in the respiratory tract (41% vs 39%), bloodstream (19% vs 20%), and genitourinary tract (4% vs 4%) infection sites. Furthermore, these similarities were all significantly different from the microbiological characteristics of patients with confirmed or suspected infection but no sepsis. This finding suggests that the Sepsis-3 definitions could help bridge the current diagnostic discrepancies in identifying children with severe sepsis. (Of note, the label severe sepsis was eliminated by the Sepsis-3 definitions in favor of sepsis, which is more commonly used in patient care.)1 One major difference with our cohort is that the incidence of severe sepsis in the SPROUT study was lower (10% vs 14%), and those patients had higher mortality (23% vs 12%).26 This difference suggests that our adaptation of the Sepsis-3 definitions with the pSOFA score likely captures a larger cohort of patients with infection and milder organ dysfunction. However, this group might still be a valuable population to consider in the context of sepsis, especially given the shared microbiological characteristics with the SPROUT cohort and a significant increase in the likelihood of dying when compared with infected patients with no sepsis. The mortality difference between our PICU population and the SPROUT cohort may also be attributable to the differences in the patient populations seen in PICUs outside the United States.26 This difference emphasizes the need for further validation of the Sepsis-3 definitions in critically ill children in other settings and populations.
There are several limitations to our study. First, our results were generated using retrospective data from a single center. Validating pSOFA in a larger, multicenter sample of critically ill children is necessary to assess the generalizability of the score. The adaptation of the Sepsis-3 definitions to children with confirmed or suspected infection using the pSOFA score or similar approaches must be validated as well in other settings and populations. Furthermore, the utility of the Sepsis-3 definitions for identifying children with sepsis for enrollment in clinical trials, reporting outcomes, and providing clinical care must be carefully examined by the pediatric community. Second, we did not evaluate the performance of the pSOFA as a longitudinal biomarker. Studies of the longitudinal performance of pSOFA would be helpful in further assessing its clinical utility.27,28 We present what we believe to be promising results for the pSOFA score and the adapted Sepsis-3 definitions in a single cohort of critically ill children, but further research is warranted.
We adapted and validated the pSOFA score, an age-adjusted pediatric version of the adult SOFA score, and used it to assess the Sepsis-3 definitions in critically ill children. The pSOFA score showed excellent discrimination for in-hospital mortality in a general PICU population and in the subgroup of patients with suspected or confirmed infection. Our assessment of the Sepsis-3 definitions in children showed promising results, but further validation in children in different settings and populations is warranted.
Corresponding Author: L. Nelson Sanchez-Pinto, MD, MBI, Division of Critical Care Medicine, Ann & Robert H. Lurie Children’s Hospital of Chicago, 225 E Chicago Ave, Chicago, IL 60611 (email@example.com).
Accepted for Publication: June 8, 2017.
Published Online: August 7, 2017. doi:10.1001/jamapediatrics.2017.2352
Author Contributions: Drs Sanchez-Pinto and Matics had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: All authors.
Administrative, technical, or material support: Sanchez-Pinto.
Study supervision: Sanchez-Pinto.
Conflict of Interest Disclosures: None reported.
Additional Contributions: The Center for Research Informatics at the University of Chicago provided the raw electronic health record data for this study.
Create a personal account or sign in to: