[Skip to Content]
[Skip to Content Landing]
Figure 1.
Patient Accrual
Patient Accrual
Figure 2.
Risk-Adjusted Odds Ratios of In-Hospital Death in the Primary Models
Risk-Adjusted Odds Ratios of In-Hospital Death in the Primary Models

Embedded table reports the total number of patients, total number of absolute deaths (%), predicted risk from adjusted models and risk difference (95% CI), and odds ratios (95% CI) comparing patients who did and did not complete the 1 hour bundle within 1 hour. Variables in the risk-adjusted model include age category, payer, protocol initiation site, diagnosis of septic shock, site of infection, platelet count <150 000/μL at protocol initiation, chronic renal disease or liver failure, diabetes, acute respiratory failure requiring mechanical ventilation at protocol initiation, serum lactate, and transfer status.

Figure 3.
Crude In-Hospital Mortality and Predicted Risk of In-Hospital Death After the Time of Sepsis Protocol Initiation
Crude In-Hospital Mortality and Predicted Risk of In-Hospital Death After the Time of Sepsis Protocol Initiation

Crude in-hospital mortality (bars) and the predicted risks of in-hospital death with 95% CIs (orange line with error bars). Predicted risks derive from model adjusted for age category, payer, protocol initiation site, diagnosis of septic shock, site of infection, platelet count <150 000/μL at protocol initiation, chronic renal failure or renal disease, diabetes, acute respiratory failure requiring mechanical ventilation, serum lactate, and transfer status across 4 hours after protocol initiation for the completion of the 1-hour bundle of sepsis care. As an interpretive example, for a typical pediatric patient with sepsis with average age and level of acuity in New York State, the completion of the 1-hour sepsis bundle within 1 hour was associated with an 8% risk of in-hospital death. In contrast, the same patient who completes the bundle at 4 hours will have a 13% predicted risk of in-hospital death.

Figure 4.
Risk and Reliability-Adjusted Rate for Each Hospital for Completion of the 1-Hour Bundle in 1 Hour, According to Hospital Rank
Risk and Reliability-Adjusted Rate for Each Hospital for Completion of the 1-Hour Bundle in 1 Hour, According to Hospital Rank

The 54 hospitals that were included in the study were ranked from lowest to highest, with higher numbers (x-axis) indicating a greater likelihood of completing the 1-hour bundle within 1 hour. Bars represents 95% CIs. Orange circles correspond to hospitals with both pediatric intensive care and cardiac surgery services. Risk and reliability adjustment accounts for both patient-level risk factors that may influence outcome and statistical variation attributed to small sample sizes at the level of the hospital. These adjustments allow for accurate comparisons between hospitals of different patient volume and case mix.

Table 1.  
Patient Characteristics
Patient Characteristics
Table 2.  
Hospital Characteristics Stratified by Quartile of Reliability-Adjusted Rate of Completing the 1-Hour Bundle Within 1 Houra
Hospital Characteristics Stratified by Quartile of Reliability-Adjusted Rate of Completing the 1-Hour Bundle Within 1 Houra
Supplement.

eMethods

List of Investigators

eFigure 1. Calibration Belt Showing Observed In-Hospital Mortality Compared to Model-Based Estimates of Expected In-Hospital Mortality

eFigure 2. Distribution of Propensity Scores for Completing the One-Hour Bundle Within One Hour Among Matched (N = 572) and Unmatched (N = 607) Patients

eFigure 3. Crude In-Hospital Mortality and Predicted Risks of In-Hospital Death, With Adjustment for Covariates Up to Four Hours After Protocol Initiation, for the Administration of Antibiotics, Blood Cultures, and Completing the Initial Bolus of Intravenous Fluids in a Typical Pediatric Patient

eFigure 4. In silico Quantitative Bias Analysis of a Hypothetical Unmeasured Confounder

eFigure 5. Reliability-Adjusted Rate for Each Hospital for Completion of Individual Bundle Elements Within One Hour

eTable 1. Comparison of Clinical Characteristics Before and After Propensity Score Matching

eTable 2. Sensitivity and Subgroup Analyses for the Primary Model

eTable 3. Risk Adjustment Model Variable Coefficients With 95% Confidence Interval for In-Hospital Mortality

eTable 4. Results of Unadjusted Regression in the Propensity Matched Cohort (N=572) and the Inverse Probability Weighted Risk-Adjusted Regression Analysis (N=1,179) for In-Hospital Mortality

eTable 5. Adjusted Hospital Length of Stay for Entire Cohort, Decedents, and Survivors From Poisson Regression Models and Linear Regression With Logarithmic Transformation of Outcome Models

eTable 6. Variable Coefficients With 95% Confidence Interval From Poisson Regression Model for Hospital Length of Stay

eReferences

1.
Balamuth  F, Weiss  SL, Neuman  MI,  et al.  Pediatric severe sepsis in US children’s hospitals.  Pediatr Crit Care Med. 2014;15(9):798-805. doi:10.1097/PCC.0000000000000225PubMedGoogle ScholarCrossref
2.
Dellinger  RP, Levy  MM, Rhodes  A,  et al; Surviving Sepsis Campaign Guidelines Committee including The Pediatric Subgroup.  Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012.  Intensive Care Med. 2013;39(2):165-228. doi:10.1007/s00134-012-2769-8PubMedGoogle ScholarCrossref
3.
Cruz  AT, Perry  AM, Williams  EA, Graf  JM, Wuestner  ER, Patel  B.  Implementation of goal-directed therapy for children with suspected sepsis in the emergency department.  Pediatrics. 2011;127(3):e758-e766. doi:10.1542/peds.2010-2895PubMedGoogle ScholarCrossref
4.
Larsen  GY, Mecham  N, Greenberg  R.  An emergency department septic shock protocol and care guideline for children initiated at triage.  Pediatrics. 2011;127(6):e1585-e1592. doi:10.1542/peds.2010-3513PubMedGoogle ScholarCrossref
5.
Dwyer  J. An infection, unnoticed, turns unstoppable. The New York Times. https://www.nytimes.com/2012/07/12/nyregion/in-rory-stauntons-fight-for-his-life-signs-that-went-unheeded.html. Published July 11, 2012. Accessed July 20, 2017.
6.
New York State Department of Health. New York State report on sepsis care improvement initiative: hospital quality performance. https://www. health.ny.gov/ press/ reports/ docs/2015_sepsis_care_improvement_initiative.pdf. Published March 2017. Accessed December 22, 2017.
7.
Brierley  J, Carcillo  JA, Choong  K,  et al.  Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine [published correction appears in Crit Care Med. 2009;37(4):1536].  Crit Care Med. 2009;37(2):666-688. doi:10.1097/CCM.0b013e31819323c6PubMedGoogle ScholarCrossref
8.
Goldstein  B, Giroir  B, Randolph  A; International Consensus Conference on Pediatric Sepsis.  International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics.  Pediatr Crit Care Med. 2005;6(1):2-8. doi:10.1097/01.PCC.0000149131.72248.E6PubMedGoogle ScholarCrossref
9.
Seymour  CW, Gesten  F, Prescott  HC,  et al.  Time to treatment and mortality during mandated emergency care for sepsis.  N Engl J Med. 2017;376(23):2235-2244. doi:10.1056/NEJMoa1703058PubMedGoogle ScholarCrossref
10.
Psaty  BM, Koepsell  TD, Lin  D,  et al.  Assessment and control for confounding by indication in observational studies.  J Am Geriatr Soc. 1999;47(6):749-754. doi:10.1111/j.1532-5415.1999.tb01603.xPubMedGoogle ScholarCrossref
11.
Odetola  FO, Gebremariam  A, Freed  GL.  Patient and hospital correlates of clinical outcomes and resource utilization in severe pediatric sepsis.  Pediatrics. 2007;119(3):487-494. doi:10.1542/peds.2006-2353PubMedGoogle ScholarCrossref
12.
Watson  RS, Carcillo  JA, Linde-Zwirble  WT, Clermont  G, Lidicker  J, Angus  DC.  The epidemiology of severe sepsis in children in the United States.  Am J Respir Crit Care Med. 2003;167(5):695-701. doi:10.1164/rccm.200207-682OCPubMedGoogle ScholarCrossref
13.
Leclerc  F, Leteurtre  S, Duhamel  A,  et al.  Cumulative influence of organ dysfunctions and septic state on mortality of critically ill children.  Am J Respir Crit Care Med. 2005;171(4):348-353. doi:10.1164/rccm.200405-630OCPubMedGoogle ScholarCrossref
14.
Dombrovskiy  VY, Martin  AA, Sunderram  J, Paz  HL.  Occurrence and outcomes of sepsis: influence of race.  Crit Care Med. 2007;35(3):763-768. doi:10.1097/01.CCM.0000256726.80998.BFPubMedGoogle ScholarCrossref
15.
New York State Department of Health. Statewide Planning and Research Cooperative System (SPARCS). https://www.health.ny.gov/statistics/sparcs/. Accessed May 18, 2018.
16.
New York State Department of Health. Health facility certification information. https://health.data.ny.gov/Health/Health-Facility-Certification-Information/2g9y-7kqm/data. Accessed January 4, 2018.
17.
Rosenberg  DI, Moss  MM; American College of Critical Care Medicine of the Society of Critical Care Medicine.  Guidelines and levels of care for pediatric intensive care units.  Crit Care Med. 2004;32(10):2117-2127. doi:10.1097/01.CCM.0000142704.36378.E9PubMedGoogle ScholarCrossref
18.
Royston  P, Sauerbrei  W.  Building multivariable regression models with continuous covariates in clinical epidemiology: with an emphasis on fractional polynomials.  Methods Inf Med. 2005;44(4):561-571. doi:10.1055/s-0038-1634008PubMedGoogle ScholarCrossref
19.
Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment.  Health Serv Res. 2010;45(6, pt 1):1614-1629. doi:10.1111/j.1475-6773.2010.01158.xPubMedGoogle ScholarCrossref
20.
Czaja  AS, Zimmerman  JJ, Nathens  AB.  Readmission and late mortality after pediatric severe sepsis.  Pediatrics. 2009;123(3):849-857. doi:10.1542/peds.2008-0856PubMedGoogle ScholarCrossref
21.
Williams  K, Thomson  D, Seto  I,  et al; StaR Child Health Group.  Standard 6: age groups for pediatric trials.  Pediatrics. 2012;129(3)(suppl 3):S153-S160. doi:10.1542/peds.2012-0055IPubMedGoogle ScholarCrossref
22.
Lash  TL, Fox  MP, Fink  AK.  Applying Quantitative Bias Analysis to Epidemiologic Data. New York, NY: Springer-Verlag; 2009.Crossref
23.
Balamuth  F, Weiss  SL, Fitzgerald  JC,  et al.  Protocolized treatment is associated with decreased organ dysfunction in pediatric severe sepsis.  Pediatr Crit Care Med. 2016;17(9):817-822. doi:10.1097/PCC.0000000000000858PubMedGoogle ScholarCrossref
24.
Davis  AL, Carcillo  JA, Aneja  RK,  et al.  American College of Critical Care Medicine clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock.  Crit Care Med. 2017;45(6):1061-1093. doi:10.1097/CCM.0000000000002425PubMedGoogle ScholarCrossref
25.
Kuhlmann  S, Mason  B, Ahlers-Schmidt  CR.  A quality improvement project to improve compliance with the joint commission children’s asthma care-3 measure.  Hosp Pediatr. 2013;3(1):45-51. doi:10.1542/hpeds.2012-0015PubMedGoogle ScholarCrossref
26.
Seymour  CW, Liu  VX, Iwashyna  TJ,  et al.  Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).  JAMA. 2016;315(8):762-774. doi:10.1001/jama.2016.0288PubMedGoogle ScholarCrossref
Original Investigation
July 24/31, 2018

Association Between the New York Sepsis Care Mandate and In-Hospital Mortality for Pediatric Sepsis

Author Affiliations
  • 1Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
  • 2The Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, Pittsburgh, Pennsylvania
  • 3IPRO, Lake Success, New York
  • 4Division of Emergency Medicine, Department of Pediatrics, Ann and Robert H. Lurie Children's Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, Illinois
  • 5Associate Editor, JAMA, Chicago, Illinois
  • 6New York State Department of Health, Albany
  • 7Department of Pediatrics, Division of Critical Care, BC Children’s Hospital and Research Institute, University of British Columbia, Vancouver, British Columbia, Canada
  • 8Division of Biostatistics, College of Public Health, The Ohio State University, Columbus
  • 9Department of Medicine, Division of Pulmonary Critical Care & Sleep, Alpert Medical School of Brown University, Providence, Rhode Island
  • 10Department of Pediatrics, Pediatric Critical Care Medicine, Stony Brook Children’s Hospital, Stony Brook, New York
  • 11Pediatric Critical Care Medicine, Seattle Children’s Hospital and Research Institute, Seattle, Washington
  • 12Division of Critical Care Medicine, Department of Anesthesiology and Critical Care, The Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia
JAMA. 2018;320(4):358-367. doi:10.1001/jama.2018.9071
Key Points

Question  Following statewide mandated care for pediatric sepsis, was the prompt completion of a 1-hour bundle associated with lower risk-adjusted in-hospital mortality?

Findings  Among 1179 pediatric patients with sepsis at 54 adult and pediatric specialty hospitals in New York State, the completion of a 1-hour sepsis bundle that included blood cultures, broad spectrum antibiotics, and a 20-mL/kg fluid bolus was significantly associated with lower risk-adjusted in-hospital mortality compared with not completing the bundle within 1 hour (odds ratio, 0.59).

Meaning  Timely completion of a 1-hour bundle of care may improve outcomes in pediatric sepsis.

Abstract

Importance  The death of a pediatric patient with sepsis motivated New York to mandate statewide sepsis treatment in 2013. The mandate included a 1-hour bundle of blood cultures, broad-spectrum antibiotics, and a 20-mL/kg intravenous fluid bolus. Whether completing the bundle elements within 1 hour improves outcomes is unclear.

Objective  To determine the risk-adjusted association between completing the 1-hour pediatric sepsis bundle and individual bundle elements with in-hospital mortality.

Design, Settings, and Participants  Statewide cohort study conducted from April 1, 2014, to December 31, 2016, in emergency departments, inpatient units, and intensive care units across New York State. A total of 1179 patients aged 18 years and younger with sepsis and septic shock reported to the New York State Department of Health who had a sepsis protocol initiated were included.

Exposures  Completion of a 1-hour sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour.

Main Outcomes and Measures  Risk-adjusted in-hospital mortality.

Results  Of 1179 patients with sepsis reported at 54 hospitals (mean [SD] age, 7.2 [6.2] years; male, 54.2%; previously healthy, 44.5%; diagnosed as having shock, 68.8%), 139 (11.8%) died. The entire sepsis bundle was completed in 1 hour in 294 patients (24.9%). Antibiotics were administered to 798 patients (67.7%), blood cultures were obtained in 740 patients (62.8%), and the fluid bolus was completed in 548 patients (46.5%) within 1 hour. Completion of the entire bundle within 1 hour was associated with lower risk-adjusted odds of in-hospital mortality (odds ratio [OR], 0.59 [95% CI, 0.38 to 0.93], P = .02; predicted risk difference [RD], 4.0% [95% CI, 0.9% to 7.0%]). However, completion of each individual bundle element within 1 hour was not significantly associated with lower risk-adjusted mortality (blood culture: OR, 0.73 [95% CI, 0.51 to 1.06], P = .10; RD, 2.6% [95% CI, −0.5% to 5.7%]; antibiotics: OR, 0.78 [95% CI, 0.55 to 1.12], P = .18; RD, 2.1% [95% CI, −1.1% to 5.2%], and fluid bolus: OR, 0.88 [95% CI, 0.56 to 1.37], P = .56; RD, 1.1% [95% CI, −2.6% to 4.8%]).

Conclusions and Relevance  In New York State following a mandate for sepsis care, completion of a sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour was associated with lower risk-adjusted in-hospital mortality among patients with pediatric sepsis and septic shock.

Introduction

Quiz Ref IDIn a consortium of 44 pediatric hospitals, it was estimated that more than 170 000 pediatric patients were hospitalized with sepsis from 2004 to 2012, among whom approximately 8% died prior to discharge.1 Contemporary clinical practice guidelines recommend prompt sepsis identification, blood culture prior to treatment, and administration of broad-spectrum antibiotics and intravenous fluids.2 In single-center studies at pediatric specialty hospitals, the combination of these elements in a bundle can reduce time to antibiotics and time to an intravenous fluid bolus.3,4 It is unknown, however, whether the timely completion of the bundle improves outcomes such as mortality or hospital length of stay.

Motivated by the death of a pediatric patient with sepsis,5 New York State mandated in 2013 that all hospitals deliver a care bundle to pediatric patients within 1 hour of sepsis recognition.6 Using the New York State Department of Health (NYSDOH) database, an analysis was performed to determine the association between completion of a sepsis bundle within 1 hour and risk-adjusted in-hospital mortality for pediatric sepsis and septic shock.

Methods

This study was approved with waiver of informed consent by the institutional review board for the NYSDOH (1156246-1).

Study Design and Setting

In 2013, the NYSDOH required hospitals to submit evidence-informed protocols and educate clinical staff about early identification and treatment of sepsis (previously termed severe sepsis) or septic shock (New York Codes, Rules, and Regulations 405.2 and 405.4). While protocols varied by hospital, all protocols were required to include a pediatric bundle consisting of the following interventions within 1 hour: blood culture collection before administering antibiotics, administration of broad-spectrum antibiotics, and completion of a 20-mL/kg fluid bolus. Data collection began on April 1, 2014. We performed a retrospective study of all pediatric sepsis cases reported from 59 acute care hospitals in the NYSDOH database from April 1, 2014, to December 31, 2016.

Population

All hospitals were required to report patient-level data for patients with sepsis and septic shock to the NYSDOH using electronic case submission that included demographics, comorbidities, characteristics of sepsis and septic shock, organ dysfunction, and outcomes. Date and time stamps for protocol initiation and completion of individual elements of the 1-hour bundle were required for patients in whom a sepsis protocol was initiated. The state performed 10% random sample audits for all patients using manual medical record review and provided feedback to hospitals on data quality and completeness. Patient-level data were linked to hospital characteristics using the NYSDOH administrative database.

Eligible patients included those younger than 18 years with sepsis or septic shock, using criteria adapted from the American College of Critical Care Medicine7 and Goldstein et al.8 We did not study patients in whom the 1-hour bundle was clinically contraindicated, neonates who were never discharged, patients for whom advanced directives limited treatment, patients and/or surrogates who declined interventions, or patients who were enrolled in a concomitant clinical trial. For patients in whom multiple sepsis episodes were present in the database, we studied data from the final encounter to preserve independence of observations. For pediatric patients transferred from a referring to a receiving hospital, only 1-hour bundle data from the receiving hospital were available in the database.

Quiz Ref IDHospitals used a variety of methods to identify pediatric sepsis (eMethods in the Supplement). These included (1) sepsis based on clinical assessment only (suspected or confirmed infection and ≥2 age-appropriate systemic inflammatory response syndrome criteria [eg, temperature, heart rate, respiratory rate, white blood cell count8], with supporting laboratories optional), (2) sepsis by both clinical criteria and abnormal laboratories, or (3) “code sepsis or rapid response” led to sepsis by clinical criteria. The regulations permitted hospitals to have flexibility in case identification to facilitate broader adoption. Most patients submitted to the NYSDOH database (>98%) were confirmed to have severe sepsis or septic shock on manual audit.9 Cases found to be erroneously entered could be removed by hospitals.

Variables

Quiz Ref IDThe primary outcome was all-cause in-hospital mortality. The primary exposure was completion of all 3 elements of the 1-hour bundle (ie, blood culture(s) collected prior to antibiotic administration, broad-spectrum antibiotics administered, and completion of at least one 20-mL/kg fluid bolus) within 1 hour of protocol initiation, as reported by hospital and clinical staff. Blood culture was recorded as time collected, antibiotic administration was recorded as time of starting administration, and intravenous fluid bolus was recorded as time of completion of 20-mL/kg crystalloid fluid administration. The NYSDOH defined “broad spectrum” according to the local hospital protocol due to variable antimicrobial resistance patterns. If elements of the 1-hour bundle occurred prior to protocol initiation, the bundle was considered completed within the first hour.

We measured covariates for a risk-adjustment model to account for confounding by indication,10 primarily that sicker children would receive the bundle more promptly. These included variables specified a priori based on clinical experience and prior pediatric studies, such as patient age; payer; comorbidity burden; location of protocol initiation (eg, emergency department, intensive care unit [ICU], or inpatient unit), site of infection (eg, respiratory, urinary, or skin); and measures of organ dysfunction including presence of shock, platelet count, or mechanical ventilation prior to protocol initiation.11-13 Race and ethnicity were included because they may be important confounders in the association between sepsis care and outcome.14 The determination of race and ethnic group was made by hospital staff at reporting hospitals with fixed categories from the NYSDOH Statewide Planning and Research Cooperative System data set.15 Using these covariates, a multivariable logistic regression for in-hospital mortality had adequate calibration (eFigure 1 in the Supplement; Hosmer-Lemeshow goodness-of-fit test for 10 groups, P = .50) and discrimination (area under the receiver operating characteristic curve, 0.83) in internal validation.

Hospital-level data were also collected to illustrate how the 1-hour bundle was completed at hospitals of different capabilities. We identified hospitals with a pediatric ICU (PICU) using data available in the NYSDOH hospital database.16 We further specified that if a hospital provided cardiac surgery or cardiac catheterization services that the ICU met criteria for a level I pediatric ICU.17

Statistical Analysis

We performed bivariate analyses of characteristics of patients who did and did not complete the 1-hour bundle within 1 hour. Continuous data are expressed as mean (SD) or median (interquartile range), depending on normality. Categorical variables are shown as proportions. Differences in predicted risk (risk difference [RD]) between groups are shown with 95% CIs.

Multivariable modeling of the association between completion of the 1-hour bundle within 1 hour and in-hospital mortality was performed using logistic regression adjusted for covariates with robust standard errors and mixed effects to account for nonindependence of cases across hospitals. Binary variables were modeled as indicator covariates and continuous variables included as linear covariates, after assessing for nonlinear relationships using fractional polynomials.18 Each exposure (ie, completion of the entire 1-hour bundle and each individual bundle element) was evaluated separately. No missing data were present for model covariates.

To predict the risk of in-hospital death across a range of times to completing the 1-hour bundle, we restricted the cohort to cases in whom the entire bundle was completed within 4 hours from protocol initiation (n = 529 [44.9%]). We then used predictive margins adjusted for covariates to predict the risk by hour. Similar models, including nonlinear models when appropriate, were used to illustrate predicted risks of death by hour for completing each individual bundle element.

We performed risk and reliability adjustment using empirical Bayes methods to determine the hospital-level rate of completing the 1-hour bundle, collecting blood cultures, administering antibiotics, and completing the initial intravenous fluid bolus within 1 hour.19 We display the ranked order of adjusted rates across hospitals in caterpillar plots.

Sensitivity Analyses

We assessed the robustness of findings in multiple sensitivity analyses. First, we performed a matched propensity score analysis to adjust for covariate imbalance between groups (eTable 1, eFigure 2, and eMethods in the Supplement). Second, we performed an inverse probability weighted least squares regression to calculate the average treatment effect for completing the 1-hour bundle within 1 hour (eMethods in the Supplement). Third, we repeated our models, accounting for discharge bias by classifying patients discharged to hospice care as in-hospital deaths. Fourth, because transferred patients may be of greater illness severity and have complex timing of bundle elements,11 we repeated models after excluding transferred patients. Fifth, we included additional confounders in the risk adjustment model: (1) an indicator variable for prior admission for sepsis because readmissions may be strongly associated with mortality20 and (2) forcing all comorbidities and acute organ dysfunctions measured by the NYSDOH in the model. Sixth, we repeated models after excluding patients who never completed all elements of the 1-hour bundle. Seventh, we repeated the analysis with a 2-element bundle that excluded the intravenous fluid bolus.

In supporting analyses, we assessed for effect modification with a likelihood ratio test for interaction between treatment and characteristic in prespecified subpopulations, including patient location when the sepsis protocol was initiated, age strata,21 presence of septic shock, or admission to a hospital with pediatric intensive care. Hospital length of stay was studied as a secondary outcome, using Poisson regression models that generated the incidence rate ratios for in-hospital death among patients who did and did not complete the bundle in 1 hour.

In addition, linear regression models with logarithmic transformation that generated a ratio of geometric means were used to evaluate hospital length of stay. One model was used for all patients, and a second model was used for survivors and decedents. Further, we performed quantitative bias analysis22 to assess the magnitude of a hypothetical, unmeasured confounder that would be necessary to account for the association between completing the 1-hour bundle within 1 hour and risk-adjusted in-hospital mortality.

All analyses were performed with Stata version 14.2 (StataCorp) with a significance threshold of less than .05 in 2-sided tests.

Results
Patient Population and Completion of the 1-Hour Bundle

Of 1669 patient visits at 59 hospitals, we excluded 297 (17.8%) who did not have a sepsis protocol initiated, 55 (3.3%) in whom the sepsis bundle was clinically contraindicated, and 138 (8.3%) who were ineligible due to duplicate visits or other reasons (Figure 1). Of the remaining 1179 eligible patients at 54 hospitals, 294 patients (24.9%) completed the 1-hour bundle within 1 hour of protocol initiation. Blood cultures were obtained within 1 hour for 740 patients (62.8%) and antibiotics were administered within 1 hour for 798 patients (67.7%) while the fluid bolus was completed within 1 hour for 548 patients (46.5%). The 1-hour bundle was completed within 1 hour more often among patients in the emergency department (69.4% vs 53.2%; P < .001) and previously healthy patients (54.8% vs 41.1%; P = .01), and less often among patients who had been transferred from another facility (13.6% vs 20.9%; P = .006). Median serum lactate level (2.2 mmol/L vs 2.1 mmol/L; P = .09) and the proportion with septic shock (70.4% vs 68.3%; P = .49; Table 1) were similar between groups. The crude mortality rate was greater among patients who did not complete the 1-hour bundle within 1 hour compared with those who did (13.2% vs 7.5%; P = .008).

Primary Analyses

In a multivariable model, the completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality (odds ratio [OR], 0.59 [95% CI, 0.38-0.93], P = .02; RD, 4.0% [95% CI, 0.9%-7.0%], Figure 2; eTable 2 in the Supplement). Coefficients of the model covariates are presented in eTable 3 in the Supplement. This association was not modified by age strata, location, presence of shock, or care at a hospital with pediatric intensive care (P > .05; eTable 2 in the Supplement). Patients who had the 1-hour bundle completed in up to 3 hours remained at lower odds of in-hospital death compared with those who completed the bundle after 3 hours (OR, 0.64 [95% CI, 0.42-0.96], P = .03; RD, 3.6% [95% CI, 0.6%-6.7%]). Figure 3 shows the crude and predicted risks of in-hospital mortality across a range of 4 hours for completing the 1-hour bundle. The mean predicted in-hospital mortality increased by 2 percentage points for each hour until the 1-hour bundle was completed. Results for individual bundle element completion within 4 hours are presented in eFigure 3 in the Supplement.

The association between individual bundle elements and outcome did not reach statistical significance. The administration of antibiotics within 1 hour was not significantly associated with lower in-hospital mortality (OR, 0.78 [95% CI, 0.55-1.12], P = .18; RD, 2.1% [95% CI, −1.1% to 5.2%]; Figure 2). Neither blood culture collection within 1 hour (OR, 0.73 [95% CI, 0.51-1.06], P = .10; RD, 2.6% [95% CI, −0.5% to 5.7%]) nor the completion of a 20-cc/kg fluid bolus within 1 hour were significantly associated with lower in-hospital mortality (OR, 0.88 [95% CI, 0.56-1.37], P = .56; RD, 1.1% [95% CI, −2.6% to 4.8%]).

Sensitivity and Supporting Analyses

In the propensity matched cohort (n = 572), completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality (OR, 0.53 [95% CI, 0.30-0.94]; P = .03; eTable 4 in the Supplement). The inverse probability weighted least squares regression estimated the reduction in mortality associated with bundle completion within 1 hour as 4.2% (95% CI, 0.6%-7.6%; P = .02; eTable 4 in the Supplement). When hospice discharges were reclassified as in-hospital deaths, timely bundle completion was associated with lower in-hospital mortality (OR, 0.56 [95% CI, 0.36-0.86], P = .008; RD, 4.6% [95% CI, 1.6%-7.6%]; eTable 2 in the Supplement). In the model that excluded transferred patients (n = 954), the completion of the 1-hour bundle was significantly associated with lower mortality (OR, 0.52 [95% CI, 0.27-0.99], P = .05; RD, 4.3% [95% CI, 1.0%-7.7%]).

We ran 2 models that included additional confounders: (1) prior admission for sepsis (OR, 0.56 [95% CI, 0.34-0.92], P = .02; RD, 4.4% [95% CI, 1.1%-7.6%]) and (2) all comorbid and organ dysfunction variables (OR, 0.54 [95% CI, 0.33-0.90], P = .02; RD, 4.5% [95% CI, 1.4%-7.7%]; eTable 2 in the Supplement). After exclusion of patients who had a protocol initiated but never completed the bundle (n = 892), the association was no longer statistically significant (OR, 0.73 [95% CI, 0.47-1.13], P = .16; RD, 2.2% [95% CI, −0.7% to 5.1%]). Completion of the 2-element bundle of blood cultures before antibiotics and administration of antibiotics within 1 hour was significantly associated with lower in-hospital mortality (OR, 0.65, [95% CI, 0.44-0.96], P = .03; RD, 3.5% [95% CI, 0.6%-6.4%]).

Completion of the 1-hour bundle within 1 hour was also associated with shorter hospital length of stay among all patients (adjusted incidence rate ratio [IRR], 0.76 [95% CI, 0.64-0.89], P = .001) and survivors (IRR, 0.71 [95% CI, 0.60-0.84], P < .001), but not significant among decedents (IRR, 1.09 [95% CI, 0.71-1.69], P = .68; eTable 5 in the Supplement). The association of bundle completion within 1 hour and shorter hospital length of stay among all patients and survivors was significant in the linear regression model with logarithmic transformation that produced a ratio of geometric means (eTable 5 in the Supplement). Coefficients of the model covariates are presented in eTable 6 in the Supplement.

To assess the sensitivity of models to unmeasured confounding, quantitative bias analysis was performed (eFigure 4 in the Supplement). Across a range of scenarios, results would be robust unless a new confounder was at least 4 times as prevalent among those who did not complete the 1-hour bundle within 1 hour compared with those who did, and unless the unmeasured confounder itself was strongly associated with lower in-hospital death (OR < 0.5).

Across hospitals, the risk and reliability-adjusted rates of completing the 1-hour bundle within 1 hour ranged from 7.3% to 46.1% (median, 32.8%; interquartile range, 22.4%-37.5%; Figure 4). After ranking hospitals from the lowest to greatest likelihood of completing the 1-hour bundle, the hospitals in the highest quartile were 2.5 times as likely to complete the 1-hour bundle within 1 hour as hospitals in the lowest quartile (37.5% vs 15.0%). Hospitals with a higher rate of completing the 1-hour bundle cared for a greater number of pediatric patients (n = 686; P = .006; Table 2), and contained a greater proportion of PICUs with specialty care such as cardiac surgery (20.4%; P = .24). We observed similar variation for each individual bundle element (eFigure 5 in the Supplement).

Discussion

Quiz Ref IDDuring a statewide mandate for pediatric sepsis care, the prompt completion of a 1-hour bundle was associated with improved survival for pediatric patients with sepsis or septic shock. The completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality and shorter hospital length of stay. However, individual bundle elements were not significantly associated with decreased mortality, and hospitals were widely variable in completing the bundle for comparable patients.

These findings corroborate single-center studies showing an association between bundled pediatric sepsis care and improved outcomes. In the emergency department, greater adherence to a pediatric sepsis protocol was associated with greater odds of less organ dysfunction after 48 hours23 and a shorter hospital length of stay.4 This analysis extends these findings to report an association between completing a sepsis bundle within 1 hour and lower mortality in a larger cohort of pediatric patients treated in multiple locations, including the emergency department, the ICU, and in-patient care areas across 54 community and pediatric specialized hospitals. The findings also support the current recommendations from the American College of Critical Care Medicine24 for bundled care in the treatment of pediatric sepsis.

Quiz Ref IDThere are many possible explanations for the association between completing a 1-hour pediatric bundle and improved outcomes. First, the individual bundle elements may each contribute to specific biologic or physiologic changes that, when combined, affect outcomes. These may include shock reversal with fluid administration, reduced pathogen load and inflammatory response from prompt antibiotics, and more accurate diagnostic information from early blood cultures. Second, it is also possible that the completion of the 1-hour bundle represents important unmeasured processes of care in a time critical situation. The completion of the bundle may be a surrogate for heightened awareness by the clinical team, differential ordering of laboratory tests such as serum lactate, and greater attentiveness to treatment response. Third, a delay in completing the bundle may be due to unmeasured, competing events in sicker patients, such as tracheal intubation, administration of blood products, or difficult intravenous access, particularly in the ICU. Fourth, it is possible the study may be underpowered to detect a significant difference between completing each individual bundle element within 1 hour or not.

Across quartiles of hospitals, there was up to a 2- to 3-fold variation in completing the 1-hour pediatric bundle. This amount of variation is similar, if not greater, than the implementation of other pediatric quality improvement measures for the acutely ill.25 Barriers to pediatric bundle completion within 1 hour may vary between units and include difficult intravenous access, clinician inexperience with pediatric care, or lack of appropriately sized equipment at primarily adult hospitals. The finding that greater rates of completing the 1-hour bundle at hospitals with more case volume and pediatric specialty services is hypothesis generating and not linear. Further qualitative study is warranted into variations in the systems of pediatric care such as electronic health record prompts for case recognition or pharmacy integration at hospitals of different sizes.

These pediatric data complement similar findings reported for adult patients with sepsis in the emergency department from the NYSDOH mandate.9 Both studies found that prompt completion of the sepsis bundle was associated with improved outcomes compared with completing the bundle after a delay. But there are important differences. First, only 25% of pediatric patients completed the 1-hour sepsis bundle within the 1-hour mandate compared with 82.5% of adults who completed the 3-hour bundle within the 3-hour adult mandate. Second, the individual elements of the pediatric bundle were not statistically significant, while individual steps in the adult bundle appeared significantly associated with outcome. The pediatric cohort, smaller than the adult cohort, may have been underpowered to detect a significant association between individual bundle element completion within 1 hour and in-hospital mortality. Third, the 3-hour adult bundle was more likely to be completed at smaller, rural hospitals, whereas the 1-hour pediatric bundle was more promptly completed at larger hospitals with greater pediatric case volume. The characteristics that may account for a typical, rural hospital performing differently for an adult bundle vs a pediatric bundle deserves dedicated study.

Limitations

This study has several limitations. First, this was a retrospective study, and results may be biased by measured and unmeasured confounding at the patient and hospital levels. The matched propensity score analysis and inverse probability weighted regression provide alternative methods to handle measured confounders and support the primary model. Quantitative bias analysis suggests that although plausible, unmeasured confounders that are common and strong enough to abrogate the primary model findings are unlikely. Further study with instrumental variables, if available, are indicated. Second, per the NYSDOH reporting guidelines, the performance of the 1-hour bundle was measured at the receiving hospital for patients cared for at multiple locations and may omit care administered at the referring hospital. Exclusion of these patients did not change the results.

Third, the moment the clinical team suspected sepsis and initiated the protocol may also be subject to misclassification, a limitation that could be addressed using electronic health record data in the future.26 Fourth, the study sample did not include neonates who were never discharged and cases not reported to the NYSDOH. Fifth, although the risk adjustment model performed well, a validated pediatric severity illness score or confounders, such as acute kidney injury, presence of central catheter, appropriateness of antimicrobials, or the variability across hospitals in sepsis case identification in children were not measured. Sixth, all hospitals were mandated to include the 3 primary sepsis elements in the bundle, and data on protocol modifications and additions were not available. Given these limitations, the true effect size of timely bundle completion may be smaller than measured in this study and could be further evaluated in designs that incorporate data from before and after the implementation of the mandate.

Conclusions

In New York State following a mandate for sepsis care, completion of a sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour was associated with lower risk-adjusted in-hospital mortality among patients with pediatric sepsis and septic shock.

Back to top
Article Information

Corresponding Author: Christopher W. Seymour, MD, MSc, Departments of Critical Care Medicine and Emergency Medicine, University of Pittsburgh School of Medicine, Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, 3550 Terrace St, Scaife Hall, No. 639, Pittsburgh, PA 15261 (seymourcw@upmc.edu).

Accepted for Publication: June 8, 2018.

Author Contributions: Dr Seymour had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design:  Evans, Friedrich, Levy, Parker, Terry, Watson, Weiss, Seymour.

Acquisition, analysis, or interpretation of data: Evans, Phillips, Alpern, Angus, Kissoon, Lemeshow, Levy, Parker, Terry, Watson, Weiss, Zimmerman, Seymour.

Drafting of the manuscript: Evans, Phillips, Kissoon, Levy, Seymour.

Critical revision of the manuscript for important intellectual content: Phillips, Alpern, Angus, Friedrich, Kissoon, Lemeshow, Levy, Parker, Terry, Watson, Weiss, Zimmerman, Seymour.

Statistical analysis: Phillips, Lemeshow, Terry, Weiss, Seymour.

Obtained funding: Friedrich.

Administrative, technical, or material support: Evans, Friedrich, Parker, Terry, Seymour.

Supervision: Angus, Friedrich, Kissoon, Seymour.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Evans reported receiving grants from the National Institutes of Health. Mr Phillips received personal fees from IPRO for conduct of statistical analyses. Dr Seymour reported receiving grants from the National Institutes of Health/National Institute of General Medical Sciences and personal fees from Beckman Coulter and Edwards Inc. No other disclosures were reported.

Funding/Support: Drs Seymour and Angus were supported in part by grants from the National Institutes of Health (R35GM119519). Dr Evans is supported in part by a grant from the National Institutes of Health (T32HL007820). Dr Weiss is supported by a grant from the National Institute of General Medical Sciences (K23GM110496).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: Derek C. Angus is Associate Editor of JAMA, but he was not involved in any of the decisions regarding review of the manuscript or its acceptance.

Additional Contributions: We thank the contributions of the New York State Sepsis Advisory Group and collaborating hospitals. We acknowledge the contributions of Colin Cooke, MD, University of Michigan, and Jason Kennedy, MS, University of Pittsburgh, to the quantitative bias analysis. Neither received compensation for their contributions.

References
1.
Balamuth  F, Weiss  SL, Neuman  MI,  et al.  Pediatric severe sepsis in US children’s hospitals.  Pediatr Crit Care Med. 2014;15(9):798-805. doi:10.1097/PCC.0000000000000225PubMedGoogle ScholarCrossref
2.
Dellinger  RP, Levy  MM, Rhodes  A,  et al; Surviving Sepsis Campaign Guidelines Committee including The Pediatric Subgroup.  Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012.  Intensive Care Med. 2013;39(2):165-228. doi:10.1007/s00134-012-2769-8PubMedGoogle ScholarCrossref
3.
Cruz  AT, Perry  AM, Williams  EA, Graf  JM, Wuestner  ER, Patel  B.  Implementation of goal-directed therapy for children with suspected sepsis in the emergency department.  Pediatrics. 2011;127(3):e758-e766. doi:10.1542/peds.2010-2895PubMedGoogle ScholarCrossref
4.
Larsen  GY, Mecham  N, Greenberg  R.  An emergency department septic shock protocol and care guideline for children initiated at triage.  Pediatrics. 2011;127(6):e1585-e1592. doi:10.1542/peds.2010-3513PubMedGoogle ScholarCrossref
5.
Dwyer  J. An infection, unnoticed, turns unstoppable. The New York Times. https://www.nytimes.com/2012/07/12/nyregion/in-rory-stauntons-fight-for-his-life-signs-that-went-unheeded.html. Published July 11, 2012. Accessed July 20, 2017.
6.
New York State Department of Health. New York State report on sepsis care improvement initiative: hospital quality performance. https://www. health.ny.gov/ press/ reports/ docs/2015_sepsis_care_improvement_initiative.pdf. Published March 2017. Accessed December 22, 2017.
7.
Brierley  J, Carcillo  JA, Choong  K,  et al.  Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine [published correction appears in Crit Care Med. 2009;37(4):1536].  Crit Care Med. 2009;37(2):666-688. doi:10.1097/CCM.0b013e31819323c6PubMedGoogle ScholarCrossref
8.
Goldstein  B, Giroir  B, Randolph  A; International Consensus Conference on Pediatric Sepsis.  International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics.  Pediatr Crit Care Med. 2005;6(1):2-8. doi:10.1097/01.PCC.0000149131.72248.E6PubMedGoogle ScholarCrossref
9.
Seymour  CW, Gesten  F, Prescott  HC,  et al.  Time to treatment and mortality during mandated emergency care for sepsis.  N Engl J Med. 2017;376(23):2235-2244. doi:10.1056/NEJMoa1703058PubMedGoogle ScholarCrossref
10.
Psaty  BM, Koepsell  TD, Lin  D,  et al.  Assessment and control for confounding by indication in observational studies.  J Am Geriatr Soc. 1999;47(6):749-754. doi:10.1111/j.1532-5415.1999.tb01603.xPubMedGoogle ScholarCrossref
11.
Odetola  FO, Gebremariam  A, Freed  GL.  Patient and hospital correlates of clinical outcomes and resource utilization in severe pediatric sepsis.  Pediatrics. 2007;119(3):487-494. doi:10.1542/peds.2006-2353PubMedGoogle ScholarCrossref
12.
Watson  RS, Carcillo  JA, Linde-Zwirble  WT, Clermont  G, Lidicker  J, Angus  DC.  The epidemiology of severe sepsis in children in the United States.  Am J Respir Crit Care Med. 2003;167(5):695-701. doi:10.1164/rccm.200207-682OCPubMedGoogle ScholarCrossref
13.
Leclerc  F, Leteurtre  S, Duhamel  A,  et al.  Cumulative influence of organ dysfunctions and septic state on mortality of critically ill children.  Am J Respir Crit Care Med. 2005;171(4):348-353. doi:10.1164/rccm.200405-630OCPubMedGoogle ScholarCrossref
14.
Dombrovskiy  VY, Martin  AA, Sunderram  J, Paz  HL.  Occurrence and outcomes of sepsis: influence of race.  Crit Care Med. 2007;35(3):763-768. doi:10.1097/01.CCM.0000256726.80998.BFPubMedGoogle ScholarCrossref
15.
New York State Department of Health. Statewide Planning and Research Cooperative System (SPARCS). https://www.health.ny.gov/statistics/sparcs/. Accessed May 18, 2018.
16.
New York State Department of Health. Health facility certification information. https://health.data.ny.gov/Health/Health-Facility-Certification-Information/2g9y-7kqm/data. Accessed January 4, 2018.
17.
Rosenberg  DI, Moss  MM; American College of Critical Care Medicine of the Society of Critical Care Medicine.  Guidelines and levels of care for pediatric intensive care units.  Crit Care Med. 2004;32(10):2117-2127. doi:10.1097/01.CCM.0000142704.36378.E9PubMedGoogle ScholarCrossref
18.
Royston  P, Sauerbrei  W.  Building multivariable regression models with continuous covariates in clinical epidemiology: with an emphasis on fractional polynomials.  Methods Inf Med. 2005;44(4):561-571. doi:10.1055/s-0038-1634008PubMedGoogle ScholarCrossref
19.
Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment.  Health Serv Res. 2010;45(6, pt 1):1614-1629. doi:10.1111/j.1475-6773.2010.01158.xPubMedGoogle ScholarCrossref
20.
Czaja  AS, Zimmerman  JJ, Nathens  AB.  Readmission and late mortality after pediatric severe sepsis.  Pediatrics. 2009;123(3):849-857. doi:10.1542/peds.2008-0856PubMedGoogle ScholarCrossref
21.
Williams  K, Thomson  D, Seto  I,  et al; StaR Child Health Group.  Standard 6: age groups for pediatric trials.  Pediatrics. 2012;129(3)(suppl 3):S153-S160. doi:10.1542/peds.2012-0055IPubMedGoogle ScholarCrossref
22.
Lash  TL, Fox  MP, Fink  AK.  Applying Quantitative Bias Analysis to Epidemiologic Data. New York, NY: Springer-Verlag; 2009.Crossref
23.
Balamuth  F, Weiss  SL, Fitzgerald  JC,  et al.  Protocolized treatment is associated with decreased organ dysfunction in pediatric severe sepsis.  Pediatr Crit Care Med. 2016;17(9):817-822. doi:10.1097/PCC.0000000000000858PubMedGoogle ScholarCrossref
24.
Davis  AL, Carcillo  JA, Aneja  RK,  et al.  American College of Critical Care Medicine clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock.  Crit Care Med. 2017;45(6):1061-1093. doi:10.1097/CCM.0000000000002425PubMedGoogle ScholarCrossref
25.
Kuhlmann  S, Mason  B, Ahlers-Schmidt  CR.  A quality improvement project to improve compliance with the joint commission children’s asthma care-3 measure.  Hosp Pediatr. 2013;3(1):45-51. doi:10.1542/hpeds.2012-0015PubMedGoogle ScholarCrossref
26.
Seymour  CW, Liu  VX, Iwashyna  TJ,  et al.  Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).  JAMA. 2016;315(8):762-774. doi:10.1001/jama.2016.0288PubMedGoogle ScholarCrossref
×