[Skip to Navigation]
Sign In
Figure 1.  Time to First Antibiotic Administration by Study Period and by Patient Subgroups Over Time
Time to First Antibiotic Administration by Study Period and by Patient Subgroups Over Time

Patient subgroups were defined by presenting temperature and blood pressure measured during the 25 hours surrounding emergency department presentation (24 hours before arrival to 1 hour after arrival). Specifically, patients were classified as normothermic (≥36 °C and ≤38 °C), hypothermic (<36 °C), or hyperthermic (>38 °C) and as hypotensive (systolic blood pressure <90 mm Hg) or normotensive (≥90 mm Hg). Circles indicate median time-to-antibiotics.

Figure 2.  Temporal Trends in Time-to-Antibiotics Among Hospitals With the Largest, Middle, and Least Decline in Time-to-Antibiotics From 2013 to 2018
Temporal Trends in Time-to-Antibiotics Among Hospitals With the Largest, Middle, and Least Decline in Time-to-Antibiotics From 2013 to 2018

Tertile 1 declined by a median 19.1 minutes per year (44 hospitals); tertile 2 declined by a median of 10.1 minutes per year (43 hospitals); tertile 3 declined by a median 2.8 minutes per year (43 hospitals). Orange dots indicate median time-to-antibiotics per year; blue lines, time-to-antibiotics per individual hospital.

Figure 3.  Variation in Median Time-to-Antibiotics by Hospital from 2013 to 2018
Variation in Median Time-to-Antibiotics by Hospital from 2013 to 2018

Individual hospitals are ordered from fastest to slowest time-to-antibiotics. The median time-to-antibiotics for each individual hospital is presented as a blue dot with corresponding 95% CIs (whiskers). The blue dotted line indicates the overall median time-to-antibiotic. The model intraclass correlation is 0.075 for 2013 to 2014, 0.081 for 2015 to 2016, and 0.068 for 2017 to 2018.

Table 1.  Characteristics of Sepsis Hospitalizations in the Department of Veterans Affairs Health Care System From 2013 to 2018
Characteristics of Sepsis Hospitalizations in the Department of Veterans Affairs Health Care System From 2013 to 2018
Table 2.  Annual Change in Time-to-Antibiotics in Primary and Sensitivity Analyses
Annual Change in Time-to-Antibiotics in Primary and Sensitivity Analyses
1.
Rhee  C, Dantes  R, Epstein  L,  et al; CDC Prevention Epicenter Program.  Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.   JAMA. 2017;318(13):1241-1249. doi:10.1001/jama.2017.13836PubMedGoogle ScholarCrossref
2.
Angus  DC, Linde-Zwirble  WT, Lidicker  J, Clermont  G, Carcillo  J, Pinsky  MR.  Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care.   Crit Care Med. 2001;29(7):1303-1310. doi:10.1097/00003246-200107000-00002 PubMedGoogle ScholarCrossref
3.
Buchman  TG, Simpson  SQ, Sciarretta  KL,  et al.  Sepsis among Medicare beneficiaries: 1. the burdens of sepsis, 2012-2018.   Crit Care Med. 2020;48(3):276-288. doi:10.1097/CCM.0000000000004224PubMedGoogle ScholarCrossref
4.
Singer  M, Deutschman  CS, Seymour  CW,  et al.  The third international consensus definitions for sepsis and septic shock (sepsis-3).   JAMA. 2016;315(8):801-810. doi:10.1001/jama.2016.0287PubMedGoogle ScholarCrossref
5.
Rhodes  A, Evans  LE, Alhazzani  W,  et al.  Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock: 2016.   Crit Care Med. 2017;45(3):486-552. doi:10.1097/CCM.0000000000002255 PubMedGoogle ScholarCrossref
6.
Dugar  S, Choudhary  C, Duggal  A.  Sepsis and septic shock: guideline-based management.   Cleve Clin J Med. 2020;87(1):53-64. doi:10.3949/ccjm.87a.18143 PubMedGoogle ScholarCrossref
7.
Levy  MM, Evans  LE, Rhodes  A.  The Surviving Sepsis Campaign bundle: 2018 update.   Intensive Care Med. 2018;44(6):925-928. doi:10.1007/s00134-018-5085-0 PubMedGoogle ScholarCrossref
8.
Angus  DC, van der Poll  T.  Severe sepsis and septic shock.   N Engl J Med. 2013;369(9):840-851. doi:10.1056/NEJMra1208623 PubMedGoogle ScholarCrossref
9.
Liu  VX, Fielding-Singh  V, Greene  JD,  et al.  The timing of early antibiotics and hospital mortality in sepsis.   Am J Respir Crit Care Med. 2017;196(7):856-863. doi:10.1164/rccm.201609-1848OC PubMedGoogle ScholarCrossref
10.
Seymour  CW, Gesten  F, Prescott  HC,  et al.  Time to treatment and mortality during mandated emergency care for sepsis.   N Engl J Med. 2017;376(23):2235-2244. doi:10.1056/NEJMoa1703058 PubMedGoogle ScholarCrossref
11.
Peltan  ID, Brown  SM, Bledsoe  JR,  et al.  ED door-to-antibiotic time and long-term mortality in sepsis.   Chest. 2019;155(5):938-946. doi:10.1016/j.chest.2019.02.008 PubMedGoogle ScholarCrossref
12.
Centers for Medicare & Medicaid Services. Severe sepsis and septic shock: management bundle (composite measure). Accessed October 30, 2020. https://cmit.cms.gov/CMIT_public/ViewMeasure?MeasureId=1017
13.
New York Codes, Rules and Regulations. 10 CRR-NY §405.4. Accessed August 6, 2021. https://regs.health.ny.gov/content/section-4054-medical-staff
14.
Levy  MM, Gesten  FC, Phillips  GS,  et al; the Results of the New York State Initiative.  Mortality changes associated with mandated public reporting for sepsis the results of the New York state initiative.   Am J Respir Crit Care Med. 2018;198(11):1406-1412. doi:10.1164/rccm.201712-2545OC PubMedGoogle ScholarCrossref
15.
Levy  MM, Dellinger  RP, Townsend  SR,  et al.  The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis.   Intensive Care Med. 2010;36(2):222-231. doi:10.1007/s00134-009-1738-3 PubMedGoogle ScholarCrossref
16.
Miller  RR  III, Dong  L, Nelson  NC,  et al; Intermountain Healthcare Intensive Medicine Clinical Program.  Multicenter implementation of a severe sepsis and septic shock treatment bundle.   Am J Respir Crit Care Med. 2013;188(1):77-82. doi:10.1164/rccm.201212-2199OC PubMedGoogle ScholarCrossref
17.
National Center for Veterans Analysis and Statistics. VA utilization profile FY 2017. Accessed October 30, 2020. https://www.va.gov/vetdata/docs/Quickfacts/VA_Utilization_Profile_2017.pdf
18.
Fihn  SD, Francis  J, Clancy  C,  et al.  Insights from advanced analytics at the veterans health administration.   Health Aff (Millwood). 2014;33(7):1203-1211. doi:10.1377/hlthaff.2014.0054 PubMedGoogle ScholarCrossref
19.
Vincent  BM, Wiitala  WL, Burns  JA, Iwashyna  TJ, Prescott  HC.  Using Veterans Affairs Corporate Data Warehouse to identify 30-day hospital readmissions.   Heal Serv Outcomes Res Methodol. 2018;18(3):143-154. doi:10.1007/s10742-018-0178-3 Google ScholarCrossref
20.
Wayne  MT, Molling  D, Wang  XQ,  et al.  Measurement of sepsis in a national cohort using three different methods to define baseline organ function.   Ann Am Thorac Soc. 2021;18(4):648-655. doi:10.1513/AnnalsATS.202009-1130OC PubMedGoogle ScholarCrossref
21.
Centers for Disease Control and Prevention. Hospital toolkit for adult sepsis surveillance. Accessed August 6, 2021. https://www.cdc.gov/sepsis/pdfs/Sepsis-Surveillance-Toolkit-Aug-2018_508.pdf
22.
Bone  RC, Balk  RA, Cerra  FB,  et al; The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.  Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis.   Chest. 1992;101(6):1644-1655. doi:10.1378/chest.101.6.1644PubMedGoogle ScholarCrossref
23.
Bell  BA, Ferron  JM, Kromrey  JD.  Cluster Size in Multilevel Models: The Impact of Sparse Data Structures on Point and Interval Estimates in Two-Level Models. Wickrama & Bryant; 2004.
24.
Austin  PC, Leckie  G.  The effect of number of clusters and cluster size on statistical power and Type I error rates when testing random effects variance components in multilevel linear and logistic regression models.   J Stat Comput Simul. 2018;88(16):3151-3163. doi:10.1080/00949655.2018.1504945 Google ScholarCrossref
25.
Maas  CJM, Hox  JJ.  Sufficient sample sizes for multilevel modeling.   Methodology. 2005;1(3):86-92. doi:10.1027/1614-2241.1.3.86 Google ScholarCrossref
26.
Clarke  P.  When can group level clustering be ignored: multilevel models versus single-level models with sparse data.   J Epidemiol Community Health. 2008;62(8):752-758. doi:10.1136/jech.2007.060798 PubMedGoogle ScholarCrossref
27.
Wang  XQ, Vincent  BM, Wiitala  WL,  et al.  Veterans Affairs patient database (VAPD 2014-2017): building nationwide granular data for clinical discovery.   BMC Med Res Methodol. 2019;19(1):94. doi:10.1186/s12874-019-0740-x PubMedGoogle ScholarCrossref
28.
Elixhauser  A, Steiner  C, Harris  DR, Coffey  RM.  Comorbidity measures for use with administrative data.   Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004 PubMedGoogle ScholarCrossref
29.
Turner  K, Burchill  C. Elixhauser Comorbidity Index macro code using ICD-10-CA. Accessed February 25, 2021. http://mchp-appserv.cpe.umanitoba.ca/Upload/SAS/_ElixhauserICD10.sas.txt
30.
Quan  H, Sundararajan  V, Halfon  P,  et al.  Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.   Med Care. 2005;43(11):1130-1139. doi:10.1097/01.mlr.0000182534.19832.83 PubMedGoogle ScholarCrossref
31.
Merlo  J, Chaix  B, Yang  M, Lynch  J, Råstam  L.  A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon.   J Epidemiol Community Health. 2005;59(6):443-449. doi:10.1136/jech.2004.023473 PubMedGoogle ScholarCrossref
32.
Sanagou  M, Wolfe  R, Forbes  A, Reid  CM.  Hospital-level associations with 30-day patient mortality after cardiac surgery: a tutorial on the application and interpretation of marginal and multilevel logistic regression.   BMC Med Res Methodol. 2012;12:28. doi:10.1186/1471-2288-12-28 PubMedGoogle ScholarCrossref
33.
Prescott  HC, Iwashyna  TJ.  Improving sepsis treatment by embracing diagnostic uncertainty.   Ann Am Thorac Soc. 2019;16(4):426-429. doi:10.1513/AnnalsATS.201809-646PS PubMedGoogle ScholarCrossref
34.
Liu  VX, Bhimarao  M, Greene  JD,  et al.  The presentation, pace, and profile of infection and sepsis patients hospitalized through the emergency department: an exploratory analysis.   Crit Care Explor. 2021;3(3):e0344. doi:10.1097/CCE.0000000000000344 PubMedGoogle Scholar
35.
Ferrer  R, Artigas  A, Levy  MM,  et al; Edusepsis Study Group.  Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain.   JAMA. 2008;299(19):2294-2303. doi:10.1001/jama.299.19.2294PubMedGoogle ScholarCrossref
36.
Zubkoff  L, Neily  J, King  BJ,  et al.  Virtual Breakthrough Series, part 1: preventing catheter-associated urinary tract infection and hospital-acquired pressure ulcers in the veterans health administration.   Jt Comm J Qual Patient Saf. 2016;42(11):485-AP2. doi:10.1016/S1553-7250(16)42091-X PubMedGoogle Scholar
37.
Zubkoff  L, Neily  J, Quigley  P,  et al.  Virtual breakthrough series, part 2: improving fall prevention practices in the veterans health administration.   Jt Comm J Qual Patient Saf. 2016;42(11):497-AP12. doi:10.1016/S1553-7250(16)42092-1 PubMedGoogle Scholar
38.
Corl  K, Levy  M, Phillips  G, Terry  K, Friedrich  M, Trivedi  AN.  Racial and ethnic disparities in care following the New York state sepsis initiative.   Health Aff (Millwood). 2019;38(7):1119-1126. doi:10.1377/hlthaff.2018.05381 PubMedGoogle ScholarCrossref
39.
Prescott  HC, Belloli  EA.  Claims-based ICU research: learning from imperfect data.   Crit Care Med. 2017;45(7):1263-1264. doi:10.1097/CCM.0000000000002396 PubMedGoogle ScholarCrossref
40.
Rhee  C, Gohil  S, Klompas  M.  Regulatory mandates for sepsis care—reasons for caution.   N Engl J Med. 2014;370(18):1673-1676. doi:10.1056/NEJMp1400276 PubMedGoogle ScholarCrossref
41.
Prescott  HC, Cope  TM, Gesten  FC,  et al.  Reporting of sepsis cases for performance measurement versus for reimbursement in New York State.   Crit Care Med. 2018;46(5):666-673. doi:10.1097/CCM.0000000000003005 PubMedGoogle ScholarCrossref
42.
Taylor  SP, Anderson  WE, Beam  K, Taylor  B, Ellerman  J, Kowalkowski  MA.  The association between antibiotic delay intervals and hospital mortality among patients treated in the emergency department for suspected sepsis.   Crit Care Med. 2021;49(5):741-747. doi:10.1097/CCM.0000000000004863 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Critical Care Medicine
    September 7, 2021

    Temporal Trends and Hospital Variation in Time-to-Antibiotics Among Veterans Hospitalized With Sepsis

    Author Affiliations
    • 1Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, University of Michigan, Ann Arbor
    • 2VA Center for Clinical Management Research, Ann Arbor, Michigan
    • 3Institute for Healthcare Policy & Innovation, University of Michigan, Ann Arbor
    • 4Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor
    • 5Salt Lake City VA Healthcare System, Salt Lake City, Utah
    • 6Department of Medicine, University of Utah, Salt Lake City
    • 7Division of Research, Kaiser Permanente Northern California, Oakland
    JAMA Netw Open. 2021;4(9):e2123950. doi:10.1001/jamanetworkopen.2021.23950
    Key Points

    Question  How has timing of antibiotics for sepsis changed over time in US Department of Veterans Affairs Hospitals?

    Findings  In this cohort of 111 385 veterans hospitalized with sepsis from 2013 to 2018, median time to antibiotics declined by 9 minutes per year. However, there was significant variation in time to antibiotics across hospitals, even after adjustment for patient characteristics.

    Meaning  These findings suggest that there is potential for performance improvement for sepsis hospitalizations, but efforts to further accelerate time-to-antibiotics must be weighed against the risks of overtreatment.

    Abstract

    Importance  It is unclear whether antimicrobial timing for sepsis has changed outside of performance incentive initiatives.

    Objective  To examine temporal trends and variation in time-to-antibiotics for sepsis in the US Department of Veterans Affairs (VA) health care system.

    Design, Setting, and Participants  This observational cohort study included 130 VA hospitals from 2013 to 2018. Participants included all patients admitted to the hospital via the emergency department with sepsis from 2013 to 2018, using a definition adapted from the Centers for Disease Control and Prevention Adult Sepsis Event definition, which requires evidence of suspected infection, acute organ dysfunction, and systemic antimicrobial therapy within 12 hours of presentation. Data were analyzed from October 6, 2020, to July 1, 2021.

    Exposures  Time from presentation to antibiotic administration.

    Main Outcomes and Measures  The main outcome was differences in time-to-antibiotics across study periods, hospitals, and patient subgroups defined by presenting temperature and blood pressure. Temporal trends in time-to-antibiotics were measured overall and by subgroups. Hospital-level variation in time-to-antibiotics was quantified after adjusting for differences in patient characteristics using multilevel linear regression models.

    Results  A total of 111 385 hospitalizations for sepsis were identified, including 107 547 men (96.6%) and 3838 women (3.4%) with a median (interquartile range [IQR]) age of 68 (62-77) years. A total of 7574 patients (6.8%) died in the hospital, and 13 855 patients (12.4%) died within 30 days. Median (IQR) time-to-antibiotics was 3.9 (2.4-6.5) hours but differed by presenting characteristics. Unadjusted median (IQR) time-to-antibiotics decreased over time, from 4.5 (2.7-7.1) hours during 2013 to 2014 to 3.5 (2.2-5.9) hours during 2017 to 2018 (P < .001). In multilevel models adjusted for patient characteristics, median time-to-antibiotics declined by 9.0 (95% CI, 8.8-9.2) minutes per calendar year. Temporal trends in time-to-antibiotics were similar across patient subgroups, but hospitals with faster baseline time-to-antibiotics had less change over time, with hospitals in the slowest tertile decreasing time-to-antibiotics by 16.6 minutes (23.1%) per year, while hospitals in the fastest tertile decreased time-to-antibiotics by 7.2 minutes (13.1%) per year. In the most recent years (2017-2018), median time-to-antibiotics ranged from 3.1 to 6.7 hours across hospitals (after adjustment for patient characteristics), 6.8% of variation in time-to-antibiotics was explained at the hospital level, and odds of receiving antibiotics within 3 hours increased by 65% (95% CI, 56%-77%) for the median patient if moving to a hospital with faster time-to-antibiotics.

    Conclusions and Relevance  This cohort study across nationwide VA hospitals found that time-to-antibiotics for sepsis has declined over time. However, there remains significant variability in time-to-antibiotics not explained by patient characteristics, suggesting potential unwarranted practice variation in sepsis treatment. Efforts to further accelerate time-to-antibiotics must be weighed against risks of overtreatment.

    Introduction

    Sepsis, life-threatening organ dysfunction secondary to infection, is a common, costly, and lethal syndrome, affecting more than 1.7 million patients annually in the United States.1-4 Sepsis quality improvement initiatives focus on rapid identification and administration of antimicrobials and fluids because there is higher quality evidence supporting these practices.5-8 Indeed, faster receipt of antimicrobial therapy has been associated with improved survival, particularly for patients who present with shock.9-11

    Over the past 15 years, there has been growing attention to the importance of timely antimicrobial administration in sepsis. The 2016 Surviving Sepsis Campaign guidelines included a strong recommendation to administer antimicrobial therapy as soon as possible, ideally within 1 hour of recognition.5 Likewise, the US Center for Medicare and Medicaid Services’ SEP-1 performance measure and New York state’s “Rory’s Regulations” both incentivize timely antimicrobial administration.10,12,13 These and other sepsis performance improvement programs have resulted in faster administration of antimicrobials for their target populations, with associated improvements in inpatient sepsis survival over time.14-16 However, it is unclear whether antimicrobial timing for sepsis has changed outside of these formal performance incentive programs, and whether temporal trends in time-to-antibiotics have differed across hospitals or among patients who present with more obvious signs of sepsis.

    In this study, we examined variation and temporal trends in time to antimicrobials for sepsis in 130 hospitals in the US Department of Veterans Affairs (VA) health care system. We hypothesized that time-to-antibiotics has decreased over time, that the magnitude of decrease has been uneven across patient subgroups and hospitals, and that time-to-antibiotics continues to vary across hospitals, even after adjustment for patient characteristics.

    Methods

    The cohort study was deemed exempt from review with a waiver of informed consent by the VA Ann Arbor institutional review board. This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for observational studies.

    Study Setting and Patient Cohort

    The US VA health care system is a comprehensive health care system that operates a diverse set of hospitals.17,18 During the study period (2013-2018), the VA used a single electronic health record, with data archived to a central data repository.18 We identified all hospitalizations19 at nationwide VA hospitals admitted through the emergency department (ED) with community-acquired sepsis.

    We defined community-acquired sepsis as in prior research,20 using a definition adapted from the Centers for Disease Control and Prevention (CDC) Adult Sepsis Event (ASE) definition.21 Specifically, we identified hospitalizations meeting the following criteria: admitted through the ED with 2 or more systemic inflammatory response (SIRS) criteria10,22; initiated on systemic antimicrobial therapy included in the CDC ASE definition21 within 12 hours of presentation10; continued on antibiotics for at least 4 consecutive days (or died prior to 4 days while receiving consecutive days of antibiotics); and had objective evidence of acute organ dysfunction20 within 48 hours of presentation. We further limited our cohort to hospitalizations at facilities with at least 15 eligible hospitalizations so that we could reliably measure variation across hospitals.23-26

    For each hospitalization, we extracted demographics, comorbidities, vital signs, laboratory values, organ supports, length of stay, time of presentation to the ED, and mortality from the central data repository.27 Time-to-antibiotics was defined as the time from ED presentation to the time of initiation of the first antibiotic. Timing of antibiotic administration was determined via barcode medication administration (BCMA) data and physician order-entry records. For patients with an eligible antimicrobial ordered during their ED stay, we used BCMA where available. However, most VA EDs do not use BCMA, so we considered the time of antimicrobial administration to be the order-entry time plus 45 minutes. However, if patients were admitted to the hospital within 45 minutes of their antimicrobial order, then we considered time of transfer from ED-to-inpatient to be the time of first antimicrobial administration. Second, for patients without an antimicrobial order in the ED, we used BCMA to determine time of first antimicrobial administration (eAppendix in the Supplement). In sensitivity analyses, we considered several alternate approaches to defining time-to-antibiotics.

    Statistical Analysis
    Temporal Trends in Time-to-Antibiotics for Sepsis

    We present time-to-antibiotic results in minutes (when <60 minutes) and hours (when >60 minutes). We focus on median (rather than mean) and interquartile range (IQR) time-to-antibiotics because of skewed distribution.

    To measure temporal trends in time-to-antibiotics for sepsis, we divided our study into early (2013-2014), middle (2015-2016), and late (2017-2018) periods. We examined the distribution of time-to-antibiotics by period, measured changes in the median time-to-antibiotics across periods, and tested for differences using a nonparametric test for trend. We compared the proportion of patients receiving antibiotics within 0 to 3, 0 to 6, and 0 to 9 hours over time. Finally, we fit multilevel linear regression models (hospitalizations nested within hospitals) estimating time-to-antibiotics, adjusting for patient characteristics (age, sex, 30 comorbid conditions identified using diagnosis codes from inpatient and outpatient encounters in the prior 1.5 years,28-30 individual SIRS criteria, lactate elevation, and 6 acute organ dysfunctions included in the CDC ASE definition21). We included a variable for calendar quarter and estimated the random slope for each hospital quarter to determine overall temporal trends in time-to-antibiotics. We calculated median time-to-antibiotics using fitted values for each patient that included the fixed-portion linear factor and contributions from the estimated random effects. In sensitivity analyses, we considered several alternate approaches to modeling. Additionally, in preliminary analyses, we assessed for a possible quadratic nonlinear association between calendar quarter and time-to-antibiotics, but we found no evidence of a nonlinear association during the study timeframe.

    Variation in Time-to-Antibiotics and Temporal Trends by Patient Subgroups

    We hypothesized that time-to-antibiotics and temporal trends in time-to-antibiotics may differ across hospitals and by patient subgroups. Specifically, we hypothesized that patients with more obvious signs of sepsis (eg, fever, hypotension) may have faster time-to-antibiotics and greater acceleration over time, that hospitals with faster baseline time-to-antibiotics in 2013 to 2014 would have less absolute change over time owing to floor effects, and that there would be clinically-significant variation in time-to-antibiotics across hospitals, even after adjustment for differences in patient characteristics.

    To test these hypotheses, we assessed variation in time-to-antibiotics and temporal trends in time-to-antibiotics by hospitals, by tertiles of hospitals (grouped by baseline time-to-antibiotics), and by patient subgroups. Patient subgroups were defined by presenting temperature and blood pressure measured during the 25 hours surrounding ED presentation (24 hours pre-ED arrival to 1 hour post-ED arrival). Specifically, patients were classified as normothermic (≥36 °C and ≤38 °C), hypothermic (<36 °C), or hyperthermic (>38 °C), and as hypotensive (systolic blood pressure <90 mm Hg) or not hypotensive (systolic blood pressure ≥90 mm Hg). We used the most abnormal measurement during the 25-hour time-window of interest to classify patients and excluded (from this analysis only) 159 patients with both hypothermic and hyperthermic temperatures recorded. We assumed normal values when no measurements were recorded.

    To quantify the variation in temporal trends across hospitals, we fit random-slope multilevel linear regression models with hospitalizations nested within hospitals, adjusting for patient characteristics as described above. We present variation in median time-to-antibiotics across hospitals in a caterpillar plot and assessed variation in slopes across hospitals. We measured the intraclass correlation (ICC) to quantify the proportion of variation in time-to-antibiotics attributable to the hospital where the patient was treated. The ICC measures the variation at the cluster level (ie, the hospital level) relative to total variation, where a value close to 0 indicates a low proportion of total variance explained at the hospital level and a value close to 1 indicates that nearly all variance in time-to-antibiotics is explained at the hospital level.31 We also calculated the median odds ratio (MOR) for receipt of antibiotics within 3 hours as a measure of variation between hospitals not explained by individual characteristics in the model.32 A MOR of 1.0 implies that odds of antibiotics within 3 hours is equivalent across hospitals; the larger the MOR, the more important hospital-level effects are in driving differences in time-to-antibiotics.

    Data management and analysis were performed in SAS version 9.4 (SAS Institute) and Stata/MP version 16.1 (StataCorp). We considered 2-sided P < .05 to be significant. Data were analyzed from October 6, 2020, to July 1, 2021.

    Results
    Study Cohort

    A total of 111 385 hospitalizations met criteria for community-acquired sepsis during the study period (eFigure 1 in the Supplement), including 107 547 men (96.6%) and 3838 women (3.4%) with median (IQR) age of 68 (62-77) years (Table 1). Included patients had a median (IQR) of 2 (1-3) comorbid conditions. The most common acute organ dysfunctions were acute kidney dysfunction (68 191 patients [61.2%]), elevated lactate (53 136 patients [47.7%]), and thrombocytopenia (15 275 patients [13.7%]). A total 76 518 patients (68.7%) received their first dose of antibiotics in the ED, while 34 867 patients (31.3%) received their first dose after admission to the hospital. Over time, more patients received antibiotics in the ED (eTable 1 in the Supplement). Time-to-antibiotics was defined by order time plus 45 minutes in 51 086 patients (45.9%), ED-to-hospital transfer time in 16 943 patients (15.2%), and BCMA time in 43,356 patients (38.9%) (8489 patients [7.6%] in the ED; 34 867 patients [31.3%] on the wards). Among 8489 patients (7.6%) of patients with BCMA data in the ED, the median (IQR) time from order to administration was 23 (15-34) minutes. A total of 7574 patients (6.8%) died in the hospital, and 13 855 patients (12.4%) died within 30 days of discharge. In a multilevel model adjusted for patient characteristics, longer time-to-antibiotics was associated with higher in-hospital mortality (OR per 1-hour from ED presentation, 1.01 [95% CI, 1.00-1.02]; P = .02) and 30-day mortality (OR per 1-hour from ED presentation, 1.02 [95% CI, 1.01-1.02]; P < .001), consistent with prior studies.9-11

    Patient characteristics by study time-period (early, middle, late) are presented in eTable 2 in the Supplement. Lactate elevation increased over time, likely reflective of greater testing, while all other acute organ functions declined (eTable 2 in the Supplement). In-hospital mortality declined from 2816 patients (8.1%) in the early period to 2330 patients (5.7%) in the late period, and 30-day mortality declined from 4876 patients (13.9%) in the early period to 4536 patients (11.2%) in the late period.

    Temporal Trend in Time-to-Antibiotics

    The median (IQR) time-to-antibiotics during the study was 3.9 (2.4-6.5) hours. Median (IQR) time-to-antibiotics declined from 4.5 (2.7-7.1) hours in the early period to 3.5 (2.2-5.9) hours in late period, an absolute change of 54.6 minutes and a relative change of 22.2% (P < .001) (Figure 1A). The proportion of patients receiving antibiotics within 0 to 3, 0 to 6, and 0 to 9 hours increased for each comparison (eFigure 2 in the Supplement). After adjusting for patient characteristics, median time-to-antibiotics declined by 9.0 (95% CI, 8.8-9.2) minutes per calendar year. Annual change in time-to-antibiotics was similar across multiple sensitivity analyses using different definitions for time-to-antibiotics and different modeling approaches (Table 2).

    Variation in Time-to-Antibiotics by Patient Characteristics

    Time-to-antibiotics varied markedly by patient characteristics. The distributions of time-to-antibiotics by patient subgroups (defined by presenting temperature and blood pressure) are shown in eFigure 3 in the Supplement. Median (IQR) time-to-antibiotics was 2.4 (1.6-4.1) hours for patients with fever and hypotension, 2.9 (1.8-4.7) hours for patients with fever and normal blood pressure, 3.6 (2.2-6.2) hours for patients without fever and with hypotension, and 4.3 (2.6-6.9) hours for patients without fever and with normal blood pressure (eFigure 3 in the Supplement). The association of individual patient characteristics with time-to-antibiotics are presented in eTable 3 in the Supplement. In fully adjusted models, lactate elevation (β = −36.9 [95% CI, −34.8 to −39.0] minutes; P < .001), abnormal white blood cell count (β = −30.9 [95% CI, −33.2 to −28.7] minutes; P < .001), abnormal temperature (β = −26.1 [95% CI, −24.1 to −28.0] minutes; P < .001), shock (β = −28.8 [95% CI, −32.0 to −25.6] minutes; P < .001), elevated respiratory rate (β = −21.2 [95% CI, −19.2 to −23.3] minutes; P < .001), and elevated heart rate (β = −17.9 [95% CI, −15.0 to −20.9] minutes; P < .001) were all associated with shorter time-to-antibiotics.

    Variation in Temporal Trends

    Time-to-antibiotics decreased over time in 112 hospitals (86.2%), but the magnitude of decline varied. Median time-to-antibiotics decreased by 19.1 (95% CI, 18.6-19.5) minutes per year in hospitals in the greatest tertile of decline, 10.1 (95% CI, 9.8-10.5) minutes per year in hospitals in the middle tertile of decline, and 2.8 (95% CI, 2.4-3.2) minutes per year for hospitals in the lowest tertile of decline, after adjustment for patient characteristics (Figure 2). Temporal trends likewise differed according to baseline time-to-antibiotics, decreasing by 16.6 minutes (23.1%) per year in hospitals with the slowest median time to antibiotics, 10.4 minutes (17.2%) per year in hospitals in the middle of the time-to-antibiotics tertile at baselines, and 7.2 minutes (13.1%) per year in hospitals with the fastest median time-to-antibiotics at baseline (P = .002 for the association of baseline tertile of time-to-antibiotics and tertile of change over time) (eTable 4 in the Supplement). Hospital characteristics were not associated with baseline time-to-antibiotics, but higher annual sepsis case-volume was associated with greater decline in time-to-antibiotics (eTable 5 and eTable 6 in the Supplement). While time-to-antibiotics varied by presenting temperature and blood pressure, declines over time were similar across these subgroups (Figure 1A; eTable 7 in the Supplement).

    Variation in Time-to-Antibiotics Across Hospitals in 2017 to 2018

    After adjusting for patient characteristics, median time-to-antibiotics varied by 118.2% across hospitals in 2017 to 2018—ranging from 3.1 to 6.7 hours (Figure 3). The ICC was 0.068, indicating that 6.8% of variation in time-to-antibiotics was explained at the hospital level. The MOR across hospitals for receipt of antibiotics within 3 hours of ED arrival was 1.65 (95% CI, 1.56-1.77), indicating a 65% increased odds of receiving antibiotics within 3 hours for the median patient if moving from a hospital with slower time-to-antibiotics to a hospital with faster time-to-antibiotics. The association between hospital and time-to-antibiotics was similar to the association of individual patient factors (eg, elevated lactate: OR, 1.61 [95% CI, 1.57-1.66]; shock: OR, 1.41 [95% CI, 1.35-1.47]). By contrast, in 2013 to 2014, median time-to-antibiotics varied by 95% (from 3.7 to 7.3 hours), the ICC was 7.5%, and the MOR for receipt of antibiotics within 3 hours was 1.73 (95% CI, 1.61-1.88).

    Discussion

    In this nationwide cohort study of patients hospitalized with sepsis at 130 VA hospitals, we found that the median time-to-antibiotics for sepsis declined over time by 9 minutes per year. This trend of decreasing time-to-antibiotics was observed in virtually all hospitals and across all patient subgroups defined by presenting temperature and blood pressure. However, the magnitude of decrease varied across hospitals, and hospitals with faster baseline time-to-antibiotics experienced less change over time on either an absolute or relative scale.

    Patient characteristics were associated with time-to-antibiotics. Most notably, lactate elevation, shock, and SIRS criteria (abnormal temperature, elevate heart rate, elevated respiratory rate, and abnormal white blood cell count) were associated with faster time-to-antibiotics, indicating that the urgency with which patient were administered antibiotics was associated with their clinical presentation. The variation in antibiotic timing observed across individual patients and the specific factors associated with faster delivery are generally consistent with a risk-based approach to antibiotic prescribing.33,34 However, time-to-antibiotics was faster for patients with fever and normal blood pressure than for patients without fever but with hypotension, suggesting that obvious signs of infection are a stronger trigger to prescribe antibiotics than shock or that competing treatments for patients with hypotension (eg, starting vasopressors) are prioritized before antibiotics, even though antibiotic delays are associated with greater mortality risk in patients with shock or hypotension.9,10

    Although time-to-antibiotics differed according to patient characteristics, change over time was similar across patient subgroups. For example, patients with fever got antibiotics faster across the full study period, but the slope of change over time was similar to patients without fever.

    While time-to-antibiotics declined overall and in nearly every hospital during the study, there remained more than 2-fold variation in median time-to-antibiotics in the most recent study years. This variation persisted after adjustment for granular patient characteristics, suggesting that sepsis practice patterns truly differ across hospitals. This may represent a potential opportunity for practice improvement going forward, but the benefits of further accelerating time-to-antibiotics must be balanced against the risk of driving antibiotic overuse in patients with noninfectious illness.33 It would be prudent to focus improvement efforts on patients with hypotension, for whom the risk of treatment delays is greatest.33

    The trend of decreasing time-to-antibiotics observed in our study is similar to trends observed in other settings, including in intensive care units in Spain after implementation of a sepsis quality improvement (QI) initiative,35 hospitals participating in the international Surviving Sepsis Campaign QI program,15 and in New York state after implementation of statewide sepsis regulations.14 While a study by Liu et al9 found no change in antibiotic timing in 21 hospitals in the Kaiser Permanente Northern California Healthcare System after implementation of a regional sepsis QI project, the short baseline time-to-antibiotics and study design choices (eg, restricting the cohort to patients who received antibiotics within 6 hours, as opposed to 12 hours) may have limited the opportunity and ability to detect change.

    The magnitude of change observed in our study (9 minutes per year) is greater than reported in all but 1 prior study,35 but our study population was less severely ill (hospital mortality rate, 6.8% vs 10% to 35% across prior studies9,10,14-16). The corresponding longer baseline time-to-antibiotics in our cohort may have provided a greater opportunity for decrease over time. Notably, however, prior studies were designed to assess changes in sepsis practice patterns after specific interventions, while ours was designed to test the change over time during a period of heightened national attention to sepsis, but not a specific sepsis intervention or quality improvement initiative. Indeed, the VA’s Nationwide Virtual Breakthrough Series36,37 on early identification and rapid treatment of sepsis in the emergency department occurred shortly after the study period (April to September 2019).

    The variation in time-to-antibiotics across hospitals in our study is generally consistent with prior studies, although differences in study design and measurement (eg, measuring baseline as ED presentation vs when meeting sepsis criteria) preclude direct comparisons. In New York state, after implementation of Rory’s Regulations, the adjusted proportion of patients receiving antibiotics within 3 hours ranged from 60% to nearly 100% across hospitals.10 However, the slope of change over time varied across hospitals in this initiative, resulting in a growing disparity in time-to-antibiotics by race as hospitals treating higher proportions of Black patients experienced less decrease over time.38 We likewise found varying slopes of change across hospitals, which may be related to a ceiling effect whereby hospitals with the fastest time-to-antibiotics at the start of our study period had the least amount of change over time.

    Limitations

    Our study has some limitations. First, there is no perfect method for identifying sepsis or defining “time zero.” We identified sepsis using the CDC surveillance definition because it identifies a stable population over time.21 Alternative methods, such as diagnostic codes, are subject to well-described temporal changes and varying sensitivities and specificities across hospitals,39 which would bias the measurement of temporal trends and cross-hospital comparisons of time-to-antibiotics.1,40 Our approach allowed for stable identification of sepsis over time and across hospitals and did not rely on physician or hospital identification of sepsis, which may result in underreporting. However, owing to increased ordering of lactate testing over time, this approach may have resulted in some differential ascertainment over time.41 Second, our definition required that patients received antibiotics within 12 hours. While the CDC ASE definition for community-acquired sepsis includes patients initiated on antibiotics within 48 hours of presentation, we used a 12-hour cutoff for consistency with prior studies of time-to-antibiotics.10,42 However, we tracked this exclusion, and only a small proportion of hospitalizations were excluded for having first antibiotic administration at more than 12 hours but less than 48 hours from ED presentation. Third, we are unable to determine the optimal time-to-antibiotics overall or for individual patients. While retrospective studies of patients with severe sepsis and septic shock have shown that faster time-to-antibiotics is associated with greater survival to hospital discharge, it is difficult to determine the presence of sepsis with certainty early in a patient’s presentation. Many patients prescribed antibiotics for potential sepsis turn out to have noninfectious causes of illness, and the benefits of early antibiotics must be balanced against the potential for overtreatment of patients who ultimately turn out to have noninfectious illness.33 Fourth, we used BCMA data to determine time of antibiotic administration where available, but for patients without these data, we estimated time-to-antibiotics based on physician order time. However, our assumption of 45 minutes from order to administration was conservative, and temporal trends were similar across several sensitivity analyses using alternate definitions for estimating time-to-antibiotics.

    Conclusions

    In this large, national cohort study of veterans hospitalized with community-acquired sepsis, time-to-antibiotics declined by 9 minutes per year in the absence of a formal incentive program. However, there remains significant variation in time-to-antibiotics across hospitals that was not explained by differences in patient characteristics.

    Back to top
    Article Information

    Accepted for Publication: July 1, 2021.

    Published: September 7, 2021. doi:10.1001/jamanetworkopen.2021.23950

    Correction: This article was corrected on September 30, 2021, to correct typographical errors in the Results sections of the Abstract and main text.

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Wayne MT et al. JAMA Network Open.

    Corresponding Author: Max T. Wayne, MD, Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, University of Michigan, 1500 E Medical Center Dr, SPC 5361, Ann Arbor, MI 48109-5361 (wmax@med.umich.edu).

    Author Contributions: Dr Prescott had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Wayne, Seelye, Jones, Liu, Prescott.

    Acquisition, analysis, or interpretation of data: Wayne, Seelye, Molling, Wang, Donnelly, Hogan, Jones, Iwashnya, Prescott.

    Drafting of the manuscript: Wayne, Seelye, Hogan.

    Critical revision of the manuscript for important intellectual content: Wayne, Seelye, Molling, Wang, Donnelly, Jones, Iwashnya, Liu, Prescott.

    Statistical analysis: Wayne, Seelye, Molling, Wang, Donnelly.

    Obtained funding: Prescott.

    Administrative, technical, or material support: Wayne, Seelye, Hogan, Jones, Prescott.

    Supervision: Seelye, Liu, Prescott.

    Conflict of Interest Disclosures: Dr Donnelly reported receiving grants from the National Heart, Lung, and Blood Institute during the conduct of the study and personal fees from American College of Emergency Physicians outside the submitted work. Dr Jones reported receiving grants from the University of Michigan during the conduct of the study. Dr Iwashnya reported receiving grants from the Department of Veterans Affairs (VA) Center for Clinical Management Research during the conduct of the study. Dr Liu reported receiving grants from the National Institute of General Medical Sciences during the conduct of the study. Dr Prescott reported grants from Agency for Healthcare Research Quality and VA Health Services Research & Development during the conduct of the study and serving on the Surviving Sepsis Campaign Guidelines panel and as physician-lead for the Michigan-statewide sepsis quality improvement consortium. No other disclosures were reported.

    Funding/Support: This work was supported by grant No. 1R01HS026725-01A1 from the Agency for Healthcare Research Quality (AHRQ), Investigator Initiated Research grant No. 17-2019 from the Department of Veterans Affairs, Health Services Research and Development Service; and grant No. R35GM128672 from the National Institute of General Medical Sciences.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The views in this work do not reflect the position or policy of the AHRQ, the views of the VA, or the US government.

    References
    1.
    Rhee  C, Dantes  R, Epstein  L,  et al; CDC Prevention Epicenter Program.  Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014.   JAMA. 2017;318(13):1241-1249. doi:10.1001/jama.2017.13836PubMedGoogle ScholarCrossref
    2.
    Angus  DC, Linde-Zwirble  WT, Lidicker  J, Clermont  G, Carcillo  J, Pinsky  MR.  Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care.   Crit Care Med. 2001;29(7):1303-1310. doi:10.1097/00003246-200107000-00002 PubMedGoogle ScholarCrossref
    3.
    Buchman  TG, Simpson  SQ, Sciarretta  KL,  et al.  Sepsis among Medicare beneficiaries: 1. the burdens of sepsis, 2012-2018.   Crit Care Med. 2020;48(3):276-288. doi:10.1097/CCM.0000000000004224PubMedGoogle ScholarCrossref
    4.
    Singer  M, Deutschman  CS, Seymour  CW,  et al.  The third international consensus definitions for sepsis and septic shock (sepsis-3).   JAMA. 2016;315(8):801-810. doi:10.1001/jama.2016.0287PubMedGoogle ScholarCrossref
    5.
    Rhodes  A, Evans  LE, Alhazzani  W,  et al.  Surviving Sepsis Campaign: international guidelines for management of sepsis and septic shock: 2016.   Crit Care Med. 2017;45(3):486-552. doi:10.1097/CCM.0000000000002255 PubMedGoogle ScholarCrossref
    6.
    Dugar  S, Choudhary  C, Duggal  A.  Sepsis and septic shock: guideline-based management.   Cleve Clin J Med. 2020;87(1):53-64. doi:10.3949/ccjm.87a.18143 PubMedGoogle ScholarCrossref
    7.
    Levy  MM, Evans  LE, Rhodes  A.  The Surviving Sepsis Campaign bundle: 2018 update.   Intensive Care Med. 2018;44(6):925-928. doi:10.1007/s00134-018-5085-0 PubMedGoogle ScholarCrossref
    8.
    Angus  DC, van der Poll  T.  Severe sepsis and septic shock.   N Engl J Med. 2013;369(9):840-851. doi:10.1056/NEJMra1208623 PubMedGoogle ScholarCrossref
    9.
    Liu  VX, Fielding-Singh  V, Greene  JD,  et al.  The timing of early antibiotics and hospital mortality in sepsis.   Am J Respir Crit Care Med. 2017;196(7):856-863. doi:10.1164/rccm.201609-1848OC PubMedGoogle ScholarCrossref
    10.
    Seymour  CW, Gesten  F, Prescott  HC,  et al.  Time to treatment and mortality during mandated emergency care for sepsis.   N Engl J Med. 2017;376(23):2235-2244. doi:10.1056/NEJMoa1703058 PubMedGoogle ScholarCrossref
    11.
    Peltan  ID, Brown  SM, Bledsoe  JR,  et al.  ED door-to-antibiotic time and long-term mortality in sepsis.   Chest. 2019;155(5):938-946. doi:10.1016/j.chest.2019.02.008 PubMedGoogle ScholarCrossref
    12.
    Centers for Medicare & Medicaid Services. Severe sepsis and septic shock: management bundle (composite measure). Accessed October 30, 2020. https://cmit.cms.gov/CMIT_public/ViewMeasure?MeasureId=1017
    13.
    New York Codes, Rules and Regulations. 10 CRR-NY §405.4. Accessed August 6, 2021. https://regs.health.ny.gov/content/section-4054-medical-staff
    14.
    Levy  MM, Gesten  FC, Phillips  GS,  et al; the Results of the New York State Initiative.  Mortality changes associated with mandated public reporting for sepsis the results of the New York state initiative.   Am J Respir Crit Care Med. 2018;198(11):1406-1412. doi:10.1164/rccm.201712-2545OC PubMedGoogle ScholarCrossref
    15.
    Levy  MM, Dellinger  RP, Townsend  SR,  et al.  The Surviving Sepsis Campaign: results of an international guideline-based performance improvement program targeting severe sepsis.   Intensive Care Med. 2010;36(2):222-231. doi:10.1007/s00134-009-1738-3 PubMedGoogle ScholarCrossref
    16.
    Miller  RR  III, Dong  L, Nelson  NC,  et al; Intermountain Healthcare Intensive Medicine Clinical Program.  Multicenter implementation of a severe sepsis and septic shock treatment bundle.   Am J Respir Crit Care Med. 2013;188(1):77-82. doi:10.1164/rccm.201212-2199OC PubMedGoogle ScholarCrossref
    17.
    National Center for Veterans Analysis and Statistics. VA utilization profile FY 2017. Accessed October 30, 2020. https://www.va.gov/vetdata/docs/Quickfacts/VA_Utilization_Profile_2017.pdf
    18.
    Fihn  SD, Francis  J, Clancy  C,  et al.  Insights from advanced analytics at the veterans health administration.   Health Aff (Millwood). 2014;33(7):1203-1211. doi:10.1377/hlthaff.2014.0054 PubMedGoogle ScholarCrossref
    19.
    Vincent  BM, Wiitala  WL, Burns  JA, Iwashyna  TJ, Prescott  HC.  Using Veterans Affairs Corporate Data Warehouse to identify 30-day hospital readmissions.   Heal Serv Outcomes Res Methodol. 2018;18(3):143-154. doi:10.1007/s10742-018-0178-3 Google ScholarCrossref
    20.
    Wayne  MT, Molling  D, Wang  XQ,  et al.  Measurement of sepsis in a national cohort using three different methods to define baseline organ function.   Ann Am Thorac Soc. 2021;18(4):648-655. doi:10.1513/AnnalsATS.202009-1130OC PubMedGoogle ScholarCrossref
    21.
    Centers for Disease Control and Prevention. Hospital toolkit for adult sepsis surveillance. Accessed August 6, 2021. https://www.cdc.gov/sepsis/pdfs/Sepsis-Surveillance-Toolkit-Aug-2018_508.pdf
    22.
    Bone  RC, Balk  RA, Cerra  FB,  et al; The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.  Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis.   Chest. 1992;101(6):1644-1655. doi:10.1378/chest.101.6.1644PubMedGoogle ScholarCrossref
    23.
    Bell  BA, Ferron  JM, Kromrey  JD.  Cluster Size in Multilevel Models: The Impact of Sparse Data Structures on Point and Interval Estimates in Two-Level Models. Wickrama & Bryant; 2004.
    24.
    Austin  PC, Leckie  G.  The effect of number of clusters and cluster size on statistical power and Type I error rates when testing random effects variance components in multilevel linear and logistic regression models.   J Stat Comput Simul. 2018;88(16):3151-3163. doi:10.1080/00949655.2018.1504945 Google ScholarCrossref
    25.
    Maas  CJM, Hox  JJ.  Sufficient sample sizes for multilevel modeling.   Methodology. 2005;1(3):86-92. doi:10.1027/1614-2241.1.3.86 Google ScholarCrossref
    26.
    Clarke  P.  When can group level clustering be ignored: multilevel models versus single-level models with sparse data.   J Epidemiol Community Health. 2008;62(8):752-758. doi:10.1136/jech.2007.060798 PubMedGoogle ScholarCrossref
    27.
    Wang  XQ, Vincent  BM, Wiitala  WL,  et al.  Veterans Affairs patient database (VAPD 2014-2017): building nationwide granular data for clinical discovery.   BMC Med Res Methodol. 2019;19(1):94. doi:10.1186/s12874-019-0740-x PubMedGoogle ScholarCrossref
    28.
    Elixhauser  A, Steiner  C, Harris  DR, Coffey  RM.  Comorbidity measures for use with administrative data.   Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004 PubMedGoogle ScholarCrossref
    29.
    Turner  K, Burchill  C. Elixhauser Comorbidity Index macro code using ICD-10-CA. Accessed February 25, 2021. http://mchp-appserv.cpe.umanitoba.ca/Upload/SAS/_ElixhauserICD10.sas.txt
    30.
    Quan  H, Sundararajan  V, Halfon  P,  et al.  Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.   Med Care. 2005;43(11):1130-1139. doi:10.1097/01.mlr.0000182534.19832.83 PubMedGoogle ScholarCrossref
    31.
    Merlo  J, Chaix  B, Yang  M, Lynch  J, Råstam  L.  A brief conceptual tutorial of multilevel analysis in social epidemiology: linking the statistical concept of clustering to the idea of contextual phenomenon.   J Epidemiol Community Health. 2005;59(6):443-449. doi:10.1136/jech.2004.023473 PubMedGoogle ScholarCrossref
    32.
    Sanagou  M, Wolfe  R, Forbes  A, Reid  CM.  Hospital-level associations with 30-day patient mortality after cardiac surgery: a tutorial on the application and interpretation of marginal and multilevel logistic regression.   BMC Med Res Methodol. 2012;12:28. doi:10.1186/1471-2288-12-28 PubMedGoogle ScholarCrossref
    33.
    Prescott  HC, Iwashyna  TJ.  Improving sepsis treatment by embracing diagnostic uncertainty.   Ann Am Thorac Soc. 2019;16(4):426-429. doi:10.1513/AnnalsATS.201809-646PS PubMedGoogle ScholarCrossref
    34.
    Liu  VX, Bhimarao  M, Greene  JD,  et al.  The presentation, pace, and profile of infection and sepsis patients hospitalized through the emergency department: an exploratory analysis.   Crit Care Explor. 2021;3(3):e0344. doi:10.1097/CCE.0000000000000344 PubMedGoogle Scholar
    35.
    Ferrer  R, Artigas  A, Levy  MM,  et al; Edusepsis Study Group.  Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain.   JAMA. 2008;299(19):2294-2303. doi:10.1001/jama.299.19.2294PubMedGoogle ScholarCrossref
    36.
    Zubkoff  L, Neily  J, King  BJ,  et al.  Virtual Breakthrough Series, part 1: preventing catheter-associated urinary tract infection and hospital-acquired pressure ulcers in the veterans health administration.   Jt Comm J Qual Patient Saf. 2016;42(11):485-AP2. doi:10.1016/S1553-7250(16)42091-X PubMedGoogle Scholar
    37.
    Zubkoff  L, Neily  J, Quigley  P,  et al.  Virtual breakthrough series, part 2: improving fall prevention practices in the veterans health administration.   Jt Comm J Qual Patient Saf. 2016;42(11):497-AP12. doi:10.1016/S1553-7250(16)42092-1 PubMedGoogle Scholar
    38.
    Corl  K, Levy  M, Phillips  G, Terry  K, Friedrich  M, Trivedi  AN.  Racial and ethnic disparities in care following the New York state sepsis initiative.   Health Aff (Millwood). 2019;38(7):1119-1126. doi:10.1377/hlthaff.2018.05381 PubMedGoogle ScholarCrossref
    39.
    Prescott  HC, Belloli  EA.  Claims-based ICU research: learning from imperfect data.   Crit Care Med. 2017;45(7):1263-1264. doi:10.1097/CCM.0000000000002396 PubMedGoogle ScholarCrossref
    40.
    Rhee  C, Gohil  S, Klompas  M.  Regulatory mandates for sepsis care—reasons for caution.   N Engl J Med. 2014;370(18):1673-1676. doi:10.1056/NEJMp1400276 PubMedGoogle ScholarCrossref
    41.
    Prescott  HC, Cope  TM, Gesten  FC,  et al.  Reporting of sepsis cases for performance measurement versus for reimbursement in New York State.   Crit Care Med. 2018;46(5):666-673. doi:10.1097/CCM.0000000000003005 PubMedGoogle ScholarCrossref
    42.
    Taylor  SP, Anderson  WE, Beam  K, Taylor  B, Ellerman  J, Kowalkowski  MA.  The association between antibiotic delay intervals and hospital mortality among patients treated in the emergency department for suspected sepsis.   Crit Care Med. 2021;49(5):741-747. doi:10.1097/CCM.0000000000004863 PubMedGoogle ScholarCrossref
    ×