qSOFA indicates quick Sequential Organ Failure Assessment.
qSOFA indicates quick Sequential Organ Failure Assessment; SIRS, systemic inflammatory response syndrome; and SOFA, Sequential [Sepsis-related] Organ Failure Assessment. The area under the receiver operating characteristic curves for qSOFA is 0.80 (95% CI, 0.74-0.85); SOFA, 0.77 (95% CI, 0.71-0.82); SIRS, 0.65 (95% CI, 0.59-0.70); and severe sepsis, 0.65 (95% CI, 0.59-0.70).
eFigure 1. Cumulative Incidence of Death
e Figure 2. ROC Curves for ICU admission
eFigure 3. ROC Curves for ICU Admission > 72h
eFigure 3. ROC Curves for ICU Admission > 72h or In-Hospital Death
eTable 1. Cox Models
Customize your JAMA Network experience by selecting one or more topics from the list below.
Freund Y, Lemachatti N, Krastinova E, et al. Prognostic Accuracy of Sepsis-3 Criteria for In-Hospital Mortality Among Patients With Suspected Infection Presenting to the Emergency Department. JAMA. 2017;317(3):301–308. doi:10.1001/jama.2016.20329
Copyright 2017 American Medical Association. All Rights Reserved.
Does the quick Sequential Organ Failure Assessment (qSOFA) score more accurately predict in-hospital mortality than the systemic inflammatory response syndrome (SIRS) or severe sepsis criteria among emergency department patients with suspected infection?
In this multicenter prospective cohort study involving 879 patients with suspected infection treated at the emergency department, the qSOFA was better at predicting in-hospital mortality with an area under the receiver operating curve (AUROC) of 0.80 than were SIRS (AUROC, 0.65) and severe sepsis (AUROC, 0.65).
Among patients presenting to the emergency department setting with suspected infection, the use of qSOFA resulted in greater prognostic accuracy for in-hospital mortality than either SIRS or severe sepsis.
An international task force recently redefined the concept of sepsis. This task force recommended the use of the quick Sequential Organ Failure Assessment (qSOFA) score instead of systemic inflammatory response syndrome (SIRS) criteria to identify patients at high risk of mortality. However, these new criteria have not been prospectively validated in some settings, and their added value in the emergency department remains unknown.
To prospectively validate qSOFA as a mortality predictor and compare the performances of the new sepsis criteria to the previous ones.
Design, Settings, and Participants
International prospective cohort study, conducted in France, Spain, Belgium, and Switzerland between May and June 2016. In the 30 participating emergency departments, for a 4-week period, consecutive patients who visited the emergency departments with suspected infection were included. All variables from previous and new definitions of sepsis were collected. Patients were followed up until hospital discharge or death.
Measurement of qSOFA, SOFA, and SIRS.
Main Outcomes and Measures
Of 1088 patients screened, 879 were included in the analysis. Median age was 67 years (interquartile range, 47-81 years), 414 (47%) were women, and 379 (43%) had respiratory tract infection. Overall in-hospital mortality was 8%: 3% for patients with a qSOFA score lower than 2 vs 24% for those with qSOFA score of 2 or higher (absolute difference, 21%; 95% CI, 15%-26%). The qSOFA performed better than both SIRS and severe sepsis in predicting in-hospital mortality, with an area under the receiver operating curve (AUROC) of 0.80 (95% CI, 0.74-0.85) vs 0.65 (95% CI, 0.59-0.70) for both SIRS and severe sepsis (P < .001; incremental AUROC, 0.15; 95% CI, 0.09-0.22). The hazard ratio of qSOFA score for death was 6.2 (95% CI, 3.8-10.3) vs 3.5 (95% CI, 2.2-5.5) for severe sepsis.
Conclusions and Relevance
Among patients presenting to the emergency department with suspected infection, the use of qSOFA resulted in greater prognostic accuracy for in-hospital mortality than did either SIRS or severe sepsis. These findings provide support for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) criteria in the emergency department setting.
clinicaltrials.gov Identifier: NCT02738164
Sepsis is a highly prevalent condition that accounts for 10% of admissions to the intensive care unit (ICU) and is associated with a 10% to 20% in-hospital mortality rate.1-5 In 2016, an international task force of experts redefined this syndrome in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).6 Due to poor specificity and sensitivity, the systemic inflammatory response syndrome (SIRS) and the previous definitions of sepsis and severe sepsis were replaced with the new state of sepsis defined as a life-threatening organ dysfunction caused by a dysregulated host response to infection. Sepsis is now identified by an increase of at least 2 points in the Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score in patients with a suspicion of infection. The quick SOFA (qSOFA) score, a surrogate for SOFA in settings in which all components of SOFA are not routinely measured, was introduced to screen for patients likely to have sepsis.
The task force derived and validated its criteria on several large patient databases, both inside and outside the ICU. They reported that qSOFA (range, 0-3; 1 point for each of the following: respiratory rate >21 breaths/min; systolic arterial blood pressure ≤100 mm Hg; or altered mental status) was a better predictor for in-hospital mortality than were SIRS or SOFA in non-ICU encounters and should be used for risk stratification and consideration for sepsis in emergency department (ED) patients with infection. However, it has not been prospectively validated or even studied specifically in the ED. For Sepsis-3 criteria to be globally endorsed, external validation is essential.
The purpose of this study was to assess the external validity of the recently developed Sepsis-3 criteria among patients presenting to the ED and to compare these criteria to prior guidelines that utilize the SIRS score and serum lactate levels.
This was an international multicenter prospective cohort study that recruited from 30 centers: 27 in France, 1 in Switzerland, 1 in Spain, and 1 in Belgium. Among those, 24 were academic centers and 6, nonacademic centers. For a 4-week period from May to June 2016, consecutive patients who had visited 1 of the recruiting EDs with a suspicion of infection were screened and followed up until death or hospital discharge after oral (or written in Belgium and Switzerland) consent was obtained. Because the study was observational, our institutional review board (IRB) (Comité de protection des personnes, Ile de France VI, Paris, France) approved the study in France, as did local IRBs in Spain, Belgium, and Switzerland. The STARD recommendations were followed for the reporting of diagnostic studies.7
We included all consecutive adult patients who presented to the ED with a clinical suspicion of infection diagnosed by the treating emergency physicians, based on the identification of an infectious source (whether clinical, radiological, or microbiological) or an equivocal presentation (for example, a febrile patient with inflammatory syndrome). After the recruitment and follow-up period was over, 2 experts in each center reviewed all files from each patient’s hospital stay and adjudicated whether the acute presentation to the ED was related to an infection. Evidence of infection was sought through the analysis of radiological studies, microbiological findings, or clinical context. In cases of disagreement, consensus was sought between the 2 experts. Patients in whom infection was not confirmed were then excluded from analysis.
We also excluded patients who refused to participate, pregnant women, prisoners or patients in custody, and low-acuity patients defined by a localized infection without general symptoms and normal vital parameters (temperature, heart rate, respiratory rate, and blood pressure), for which laboratory examinations were not deemed necessary by the emergency physicians (for example tonsillitis, skin abscess, or cystitis).
For each recruited patient, the emergency physician collected the 3 components of the qSOFA in the ED at their worst level during the ED stay (namely highest respiratory rate, lowest systolic blood pressure, and lowest Glasgow Coma Scale [GCS] score). Because the definition of altered consciousness is not equivalent to a GCS score less than 15, the presence of an altered mental status was recorded independently of GCS. The presence of altered mental status was determined clinically by the treating physician. We also recorded data to assess the severity of sepsis using the previous definitions of sepsis (ie, blood lactate and components of the SIRS) and components of the SOFA score when available. Other variables collected by the experts after chart review included the site of infection, means of confirmation (clinical, radiological, or microbiological), and vasopressor administration.
The primary end point was in-hospital mortality. Because this end point could be equivocal for some patients (for example patients transferred to another facility), this end point was adjudicated by 2 experts blinded to each other after reviewing all available medical records. In cases of disagreement, consensus was sought between the 2 experts. For patients who were still hospitalized after 28 days and outside of ICU, we considered that they did not meet the end point of in-hospital mortality. Secondary end points included admission to ICU, length of ICU stay of more than 72 hours, and a composite of death or ICU stay of more than 72 hours.
All Gaussian distributed variables are expressed as mean (SD), and nonnormally distributed variables as median (interquartile range [IQR]). Categorical variables are expressed as number and percentage. We handled missing values for the SOFA score by assuming that they were within the normal range for each value.
To assess the performances of the qSOFA to predict the primary end point, we calculated diagnostic performances (sensitivity, specificity, and negative and positive predictive values) for a qSOFA score of 2 or higher. We constructed a receiver operating characteristic (ROC) curve and calculated the corresponding area under the ROC curve (AUROC). Performances of qSOFA and SOFA to predict the primary and secondary end points were compared with those of SIRS and the previous definition of severe sepsis, namely at least 2 elements of SIRS and a blood lactate level of more than 2 mmol/L (18 mg/dL). The respective hazard ratios (HRs) for in-hospital death of qSOFA and SIRS, which were dichotomized to less than 2 and 2 or more, were estimated with a Cox proportional hazards model after adjustment for measured confounders. The model fit was assessed by the calculation of the concordance probability, which is defined as the probability that predictions and outcomes are concordant. We used the Harrell C coefficient, which is defined as the proportion of all usable subject pairs in which the predictions and outcomes are concordant.
In line with Seymour et al,8 the added value of hyperlactatemia to qSOFA (qSOFA + 1 if lactate >2 mmol/L ) was also tested and compared with the qSOFA score alone. To assess whether the inclusion criteria and primary end point were valid, interrater agreement between the 2 blinded experts who adjudicated these 2 variables was achieved using the Cohen κ statistic.
To validate the results of the Sepsis-3 consensus article, the aim was to confirm the hypothesis that patients with a qSOFA score of 2 or higher have an in-hospital mortality rate of at least 10%.6 This percentage corresponds to the reported overall mortality rate of infected patients with a SOFA score of 2. For this reason, a difference in mortality rate of 10% was considered clinically significant in the Sepsis-3 consensus statement.6,8 With an estimated overall mortality of 3%,8 an assumption that 80% of included patients would have a qSOFA score of less than 2, and power set at 90%, a target recruitment number of 840 patients was calculated.
All statistical analyses were 2-tailed, and a P value less than .05 was required for statistical significance. All analyses were performed with NCSS 10.0 (Statistical Solution).
A total of 1088 patients were included from 30 EDs during the recruitment period. Following adjudication, 60 patients (6%) were excluded because they did not have infection, and 149 patients were excluded because of missing values required to calculate qSOFA score, leaving 879 included for the final analysis (Figure 1). A component of the SOFA was missing in 260 patients. The identified infection source was clinical in 79% of patients, radiological in 50%, and microbiological in 37%.
The median age was 67 years (IQR, 48-81 years). The most common site of infection was respiratory (43% cases). Baseline characteristics are summarized in Table 1. The qSOFA score was 2 or higher for 218 patients (25%), SOFA was 2 or higher for 297 patients (34%), SIRS was 2 or higher for 653 patients (74%), and 176 patients (20%) fulfilled the previous criteria of severe sepsis (≥2 elements of SIRS and a blood lactate concentration of >2 mmol/L ). Interrater agreement for the diagnosis of infection had a Cohen κ of 0.87 (95% CI, 0.81-0.93).
Overall, in-hospital mortality was 8%. For patients with qSOFA scores less than 2, the mortality rate was 3% (95% CI, 2%-5%) vs 24% (95% CI, 18%-30%) for patients with a qSOFA score of 2 or higher (absolute difference, 21%; 95% CI, 15%-26%). Secondary end points are reported in Table 2. Interrater agreement for the primary end point had a Cohen κ of 0.99 (95% CI, 0.96-1.0). Cumulative incidence of death according to qSOFA is reported in eFigure 1 in the Supplement. An AUROC curve for the prediction of in-hospital death was constructed with new and former definitions of sepsis, namely qSOFA, SOFA, SIRS, and severe sepsis (Figure 2). The highest AUROCs were for the qSOFA score (0.80; 95% CI, 0.74-0.85) and the SOFA score (0.77; 95% CI, 0.71-0.82) compared with 0.65 (95% CI, 0.59-0.70) for SIRS and 0.65 (95% CI, 0.59-0.70) for severe sepsis (P < .001, compared with qSOFA). The incremental AUROC for qSOFA compared with SIRS or severe sepsis was 0.15 (95% CI, 0.09-0.22). We found similar results for the prediction of ICU admission, ICU admission of more than 72 hours, and a composite of death or ICU admission of more than 72 hours (eFigures 2, 3, and 4 in the Supplement).
Prognostic performances of these criteria are reported in Table 3. For the prediction of in-hospital mortality, qSOFA had a sensitivity of 70% (95% CI, 59%-80%) and specificity of 79% (76%-82%); SOFA had a sensitivity of 73% (95% CI, 61%-83%) and a specificity of 70% (95% CI, 67%-73%). The positive likelihood ratio was 3.40 (95% CI, 2.80-4.17) for qSOFA and 2.40 (95% CI, 2.00-2.90) for SOFA. Both assessments had a negative predictive value of 97% (95% CI, 95%-98%).
After adjustment for age and site of infection (respiratory vs others) and using a Cox model, we found that a qSOFA of 2 or higher was associated with in-hospital mortality with an HR of 6.2 (95% CI, 3.8-10.3; Harrell C, 0.83). With the previous definition of severe sepsis, the HR was 3.5 (95% CI, 2.2-5.5). Other adjusted models for the prediction of in-hospital mortality confirmed the good results of Sepsis-3 criteria (eTable in the Supplement).
The AUROC of blood lactate was 0.70 (95% CI, 0.63-0.77). We found no value in adding lactate to qSOFA for the prediction of in-hospital mortality, with a similar AUROC for both: 0.80 (95% CI, 0.75-0.85) for qSOFA and lactate and 0.80 (95% CI, 0.74-0.85) for qSOFA alone.
In addition, only 30 patients fulfilled the septic shock criteria (presence of hypotension that requires vasoactive drug administration), with a mortality of 40% vs 7% for others (absolute difference, 32%; 95% CI, 15%-50%).
This international cohort study recruited 879 emergency patients with infection in 4 European countries to prospectively validate the new Sepsis-3 criteria, especially the qSOFA score. The latter aimed at identifying patients with sepsis, which is a life-threatening situation. This index was derived from large retrospective databases and requires prospective validation.9 The Sepsis-3 task force estimated that patients with sepsis would have an in-hospital mortality rate greater than 10%. In the present study, patients with a qSOFA score of 2 or higher had an in-hospital mortality rate of 24% compared with 3% for patients with a qSOFA score of less than 2.
This international study prospectively assessed qSOFA and validated the findings from the derivation cohort. Compared with previous criteria (SIRS and severe sepsis), qSOFA had better discriminative value and hazard ratio for predicting death, ICU admission, and ICU stay longer than 72 hours. The strong prognostic accuracy of qSOFA for mortality was confirmed with an AUROC of 0.80 (95% CI, 0.74-0.85), which was greater than that of SIRS and severe sepsis (AUROC, 0.65; 95% CI, 0.59-0.70 for both). This is in line with the Sepsis 3 task force study that reported an AUROC of 0.81 for qSOFA for non-ICU encounters.8 Recently, 2 retrospective studies also confirmed the good prognostic ability of qSOFA to predict mortality and ICU admission.10,11
Following the publication of Sepsis-3, a prospective validation study focused on ED patients was required to support the new recommendations and assist in changing the paradigm. In the cohort with a SIRS score of 2 or more reported herein, the mortality was 11%, and the high sensitivity (93%; 95% CI, 85%-98%) was associated with a poor specificity (27%; 95% CI, 24%-31%). Nearly 75% of patients had at least 2 points of SIRS, but far fewer had life-threatening organ dysfunction. Similarly, previous studies reported that 68% to 93% of patients admitted in the ICU had at least 2 elements of SIRS.12-14 This indicates that having 2 or more elements of SIRS does not discriminate well enough for organ dysfunction. The very low mortality rate of patients with qSOFA score less than 2 is a strong argument to replace SIRS without the risk of missing critically ill patients. Moreover, there was no difference in the rate of the false-negative of SIRS and qSOFA for the prediction of death or ICU stay of more than 72 hours (7%; 95% CI, 4%-10% and 9%; 95% CI, 7%-11%). Although qSOFA was not meant to replace SIRS in the definition of sepsis but rather help clinicians for early detection of sepsis,15 these results suggest that ED patients with infection and a qSOFA score of 2 or more should be considered for sepsis even in the absence of a SOFA score or more 2. More than 70% of patients with a qSOFA of 2 or more had a SOFA score of at least 2 points as previously reported.8
Although blood lactate was known to be associated with severe outcome in patients with sepsis,16-19 there was no added value of hyperlactatemia to qSOFA in this study. This confirms the findings of the Sepsis-3 task force, which suggested qSOFA performs effectively and there is no added value when stratified by blood lactate level. This along with other findings could result in a complete change of the current clinical approach because the severity of sepsis up until now has been assessed in ED patients using lactate levels.20,21
In addition to its improved performance, qSOFA may be practical for use in the ED. The endorsement of the Sepsis-3 criteria would allow not only a more accurate recognition of critically ill patients but also an earlier detection because qSOFA can be assessed immediately upon arrival and does not require any supplemental investigation such as leukocytosis or blood lactate.
The work presented by the Sepsis-3 task force included 2 major shortcomings that might have contributed to the reluctance of physicians to adopt them: they were not prospectively validated, and they did not involve emergency patient cohorts or emergency physicians. This was particularly criticized because two-thirds of patients with sepsis come through the ED. One of the strengths of this study is that it prospectively validates the task forces findings and highlights how these findings particularly apply to ED patients with even stronger results.
Our study has some limitations. First, we did not follow up discharged patients and only focused on in-hospital mortality, which was because we used the Sepsis-3 primary end point of in-hospital mortality. It is possible that a discharged patient could have been readmitted or could have died in the first 28 days. Second, the worst value of qSOFA criteria during the ED stay of the patient was recorded. This could have biased the results to a higher qSOFA score. Because the qSOFA can vary in a short time frame, these results could not be extrapolated to the detection of sepsis at the time of the arrival, for instance to be used as a nurse triage tool. A specific study on the value at ED entry should be performed to answer this question. Third, there was a substantial part of missing data regarding laboratory results, so the calculation of SOFA may not be accurate. It is possible that with more complete data, the SOFA score may actually perform better than qSOFA. However, qSOFA seems much more appropriate in the ED as an early detection tool. Similarly, one-third of patients with at least 2 SIRS criteria did not have blood lactate measurement, resulting in a possible misclassification in the severe sepsis category. Fourth, we did not exclude patients with “do not attempt resuscitation” status or with set limitations of care, and this could have skewed the mortality rate. Fifth, although the study was adequately powered, only 74 patients met the primary end point, which may be considered relatively low. Sixth, experts could not have been blinded to the value of the components of the scores, and this could have influenced their adjudication as to whether the ED presentation was related to an infection. This could be a source of incorporation bias.
Among patients presenting to the ED with suspected infection, the use of qSOFA resulted in greater prognostic accuracy for in-hospital mortality than did either SIRS or severe sepsis. These findings provide support for the Sepsis-3 criteria in the ED setting.
Corresponding Author: Yonathan Freund, MD, PhD, Service d’Accueil des Urgences, Hôpital Pitié-Salpêtrière, 83 Boulevard de l’hôpital, 75013 Paris, France (firstname.lastname@example.org).
Author Contributions: Dr Freund had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Freund, Riou, Beaune.
Acquisition, analysis, or interpretation of data: Freund, Lemachatti, Krastinova, Van Laer, Claessens, Avondo, Occelli, Feral-Pierssens, Truchot, Ortega, Carneiro, Pernet, Claret, Dami, Bloom, Beaune.
Drafting of the manuscript: Freund.
Critical revision of the manuscript for important intellectual content: All authors.
Supervision: Freund, Lemachatti, Riou, Beaune.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Freund reported receiving lecturer fees from Daichi-Sankyo. Dr Riou reported receiving personal fees from ThermoFisher Brahms, CrossJect, Sanofi, and LFB. No other disclosures were reported.
French Society of Emergency Medicine Collaborators Group: Patrick Ray, MD, PhD, Hôpital Tenon, Paris, France; Youri Yordanov, MD, and Pierre-Alexis Raynal, Hôpital Saint-Antoine, Paris, France; Jérôme Bokobza, MD, Hôpital Lariboisière, Paris, France; Frédéric Adnet, MD, PhD, Frédéric Lapostolle, MD, PhD, and Aurélien Guenin, MD, Hôpital Avicenne, Bobigny, France; Florence Dumas, MD, PhD, and Benoît Doumenc, MD, Hôpital Cochin, Paris, France; Sandrine Charpentier, MD, PhD, CHU Toulouse, France; Hery Andrianjafy, MD, Hôpital Nord-Essone, Longjumeau, France; Pierre-Marie Roy, MD, PhD, CHU Angers, France; Tahar Chouihed, MD, CHU Nancy, France; Jacques Levraut, MD, PhD, CHU Nice, France; Pierre-Arnaud Fort, MD, Centre hospitalier d’Agen, France; Maxime Maignan, MD, PhD, CHU Grenoble, France; Sabrina Kepka, MD, CHU Strasbourg, France; Laurent Jacquin, MD, Hospices Civils de Lyon, Lyon, France; Mathias Wargon, MD, PhD, Hôpital Saint Camille, Bry sur Marne, France; Mehdi Khellaf, MD, PhD, Hôpital Henri-Mondor, Créteil, France; Agnès Ricard-Hibon, MD, PhD, and Aurélie Ferré, MD, CHIPO, Beaumont sur Oise, France; Said Laribi, MD, PhD, CHU Tours, France; Carlos El Khoury, MD, Centre Hospitalier, Vienne, France; Carl Ogereau, MD, Hôpital Saint Louis, Paris, France; Luc-Marie Joly, MD, PhD, CHU Rouen, France; Andrea Penaloza, MD, PhD, Cliniques universitaires St Luc, Bruxelles, Belgium; Oscar Miro, MD, PhD, Hospital clinic, Barcelona, Spain; and Olivier Hugli, MD, PhD, CHUV, Lausanne, Switzerland.
Additional Contributions: We thank Pierre-Yves Gueugniaud, MD, PhD, and Agnès Ricard-Hibon, MD, PhD, (president and vice-president of the French society of emergency medicine SFMU) for their help and support. We also thank Tabassome Simon, MD, PhD (Hopital Saint-Antoine, Paris, France) for her constant support, and Anders Boyd, MD, PhD (INSERM UMRS_S1136, Paris, France) for statistical expertise. No compensation was received by any of the acknowledged colleagues for their role in the study.