Association of Postoperative Undertriage to Hospital Wards With Mortality and Morbidity | Surgery | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Table 1.  Patient Characteristics
Patient Characteristics
Table 2.  Admitted Patient Perioperative Factors
Admitted Patient Perioperative Factors
Table 3.  Postoperative Outcomes
Postoperative Outcomes
1.
Elixhauser  A, Andrews  RM.  Profile of inpatient operating room procedures in US hospitals in 2007.   Arch Surg. 2010;145(12):1201-1208. doi:10.1001/archsurg.2010.269PubMedGoogle ScholarCrossref
2.
Pearse  RM, Harrison  DA, James  P,  et al.  Identification and characterisation of the high-risk surgical population in the United Kingdom.   Crit Care. 2006;10(3):R81. doi:10.1186/cc4928PubMedGoogle ScholarCrossref
3.
Jhanji  S, Thomas  B, Ely  A, Watson  D, Hinds  CJ, Pearse  RM.  Mortality and utilisation of critical care resources amongst high-risk surgical patients in a large NHS trust.   Anaesthesia. 2008;63(7):695-700. doi:10.1111/j.1365-2044.2008.05560.xPubMedGoogle ScholarCrossref
4.
Skogvoll  E, Isern  E, Sangolt  GK, Gisvold  SE.  In-hospital cardiopulmonary resuscitation: 5 years’ incidence and survival according to the Utstein template.   Acta Anaesthesiol Scand. 1999;43(2):177-184. doi:10.1034/j.1399-6576.1999.430210.xPubMedGoogle ScholarCrossref
5.
Merchant  RM, Yang  L, Becker  LB,  et al; American Heart Association Get With The Guidelines-Resuscitation Investigators.  Incidence of treated cardiac arrest in hospitalized patients in the United States.   Crit Care Med. 2011;39(11):2401-2406. doi:10.1097/CCM.0b013e3182257459PubMedGoogle ScholarCrossref
6.
Perman  SM, Stanton  E, Soar  J,  et al; American Heart Association’s Get With the Guidelines—Resuscitation (formerly the National Registry of Cardiopulmonary Resuscitation) Investigators.  Location of in-hospital cardiac arrest in the United States–variability in event rate and outcomes.   J Am Heart Assoc. 2016;5(10):e003638. doi:10.1161/JAHA.116.003638PubMedGoogle Scholar
7.
Sandroni  C, Nolan  J, Cavallaro  F, Antonelli  M.  In-hospital cardiac arrest: incidence, prognosis and possible measures to improve survival.   Intensive Care Med. 2007;33(2):237-245. doi:10.1007/s00134-006-0326-zPubMedGoogle ScholarCrossref
8.
Schein  RM, Hazday  N, Pena  M, Ruben  BH, Sprung  CL.  Clinical antecedents to in-hospital cardiopulmonary arrest.   Chest. 1990;98(6):1388-1392. doi:10.1378/chest.98.6.1388PubMedGoogle ScholarCrossref
9.
Bihorac  A, Ozrazgat-Baslanti  T, Ebadi  A,  et al.  MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery.   Ann Surg. 2019;269(4):652-662. doi:10.1097/SLA.0000000000002706PubMedGoogle ScholarCrossref
10.
Datta  S, Loftus  TJ, Ruppert  MM,  et al.  Added value of intraoperative data for predicting postoperative complications: the MySurgeryRisk postop extension.   J Surg Res. 2020;254:350-363. doi:10.1016/j.jss.2020.05.007PubMedGoogle ScholarCrossref
11.
Finlayson  SG, Subbaswamy  A, Singh  K,  et al.  The clinician and dataset shift in artificial intelligence.   N Engl J Med. 2021;385(3):283-286. doi:10.1056/NEJMc2104626PubMedGoogle ScholarCrossref
12.
Boyd  O, Jackson  N.  How is risk defined in high-risk surgical patient management?   Crit Care. 2005;9(4):390-396. doi:10.1186/cc3057PubMedGoogle ScholarCrossref
13.
sklearn.neighbors.NearestNeighbors. Scikit Learn. Accessed October 15, 2021. https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html
14.
Cutti  S, Klersy  C, Favalli  V,  et al.  A multidimensional approach of Surgical Mortality Assessment and Stratification (SMATT score).   Sci Rep. 2020;10(1):10964. doi:10.1038/s41598-020-67164-6PubMedGoogle ScholarCrossref
15.
Subbe  CP, Williams  E, Fligelstone  L, Gemmell  L.  Does earlier detection of critically ill patients on surgical wards lead to better outcomes?   Ann R Coll Surg Engl. 2005;87(4):226-232. doi:10.1308/003588405X50921PubMedGoogle ScholarCrossref
16.
Taenzer  AH, Pyke  JB, McGrath  SP, Blike  GT.  Impact of pulse oximetry surveillance on rescue events and intensive care unit transfers: a before-and-after concurrence study.   Anesthesiology. 2010;112(2):282-287. doi:10.1097/ALN.0b013e3181ca7a9bPubMedGoogle ScholarCrossref
17.
Brown  H, Terrence  J, Vasquez  P, Bates  DW, Zimlichman  E.  Continuous monitoring in an inpatient medical-surgical unit: a controlled clinical trial.   Am J Med. 2014;127(3):226-232. doi:10.1016/j.amjmed.2013.12.004PubMedGoogle ScholarCrossref
18.
Sauro  KM, Soo  A, de Grood  C,  et al.  Adverse events after transition from ICU to hospital ward: a multicenter cohort study.   Crit Care Med. 2020;48(7):946-953. doi:10.1097/CCM.0000000000004327PubMedGoogle Scholar
19.
Finlayson  EV, Birkmeyer  JD.  Operative mortality with elective surgery in older adults.   Eff Clin Pract. 2001;4(4):172-177.PubMedGoogle Scholar
20.
Teh  SH, Nagorney  DM, Stevens  SR,  et al.  Risk factors for mortality after surgery in patients with cirrhosis.   Gastroenterology. 2007;132(4):1261-1269. doi:10.1053/j.gastro.2007.01.040PubMedGoogle ScholarCrossref
21.
Pan  Y, Wang  W, Wang  J, Yang  L, Ding  F; ISN AKF 0by25 China Consortium.  Incidence and risk factors of in-hospital mortality from AKI after non-cardiovascular operation: a nationwide survey in China.   Sci Rep. 2017;7(1):13953. doi:10.1038/s41598-017-13763-9PubMedGoogle ScholarCrossref
22.
Oh  TK, Jeon  YT, Do  SH, Hwang  JW.  Pre-operative assessment of 30-day mortality risk after major surgery: the role of the quick sequential organ failure assessment: a retrospective observational study.   Eur J Anaesthesiol. 2019;36(9):688-694. doi:10.1097/EJA.0000000000000957PubMedGoogle ScholarCrossref
23.
Forte  ML, Virnig  BA, Swiontkowski  MF,  et al.  Ninety-day mortality after intertrochanteric hip fracture: does provider volume matter?   J Bone Joint Surg Am. 2010;92(4):799-806. doi:10.2106/JBJS.H.01204PubMedGoogle ScholarCrossref
24.
Kim  TH, Um  SH, Yim  SY,  et al.  The risk of perioperative adverse events in patients with chronic liver disease.   Liver Int. 2015;35(3):713-723. doi:10.1111/liv.12529PubMedGoogle ScholarCrossref
25.
Nouraei  SAR, Middleton  SE, Hudovsky  A,  et al.  A national analysis of the outcome of major head and neck cancer surgery: implications for surgeon-level data publication.   Clin Otolaryngol. 2013;38(6):502-511. doi:10.1111/coa.12185PubMedGoogle ScholarCrossref
26.
Wolters  U, Wolf  T, Stützer  H, Schröder  T.  ASA classification and perioperative variables as predictors of postoperative outcome.   Br J Anaesth. 1996;77(2):217-222. doi:10.1093/bja/77.2.217PubMedGoogle ScholarCrossref
27.
Brennan  M, Puri  S, Ozrazgat-Baslanti  T,  et al.  Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.   Surgery. 2019;165(5):1035-1045. doi:10.1016/j.surg.2019.01.002PubMedGoogle ScholarCrossref
28.
Gawande  AA, Kwaan  MR, Regenbogen  SE, Lipsitz  SA, Zinner  MJ.  An Apgar score for surgery.   J Am Coll Surg. 2007;204(2):201-208. doi:10.1016/j.jamcollsurg.2006.11.011PubMedGoogle ScholarCrossref
29.
Carmichael  H, Moore  A, Steward  L, Velopulos  CG.  Disparities in emergency versus elective surgery: comparing measures of neighborhood social vulnerability.   J Surg Res. 2020;256:397-403. doi:10.1016/j.jss.2020.07.002PubMedGoogle ScholarCrossref
30.
Kazaure  H, Roman  S, Sosa  JA.  High mortality in surgical patients with do-not-resuscitate orders: analysis of 8256 patients.   Arch Surg. 2011;146(8):922-928. doi:10.1001/archsurg.2011.69PubMedGoogle ScholarCrossref
31.
Ghaferi  AA, Birkmeyer  JD, Dimick  JB.  Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients.   Ann Surg. 2009;250(6):1029-1034. doi:10.1097/SLA.0b013e3181bef697PubMedGoogle ScholarCrossref
32.
Ghaferi  AA, Osborne  NH, Birkmeyer  JD, Dimick  JB.  Hospital characteristics associated with failure to rescue from complications after pancreatectomy.   J Am Coll Surg. 2010;211(3):325-330. doi:10.1016/j.jamcollsurg.2010.04.025PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,241
    Citations 0
    Original Investigation
    Health Informatics
    November 10, 2021

    Association of Postoperative Undertriage to Hospital Wards With Mortality and Morbidity

    Author Affiliations
    • 1Department of Surgery, University of Florida Health, Gainesville
    • 2Precision and Intelligent Systems in Medicine Research Partnership, University of Florida, Gainesville
    • 3Department of Medicine, University of Florida Health, Gainesville
    • 4Department of Anesthesiology, University of Florida Health, Gainesville
    • 5Department of Orthopaedic Surgery and Sports Medicine, University of Florida Health, Gainesville
    • 6Department of Information Systems and Operations Management, University of Florida Health, Gainesville
    • 7Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, Gainesville
    • 8Department of Biomedical Engineering, University of Florida, Gainesville
    • 9Department of Computer and Information Science and Engineering, University of Florida, Gainesville
    • 10Department of Electrical and Computer Engineering, University of Florida, Gainesville
    JAMA Netw Open. 2021;4(11):e2131669. doi:10.1001/jamanetworkopen.2021.31669
    Key Points

    Question  Is postoperative undertriage associated with increased mortality, morbidity, and resource use?

    Findings  In this cross-sectional study of 14 890 postoperative admissions, undertriage to hospital wards was associated with increased mortality and morbidity compared with admissions that had similar risk profiles and were triaged to intensive care units. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features for real-time machine-learning models.

    Meaning  These findings suggest that there is a rationale and framework for clinical decision support platforms to augment postoperative triage decisions.

    Abstract

    Importance  Undertriaging patients who are at increased risk for postoperative complications after surgical procedures to low-acuity hospital wards (ie, floors) rather than highly vigilant intensive care units (ICUs) may be associated with risk of unrecognized decompensation and worse patient outcomes, but evidence for these associations is lacking.

    Objective  To test the hypothesis that postoperative undertriage is associated with increased mortality and morbidity compared with risk-matched ICU admission.

    Design, Setting, and Participants  This longitudinal cross-sectional study was conducted using data from the University of Florida Integrated Data Repository on admissions to a university hospital. Included patients were individuals aged 18 years or older who were admitted after a surgical procedure from June 1, 2014, to August 20, 2020. Data were analyzed from April through August 2021.

    Exposures  Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among all inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative electronic health record features available at surgical procedure end time. A nearest neighbors algorithm was used to identify a risk-matched control group of ICU admissions.

    Main Outcomes and Measures  The primary outcomes of hospital mortality and morbidity were compared among appropriately triaged ward admissions, undertriaged wards admissions, and a risk-matched control group of ICU admissions.

    Results  Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged (5927 [53.7%] women; median [IQR] age, 59 [44-70] years) and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the control group, had increased median [IQR] age (64 [54-74] years vs 62 [50-73] years; P = .001) and increased proportions of women (649 [49.7%] women vs 1080 [44.0%] women; P < .001) and admitted patients with do not resuscitate orders before first surgical procedure (53 admissions [4.1%] vs 27 admissions [1.1%]); P < .001); 207 admissions that were undertriaged (15.8%) had subsequent ICU admission. In the validation cohort, hospital mortality and prolonged ICU stay estimations had areas under the receiver operating characteristic curve of 0.92 (95% CI, 0.91-0.93) and 0.92 (95% CI, 0.92-0.92), respectively. The undertriaged group, compared with the control group, had similar incidence of prolonged mechanical ventilation (32 admissions [2.5%] vs 53 admissions [2.2%]; P = .60), decreased median (IQR) total costs for admission ($26 900 [$18 400-$42 300] vs $32 700 [$22 700-$48 500]; P < .001), increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 6.0 [3.3-9.3] days, P < .001), and increased incidence of hospital mortality (19 admissions [1.5%] vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 49 admissions [2.0%]; P = .01), and acute kidney injury (341 admissions [26.1%] vs 477 admissions [19.5%]; P < .001).

    Conclusions and Relevance  This study found that admitted patients at increased risk for postoperative complications who were undertriaged to hospital wards had increased mortality and morbidity compared with a risk-matched control group of admissions to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features in real-time machine-learning models.

    Introduction

    Approximately 15 million major inpatient surgical procedures are performed in the United States annually.1 Immediately after surgical procedures, surgeons must determine whether patients can be triaged safely to hospital wards (ie, floors) with high patient-to-clinician ratios and infrequent assessments. When patients with high acuity are undertriaged to hospital wards, postoperative complications may progress to critical illness and cardiac arrest between patient assessments that are typically spaced every 4 hours.2-6 Approximately 41% of hospital arrests occur on wards, at 0.1 per 1000 bed-days, with 80% to 85% hospital mortality.4,6,7 The underlying disease process is often reversible, suggesting opportunities for prevention with appropriate surveillance and early intervention.8

    Despite the intuitive importance of undertriage, there is a paucity of evidence regarding associations between postoperative undertriage and patient outcomes. Among hospital admissions involving surgical procedures in the United Kingdom, 85% of all patients undergoing high-risk procedures (ie, those with mortality rates ≥5%) were admitted to hospital wards; patients with subsequent intensive care unit (ICU) admission had a 37% incidence of mortality.2 In a similar analysis, 74% of postoperative deaths occurred outside the ICU.3 These findings are compelling, but they use procedure-associated mortality alone to identify patients at increased risk. Granular, patient-level acuity assessments may further elucidate the sequelae associated with postoperative undertriage.

    Using a longitudinal cohort of postoperative admissions, we tested the hypothesis that postoperative undertriage would be associated with increased mortality and morbidity compared with risk-matched ICU admission. Undertriaged ward admissions were identified by a validated machine-learning model using automated electronic health record (EHR) data to generate risk estimations at surgical procedure end time, when triage decisions were finalized.9,10 We compared mortality and morbidity among appropriately triaged ward admissions, undertriaged wards admissions, and risk-matched ICU admissions.

    Methods

    The University of Florida Institutional Review Board approved this cross-sectional study with full waiver of informed consent with the justification that this retrospective study involved no interventions and it would be difficult to contact thousands patients who received care many years prior to enrollment. This study complies with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Study Design

    We generated a longitudinal cohort of patients aged 18 years or older undergoing inpatient surgical procedures at a university hospital. Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative estimator features extracted directly from EHRs, generating estimations at surgical procedure end time; risk thresholds were applied to these estimations.9,10 A nearest neighbors algorithm was used to identify a risk-matched control ICU admissions. Matching was performed on composite risk for hospital mortality or prolonged ICU stay represented by the mean z score across both outcomes. The primary analysis compared mortality and morbidity among admissions that were appropriately triaged to hospital wards or undertriaged to hospital wards and members of a risk-matched control group.

    Data Source

    The University of Florida Integrated Data Repository functioned as honest broker in assembling the study cohort, consisting of 71 065 hospital admissions for inpatient surgical procedures at University of Florida Health in Gainesville, Florida, from June 1, 2014, to August 20, 2020. Derivation of the study population is illustrated in eFigure 1 in the Supplement. Briefly, we included inpatient surgical admissions and excluded organ donation surgical procedures, anesthesia outside the operating room, and admissions lasting less than 24 hours, which are often considered observation only, failing to represent major inpatient surgical procedures. Deaths within 24 hours were excluded to avoid excessively narrow estimation windows. The study cohort was split chronologically into development and validation cohorts to allow nonrandom variation between data sets and mitigate potentially adverse outcomes associated with data set drift for algorithm estimations, consistent with Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline recommendations under the type 2b analysis category.11 The final data set included granular information from preoperative, intraoperative, and postoperative phases of care.9,10 Descriptions of all data elements and their preprocessing steps are provided in eMethods in the Supplement.

    Model Development and Validation

    The model used a data transformer layer to perform preprocessing, feature transformation, and feature selection techniques.9,10 A data analytics layer computed separate, patient admission–specific risk probabilities for hospital mortality and prolonged ICU stay using a random forest algorithm.10 The model was trained on a development cohort of 49 687 postoperative admissions, using only preoperative and intraoperative data as input features. The trained model made estimations on 20 940 validation cohort admissions; results are reported exclusively from this separate cohort.

    Identifying Undertriaged Ward Admissions

    In identifying undertriage, the top quartile (ie, ≥75th percentile) cutoff for hospital mortality risk was chosen because it corresponds to 5% incidence of mortality, consistent with definitions of high-risk surgical procedures in peer-reviewed literature.2,3,12 Regarding risk of prolonged ICU stay, for which there is no precedent in peer-reviewed literature, the top quartile cutoff was chosen to maintain consistency with the mortality cutoff.

    Identifying Risk-Matched Control ICU Admissions

    To identify a control population of postoperative ICU admissions, we fit a nearest neighbors, brute-force algorithm13 on estimations for hospital mortality and prolonged ICU stay among ICU admissions that did not receive immediate postoperative mechanical ventilation or vasopressor support (ie, within 2 hours of surgical procedure end time) and applied the fit model to the undertriaged group using a maximum Minkowski distance of 1 to identify the 3 nearest neighbors for each admission. Matching was performed on composite risk for hospital mortality or prolonged ICU stay represented by the mean z score across these outcomes. This method matched 2452 control admissions with 1306 undertriage admissions.

    Hospital Stations

    We extracted EHR-embedded hospital station labels with entrance and exit date times. Level of care assignments allowed ICU admission under intensive status, requiring patient-to-nurse ratios of 2:1 or less, or intermediate status, requiring patient-to-nurse ratios of 3:1 or less. The ICU was considered as a single destination that provided low patient-to-nurse ratios and consistent availability of critical care resources and personnel.

    Statistical Analysis

    We performed a power analysis using data from a peer-reviewed study of 121 290 inpatient surgical procedures.14 Planned surgical complexity grade 4 was observed among 35 931 hospital admissions and was associated with hospital mortality among 1863 admissions (5.18%), corresponding to high-risk surgical procedures definitions.2,3,12 Planned surgical complexity grade 1 was observed among 13 484 hospital admissions and was associated with hospital mortality among 49 admissions (0.36%). Accounting for imbalanced cohorts and hospital mortality as low as among 16 admissions (1.24%) in the high-risk group, our sample would detect a difference in hospital mortality of 1.19 percentage points, with 80% power and α = .05. Other outcomes reported were exploratory. While conducting this study, we observed that the undertriaged group had increased area deprivation indices and increased proportions of patients identifying as Black or African American and with preoperative do not resuscitate (DNR) orders; we performed exploratory, post hoc, secondary analyses for these 3 variables. Race data were collected from electronic health records using self-reported classifications. Race was assessed to determine whether there were associations among race, postoperative triage decisions, and patient outcomes. All primary outcome analyses were adjusted for multiple comparisons using the Benjamini-Hochberg procedure.

    For model training, missing values were replaced with a distinct missing indicator; for reporting results, missing values were imputed with medians. Continuous variables were reported as median values with IQR and compared by the Kruskal-Wallis test. Discrete variables were reported as raw numbers with percentages and compared by Fisher exact test. Model performance was assessed by calculating area under the receiver operating characteristic curves (AUROCs) and area under the precision recall curves (AUPRCs) with 95% CIs. All hypothesis tests were 2-sided with α = .05. The power analysis was performed using PASS statistical software (NCSS). Other statistical analyses were performed with Python programming language version 3.8.8 (Python Software Foundation). Data were analyzed from April through August 2021.

    Results
    Study Population Characteristics

    Patient characteristics are summarized in Table 1. Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the appropriately triaged group, had a decreased proportion of women (649 [49.7%] women vs 5927 [53.7%] women; P = .007), increased median (IQR) age (64 [54-74] vs 59 [44-70] years; P < .001), and an increased proportion of admitted patients self-identifying as Black or African American (261 admissions [20.0%] vs 1627 admission [14.7%]; P < .001), with a decreased proportion of admitted patients self-identifying as White (956 admissions [73.2%] vs 8483 admissions [76.8%]; P = .004) and a similar proportion of admitted patients self-identifying as Hispanic (2 admissions [0.2%] vs 20 admissions [0.2%]; P > .99). Area deprivation indices were increased among undertriaged admissions, suggesting socioeconomic vulnerability. The undertriaged group had increased median [IQR] Charlson Comorbidity Index score (2.0 [1.0-4.0] vs 1.0 [0-2.0]; P < .001) and increased proportions of admitted patients undergoing emergent admission (868 admissions [66.5%] vs 3369 admissions [30.5%]; P < .001), preoperative red blood cell transfusion (87 admissions [6.7%] vs 84 admissions [0.8%]; P < .001), and emergent surgical procedures (298 admissions [22.8%] vs 1352 admissions [12.2%]; P < .001).

    The undertriaged group, compared with the control group, had increased median [IQR] age (vs 62 [50-73] years; P = .001) and increased proportions of women (vs 1080 [44.0%] women; P < .001) and admitted patients self-identifying as Black or African American (vs 331 admissions [13.5%]; P < .001), with a decreased proportion of admitted patients self-identifying as White (vs 1946 admissions [79.4%]; P < .001) and a similar proportion of admitted patients self-identifying as Hispanic (vs 2 admissions [0.1%]; P = .61). Area deprivation indices were increased among undertriaged admissions compared with admissions in the control group. American Society of Anesthesiologists class distributions were similar between these 2 groups, but the undertriaged group had an increased median (IQR) Charlson Comorbidity Index score (vs 2.0 [0-3.0]; P < .001) and decreased median (IQR) Sequential Organ Failure Assessment score on admission (0 [0-0] vs 1.0 [0-2.0]; P < .001). The undertriaged group had increased proportions of admitted patients undergoing emergent admission (vs 848 admissions [34.6%]; P < .001) and emergent surgical procedures (vs 444 admissions [18.1%]; P < .001) and with preoperative red blood cell transfusions (vs 68 admissions [2.8%]; P < .001). Primary surgical services for each admission and service-specific mortality rates are listed in eTable 1 and eTable 2 in the Supplement, respectively.

    Triage Classifications

    In the validation cohort, hospital mortality estimations had an AUROC of 0.92 (95% CI, 0.91-0.93) and an AUPRC of 0.26 (95% CI, 0.23-0.30); prolonged ICU stay estimations had an AUROC of 0.92 (95% CI, 0.92-0.92) and an AUPRC of 0.85 (95% CI, 0.85-0.86). According to these estimations, 11 042 postoperative ward admissions were appropriate and 1306 admissions were undertriaged, representing 10.6% of all ward admissions. The 20 most important features (as defined by weight, which was derived by calculating change in node impurity and the probability of reaching each node) for estimating hospital mortality and prolonged ICU stay are listed in eTable 3 and eTable 4 in the Supplement, respectively. Primary procedure, scheduled postoperative location, and intraoperative minimum alveolar concentration measurements or inhalational anesthetic duration were among the 5 most important features for both estimations.

    Perioperative Factors

    Perioperative factors are summarized in Table 2. Increased proportions of undertriaged admissions, compared with appropriately triaged admissions, were planned for postoperative ICU admission under intensive status (91 admissions [7.0%] vs 284 admissions [2.6%]; P < .001) and intermediate status (19 admissions [1.5%] vs 66 admissions [0.6%]; P = .002). Immediately after the surgical procedure, undertriaged admissions had increased median (IQR) heart rate (82.1 beats per minute [bpm; 72.2-91.4 bpm] vs 79.8 bpm [71.2-89.0 bpm]; P < .001) and decreased median (IQR) systolic blood pressure (124.3 [111.0-138.9] mm Hg vs 126.1 [114.5-138.0] mm Hg; P < .001) but similar respiratory rate, oxygen saturation, and Glasgow coma scale eye opening response. An increased proportion of undertriaged admissions had cardiac telemetry ordered prior to ward admission (225 admissions [17.2%] vs 460 admissions [4.2%]; P < .001) compared with appropriately triaged admissions, while the proportion with a continuous pulse oximetry order was similar between these 2 groups (1094 admissions [83.8%] vs 9408 admissions [85.2%]; P = .18). The undertriaged group had an increased proportion of admitted patients with DNR orders placed preoperatively (53 admissions [4.1%] vs 55 admissions [0.5%]; P < .001) and during admission (91 admissions [7.0%] vs 78 admissions [0.7%]; P < .001) compared with the appropriately triaged group.

    The undertriaged group, compared with the control group, had similar immediate postoperative diastolic blood pressure but decreased median (IQR) systolic blood pressure (vs 131.0 [116.4-145.4] mm Hg; P < .001), respiratory rate (16.2 breaths per minute [14.3-17.9 breaths per minute] vs 16.4 breaths per minute [14.5-18.6 breaths per minute]; P < .001), and oxygen saturation (95.9% [94.4%-97.6%] vs 96.2% [94.8%-97.8%]; P < .001) and increased median (IQR) heart rate (82.1 bpm [72.2-91.4 bpm] vs 80.5 bpm [70.2-91.2 bpm], P = .01) and temperature (37.3 °C [37.1 °C-37.5 °C] vs 37.2 °C [37.0 °C-37.5 °C ]; P < .001). A decreased proportion of admitted patients who were undertriaged had a Glasgow coma scale eye opening response score less than 4 (133 admissions [10.2%] vs 398 admissions [16.2%]; P < .001) compared with admitted patients in the control group. An increased proportion of admitted patients who were undertriaged had a DNR order placed preoperatively (vs 27 admissions [1.1%]; P < .001) and during admission (vs 59 admissions [2.4%]; P < .001).

    Patient Outcomes

    Patient outcomes are summarized in Table 3. The undertriaged group, compared with the appropriately triaged group, had increased incidence of ICU transfer (207 admissions [15.8%] vs 298 admissions [2.7%]; P < .001) at later median (IQR) postoperative times (71.0 [31.5-144.0] hours vs 35.0 [21.2-70.0] hours; P < .001). Undertriaged admissions had increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 3.0 [1.5-5.1] days; P < .001) and increased incidence of prolonged ICU admission (165 admissions [12.6%] vs 212 admissions [1.9%]; P < .001), prolonged mechanical ventilation (32 admissions [2.5%] vs 19 admissions [0.2%]; P < .001), postoperative red blood cell transfusion (325 admissions [24.9%] vs 475 admissions [4.3%]; P < .001), arterial catheter placement (173 admissions [13.2%] vs 370 admissions [3.4%]; P < .001), central venous catheter placement (37 admissions [2.8%] vs 49 admissions [0.4%]; P < .001), second surgical procedure during admission (360 admissions [27.6%] vs 536 admissions [4.9%]; P < .001), hospital mortality (19 admissions [1.5%] vs 5 admissions [0.05%]; P < .001), cardiac arrest (7 admissions [0.5%] vs 7 admissions [0.1%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 48 admissions [0.4%]; P < .001), and acute kidney injury (341 admissions [26.1%] vs 965 admissions [8.7%]; P < .001) compared with appropriately triaged admissions. The undertriaged group had increased median (IQR) total charges per admission ($101 300 [$70 300-$155 000] vs $65 700 [$47 600-$88 300]; P < .001) and total costs per admission ($26 900 [$18 400-$42 300] vs $15 900 [$11 300-$22 500]; P < .001).

    The undertriaged group, compared with the control group, had decreased proportions of admitted patients with prolonged ICU admission (vs 1407 admissions [57.4%]; P < .001), similar proportions of admitted patients with prolonged mechanical ventilation (vs 53 admissions [2.2%]; P = .60), decreased proportions of admitted patients receiving postoperative red blood cell transfusion (275 admissions [21.1%] vs 337 admissions [13.7%]; P < .001), and increased median (IQR) hospital length of stay (vs 6.0 [3.3-9.3] days; P < .001). There was also an increased proportion of admitted patients who were undertriaged undergoing postoperative arterial catheter placement (vs 261 admissions [10.6%]; P = .02), second surgical procedure during admission (vs 353 admissions [14.4%]; P < .001), and emergent second surgical procedure (vs 91 admissions [3.7%]; P < .001) compared with admitted patients in the control group. Intervals between first and second surgical procedures were similar between undertriaged and control groups (median [IQR], 3.8 [2.0-6.0] days vs 3.3 [1.6-6.3] days; P = .23). The undertriaged group, compared with the control group, had increased incidence of hospital mortality (vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (vs 49 admissions [2.0%]; P = .01), and acute kidney injury (vs 477 admissions [19.5%]; P < .001), which was primarily associated with increased incidence of persistent acute kidney injury with kidney recovery (97 admissions [7.4%] vs 91 admissions [3.7%]; P < .001). Undertriaged admissions had decreased median (IQR) total charges per admission (vs $120 200 [$84 500-$163 700]; P < .001) and total costs per admission (vs $32 700 [$22 700-$48 500]; P < .001) compared with the control group.

    Secondary Analyses

    Exploratory secondary analyses investigating whether there were associations between self-identification as Black or African American, area deprivation indices, and preoperative DNR orders and outcomes are illustrated in eFigure 2, eFigure 3, and eFigure 4 in the Supplement, respectively; we provide analytic details in figure legends. Appropriately triaged admissions were excluded from these analyses owing to substantial differences in baseline characteristics and outcomes. Secondary analysis outcomes were hospital mortality or discharge to hospice, unplanned intubation, and acute kidney injury because these outcomes differed between undertriaged and control groups in primary analyses.

    There were no associations of area deprivation indices or self-identification as Black or African American with selected outcomes. Admitted patients with preoperative DNR orders in the undertriaged and control groups had increased incidence of hospital death or discharge to hospice (5 admissions [6.2%] vs 68 admissions [1.8%]; P = .02) and acute kidney injury (32 admissions [40.0%] vs 786 admissions [21.4%]; P < .001) compared with admitted patients without DNR orders. The undertriaged group had an increased proportion of admitted patients with both preoperative DNR orders and postoperative acute kidney injury (19 admissions [1.5%] vs 13 admissions [0.5%]; P = .005) compared with the control group; the proportion of admitted patients with both preoperative DNR orders and death or hospice were similar between the undertriage and control groups (3 admissions [0.2%] vs 2 admissions [0.1%]; P = .35). There were no associations between preoperative DNR orders and unplanned intubation.

    Discussion

    This cross-sectional study found that approximately 11% of postoperative ward admissions had increased risk of postoperative complications and had increased short-term mortality and morbidity compared with risk-matched ICU admissions. The observed increased incidence of preoperative DNR orders among undertriaged admissions may have been associated with decreased escalation of care in response to impending or evolving postoperative complications; deeper analyses of palliative care consultations and clinical notes are needed to address this hypothesis. Undertriaged admissions had increased incidence of cardiac telemetry; among patients admitted to wards who were at increased risk of complications, continuous monitoring may be associated with increased early detection of decompensation without requiring ICU resources.15-17 Additionally, we found that patients who were at increased risk of postoperative complications and undertriaged were accurately identified using automated EHR data from preoperative and intraoperative phases of care, suggesting opportunities for data-driven decision support. These results could be operationalized by generating EHR advisories triggered by ward admission orders placed for patients who have received surgical procedures who are at increased risk of complications.

    Previous work has characterized postoperative triage patterns by classifying patients at increased risk of complications using a single parameter: high-risk procedures with mortality rates of 5% or greater. Using this approach in a National Health Service study, Pearse et al2 provided a sentinel report that patients undergoing high-risk surgical procedures accounted for 13% of surgical procedures and 84% of deaths. Notably, fewer than 15% of patients at increased risk of complications were triaged to ICUs postoperatively. In a subsequent study by Jhanji et al,3 35% of patients at increased risk of complications were admitted to an ICU at any point after their surgical procedures. Overall, the high-risk mortality rate was 12%, but among patients at increased risk of complications who were initially triaged to a ward and subsequently transferred to the ICU, mortality was 30%. Others have assessed the ability of physicians to estimate adverse events when making triage decisions. In a multicenter study of patients admitted to ICUs for medical and surgical procedures who were transferred to hospital wards, physicians surveyed at the time of ICU discharge exhibited mediocre performance in estimating adverse events, ICU readmissions, and hospital mortality.18 A similar phenomenon may occur at surgical procedure end time, although further investigation is needed to address this hypothesis.

    The methods presented in our study are unique, which hinders direct comparisons with previous work, but our results are generally consistent with those in peer-reviewed literature. Advanced age, demographic variables, comorbidities, American Society of Anesthesiologists physical status classifications, and organ dysfunction have all been previously described as major risk factors associated with postoperative mortality and morbidity.9,10,19-28 Analyses including area deprivation indices in postoperative outcomes are sparse, but Carmichael et al29 found that increased area deprivation indices, along with other indicators associated with social vulnerability, were associated with increased odds of undergoing emergency surgical procedures. We observed increased area deprivation indices among undertriaged admissions, which may be associated with overlap with emergency admission priority and surgical procedure priority or an odious association with biased care or unobserved disease; further investigation of these potential etiologies is needed. We observed that DNR orders were associated with increased incidence of postoperative mortality, as was previously described by Kazaure et al.30 Overall, although our postoperative acuity assessment methods are unique, results based on our methods are similar to results from similar studies.

    Limitations

    This study has several limitations. Our study was limited by its single institution design and may not be generalizable to other practice settings. In addition, the retrospective analysis of EHR data is associated with risk for selection bias. We sought to minimize selection bias by including all consecutive admissions meeting inclusion criteria. These analyses excluded decisions for postoperative admission vs discharge home (ie, ambulatory surgical procedures), which require further investigation. Time-splitting development and validation cohorts allow bias from practice changes over time but may also mitigate the potential for model performance degradation from data set drift when applied prospectively.11 Although model discrimination was strong, precision in estimating hospital mortality was poor, as is often observed when estimating rare outcomes. Additionally, our power analysis was targeted loosely, owing to the lack of similar studies in peer-reviewed literature. This underscores a greater problem: despite the severe consequences associated with postoperative undertriage, there is sparse evidence regarding this topic, which hinders the development of decision-support tools. In the absence of efficient, effective decision-support platforms to augment postoperative triage decisions, surgeons must rely on individual judgement and hypothetical-deductive reasoning, which is highly variable and error-prone and may contribute to variability in surgical care costs and the incidence of failure to rescue across US hospitals.31,32

    Conclusions

    This study found that patients at increased risk for postoperative complications who underwent surgical procedures and were undertriaged to hospital wards had increased mortality and morbidity compared with patients in a risk-matched control group admitted to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative EHR data as features in machine-learning models that made data-driven, patient-level risk assessments. These findings may provide a framework and rationale for real-time clinical decision support platforms to augment postoperative triage decisions.

    Back to top
    Article Information

    Accepted for Publication: August 26, 2021.

    Published: November 10, 2021. doi:10.1001/jamanetworkopen.2021.31669

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Loftus TJ et al. JAMA Network Open.

    Corresponding Author: Tyler J. Loftus, MD, Department of Surgery, University of Florida Health, PO Box 100108, Gainesville, FL 32610-0108 (tyler.loftus@surgery.ufl.edu).

    Author Contributions: Dr Loftus had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Loftus, Efron, Hogan, Rashidi, Upchurch, Bihorac.

    Acquisition, analysis, or interpretation of data: Loftus, Ruppert, Ozrazgat-Baslanti, Balch, Tighe.

    Drafting of the manuscript: Loftus, Balch, Efron.

    Critical revision of the manuscript for important intellectual content: Ruppert, Ozrazgat-Baslanti, Balch, Tighe, Hogan, Rashidi, Upchurch, Bihorac.

    Statistical analysis: Loftus, Ruppert, Ozrazgat-Baslanti.

    Obtained funding: Loftus, Rashidi.

    Administrative, technical, or material support: Ruppert, Balch, Hogan.

    Supervision: Loftus, Efron, Tighe, Hogan, Upchurch, Bihorac.

    Conflict of Interest Disclosures: Dr Rashidi reported having a patent pending with the University of Florida. No other disclosures were reported.

    Funding/Support: Dr Loftus was supported by grant K23GM140268 from the National Institute of General Medical Sciences (NIGMS) of the National Institutes of Health (NIH). Dr Ozrazgat-Baslanti was supported by grant K01 DK120784 from the National Institute of Diabetes and Digestive and Kidney Diseases of the NIH, grant R01GM110240 from the NIGMS, grant P50 GM-111152 from the NIGMS awarded to the Sepsis and Critical Illness Research Center, and grant AWD09459 from University of Florida Research and the Gatorade Trust of the University of Florida. Dr Tighe was supported by grant R01GM114290 from the NIGMS and grant R01AG121647 from the National Institute on Aging (NIA). Dr Rashidi was supported by Faculty Early Career Development Program award 1750192 from the National Science Foundation, grants P30AG028740 and R01AG05533 from the NIA, grant 1R21EB027344 from the National Institute of Biomedical Imaging and Bioengineering (NIBIB), and grant R01GM-110240 from the NIGMS. Dr Bihorac was supported by grant R01GM110240 from the NIGMS and grant 1R21EB027344 from the NIBIB. This work was supported in part by grant UL1TR000064 from the National Center for Advancing Translational Sciences and Clinical and Translational Sciences via the University of Florida.

    Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

    References
    1.
    Elixhauser  A, Andrews  RM.  Profile of inpatient operating room procedures in US hospitals in 2007.   Arch Surg. 2010;145(12):1201-1208. doi:10.1001/archsurg.2010.269PubMedGoogle ScholarCrossref
    2.
    Pearse  RM, Harrison  DA, James  P,  et al.  Identification and characterisation of the high-risk surgical population in the United Kingdom.   Crit Care. 2006;10(3):R81. doi:10.1186/cc4928PubMedGoogle ScholarCrossref
    3.
    Jhanji  S, Thomas  B, Ely  A, Watson  D, Hinds  CJ, Pearse  RM.  Mortality and utilisation of critical care resources amongst high-risk surgical patients in a large NHS trust.   Anaesthesia. 2008;63(7):695-700. doi:10.1111/j.1365-2044.2008.05560.xPubMedGoogle ScholarCrossref
    4.
    Skogvoll  E, Isern  E, Sangolt  GK, Gisvold  SE.  In-hospital cardiopulmonary resuscitation: 5 years’ incidence and survival according to the Utstein template.   Acta Anaesthesiol Scand. 1999;43(2):177-184. doi:10.1034/j.1399-6576.1999.430210.xPubMedGoogle ScholarCrossref
    5.
    Merchant  RM, Yang  L, Becker  LB,  et al; American Heart Association Get With The Guidelines-Resuscitation Investigators.  Incidence of treated cardiac arrest in hospitalized patients in the United States.   Crit Care Med. 2011;39(11):2401-2406. doi:10.1097/CCM.0b013e3182257459PubMedGoogle ScholarCrossref
    6.
    Perman  SM, Stanton  E, Soar  J,  et al; American Heart Association’s Get With the Guidelines—Resuscitation (formerly the National Registry of Cardiopulmonary Resuscitation) Investigators.  Location of in-hospital cardiac arrest in the United States–variability in event rate and outcomes.   J Am Heart Assoc. 2016;5(10):e003638. doi:10.1161/JAHA.116.003638PubMedGoogle Scholar
    7.
    Sandroni  C, Nolan  J, Cavallaro  F, Antonelli  M.  In-hospital cardiac arrest: incidence, prognosis and possible measures to improve survival.   Intensive Care Med. 2007;33(2):237-245. doi:10.1007/s00134-006-0326-zPubMedGoogle ScholarCrossref
    8.
    Schein  RM, Hazday  N, Pena  M, Ruben  BH, Sprung  CL.  Clinical antecedents to in-hospital cardiopulmonary arrest.   Chest. 1990;98(6):1388-1392. doi:10.1378/chest.98.6.1388PubMedGoogle ScholarCrossref
    9.
    Bihorac  A, Ozrazgat-Baslanti  T, Ebadi  A,  et al.  MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery.   Ann Surg. 2019;269(4):652-662. doi:10.1097/SLA.0000000000002706PubMedGoogle ScholarCrossref
    10.
    Datta  S, Loftus  TJ, Ruppert  MM,  et al.  Added value of intraoperative data for predicting postoperative complications: the MySurgeryRisk postop extension.   J Surg Res. 2020;254:350-363. doi:10.1016/j.jss.2020.05.007PubMedGoogle ScholarCrossref
    11.
    Finlayson  SG, Subbaswamy  A, Singh  K,  et al.  The clinician and dataset shift in artificial intelligence.   N Engl J Med. 2021;385(3):283-286. doi:10.1056/NEJMc2104626PubMedGoogle ScholarCrossref
    12.
    Boyd  O, Jackson  N.  How is risk defined in high-risk surgical patient management?   Crit Care. 2005;9(4):390-396. doi:10.1186/cc3057PubMedGoogle ScholarCrossref
    13.
    sklearn.neighbors.NearestNeighbors. Scikit Learn. Accessed October 15, 2021. https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html
    14.
    Cutti  S, Klersy  C, Favalli  V,  et al.  A multidimensional approach of Surgical Mortality Assessment and Stratification (SMATT score).   Sci Rep. 2020;10(1):10964. doi:10.1038/s41598-020-67164-6PubMedGoogle ScholarCrossref
    15.
    Subbe  CP, Williams  E, Fligelstone  L, Gemmell  L.  Does earlier detection of critically ill patients on surgical wards lead to better outcomes?   Ann R Coll Surg Engl. 2005;87(4):226-232. doi:10.1308/003588405X50921PubMedGoogle ScholarCrossref
    16.
    Taenzer  AH, Pyke  JB, McGrath  SP, Blike  GT.  Impact of pulse oximetry surveillance on rescue events and intensive care unit transfers: a before-and-after concurrence study.   Anesthesiology. 2010;112(2):282-287. doi:10.1097/ALN.0b013e3181ca7a9bPubMedGoogle ScholarCrossref
    17.
    Brown  H, Terrence  J, Vasquez  P, Bates  DW, Zimlichman  E.  Continuous monitoring in an inpatient medical-surgical unit: a controlled clinical trial.   Am J Med. 2014;127(3):226-232. doi:10.1016/j.amjmed.2013.12.004PubMedGoogle ScholarCrossref
    18.
    Sauro  KM, Soo  A, de Grood  C,  et al.  Adverse events after transition from ICU to hospital ward: a multicenter cohort study.   Crit Care Med. 2020;48(7):946-953. doi:10.1097/CCM.0000000000004327PubMedGoogle Scholar
    19.
    Finlayson  EV, Birkmeyer  JD.  Operative mortality with elective surgery in older adults.   Eff Clin Pract. 2001;4(4):172-177.PubMedGoogle Scholar
    20.
    Teh  SH, Nagorney  DM, Stevens  SR,  et al.  Risk factors for mortality after surgery in patients with cirrhosis.   Gastroenterology. 2007;132(4):1261-1269. doi:10.1053/j.gastro.2007.01.040PubMedGoogle ScholarCrossref
    21.
    Pan  Y, Wang  W, Wang  J, Yang  L, Ding  F; ISN AKF 0by25 China Consortium.  Incidence and risk factors of in-hospital mortality from AKI after non-cardiovascular operation: a nationwide survey in China.   Sci Rep. 2017;7(1):13953. doi:10.1038/s41598-017-13763-9PubMedGoogle ScholarCrossref
    22.
    Oh  TK, Jeon  YT, Do  SH, Hwang  JW.  Pre-operative assessment of 30-day mortality risk after major surgery: the role of the quick sequential organ failure assessment: a retrospective observational study.   Eur J Anaesthesiol. 2019;36(9):688-694. doi:10.1097/EJA.0000000000000957PubMedGoogle ScholarCrossref
    23.
    Forte  ML, Virnig  BA, Swiontkowski  MF,  et al.  Ninety-day mortality after intertrochanteric hip fracture: does provider volume matter?   J Bone Joint Surg Am. 2010;92(4):799-806. doi:10.2106/JBJS.H.01204PubMedGoogle ScholarCrossref
    24.
    Kim  TH, Um  SH, Yim  SY,  et al.  The risk of perioperative adverse events in patients with chronic liver disease.   Liver Int. 2015;35(3):713-723. doi:10.1111/liv.12529PubMedGoogle ScholarCrossref
    25.
    Nouraei  SAR, Middleton  SE, Hudovsky  A,  et al.  A national analysis of the outcome of major head and neck cancer surgery: implications for surgeon-level data publication.   Clin Otolaryngol. 2013;38(6):502-511. doi:10.1111/coa.12185PubMedGoogle ScholarCrossref
    26.
    Wolters  U, Wolf  T, Stützer  H, Schröder  T.  ASA classification and perioperative variables as predictors of postoperative outcome.   Br J Anaesth. 1996;77(2):217-222. doi:10.1093/bja/77.2.217PubMedGoogle ScholarCrossref
    27.
    Brennan  M, Puri  S, Ozrazgat-Baslanti  T,  et al.  Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: a pilot usability study.   Surgery. 2019;165(5):1035-1045. doi:10.1016/j.surg.2019.01.002PubMedGoogle ScholarCrossref
    28.
    Gawande  AA, Kwaan  MR, Regenbogen  SE, Lipsitz  SA, Zinner  MJ.  An Apgar score for surgery.   J Am Coll Surg. 2007;204(2):201-208. doi:10.1016/j.jamcollsurg.2006.11.011PubMedGoogle ScholarCrossref
    29.
    Carmichael  H, Moore  A, Steward  L, Velopulos  CG.  Disparities in emergency versus elective surgery: comparing measures of neighborhood social vulnerability.   J Surg Res. 2020;256:397-403. doi:10.1016/j.jss.2020.07.002PubMedGoogle ScholarCrossref
    30.
    Kazaure  H, Roman  S, Sosa  JA.  High mortality in surgical patients with do-not-resuscitate orders: analysis of 8256 patients.   Arch Surg. 2011;146(8):922-928. doi:10.1001/archsurg.2011.69PubMedGoogle ScholarCrossref
    31.
    Ghaferi  AA, Birkmeyer  JD, Dimick  JB.  Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients.   Ann Surg. 2009;250(6):1029-1034. doi:10.1097/SLA.0b013e3181bef697PubMedGoogle ScholarCrossref
    32.
    Ghaferi  AA, Osborne  NH, Birkmeyer  JD, Dimick  JB.  Hospital characteristics associated with failure to rescue from complications after pancreatectomy.   J Am Coll Surg. 2010;211(3):325-330. doi:10.1016/j.jamcollsurg.2010.04.025PubMedGoogle ScholarCrossref
    ×