Association of Postoperative Undertriage to Hospital Wards With Mortality and Morbidity

Key Points

Question  Is postoperative undertriage associated with increased mortality, morbidity, and resource use?

Findings  In this cross-sectional study of 14 890 postoperative admissions, undertriage to hospital wards was associated with increased mortality and morbidity compared with admissions that had similar risk profiles and were triaged to intensive care units. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features for real-time machine-learning models.

Meaning  These findings suggest that there is a rationale and framework for clinical decision support platforms to augment postoperative triage decisions.

Importance  Undertriaging patients who are at increased risk for postoperative complications after surgical procedures to low-acuity hospital wards (ie, floors) rather than highly vigilant intensive care units (ICUs) may be associated with risk of unrecognized decompensation and worse patient outcomes, but evidence for these associations is lacking.

Objective  To test the hypothesis that postoperative undertriage is associated with increased mortality and morbidity compared with risk-matched ICU admission.

Design, Setting, and Participants  This longitudinal cross-sectional study was conducted using data from the University of Florida Integrated Data Repository on admissions to a university hospital. Included patients were individuals aged 18 years or older who were admitted after a surgical procedure from June 1, 2014, to August 20, 2020. Data were analyzed from April through August 2021.

Exposures  Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among all inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative electronic health record features available at surgical procedure end time. A nearest neighbors algorithm was used to identify a risk-matched control group of ICU admissions.

Main Outcomes and Measures  The primary outcomes of hospital mortality and morbidity were compared among appropriately triaged ward admissions, undertriaged wards admissions, and a risk-matched control group of ICU admissions.

Results  Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged (5927 [53.7%] women; median [IQR] age, 59 [44-70] years) and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the control group, had increased median [IQR] age (64 [54-74] years vs 62 [50-73] years; P = .001) and increased proportions of women (649 [49.7%] women vs 1080 [44.0%] women; P < .001) and admitted patients with do not resuscitate orders before first surgical procedure (53 admissions [4.1%] vs 27 admissions [1.1%]); P < .001); 207 admissions that were undertriaged (15.8%) had subsequent ICU admission. In the validation cohort, hospital mortality and prolonged ICU stay estimations had areas under the receiver operating characteristic curve of 0.92 (95% CI, 0.91-0.93) and 0.92 (95% CI, 0.92-0.92), respectively. The undertriaged group, compared with the control group, had similar incidence of prolonged mechanical ventilation (32 admissions [2.5%] vs 53 admissions [2.2%]; P = .60), decreased median (IQR) total costs for admission ($26 900 [$18 400-$42 300] vs $32 700 [$22 700-$48 500]; P < .001), increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 6.0 [3.3-9.3] days, P < .001), and increased incidence of hospital mortality (19 admissions [1.5%] vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 49 admissions [2.0%]; P = .01), and acute kidney injury (341 admissions [26.1%] vs 477 admissions [19.5%]; P < .001).

Conclusions and Relevance  This study found that admitted patients at increased risk for postoperative complications who were undertriaged to hospital wards had increased mortality and morbidity compared with a risk-matched control group of admissions to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative data as features in real-time machine-learning models.

Approximately 15 million major inpatient surgical procedures are performed in the United States annually.¹ Immediately after surgical procedures, surgeons must determine whether patients can be triaged safely to hospital wards (ie, floors) with high patient-to-clinician ratios and infrequent assessments. When patients with high acuity are undertriaged to hospital wards, postoperative complications may progress to critical illness and cardiac arrest between patient assessments that are typically spaced every 4 hours.^2-6 Approximately 41% of hospital arrests occur on wards, at 0.1 per 1000 bed-days, with 80% to 85% hospital mortality.^4,6,7 The underlying disease process is often reversible, suggesting opportunities for prevention with appropriate surveillance and early intervention.⁸

Despite the intuitive importance of undertriage, there is a paucity of evidence regarding associations between postoperative undertriage and patient outcomes. Among hospital admissions involving surgical procedures in the United Kingdom, 85% of all patients undergoing high-risk procedures (ie, those with mortality rates ≥5%) were admitted to hospital wards; patients with subsequent intensive care unit (ICU) admission had a 37% incidence of mortality.² In a similar analysis, 74% of postoperative deaths occurred outside the ICU.³ These findings are compelling, but they use procedure-associated mortality alone to identify patients at increased risk. Granular, patient-level acuity assessments may further elucidate the sequelae associated with postoperative undertriage.

Using a longitudinal cohort of postoperative admissions, we tested the hypothesis that postoperative undertriage would be associated with increased mortality and morbidity compared with risk-matched ICU admission. Undertriaged ward admissions were identified by a validated machine-learning model using automated electronic health record (EHR) data to generate risk estimations at surgical procedure end time, when triage decisions were finalized.^9,10 We compared mortality and morbidity among appropriately triaged ward admissions, undertriaged wards admissions, and risk-matched ICU admissions.

The University of Florida Institutional Review Board approved this cross-sectional study with full waiver of informed consent with the justification that this retrospective study involved no interventions and it would be difficult to contact thousands patients who received care many years prior to enrollment. This study complies with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

We generated a longitudinal cohort of patients aged 18 years or older undergoing inpatient surgical procedures at a university hospital. Ward admissions were considered undertriaged if their estimated risk for hospital mortality or prolonged ICU stay (ie, ≥48 hours) was in the top quartile among inpatient surgical procedures according to a validated machine-learning model using preoperative and intraoperative estimator features extracted directly from EHRs, generating estimations at surgical procedure end time; risk thresholds were applied to these estimations.^9,10 A nearest neighbors algorithm was used to identify a risk-matched control ICU admissions. Matching was performed on composite risk for hospital mortality or prolonged ICU stay represented by the mean z score across both outcomes. The primary analysis compared mortality and morbidity among admissions that were appropriately triaged to hospital wards or undertriaged to hospital wards and members of a risk-matched control group.

The University of Florida Integrated Data Repository functioned as honest broker in assembling the study cohort, consisting of 71 065 hospital admissions for inpatient surgical procedures at University of Florida Health in Gainesville, Florida, from June 1, 2014, to August 20, 2020. Derivation of the study population is illustrated in eFigure 1 in the Supplement. Briefly, we included inpatient surgical admissions and excluded organ donation surgical procedures, anesthesia outside the operating room, and admissions lasting less than 24 hours, which are often considered observation only, failing to represent major inpatient surgical procedures. Deaths within 24 hours were excluded to avoid excessively narrow estimation windows. The study cohort was split chronologically into development and validation cohorts to allow nonrandom variation between data sets and mitigate potentially adverse outcomes associated with data set drift for algorithm estimations, consistent with Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline recommendations under the type 2b analysis category.¹¹ The final data set included granular information from preoperative, intraoperative, and postoperative phases of care.^9,10 Descriptions of all data elements and their preprocessing steps are provided in eMethods in the Supplement.

The model used a data transformer layer to perform preprocessing, feature transformation, and feature selection techniques.^9,10 A data analytics layer computed separate, patient admission–specific risk probabilities for hospital mortality and prolonged ICU stay using a random forest algorithm.¹⁰ The model was trained on a development cohort of 49 687 postoperative admissions, using only preoperative and intraoperative data as input features. The trained model made estimations on 20 940 validation cohort admissions; results are reported exclusively from this separate cohort.

In identifying undertriage, the top quartile (ie, ≥75th percentile) cutoff for hospital mortality risk was chosen because it corresponds to 5% incidence of mortality, consistent with definitions of high-risk surgical procedures in peer-reviewed literature.^2,3,12 Regarding risk of prolonged ICU stay, for which there is no precedent in peer-reviewed literature, the top quartile cutoff was chosen to maintain consistency with the mortality cutoff.

To identify a control population of postoperative ICU admissions, we fit a nearest neighbors, brute-force algorithm¹³ on estimations for hospital mortality and prolonged ICU stay among ICU admissions that did not receive immediate postoperative mechanical ventilation or vasopressor support (ie, within 2 hours of surgical procedure end time) and applied the fit model to the undertriaged group using a maximum Minkowski distance of 1 to identify the 3 nearest neighbors for each admission. Matching was performed on composite risk for hospital mortality or prolonged ICU stay represented by the mean z score across these outcomes. This method matched 2452 control admissions with 1306 undertriage admissions.

We extracted EHR-embedded hospital station labels with entrance and exit date times. Level of care assignments allowed ICU admission under intensive status, requiring patient-to-nurse ratios of 2:1 or less, or intermediate status, requiring patient-to-nurse ratios of 3:1 or less. The ICU was considered as a single destination that provided low patient-to-nurse ratios and consistent availability of critical care resources and personnel.

We performed a power analysis using data from a peer-reviewed study of 121 290 inpatient surgical procedures.¹⁴ Planned surgical complexity grade 4 was observed among 35 931 hospital admissions and was associated with hospital mortality among 1863 admissions (5.18%), corresponding to high-risk surgical procedures definitions.^2,3,12 Planned surgical complexity grade 1 was observed among 13 484 hospital admissions and was associated with hospital mortality among 49 admissions (0.36%). Accounting for imbalanced cohorts and hospital mortality as low as among 16 admissions (1.24%) in the high-risk group, our sample would detect a difference in hospital mortality of 1.19 percentage points, with 80% power and α = .05. Other outcomes reported were exploratory. While conducting this study, we observed that the undertriaged group had increased area deprivation indices and increased proportions of patients identifying as Black or African American and with preoperative do not resuscitate (DNR) orders; we performed exploratory, post hoc, secondary analyses for these 3 variables. Race data were collected from electronic health records using self-reported classifications. Race was assessed to determine whether there were associations among race, postoperative triage decisions, and patient outcomes. All primary outcome analyses were adjusted for multiple comparisons using the Benjamini-Hochberg procedure.

For model training, missing values were replaced with a distinct missing indicator; for reporting results, missing values were imputed with medians. Continuous variables were reported as median values with IQR and compared by the Kruskal-Wallis test. Discrete variables were reported as raw numbers with percentages and compared by Fisher exact test. Model performance was assessed by calculating area under the receiver operating characteristic curves (AUROCs) and area under the precision recall curves (AUPRCs) with 95% CIs. All hypothesis tests were 2-sided with α = .05. The power analysis was performed using PASS statistical software (NCSS). Other statistical analyses were performed with Python programming language version 3.8.8 (Python Software Foundation). Data were analyzed from April through August 2021.

Patient characteristics are summarized in Table 1. Among 12 348 postoperative ward admissions, 11 042 admissions (89.4%) were appropriately triaged and 1306 admissions (10.6%) were undertriaged and matched with a control group of 2452 ICU admissions. The undertriaged group, compared with the appropriately triaged group, had a decreased proportion of women (649 [49.7%] women vs 5927 [53.7%] women; P = .007), increased median (IQR) age (64 [54-74] vs 59 [44-70] years; P < .001), and an increased proportion of admitted patients self-identifying as Black or African American (261 admissions [20.0%] vs 1627 admission [14.7%]; P < .001), with a decreased proportion of admitted patients self-identifying as White (956 admissions [73.2%] vs 8483 admissions [76.8%]; P = .004) and a similar proportion of admitted patients self-identifying as Hispanic (2 admissions [0.2%] vs 20 admissions [0.2%]; P > .99). Area deprivation indices were increased among undertriaged admissions, suggesting socioeconomic vulnerability. The undertriaged group had increased median [IQR] Charlson Comorbidity Index score (2.0 [1.0-4.0] vs 1.0 [0-2.0]; P < .001) and increased proportions of admitted patients undergoing emergent admission (868 admissions [66.5%] vs 3369 admissions [30.5%]; P < .001), preoperative red blood cell transfusion (87 admissions [6.7%] vs 84 admissions [0.8%]; P < .001), and emergent surgical procedures (298 admissions [22.8%] vs 1352 admissions [12.2%]; P < .001).

The undertriaged group, compared with the control group, had increased median [IQR] age (vs 62 [50-73] years; P = .001) and increased proportions of women (vs 1080 [44.0%] women; P < .001) and admitted patients self-identifying as Black or African American (vs 331 admissions [13.5%]; P < .001), with a decreased proportion of admitted patients self-identifying as White (vs 1946 admissions [79.4%]; P < .001) and a similar proportion of admitted patients self-identifying as Hispanic (vs 2 admissions [0.1%]; P = .61). Area deprivation indices were increased among undertriaged admissions compared with admissions in the control group. American Society of Anesthesiologists class distributions were similar between these 2 groups, but the undertriaged group had an increased median (IQR) Charlson Comorbidity Index score (vs 2.0 [0-3.0]; P < .001) and decreased median (IQR) Sequential Organ Failure Assessment score on admission (0 [0-0] vs 1.0 [0-2.0]; P < .001). The undertriaged group had increased proportions of admitted patients undergoing emergent admission (vs 848 admissions [34.6%]; P < .001) and emergent surgical procedures (vs 444 admissions [18.1%]; P < .001) and with preoperative red blood cell transfusions (vs 68 admissions [2.8%]; P < .001). Primary surgical services for each admission and service-specific mortality rates are listed in eTable 1 and eTable 2 in the Supplement, respectively.

In the validation cohort, hospital mortality estimations had an AUROC of 0.92 (95% CI, 0.91-0.93) and an AUPRC of 0.26 (95% CI, 0.23-0.30); prolonged ICU stay estimations had an AUROC of 0.92 (95% CI, 0.92-0.92) and an AUPRC of 0.85 (95% CI, 0.85-0.86). According to these estimations, 11 042 postoperative ward admissions were appropriate and 1306 admissions were undertriaged, representing 10.6% of all ward admissions. The 20 most important features (as defined by weight, which was derived by calculating change in node impurity and the probability of reaching each node) for estimating hospital mortality and prolonged ICU stay are listed in eTable 3 and eTable 4 in the Supplement, respectively. Primary procedure, scheduled postoperative location, and intraoperative minimum alveolar concentration measurements or inhalational anesthetic duration were among the 5 most important features for both estimations.

Perioperative factors are summarized in Table 2. Increased proportions of undertriaged admissions, compared with appropriately triaged admissions, were planned for postoperative ICU admission under intensive status (91 admissions [7.0%] vs 284 admissions [2.6%]; P < .001) and intermediate status (19 admissions [1.5%] vs 66 admissions [0.6%]; P = .002). Immediately after the surgical procedure, undertriaged admissions had increased median (IQR) heart rate (82.1 beats per minute [bpm; 72.2-91.4 bpm] vs 79.8 bpm [71.2-89.0 bpm]; P < .001) and decreased median (IQR) systolic blood pressure (124.3 [111.0-138.9] mm Hg vs 126.1 [114.5-138.0] mm Hg; P < .001) but similar respiratory rate, oxygen saturation, and Glasgow coma scale eye opening response. An increased proportion of undertriaged admissions had cardiac telemetry ordered prior to ward admission (225 admissions [17.2%] vs 460 admissions [4.2%]; P < .001) compared with appropriately triaged admissions, while the proportion with a continuous pulse oximetry order was similar between these 2 groups (1094 admissions [83.8%] vs 9408 admissions [85.2%]; P = .18). The undertriaged group had an increased proportion of admitted patients with DNR orders placed preoperatively (53 admissions [4.1%] vs 55 admissions [0.5%]; P < .001) and during admission (91 admissions [7.0%] vs 78 admissions [0.7%]; P < .001) compared with the appropriately triaged group.

The undertriaged group, compared with the control group, had similar immediate postoperative diastolic blood pressure but decreased median (IQR) systolic blood pressure (vs 131.0 [116.4-145.4] mm Hg; P < .001), respiratory rate (16.2 breaths per minute [14.3-17.9 breaths per minute] vs 16.4 breaths per minute [14.5-18.6 breaths per minute]; P < .001), and oxygen saturation (95.9% [94.4%-97.6%] vs 96.2% [94.8%-97.8%]; P < .001) and increased median (IQR) heart rate (82.1 bpm [72.2-91.4 bpm] vs 80.5 bpm [70.2-91.2 bpm], P = .01) and temperature (37.3 °C [37.1 °C-37.5 °C] vs 37.2 °C [37.0 °C-37.5 °C ]; P < .001). A decreased proportion of admitted patients who were undertriaged had a Glasgow coma scale eye opening response score less than 4 (133 admissions [10.2%] vs 398 admissions [16.2%]; P < .001) compared with admitted patients in the control group. An increased proportion of admitted patients who were undertriaged had a DNR order placed preoperatively (vs 27 admissions [1.1%]; P < .001) and during admission (vs 59 admissions [2.4%]; P < .001).

Patient outcomes are summarized in Table 3. The undertriaged group, compared with the appropriately triaged group, had increased incidence of ICU transfer (207 admissions [15.8%] vs 298 admissions [2.7%]; P < .001) at later median (IQR) postoperative times (71.0 [31.5-144.0] hours vs 35.0 [21.2-70.0] hours; P < .001). Undertriaged admissions had increased median (IQR) hospital length of stay (8.1 [5.1-13.6] days vs 3.0 [1.5-5.1] days; P < .001) and increased incidence of prolonged ICU admission (165 admissions [12.6%] vs 212 admissions [1.9%]; P < .001), prolonged mechanical ventilation (32 admissions [2.5%] vs 19 admissions [0.2%]; P < .001), postoperative red blood cell transfusion (325 admissions [24.9%] vs 475 admissions [4.3%]; P < .001), arterial catheter placement (173 admissions [13.2%] vs 370 admissions [3.4%]; P < .001), central venous catheter placement (37 admissions [2.8%] vs 49 admissions [0.4%]; P < .001), second surgical procedure during admission (360 admissions [27.6%] vs 536 admissions [4.9%]; P < .001), hospital mortality (19 admissions [1.5%] vs 5 admissions [0.05%]; P < .001), cardiac arrest (7 admissions [0.5%] vs 7 admissions [0.1%]; P < .001), unplanned intubation (45 admissions [3.4%] vs 48 admissions [0.4%]; P < .001), and acute kidney injury (341 admissions [26.1%] vs 965 admissions [8.7%]; P < .001) compared with appropriately triaged admissions. The undertriaged group had increased median (IQR) total charges per admission ($101 300 [$70 300-$155 000] vs $65 700 [$47 600-$88 300]; P < .001) and total costs per admission ($26 900 [$18 400-$42 300] vs $15 900 [$11 300-$22 500]; P < .001).

The undertriaged group, compared with the control group, had decreased proportions of admitted patients with prolonged ICU admission (vs 1407 admissions [57.4%]; P < .001), similar proportions of admitted patients with prolonged mechanical ventilation (vs 53 admissions [2.2%]; P = .60), decreased proportions of admitted patients receiving postoperative red blood cell transfusion (275 admissions [21.1%] vs 337 admissions [13.7%]; P < .001), and increased median (IQR) hospital length of stay (vs 6.0 [3.3-9.3] days; P < .001). There was also an increased proportion of admitted patients who were undertriaged undergoing postoperative arterial catheter placement (vs 261 admissions [10.6%]; P = .02), second surgical procedure during admission (vs 353 admissions [14.4%]; P < .001), and emergent second surgical procedure (vs 91 admissions [3.7%]; P < .001) compared with admitted patients in the control group. Intervals between first and second surgical procedures were similar between undertriaged and control groups (median [IQR], 3.8 [2.0-6.0] days vs 3.3 [1.6-6.3] days; P = .23). The undertriaged group, compared with the control group, had increased incidence of hospital mortality (vs 17 admissions [0.7%]; P = .04), discharge to hospice (23 admissions [1.8%] vs 14 admissions [0.6%]; P < .001), unplanned intubation (vs 49 admissions [2.0%]; P = .01), and acute kidney injury (vs 477 admissions [19.5%]; P < .001), which was primarily associated with increased incidence of persistent acute kidney injury with kidney recovery (97 admissions [7.4%] vs 91 admissions [3.7%]; P < .001). Undertriaged admissions had decreased median (IQR) total charges per admission (vs $120 200 [$84 500-$163 700]; P < .001) and total costs per admission (vs $32 700 [$22 700-$48 500]; P < .001) compared with the control group.

Exploratory secondary analyses investigating whether there were associations between self-identification as Black or African American, area deprivation indices, and preoperative DNR orders and outcomes are illustrated in eFigure 2, eFigure 3, and eFigure 4 in the Supplement, respectively; we provide analytic details in figure legends. Appropriately triaged admissions were excluded from these analyses owing to substantial differences in baseline characteristics and outcomes. Secondary analysis outcomes were hospital mortality or discharge to hospice, unplanned intubation, and acute kidney injury because these outcomes differed between undertriaged and control groups in primary analyses.

There were no associations of area deprivation indices or self-identification as Black or African American with selected outcomes. Admitted patients with preoperative DNR orders in the undertriaged and control groups had increased incidence of hospital death or discharge to hospice (5 admissions [6.2%] vs 68 admissions [1.8%]; P = .02) and acute kidney injury (32 admissions [40.0%] vs 786 admissions [21.4%]; P < .001) compared with admitted patients without DNR orders. The undertriaged group had an increased proportion of admitted patients with both preoperative DNR orders and postoperative acute kidney injury (19 admissions [1.5%] vs 13 admissions [0.5%]; P = .005) compared with the control group; the proportion of admitted patients with both preoperative DNR orders and death or hospice were similar between the undertriage and control groups (3 admissions [0.2%] vs 2 admissions [0.1%]; P = .35). There were no associations between preoperative DNR orders and unplanned intubation.

This cross-sectional study found that approximately 11% of postoperative ward admissions had increased risk of postoperative complications and had increased short-term mortality and morbidity compared with risk-matched ICU admissions. The observed increased incidence of preoperative DNR orders among undertriaged admissions may have been associated with decreased escalation of care in response to impending or evolving postoperative complications; deeper analyses of palliative care consultations and clinical notes are needed to address this hypothesis. Undertriaged admissions had increased incidence of cardiac telemetry; among patients admitted to wards who were at increased risk of complications, continuous monitoring may be associated with increased early detection of decompensation without requiring ICU resources.^15-17 Additionally, we found that patients who were at increased risk of postoperative complications and undertriaged were accurately identified using automated EHR data from preoperative and intraoperative phases of care, suggesting opportunities for data-driven decision support. These results could be operationalized by generating EHR advisories triggered by ward admission orders placed for patients who have received surgical procedures who are at increased risk of complications.

Previous work has characterized postoperative triage patterns by classifying patients at increased risk of complications using a single parameter: high-risk procedures with mortality rates of 5% or greater. Using this approach in a National Health Service study, Pearse et al² provided a sentinel report that patients undergoing high-risk surgical procedures accounted for 13% of surgical procedures and 84% of deaths. Notably, fewer than 15% of patients at increased risk of complications were triaged to ICUs postoperatively. In a subsequent study by Jhanji et al,³ 35% of patients at increased risk of complications were admitted to an ICU at any point after their surgical procedures. Overall, the high-risk mortality rate was 12%, but among patients at increased risk of complications who were initially triaged to a ward and subsequently transferred to the ICU, mortality was 30%. Others have assessed the ability of physicians to estimate adverse events when making triage decisions. In a multicenter study of patients admitted to ICUs for medical and surgical procedures who were transferred to hospital wards, physicians surveyed at the time of ICU discharge exhibited mediocre performance in estimating adverse events, ICU readmissions, and hospital mortality.¹⁸ A similar phenomenon may occur at surgical procedure end time, although further investigation is needed to address this hypothesis.

The methods presented in our study are unique, which hinders direct comparisons with previous work, but our results are generally consistent with those in peer-reviewed literature. Advanced age, demographic variables, comorbidities, American Society of Anesthesiologists physical status classifications, and organ dysfunction have all been previously described as major risk factors associated with postoperative mortality and morbidity.^9,10,19-28 Analyses including area deprivation indices in postoperative outcomes are sparse, but Carmichael et al²⁹ found that increased area deprivation indices, along with other indicators associated with social vulnerability, were associated with increased odds of undergoing emergency surgical procedures. We observed increased area deprivation indices among undertriaged admissions, which may be associated with overlap with emergency admission priority and surgical procedure priority or an odious association with biased care or unobserved disease; further investigation of these potential etiologies is needed. We observed that DNR orders were associated with increased incidence of postoperative mortality, as was previously described by Kazaure et al.³⁰ Overall, although our postoperative acuity assessment methods are unique, results based on our methods are similar to results from similar studies.

This study has several limitations. Our study was limited by its single institution design and may not be generalizable to other practice settings. In addition, the retrospective analysis of EHR data is associated with risk for selection bias. We sought to minimize selection bias by including all consecutive admissions meeting inclusion criteria. These analyses excluded decisions for postoperative admission vs discharge home (ie, ambulatory surgical procedures), which require further investigation. Time-splitting development and validation cohorts allow bias from practice changes over time but may also mitigate the potential for model performance degradation from data set drift when applied prospectively.¹¹ Although model discrimination was strong, precision in estimating hospital mortality was poor, as is often observed when estimating rare outcomes. Additionally, our power analysis was targeted loosely, owing to the lack of similar studies in peer-reviewed literature. This underscores a greater problem: despite the severe consequences associated with postoperative undertriage, there is sparse evidence regarding this topic, which hinders the development of decision-support tools. In the absence of efficient, effective decision-support platforms to augment postoperative triage decisions, surgeons must rely on individual judgement and hypothetical-deductive reasoning, which is highly variable and error-prone and may contribute to variability in surgical care costs and the incidence of failure to rescue across US hospitals.^31,32

This study found that patients at increased risk for postoperative complications who underwent surgical procedures and were undertriaged to hospital wards had increased mortality and morbidity compared with patients in a risk-matched control group admitted to ICUs. Postoperative undertriage was identifiable using automated preoperative and intraoperative EHR data as features in machine-learning models that made data-driven, patient-level risk assessments. These findings may provide a framework and rationale for real-time clinical decision support platforms to augment postoperative triage decisions.

Accepted for Publication: August 26, 2021.

Published: November 10, 2021. doi:10.1001/jamanetworkopen.2021.31669

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Loftus TJ et al. JAMA Network Open.

Corresponding Author: Tyler J. Loftus, MD, Department of Surgery, University of Florida Health, PO Box 100108, Gainesville, FL 32610-0108 (tyler.loftus@surgery.ufl.edu).

Author Contributions: Dr Loftus had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Loftus, Efron, Hogan, Rashidi, Upchurch, Bihorac.

Acquisition, analysis, or interpretation of data: Loftus, Ruppert, Ozrazgat-Baslanti, Balch, Tighe.

Drafting of the manuscript: Loftus, Balch, Efron.

Critical revision of the manuscript for important intellectual content: Ruppert, Ozrazgat-Baslanti, Balch, Tighe, Hogan, Rashidi, Upchurch, Bihorac.

Statistical analysis: Loftus, Ruppert, Ozrazgat-Baslanti.

Obtained funding: Loftus, Rashidi.

Administrative, technical, or material support: Ruppert, Balch, Hogan.

Supervision: Loftus, Efron, Tighe, Hogan, Upchurch, Bihorac.

Conflict of Interest Disclosures: Dr Rashidi reported having a patent pending with the University of Florida. No other disclosures were reported.

Funding/Support: Dr Loftus was supported by grant K23GM140268 from the National Institute of General Medical Sciences (NIGMS) of the National Institutes of Health (NIH). Dr Ozrazgat-Baslanti was supported by grant K01 DK120784 from the National Institute of Diabetes and Digestive and Kidney Diseases of the NIH, grant R01GM110240 from the NIGMS, grant P50 GM-111152 from the NIGMS awarded to the Sepsis and Critical Illness Research Center, and grant AWD09459 from University of Florida Research and the Gatorade Trust of the University of Florida. Dr Tighe was supported by grant R01GM114290 from the NIGMS and grant R01AG121647 from the National Institute on Aging (NIA). Dr Rashidi was supported by Faculty Early Career Development Program award 1750192 from the National Science Foundation, grants P30AG028740 and R01AG05533 from the NIA, grant 1R21EB027344 from the National Institute of Biomedical Imaging and Bioengineering (NIBIB), and grant R01GM-110240 from the NIGMS. Dr Bihorac was supported by grant R01GM110240 from the NIGMS and grant 1R21EB027344 from the NIBIB. This work was supported in part by grant UL1TR000064 from the National Center for Advancing Translational Sciences and Clinical and Translational Sciences via the University of Florida.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.