Advances in Appropriate Postoperative Triage and the Role of Real-time Machine-Learning Models: The Goldilocks Dilemma

It is critical in health care to neither overuse nor underuse resources. One of the key decisions in surgical treatment is to determine whether a patient can safely undergo an ambulatory procedure or requires hospital admission and observation. Patients admitted after a procedure undergo further stratification by admission status: to observation, inpatient ward, high-dependency unit, or intensive care unit (ICU). It is essential to accurately determine the optimal location for a patient after surgical treatment, balancing resource allocation with perioperative risks. A dynamic tension exists between overtriage to higher levels of care when not necessary, given that this is costly and is not associated with meaningful changes in perioperative outcomes, and conversely undertriage, which can also be associated with increased costs but may additionally be associated with worse perioperative outcomes. The Goldilocks admission dilemma of too hot, too cold, or just right has not yet been optimized. In this longitudinal, cross-sectional study, Loftus and associates¹ developed a machine-learning model to identify undertriage to hospital wards among patients after surgical procedures. They defined patients who were undertriaged as those who were sent to the floor but found to be in the top quartile of hospital mortality risk and who subsequently required a prolonged ICU stay of more than 48 hours. Out of more than 12 000 postoperative ward admissions during the 6-year study period, 10.6% of admissions were identified as undertriaged. By using this model, the authors found that, compared with risk-matched ICU admissions, undertriaged postoperative admissions had an increased risk of mortality and morbidity, namely unplanned intubation, acute kidney injury, and longer hospital length of stay.

Existing models for postoperative triage have been primarily based on high-risk surgical procedures, as defined by estimated mortality rates of 5% or more,² or guided by basic tools, such as the Modified Early Warning Score (MEWS), calculated in the preoperative and postoperative setting.³ The study by Loftus et al¹ is unique in that it takes machine-learning algorithms from preoperative and intraoperative data to estimate patients at increased risk of postoperative complications. In developing this model, the authors found that primary procedure, scheduled postoperative location, intraoperative minimum alveolar anesthetic concentration measurements, and duration of inhalation anesthetic were among the most important features of estimators associated with mortality and prolonged ICU stay.

The authors point out that compared with appropriately triaged postoperative ward admissions and risk-matched control ICU admissions groups, the undertriaged admissions group was older, had an increased proportion of admitted patients identifying as Black or African American, and had increased area deprivation indices, suggesting increased socioeconomic disadvantage. The undertriaged admissions group also had increased Charlson Comorbidity Index scores, as well as increased proportions of admitted patients undergoing emergent admission and emergent surgical procedures. In post hoc exploratory, secondary analyses, however, self-identification as Black or African American and increased area deprivation indices were not associated with worse outcomes (ie, hospital mortality, unplanned intubation, and acute kidney injury) despite undertriage to the wards.¹ These findings, however, may encourage clinicians to pause to make appropriate postoperative triage decisions when confronted with certain patient populations.

As previously mentioned, MEWS has been applied to triage patients to the appropriate postoperative level of care, particularly after emergency abdominal surgical procedures.³ As a bedside evaluation of 5 physiological parameters, a MEWS score of 5 or more was considered to meet criteria for postoperative ICU admission, whereas a score of 3 to 4 warranted admission to a high-dependency unit (ie, a step-down unit). In an Italian study,³ use of MEWS for postoperative triage was associated with a significantly decreased rate of ICU admissions without a difference in mortality rate, suggesting the tool’s utility in preventing overtriage to the ICU. With more sophisticated machine-learning models, such as that developed by Loftus and associates,¹ one could anticipate not only avoiding undertriage to wards, which may be wrought with increased mortality and morbidity, but also preventing overtriage to the ICU in the setting of increased health care costs and overuse of resources.

It seems critical, however, that real-time machine-learning models be tested against the clinical judgment of clinicians. “To err is human,” a phrase corroborated by findings from a recent multicenter study on the transition of patients from the ICU to wards.⁴ The study found that at the time of transfer out of the ICU, ward and ICU physicians had poor accuracy in estimating adverse events, readmissions to the ICU, and hospital mortality. Alternatively, even early artificial intelligence tools using electronic health record data for detecting clinical deterioration in patients, such as the Rothman Index (RI), have weaknesses compared with clinical acumen. Some individuals argue that certain critical variables are not incorporated into the equation, while others claim that clinicians using their judgment would have already suspected clinical worsening before a model would detect such an outcome. A prospective observational study comparing RI with physician judgment in estimating deterioration among internal medicine patients found that there was no significant difference in estimation.⁵ A combined model using RI and physician judgment, however, was associated with improved performance compared with either method alone. Tools like MEWS and RI are not specific to patients undergoing surgical procedures, but other machine-learning algorithms, such as MySurgeryRisk, have been developed for estimating preoperative risk for major surgical complications.⁶ Although it was found that physicians’ initial risk assessments were not as accurate as the algorithm in estimating most complications, physicians improved their risk assessment after interacting with the algorithm. These data-driven, patient-level risk assessment models seem promising, not in substitution for clinical judgment, but in supplementation of it. Given that Loftus et al¹ present a unique postoperative triage algorithm, appropriate studies comparing the model with clinical judgment may be warranted to investigate whether machine-learning models may augment physicians’ decision-making. An optimal system would perform real-time analysis to provide an informed recommendation for admission location while the patient was still in the operating room or recovery unit. This may minimize use of resources among patients at decreased risk of postoperative complications and identify patients at increased risk of complications for closer observation and early interventions to decrease perioperative morbidity and mortality. Real-time, big data machine-learning algorithms may eventually serve a critical role in determining patient admission location after the operating room.

Published: November 10, 2021. doi:10.1001/jamanetworkopen.2021.33843

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Ko A et al. JAMA Network Open.

Corresponding Author: Sherry M. Wren, MD, Division of General Surgery, Department of Surgery, Stanford University, G112 3801 Miranda Ave, Palo Alto, CA 94304 (swren@stanford.edu).

Conflict of Interest Disclosures: None reported.