Evaluation of a Risk Stratification Model Using Preoperative and Intraoperative Data for Major Morbidity or Mortality After Cardiac Surgical Treatment

Postoperative risk–stratification models for major surgical procedures are the standard of care for operative care planning. Risk models, such as the Society of Thoracic Surgeons (STS) Adult Cardiac Surgery Database (ACSD) coronary artery bypass risk calculator, rely on preoperative clinical data to stratify patients by risk for complications.^1,2 While operative strategy and intraoperative events may influence outcomes, the degree to which intraoperative data could be used to improve postoperative risk stratification is not well characterized.³ Accordingly, we used information from a single-center STS-ACSD registry to investigate whether use of intraoperative variables was associated with improved accuracy in postoperative risk stratification.

The institutional review board of Yale University approved this cohort study and waived the informed consent requirement for the use of retrospective data from the STS registry given the retrospective nature of the study and the deidentified nature of the analysis. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline. Data were extracted from our local STS ACSD database, version 2.81, from 2011 to 2017 for patients who received coronary artery bypass with or without a concurrent valve procedure.

We adapted initial preoperative variable selection, adverse events, and data imputation from O’Brien et al¹ and Shahian et al.² Intraoperative candidate variables are defined in the ACSD version 2.81 data collection form. From the candidate variables, we used least absolute shrinkage and selection operator (LASSO) regularization⁴ for variable selection to develop 3 separate logistic regression models that made use of preoperative variables, intraoperative variables, or a combination of preoperative and intraoperative variables. We evaluated the ability of these 3 models to stratify patients by the probability that they would experience a composite of major morbidity or mortality events (eg, operative mortality or reoperation), as previously described.² Performance was assessed using the area under the receiver operator characteristic curve (AUROC) and F1 score (ie, the harmonic mean of positive predictive value and sensitivity) across 100 random splits of the data into training and validation sets. Differences in mean performance metrics between models were assessed using 2-sided unpaired independent t tests with Bonferroni corrections to account for multiple comparisons, and significance was set at P = .016. Data preprocessing and statistical analysis were performed with Python version 2.7 (Python Software Foundation) and the scikit-learn software package version 0.19.1.⁵ Data analysis was performed October 9, 2018, to May 20, 2019.

Of 2905 individuals in the analysis (mean [SD] age, 67.8 [10.6] years; 2193 [75.4%] men), 465 patients experienced an adverse event, for a composite event rate of 15.9%. Of preoperative and intraoperative candidate variables, LASSO regularization identified important variables (ie, those with higher coefficients weighted more heavily by the model) for each of the 3 models (Table).⁴ The magnitude of importance was determined by the distance of the variable coefficient from zero. The intraoperative-only model, compared with the baseline preoperative model, had significantly lower mean (SD) AUROC (0.74 [0.02] vs 0.75 [0.02]; P < .001) and F1 score (0.27 [0.04] vs 0.34 [0.04]; P < .001). Of 31 variables in the model with combined variable selection, 15 (49%) were preoperative variables and 16 (51%) were intraoperative variables. The combined model demonstrated the best overall model performance, with significantly increased mean (SD) AUROC (0.79 [0.02]; P < .001) (Figure, A) and mean (SD) F1 (0.41 [0.04]; P < .001) (Figure, B) compared with the preoperative and intraoperative models.

This cohort study found that intraoperative data combined with preoperative data were associated with increased performance of postoperative risk stratification for a composite profile of adverse events compared with preoperative data alone. While intraoperative variables may be associated with different levels of overall quality of care provided, the use of intraoperative variables, as proposed in this study, remains limited to postoperative care guidance. In contrast, assessment of surgical quality should be limited to information available up to the time of surgical treatment. As postoperative treatment planning becomes increasingly dependent on multidisciplinary care teams, continuously updated perioperative risk models represent a novel opportunity for personalized perioperative care.⁶ Limitations of this study include the limited number of composite adverse events, the need for validation at other sites, the lack of rigorous assumption checks for intraoperative variables, and the inclusion of procedures with concomitant valve replacement, which limits generalizability. Subsequent studies may use national data repositories to generalize to larger populations, identify novel intraoperative variables, and investigate whether updated risk models in the recovery area are associated with improved postoperative outcomes.

Accepted for Publication: October 9, 2020.

Published: December 7, 2020. doi:10.1001/jamanetworkopen.2020.28361

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Durant TJS et al. JAMA Network Open.

Corresponding Author: Wade L. Schulz, MD, PhD, Center for Outcomes Research and Evaluation, Yale New Haven Hospital, 1 Church St, Ste 200, New Haven, CT 06510 (harlan.krumholz@yale.edu).

Author Contributions: Drs Durant and Huang had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs Durant and Jean contributed equally.

Concept and design: Durant, Jean, Schulz, Geirsson, Krumholz.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Durant, Jean, Coppi, Schulz.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Durant, Jean, Huang, Coppi.

Obtained funding: Jean.

Administrative, technical, or material support: Schulz, Geirsson.

Supervision: Schulz, Geirsson, Krumholz.

Conflict of Interest Disclosures: Dr Schulz reported being the founder of Refactor Health and receiving personal fees from Hugo Health and Interpace Diagnostics outside the submitted work. Dr Krumholz reported receiving personal fees from UnitedHealth, IBM Watson Health, Element Science, Aetna, Facebook, Siegfried and Jensen, Arnold and Porter, Martin/Baughman, and the Chinese National Center for Cardiovascular Diseases; being co-founder of Hugo Health and founder of Refactor Health; serving as a venture partner for F-Prime; and receiving contracts (paid to Yale University) from the Centers for Medicare & Medicaid Services and grants (paid to Yale University) from Medtronic, the Food and Drug Administration, Johnson and Johnson, and the Shenzhen Center for Health Information. No other disclosures were reported.

Additional Contributions: Makoto Mori, MD (Yale New Haven Hospital; Yale University School of Medicine), provided creative input on this study and assisted in developing future research in this area. He was not compensated for this work.