Use of Machine Learning Models to Predict Death After Acute Myocardial Infarction | Acute Coronary Syndromes | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Original Investigation
March 10, 2021

Use of Machine Learning Models to Predict Death After Acute Myocardial Infarction

Author Affiliations
  • 1Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut
  • 2Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, Connecticut
  • 3Department of Internal Medicine, Massachusetts General Hospital, Boston
  • 4Department of Computer Science and Engineering, Texas A&M University, College Station
  • 5Saint Luke's Mid America Heart Institute, Kansas City, Missouri
  • 6Division of Cardiology, Department of Internal Medicine, University of Missouri, Kansas City
  • 7Division of Cardiology, Department of Internal Medicine, University of Colorado Anschutz Medical Campus, Aurora
  • 8Department of Biostatistics, T. H. Chan School of Public Health, Harvard University, Boston, Massachusetts
  • 9Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
  • 10Department of Health Policy and Management, Yale School of Public Health, New Haven, Connecticut
JAMA Cardiol. 2021;6(6):633-641. doi:10.1001/jamacardio.2021.0122
Key Points

Question  Do contemporary machine learning methods improve prediction of in-hospital death after hospitalization for acute myocardial infarction (AMI)?

Findings  In this cohort study of 755 402 patients with AMI in a nationwide registry, machine learning models that used the same data inputs as logistic regression were not associated with substantially improved prediction of in-hospital mortality after AMI. Two of these models, extreme gradient descent boosting and meta-classifier, however, were associated with improved calibration across the risk spectrum, reclassifying 1 in every 4 patients deemed to be at moderate or high risk for death in logistic regression accurately as low risk, consistent with the actual observed risk.

Meaning  These findings suggest that machine learning models are not associated with substantially better prediction of risk of death after AMI but may offer greater resolution of risk, which can better clarify the individual risk for adverse outcomes.

Abstract

Importance  Accurate prediction of adverse outcomes after acute myocardial infarction (AMI) can guide the triage of care services and shared decision-making, and novel methods hold promise for using existing data to generate additional insights.

Objective  To evaluate whether contemporary machine learning methods can facilitate risk prediction by including a larger number of variables and identifying complex relationships between predictors and outcomes.

Design, Setting, and Participants  This cohort study used the American College of Cardiology Chest Pain-MI Registry to identify all AMI hospitalizations between January 1, 2011, and December 31, 2016. Data analysis was performed from February 1, 2018, to October 22, 2020.

Main Outcomes and Measures  Three machine learning models were developed and validated to predict in-hospital mortality based on patient comorbidities, medical history, presentation characteristics, and initial laboratory values. Models were developed based on extreme gradient descent boosting (XGBoost, an interpretable model), a neural network, and a meta-classifier model. Their accuracy was compared against the current standard developed using a logistic regression model in a validation sample.

Results  A total of 755 402 patients (mean [SD] age, 65 [13] years; 495 202 [65.5%] male) were identified during the study period. In independent validation, 2 machine learning models, gradient descent boosting and meta-classifier (combination including inputs from gradient descent boosting and a neural network), marginally improved discrimination compared with logistic regression (C statistic, 0.90 for best performing machine learning model vs 0.89 for logistic regression). Nearly perfect calibration in independent validation data was found in the XGBoost (slope of predicted to observed events, 1.01; 95% CI, 0.99-1.04) and the meta-classifier model (slope of predicted-to-observed events, 1.01; 95% CI, 0.99-1.02), with more precise classification across the risk spectrum. The XGBoost model reclassified 32 393 of 121 839 individuals (27%) and the meta-classifier model reclassified 30 836 of 121 839 individuals (25%) deemed at moderate to high risk for death in logistic regression as low risk, which were more consistent with the observed event rates.

Conclusions and Relevance  In this cohort study using a large national registry, none of the tested machine learning models were associated with substantive improvement in the discrimination of in-hospital mortality after AMI, limiting their clinical utility. However, compared with logistic regression, XGBoost and meta-classifier models, but not the neural network, offered improved resolution of risk for high-risk individuals.

×