[Skip to Content]
[Skip to Content Landing]
Views 922
Citations 0
Original Investigation
December 2016

Evaluation of a Prediction Model for the Development of Atrial Fibrillation in a Repository of Electronic Medical Records

Author Affiliations
  • 1Division of Cardiovascular Medicine, Vanderbilt University School of Medicine, Nashville, Tennessee
  • 2Department of Biostatistics, Vanderbilt University School of Medicine, Nashville, Tennessee
  • 3Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee
  • 4School of Biomedical Informatics, University of Texas Health Science Center at Houston
  • 5Department of Epidemiology, University of Washington, Seattle
  • 6Cardiac Arrhythmia Service, Massachusetts General Hospital, Boston
  • 7Framingham Heart Study, National Heart Lung and Blood Institute and Boston University, Framingham, Massachusetts
  • 8Department of Medicine, Boston University School of Medicine, Boston, Massachusetts
  • 9Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia
  • 10Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
  • 11Department of Biostatistics, Osaka University, Suita, Japan
  • 12Division of Cardiology, University of Illinois at Chicago
JAMA Cardiol. 2016;1(9):1007-1013. doi:10.1001/jamacardio.2016.3366
Key Points

Question  Can the atrial fibrillation (AF) risk prediction model developed by the Calibration of the Cohorts for Heart and Aging Research in Genomic Epidemiology–Atrial Fibrillation (CHARGE-AF) investigators be externally validated using a large repository of electronic medical records (EMRs)?

Findings  In this prediction model study of EMRs of 33 494 patients, 7.3% developed AF during a 5-year period. The CHARGE-AF model was a poor predictor, with underprediction of AF among low-risk individuals and overprediction of AF in high-risk individuals.

Meaning  Application of a risk model derived from prospective cohort studies to an EMR setting has inherent difficulties.

Abstract

Importance  Atrial fibrillation (AF) contributes to substantial morbidity, mortality, and health care expenditures. Accurate prediction of incident AF would enhance AF management and potentially improve patient outcomes.

Objective  To validate the AF risk prediction model originally developed by the Cohorts for Heart and Aging Research in Genomic Epidemiology–Atrial Fibrillation (CHARGE-AF) investigators using a large repository of electronic medical records (EMRs).

Design, Setting, and Participants  In this prediction model study, deidentified EMRs of 33 494 individuals 40 years or older who were white or African American and had no history of AF were reviewed and analyzed. The participants were followed up in the internal medicine outpatient clinics at Vanderbilt University Medical Center for incident AF from December 31, 2005, until December 31, 2010. Adjusting for differences in baseline hazard, the CHARGE-AF Cox proportional hazards model regression coefficients were applied to the EMR cohort. A simple version of the model with no echocardiographic variables was also evaluated. Data were analyzed from October 31, 2013, to January 31, 2014.

Main Outcomes and Measures  Incident AF. Predictors in the model included age, race, height, weight, systolic and diastolic blood pressure, treatment for hypertension, smoking status, type 2 diabetes, heart failure, history of myocardial infarction, left ventricular hypertrophy, and PR interval.

Results  Among the 33 494 participants, the median age was 57 (interquartile range, 49-67) years; 57% of patients were women, 43% were men, 85.7% were white, and 14.3% were African American. During the mean (SD) follow-up of 4.8 (0.9) years, 2455 individuals (7.3%) developed AF. Both models had poor calibration in the EMR cohort, with underprediction of AF among low-risk individuals and overprediction of AF among high-risk individuals (10th and 90th percentiles for predicted probability of incident AF, 0.005 and 0.179, respectively). The full CHARGE-AF model had a C index of 0.708 (95% CI, 0.699-0.718) in our cohort. The simple model had similar discrimination (C index, 0.709; 95% CI, 0.699-0.718; P = .70 for difference between models).

Conclusions and Relevance  Despite reasonable discrimination, the CHARGE-AF models showed poor calibration in this EMR cohort. This study highlights the difficulties of applying a risk model derived from prospective cohort studies to an EMR cohort and suggests that these AF risk prediction models be used with caution in the EMR setting. Future risk models may need to be developed and validated within EMR cohorts.

×