Predictive Accuracy of a Polygenic Risk Score Compared With a Clinical Risk Score for Incident Coronary Heart Disease | Cardiology | JAMA | JAMA Network
[Skip to Navigation]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
Benjamin  EJ, Muntner  P, Alonso  A,  et al; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee.  Heart disease and stroke statistics-2019 update: a report from the American Heart Association.   Circulation. 2019;139(10):e56-e528. doi:10.1161/CIR.0000000000000659PubMedGoogle ScholarCrossref
Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.   Circulation. 2014;129(25)(suppl 2):S49-S73. doi:10.1161/01.cir.0000437741.48606.98PubMedGoogle ScholarCrossref
Muntner  P, Colantonio  LD, Cushman  M,  et al.  Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations.   JAMA. 2014;311(14):1406-1415. doi:10.1001/jama.2014.2630PubMedGoogle ScholarCrossref
Greenland  P, Hassan  S.  Precision preventive medicine-ready for prime time?   JAMA Intern Med. 2019;179(5):605-606. doi:10.1001/jamainternmed.2019.0142PubMedGoogle ScholarCrossref
Torkamani  A, Wineinger  NE, Topol  EJ.  The personal and clinical utility of polygenic risk scores.   Nat Rev Genet. 2018;19(9):581-590. doi:10.1038/s41576-018-0018-xPubMedGoogle ScholarCrossref
Khera  AV, Chaffin  M, Wade  KH,  et al.  Polygenic prediction of weight and obesity trajectories from birth to adulthood.   Cell. 2019;177(3):587-596.e9. doi:10.1016/j.cell.2019.03.028PubMedGoogle ScholarCrossref
Khera  AV, Chaffin  M, Aragam  KG,  et al.  Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations.   Nat Genet. 2018;50(9):1219-1224. doi:10.1038/s41588-018-0183-zPubMedGoogle ScholarCrossref
Inouye  M, Abraham  G, Nelson  CP,  et al; UK Biobank CardioMetabolic Consortium CHD Working Group.  Genomic risk prediction of coronary artery disease in 480,000 adults: implications for primary prevention.   J Am Coll Cardiol. 2018;72(16):1883-1893. doi:10.1016/j.jacc.2018.07.079PubMedGoogle ScholarCrossref
Iribarren  C, Lu  M, Jorgenson  E,  et al.  Clinical utility of multimarker genetic risk scores for prediction of incident coronary heart disease: a cohort study among over 51 000 individuals of European ancestry.   Circ Cardiovasc Genet. 2016;9(6):531-540. doi:10.1161/CIRCGENETICS.116.001522PubMedGoogle ScholarCrossref
Wünnemann  F, Sin Lo  K, Langford-Avelar  A,  et al.  Validation of genome-wide polygenic risk scores for coronary artery disease in French Canadians.   Circ Genom Precis Med. 2019;12(6):e002481. doi:10.1161/CIRCGEN.119.002481PubMedGoogle Scholar
Knowles  JW, Ashley  EA.  Cardiovascular disease: the rise of the genetic risk score.   PLoS Med. 2018;15(3):e1002546. doi:10.1371/journal.pmed.1002546PubMedGoogle Scholar
Warren  M.  The approach to predictive medicine that is taking genomics research by storm.   Nature. 2018;562(7726):181-183. doi:10.1038/d41586-018-06956-3PubMedGoogle ScholarCrossref
The ARIC investigators.  The Atherosclerosis Risk in Communities (ARIC) study: design and objectives. The ARIC investigators.   Am J Epidemiol. 1989;129(4):687-702. doi:10.1093/oxfordjournals.aje.a115184PubMedGoogle ScholarCrossref
Bild  DE, Bluemke  DA, Burke  GL,  et al.  Multi-Ethnic Study of Atherosclerosis: objectives and design.   Am J Epidemiol. 2002;156(9):871-881. doi:10.1093/aje/kwf113PubMedGoogle ScholarCrossref
Nikpay  M, Goel  A, Won  H-H,  et al.  A comprehensive 1,000 Genomes-based genome-wide association meta-analysis of coronary artery disease.   Nat Genet. 2015;47(10):1121-1130. doi:10.1038/ng.3396PubMedGoogle ScholarCrossref
Pritchard  JK, Stephens  M, Donnelly  P.  Inference of population structure using multilocus genotype data.   Genetics. 2000;155(2):945-959.PubMedGoogle Scholar
Mosley  JD, van Driest  SL, Wells  QS,  et al.  Defining a contemporary ischemic heart disease genetic risk profile using historical data.   Circ Cardiovasc Genet. 2016;9(6):521-530. doi:10.1161/CIRCGENETICS.116.001530PubMedGoogle ScholarCrossref
Purcell  S, Neale  B, Todd-Brown  K,  et al.  PLINK: a tool set for whole-genome association and population-based linkage analyses.   Am J Hum Genet. 2007;81(3):559-575. doi:10.1086/519795PubMedGoogle ScholarCrossref
Patterson  N, Price  AL, Reich  D.  Population structure and eigenanalysis.   PLoS Genet. 2006;2(12):e190. doi:10.1371/journal.pgen.0020190PubMedGoogle Scholar
Zheng  X, Levine  D, Shen  J, Gogarten  SM, Laurie  C, Weir  BS.  A high-performance computing toolset for relatedness and principal component analysis of SNP data.   Bioinformatics. 2012;28(24):3326-3328. doi:10.1093/bioinformatics/bts606PubMedGoogle ScholarCrossref
Vilhjálmsson  BJ, Yang  J, Finucane  HK,  et al; Schizophrenia Working Group of the Psychiatric Genomics Consortium, Discovery, Biology, and Risk of Inherited Variants in Breast Cancer (DRIVE) study.  Modeling linkage disequilibrium increases accuracy of polygenic risk scores.   Am J Hum Genet. 2015;97(4):576-592. doi:10.1016/j.ajhg.2015.09.001PubMedGoogle ScholarCrossref
Harrell  FE  Jr, Califf  RM, Pryor  DB, Lee  KL, Rosati  RA.  Evaluating the yield of medical tests.   JAMA. 1982;247(18):2543-2546. doi:10.1001/jama.1982.03320430047030PubMedGoogle ScholarCrossref
Demler  OV, Paynter  NP, Cook  NR.  Tests of calibration and goodness-of-fit in the survival setting.   Stat Med. 2015;34(10):1659-1680. doi:10.1002/sim.6428PubMedGoogle ScholarCrossref
Pencina  MJ, D’Agostino  RB  Sr, D’Agostino  RB  Jr, Vasan  RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.   Stat Med. 2008;27(2):157-172. doi:10.1002/sim.2929PubMedGoogle ScholarCrossref
Kerr  KF, Wang  Z, Janes  H, McClelland  RL, Psaty  BM, Pepe  MS.  Net reclassification indices for evaluating risk prediction instruments: a critical review.   Epidemiology. 2014;25(1):114-121. doi:10.1097/EDE.0000000000000018PubMedGoogle ScholarCrossref
Baker  SG, Schuit  E, Steyerberg  EW,  et al.  How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.   Stat Med. 2014;33(22):3946-3959. doi:10.1002/sim.6195PubMedGoogle ScholarCrossref
Hajek  C, Guo  X, Yao  J,  et al.  Coronary heart disease genetic risk score predicts cardiovascular disease risk in men, not women.   Circ Genom Precis Med. 2018;11(10):e002324. doi:10.1161/CIRCGEN.118.002324PubMedGoogle Scholar
Janes  H, Pepe  MS, Gu  W.  Assessing the value of risk predictions by using risk stratification tables.   Ann Intern Med. 2008;149(10):751-760. doi:10.7326/0003-4819-149-10-200811180-00009PubMedGoogle ScholarCrossref
Wald  NJ, Old  R.  The illusion of polygenic disease risk prediction.   Genet Med. 2019;21(8):1705-1707. doi:10.1038/s41436-018-0418-5PubMedGoogle ScholarCrossref
Pepe  MS, Janes  H, Longton  G, Leisenring  W, Newcomb  P.  Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.   Am J Epidemiol. 2004;159(9):882-890. doi:10.1093/aje/kwh101PubMedGoogle ScholarCrossref
Homocysteine Studies Collaboration.  Homocysteine and risk of ischemic heart disease and stroke: a meta-analysis.   JAMA. 2002;288(16):2015-2022. doi:10.1001/jama.288.16.2015PubMedGoogle ScholarCrossref
Kaptoge  S, Di Angelantonio  E, Pennells  L,  et al; Emerging Risk Factors Collaboration.  C-reactive protein, fibrinogen, and cardiovascular disease prediction.   N Engl J Med. 2012;367(14):1310-1320. doi:10.1056/NEJMoa1107477PubMedGoogle ScholarCrossref
Karmali  KN, Goff  DC  Jr, Ning  H, Lloyd-Jones  DM.  A systematic examination of the 2013 ACC/AHA pooled cohort risk assessment tool for atherosclerotic cardiovascular disease.   J Am Coll Cardiol. 2014;64(10):959-968. doi:10.1016/j.jacc.2014.06.1186PubMedGoogle ScholarCrossref
Perak  AM, Ning  H, de Ferranti  SD, Gooding  HC, Wilkins  JT, Lloyd-Jones  DM.  Long-term risk of atherosclerotic cardiovascular disease in US adults with the familial hypercholesterolemia phenotype.   Circulation. 2016;134(1):9-19. doi:10.1161/CIRCULATIONAHA.116.022335PubMedGoogle ScholarCrossref
Wang  TJ, Gona  P, Larson  MG,  et al.  Multiple biomarkers for the prediction of first major cardiovascular events and death.   N Engl J Med. 2006;355(25):2631-2639. doi:10.1056/NEJMoa055373PubMedGoogle ScholarCrossref
Knowles  JW, Zarafshar  S, Pavlovic  A,  et al.  Impact of a genetic risk score for coronary artery disease on reducing cardiovascular risk: a pilot randomized controlled study.   Front Cardiovasc Med. 2017;4:53. doi:10.3389/fcvm.2017.00053PubMedGoogle ScholarCrossref
Kullo  IJ, Jouni  H, Austin  EE,  et al.  Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES Clinical Trial).   Circulation. 2016;133(12):1181-1188. doi:10.1161/CIRCULATIONAHA.115.020109PubMedGoogle ScholarCrossref
De La Vega  FM, Bustamante  CD.  Polygenic risk scores: a biased prediction?   Genome Med. 2018;10(1):100. doi:10.1186/s13073-018-0610-xPubMedGoogle ScholarCrossref
Original Investigation
February 18, 2020

Predictive Accuracy of a Polygenic Risk Score Compared With a Clinical Risk Score for Incident Coronary Heart Disease

Author Affiliations
  • 1Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
  • 2Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee
  • 3Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center, Department of Pediatrics, Torrance, California
  • 4Department of Pharmacology, Vanderbilt University, Nashville, Tennessee
  • 5Vanderbilt O'Brien Center for Kidney Disease, Vanderbilt University Medical Center, Nashville, Tennessee
  • 6Departments of Medicine, Epidemiology and Health Services, University of Washington School of Public Health; and Kaiser Permanente Washington Health Research Institute, Seattle, Washington
  • 7Department of Public Health Sciences, Center for Public Health Genomics, Charlottesville, Virginia
  • 8Department of Medicine, Johns Hopkins University, Baltimore, Maryland
  • 9Department of Epidemiology, Johns Hopkins University, Baltimore, Maryland
  • 10Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center, Department of Medicine, Torrance, California12
  • 11Beth Israel Deaconess Medical Center, Division of Cardiovascular Medicine, Boston, Massachusetts
  • 12Department of Internal Medicine, University of Texas Southwestern Medical Center
JAMA. 2020;323(7):627-635. doi:10.1001/jama.2019.21782
Key Points

Question  Does a polygenic predictor of coronary heart disease (CHD) that incorporates millions of common single-nucleotide polymorphisms (SNPs) improve risk stratification compared with a guideline-based risk equation?

Findings  In a retrospective cohort study that included 7237 middle-aged participants of European ancestry free of clinical CHD at baseline, a polygenic risk score added to the 2013 American College of Cardiology and American Heart Association pooled cohort equations did not significantly improve discriminative accuracy (measured by C statistic), calibration (comparing observed vs expected event probabilities), or net reclassification improvement (using a 10-year risk threshold of 7.5%).

Meaning  Addition of a polygenic risk score to a clinical risk score for incident CHD may not provide important information in a white middle-aged population.


Importance  Polygenic risk scores comprising millions of single-nucleotide polymorphisms (SNPs) could be useful for population-wide coronary heart disease (CHD) screening.

Objective  To determine whether a polygenic risk score improves prediction of CHD compared with a guideline-recommended clinical risk equation.

Design, Setting, and Participants  A retrospective cohort study of the predictive accuracy of a previously validated polygenic risk score was assessed among 4847 adults of white European ancestry, aged 45 through 79 years, participating in the Atherosclerosis Risk in Communities (ARIC) study and 2390 participating in the Multi-Ethnic Study of Atherosclerosis (MESA) from 1996 through December 31, 2015, the final day of follow-up. The performance of the polygenic risk score was compared with that of the 2013 American College of Cardiology and American Heart Association pooled cohort equations.

Exposures  Genetic risk was computed for each participant by summing the product of the weights and allele dosage across 6 630 149 SNPs. Weights were based on an international genome-wide association study.

Main Outcomes and Measures  Prediction of 10-year first CHD events (including myocardial infarctions, fatal coronary events, silent infarctions, revascularization procedures, or resuscitated cardiac arrest) assessed using measures of model discrimination, calibration, and net reclassification improvement (NRI).

Results  The study population included 4847 adults from the ARIC study (mean [SD] age, 62.9 [5.6] years; 56.4% women) and 2390 adults from the MESA cohort (mean [SD] age, 61.8 [9.6] years; 52.2% women). Incident CHD events occurred in 696 participants (14.4%) and 227 participants (9.5%), respectively, over median follow-up of 15.5 years (interquartile range [IQR], 6.3 years) and 14.2 (IQR, 2.5 years) years. The polygenic risk score was significantly associated with 10-year CHD incidence in ARIC with hazard ratios per SD increment of 1.24 (95% CI, 1.15 to 1.34) and in MESA, 1.38 (95% CI, 1.21 to 1.58). Addition of the polygenic risk score to the pooled cohort equations did not significantly increase the C statistic in either cohort (ARIC, change in C statistic, −0.001; 95% CI, −0.009 to 0.006; MESA, 0.021; 95% CI, −0.0004 to 0.043). At the 10-year risk threshold of 7.5%, the addition of the polygenic risk score to the pooled cohort equations did not provide significant improvement in reclassification in either ARIC (NRI, 0.018, 95% CI, −0.012 to 0.036) or MESA (NRI, 0.001, 95% CI, −0.038 to 0.076). The polygenic risk score did not significantly improve calibration in either cohort.

Conclusions and Relevance  In this analysis of 2 cohorts of US adults, the polygenic risk score was associated with incident coronary heart disease events but did not significantly improve discrimination, calibration, or risk reclassification compared with conventional predictors. These findings suggest that a polygenic risk score may not enhance risk prediction in a general, white middle-aged population.