Predictive Accuracy of a Polygenic Risk Score Compared With a Clinical Risk Score for Incident Coronary Heart Disease | Cardiology | JAMA | JAMA Network
[Skip to Navigation]
Figure 1.  Overview of the Atherosclerosis Risk in Communities (ARIC) and Multi-Ethnic Study of Atherosclerosis (MESA) Study Population Selection
Overview of the Atherosclerosis Risk in Communities (ARIC) and Multi-Ethnic Study of Atherosclerosis (MESA) Study Population Selection

aFor the ARIC population, participants with a diagnosis of coronary heart disease (CHD) at the time of their visit 1 examination (prevalent cases) or with a diagnosis of CHD that occurred between their visit 1 and visit 4 (10 year) examinations (incident cases) were excluded from the primary analyses of incident CHD risk. The primary analyses used visit 4 as the starting point and examined incident events that occurred after that visit. SNP indicates single-nucleotide polymorphism.

Figure 2.  Reclassification of 10-Year Predicted Coronary Heart Disease Risk
Reclassification of 10-Year Predicted Coronary Heart Disease Risk

Columns and rows refer to categories of 10-year predicted risk. The numbers represent the counts of individuals assigned to the indicated risk category. The number of events differs from those in the main analyses because the table is restricted to events occurring over the first 10 years of follow-up. The standard coronary heart disease (CHD) model includes sex, age, 5 principal components, and the binary classifier for low or high risk are based on the pooled risk equations. Thus, among the 496 individuals in the Atherosclerosis Risk in Communities (ARIC) cohort with events, 2 were correctly up classified and 8 individuals were incorrectly down classified. The net proportion of correct reclassifications for events is –6/496 = –0.012. Among ARIC nonevents, 146 were correctly down classified and 27 were incorrectly up classified. The net proportion of correct reclassifications for nonevents is 119/3672 = 0.032. The overall net reclassification improvement is defined as the sum of the net reclassifications for events and nonevents (eg, –0.012 + 0.032 = 0.02). MESA indicates Multi-Ethnic Study of Atherosclerosis.

Table 1.  Baseline Characteristics of the ARIC and MESA Cohorts
Baseline Characteristics of the ARIC and MESA Cohorts
Table 2.  Odds Ratios Associated With CHD Cases Prior to Visit 4 for Selected Polygenic Risk Score Percentiles Among ARIC Participants
Odds Ratios Associated With CHD Cases Prior to Visit 4 for Selected Polygenic Risk Score Percentiles Among ARIC Participants
Table 3.  Hazard Ratios for ARIC and MESA Incident CHD Events for Selected Polygenic Risk Score Strata
Hazard Ratios for ARIC and MESA Incident CHD Events for Selected Polygenic Risk Score Strata
Table 4.  C Statistics Evaluating the Performance of the Polygenic Risk Score in ARIC and MESA
C Statistics Evaluating the Performance of the Polygenic Risk Score in ARIC and MESA
Table 5.  Reclassification Based on the Net Reclassification Improvement
Reclassification Based on the Net Reclassification Improvement
1.
Benjamin  EJ, Muntner  P, Alonso  A,  et al; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee.  Heart disease and stroke statistics-2019 update: a report from the American Heart Association.   Circulation. 2019;139(10):e56-e528. doi:10.1161/CIR.0000000000000659PubMedGoogle ScholarCrossref
2.
Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.   Circulation. 2014;129(25)(suppl 2):S49-S73. doi:10.1161/01.cir.0000437741.48606.98PubMedGoogle ScholarCrossref
3.
Muntner  P, Colantonio  LD, Cushman  M,  et al.  Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations.   JAMA. 2014;311(14):1406-1415. doi:10.1001/jama.2014.2630PubMedGoogle ScholarCrossref
4.
Greenland  P, Hassan  S.  Precision preventive medicine-ready for prime time?   JAMA Intern Med. 2019;179(5):605-606. doi:10.1001/jamainternmed.2019.0142PubMedGoogle ScholarCrossref
5.
Torkamani  A, Wineinger  NE, Topol  EJ.  The personal and clinical utility of polygenic risk scores.   Nat Rev Genet. 2018;19(9):581-590. doi:10.1038/s41576-018-0018-xPubMedGoogle ScholarCrossref
6.
Khera  AV, Chaffin  M, Wade  KH,  et al.  Polygenic prediction of weight and obesity trajectories from birth to adulthood.   Cell. 2019;177(3):587-596.e9. doi:10.1016/j.cell.2019.03.028PubMedGoogle ScholarCrossref
7.
Khera  AV, Chaffin  M, Aragam  KG,  et al.  Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations.   Nat Genet. 2018;50(9):1219-1224. doi:10.1038/s41588-018-0183-zPubMedGoogle ScholarCrossref
8.
Inouye  M, Abraham  G, Nelson  CP,  et al; UK Biobank CardioMetabolic Consortium CHD Working Group.  Genomic risk prediction of coronary artery disease in 480,000 adults: implications for primary prevention.   J Am Coll Cardiol. 2018;72(16):1883-1893. doi:10.1016/j.jacc.2018.07.079PubMedGoogle ScholarCrossref
9.
Iribarren  C, Lu  M, Jorgenson  E,  et al.  Clinical utility of multimarker genetic risk scores for prediction of incident coronary heart disease: a cohort study among over 51 000 individuals of European ancestry.   Circ Cardiovasc Genet. 2016;9(6):531-540. doi:10.1161/CIRCGENETICS.116.001522PubMedGoogle ScholarCrossref
10.
Wünnemann  F, Sin Lo  K, Langford-Avelar  A,  et al.  Validation of genome-wide polygenic risk scores for coronary artery disease in French Canadians.   Circ Genom Precis Med. 2019;12(6):e002481. doi:10.1161/CIRCGEN.119.002481PubMedGoogle Scholar
11.
Knowles  JW, Ashley  EA.  Cardiovascular disease: the rise of the genetic risk score.   PLoS Med. 2018;15(3):e1002546. doi:10.1371/journal.pmed.1002546PubMedGoogle Scholar
12.
Warren  M.  The approach to predictive medicine that is taking genomics research by storm.   Nature. 2018;562(7726):181-183. doi:10.1038/d41586-018-06956-3PubMedGoogle ScholarCrossref
13.
The ARIC investigators.  The Atherosclerosis Risk in Communities (ARIC) study: design and objectives. The ARIC investigators.   Am J Epidemiol. 1989;129(4):687-702. doi:10.1093/oxfordjournals.aje.a115184PubMedGoogle ScholarCrossref
14.
Bild  DE, Bluemke  DA, Burke  GL,  et al.  Multi-Ethnic Study of Atherosclerosis: objectives and design.   Am J Epidemiol. 2002;156(9):871-881. doi:10.1093/aje/kwf113PubMedGoogle ScholarCrossref
15.
Nikpay  M, Goel  A, Won  H-H,  et al.  A comprehensive 1,000 Genomes-based genome-wide association meta-analysis of coronary artery disease.   Nat Genet. 2015;47(10):1121-1130. doi:10.1038/ng.3396PubMedGoogle ScholarCrossref
16.
Pritchard  JK, Stephens  M, Donnelly  P.  Inference of population structure using multilocus genotype data.   Genetics. 2000;155(2):945-959.PubMedGoogle Scholar
17.
Mosley  JD, van Driest  SL, Wells  QS,  et al.  Defining a contemporary ischemic heart disease genetic risk profile using historical data.   Circ Cardiovasc Genet. 2016;9(6):521-530. doi:10.1161/CIRCGENETICS.116.001530PubMedGoogle ScholarCrossref
18.
Purcell  S, Neale  B, Todd-Brown  K,  et al.  PLINK: a tool set for whole-genome association and population-based linkage analyses.   Am J Hum Genet. 2007;81(3):559-575. doi:10.1086/519795PubMedGoogle ScholarCrossref
19.
Patterson  N, Price  AL, Reich  D.  Population structure and eigenanalysis.   PLoS Genet. 2006;2(12):e190. doi:10.1371/journal.pgen.0020190PubMedGoogle Scholar
20.
Zheng  X, Levine  D, Shen  J, Gogarten  SM, Laurie  C, Weir  BS.  A high-performance computing toolset for relatedness and principal component analysis of SNP data.   Bioinformatics. 2012;28(24):3326-3328. doi:10.1093/bioinformatics/bts606PubMedGoogle ScholarCrossref
21.
Vilhjálmsson  BJ, Yang  J, Finucane  HK,  et al; Schizophrenia Working Group of the Psychiatric Genomics Consortium, Discovery, Biology, and Risk of Inherited Variants in Breast Cancer (DRIVE) study.  Modeling linkage disequilibrium increases accuracy of polygenic risk scores.   Am J Hum Genet. 2015;97(4):576-592. doi:10.1016/j.ajhg.2015.09.001PubMedGoogle ScholarCrossref
22.
Harrell  FE  Jr, Califf  RM, Pryor  DB, Lee  KL, Rosati  RA.  Evaluating the yield of medical tests.   JAMA. 1982;247(18):2543-2546. doi:10.1001/jama.1982.03320430047030PubMedGoogle ScholarCrossref
23.
Demler  OV, Paynter  NP, Cook  NR.  Tests of calibration and goodness-of-fit in the survival setting.   Stat Med. 2015;34(10):1659-1680. doi:10.1002/sim.6428PubMedGoogle ScholarCrossref
24.
Pencina  MJ, D’Agostino  RB  Sr, D’Agostino  RB  Jr, Vasan  RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.   Stat Med. 2008;27(2):157-172. doi:10.1002/sim.2929PubMedGoogle ScholarCrossref
25.
Kerr  KF, Wang  Z, Janes  H, McClelland  RL, Psaty  BM, Pepe  MS.  Net reclassification indices for evaluating risk prediction instruments: a critical review.   Epidemiology. 2014;25(1):114-121. doi:10.1097/EDE.0000000000000018PubMedGoogle ScholarCrossref
26.
Baker  SG, Schuit  E, Steyerberg  EW,  et al.  How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.   Stat Med. 2014;33(22):3946-3959. doi:10.1002/sim.6195PubMedGoogle ScholarCrossref
27.
Hajek  C, Guo  X, Yao  J,  et al.  Coronary heart disease genetic risk score predicts cardiovascular disease risk in men, not women.   Circ Genom Precis Med. 2018;11(10):e002324. doi:10.1161/CIRCGEN.118.002324PubMedGoogle Scholar
28.
Janes  H, Pepe  MS, Gu  W.  Assessing the value of risk predictions by using risk stratification tables.   Ann Intern Med. 2008;149(10):751-760. doi:10.7326/0003-4819-149-10-200811180-00009PubMedGoogle ScholarCrossref
29.
Wald  NJ, Old  R.  The illusion of polygenic disease risk prediction.   Genet Med. 2019;21(8):1705-1707. doi:10.1038/s41436-018-0418-5PubMedGoogle ScholarCrossref
30.
Pepe  MS, Janes  H, Longton  G, Leisenring  W, Newcomb  P.  Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.   Am J Epidemiol. 2004;159(9):882-890. doi:10.1093/aje/kwh101PubMedGoogle ScholarCrossref
31.
Homocysteine Studies Collaboration.  Homocysteine and risk of ischemic heart disease and stroke: a meta-analysis.   JAMA. 2002;288(16):2015-2022. doi:10.1001/jama.288.16.2015PubMedGoogle ScholarCrossref
32.
Kaptoge  S, Di Angelantonio  E, Pennells  L,  et al; Emerging Risk Factors Collaboration.  C-reactive protein, fibrinogen, and cardiovascular disease prediction.   N Engl J Med. 2012;367(14):1310-1320. doi:10.1056/NEJMoa1107477PubMedGoogle ScholarCrossref
33.
Karmali  KN, Goff  DC  Jr, Ning  H, Lloyd-Jones  DM.  A systematic examination of the 2013 ACC/AHA pooled cohort risk assessment tool for atherosclerotic cardiovascular disease.   J Am Coll Cardiol. 2014;64(10):959-968. doi:10.1016/j.jacc.2014.06.1186PubMedGoogle ScholarCrossref
34.
Perak  AM, Ning  H, de Ferranti  SD, Gooding  HC, Wilkins  JT, Lloyd-Jones  DM.  Long-term risk of atherosclerotic cardiovascular disease in US adults with the familial hypercholesterolemia phenotype.   Circulation. 2016;134(1):9-19. doi:10.1161/CIRCULATIONAHA.116.022335PubMedGoogle ScholarCrossref
35.
Wang  TJ, Gona  P, Larson  MG,  et al.  Multiple biomarkers for the prediction of first major cardiovascular events and death.   N Engl J Med. 2006;355(25):2631-2639. doi:10.1056/NEJMoa055373PubMedGoogle ScholarCrossref
36.
Knowles  JW, Zarafshar  S, Pavlovic  A,  et al.  Impact of a genetic risk score for coronary artery disease on reducing cardiovascular risk: a pilot randomized controlled study.   Front Cardiovasc Med. 2017;4:53. doi:10.3389/fcvm.2017.00053PubMedGoogle ScholarCrossref
37.
Kullo  IJ, Jouni  H, Austin  EE,  et al.  Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES Clinical Trial).   Circulation. 2016;133(12):1181-1188. doi:10.1161/CIRCULATIONAHA.115.020109PubMedGoogle ScholarCrossref
38.
De La Vega  FM, Bustamante  CD.  Polygenic risk scores: a biased prediction?   Genome Med. 2018;10(1):100. doi:10.1186/s13073-018-0610-xPubMedGoogle ScholarCrossref
Original Investigation
February 18, 2020

Predictive Accuracy of a Polygenic Risk Score Compared With a Clinical Risk Score for Incident Coronary Heart Disease

Author Affiliations
  • 1Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee
  • 2Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee
  • 3Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center, Department of Pediatrics, Torrance, California
  • 4Department of Pharmacology, Vanderbilt University, Nashville, Tennessee
  • 5Vanderbilt O'Brien Center for Kidney Disease, Vanderbilt University Medical Center, Nashville, Tennessee
  • 6Departments of Medicine, Epidemiology and Health Services, University of Washington School of Public Health; and Kaiser Permanente Washington Health Research Institute, Seattle, Washington
  • 7Department of Public Health Sciences, Center for Public Health Genomics, Charlottesville, Virginia
  • 8Department of Medicine, Johns Hopkins University, Baltimore, Maryland
  • 9Department of Epidemiology, Johns Hopkins University, Baltimore, Maryland
  • 10Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center, Department of Medicine, Torrance, California12
  • 11Beth Israel Deaconess Medical Center, Division of Cardiovascular Medicine, Boston, Massachusetts
  • 12Department of Internal Medicine, University of Texas Southwestern Medical Center
JAMA. 2020;323(7):627-635. doi:10.1001/jama.2019.21782
Key Points

Question  Does a polygenic predictor of coronary heart disease (CHD) that incorporates millions of common single-nucleotide polymorphisms (SNPs) improve risk stratification compared with a guideline-based risk equation?

Findings  In a retrospective cohort study that included 7237 middle-aged participants of European ancestry free of clinical CHD at baseline, a polygenic risk score added to the 2013 American College of Cardiology and American Heart Association pooled cohort equations did not significantly improve discriminative accuracy (measured by C statistic), calibration (comparing observed vs expected event probabilities), or net reclassification improvement (using a 10-year risk threshold of 7.5%).

Meaning  Addition of a polygenic risk score to a clinical risk score for incident CHD may not provide important information in a white middle-aged population.

Abstract

Importance  Polygenic risk scores comprising millions of single-nucleotide polymorphisms (SNPs) could be useful for population-wide coronary heart disease (CHD) screening.

Objective  To determine whether a polygenic risk score improves prediction of CHD compared with a guideline-recommended clinical risk equation.

Design, Setting, and Participants  A retrospective cohort study of the predictive accuracy of a previously validated polygenic risk score was assessed among 4847 adults of white European ancestry, aged 45 through 79 years, participating in the Atherosclerosis Risk in Communities (ARIC) study and 2390 participating in the Multi-Ethnic Study of Atherosclerosis (MESA) from 1996 through December 31, 2015, the final day of follow-up. The performance of the polygenic risk score was compared with that of the 2013 American College of Cardiology and American Heart Association pooled cohort equations.

Exposures  Genetic risk was computed for each participant by summing the product of the weights and allele dosage across 6 630 149 SNPs. Weights were based on an international genome-wide association study.

Main Outcomes and Measures  Prediction of 10-year first CHD events (including myocardial infarctions, fatal coronary events, silent infarctions, revascularization procedures, or resuscitated cardiac arrest) assessed using measures of model discrimination, calibration, and net reclassification improvement (NRI).

Results  The study population included 4847 adults from the ARIC study (mean [SD] age, 62.9 [5.6] years; 56.4% women) and 2390 adults from the MESA cohort (mean [SD] age, 61.8 [9.6] years; 52.2% women). Incident CHD events occurred in 696 participants (14.4%) and 227 participants (9.5%), respectively, over median follow-up of 15.5 years (interquartile range [IQR], 6.3 years) and 14.2 (IQR, 2.5 years) years. The polygenic risk score was significantly associated with 10-year CHD incidence in ARIC with hazard ratios per SD increment of 1.24 (95% CI, 1.15 to 1.34) and in MESA, 1.38 (95% CI, 1.21 to 1.58). Addition of the polygenic risk score to the pooled cohort equations did not significantly increase the C statistic in either cohort (ARIC, change in C statistic, −0.001; 95% CI, −0.009 to 0.006; MESA, 0.021; 95% CI, −0.0004 to 0.043). At the 10-year risk threshold of 7.5%, the addition of the polygenic risk score to the pooled cohort equations did not provide significant improvement in reclassification in either ARIC (NRI, 0.018, 95% CI, −0.012 to 0.036) or MESA (NRI, 0.001, 95% CI, −0.038 to 0.076). The polygenic risk score did not significantly improve calibration in either cohort.

Conclusions and Relevance  In this analysis of 2 cohorts of US adults, the polygenic risk score was associated with incident coronary heart disease events but did not significantly improve discrimination, calibration, or risk reclassification compared with conventional predictors. These findings suggest that a polygenic risk score may not enhance risk prediction in a general, white middle-aged population.

Introduction

Quiz Ref IDEarly identification and treatment of individuals at risk of coronary heart disease (CHD) has been an important contributor to reductions in cardiovascular morbidity and mortality since 1970.1 The American College of Cardiology and American Heart Association (ACC/AHA) cardiovascular prevention guidelines suggest that therapy with lipid-lowering statin medications be considered for individuals with an estimated 10-year risk of atherosclerotic events greater than 7.5% based on the 2013 ACC/AHA pooled cohort equations.2 However, many individuals who develop CHD have an estimated 10-year cardiovascular risk of less than 7.5%.3 Conversely, only a minority of those judged to be at high risk actually have events over the subsequent decades. Thus, there is considerable interest in identifying strategies to enhance risk stratification in order to minimize overtreatment and undertreatment, improve communication with patients about risk, and promote further health gains.4

Quiz Ref IDRecently, CHD risk scores based on common genetic variation have been developed using single-nucleotide polymorphisms (SNPs) derived from genome-wide association studies (GWAS).5 Such classifiers may now incorporate millions of SNPs.6 In cross-sectional studies, individuals falling in the highest deciles of these polygenic risk scores have odds ratios for prevalent CHD of 3 to 4 compared with lower risk individuals.7,8 The risk associated with elevated polygenic risk scores has been more modest in studies focusing on incident events.9,10 Nevertheless, there has been substantial interest in the possibility of incorporating polygenic risk scores into population-wide screening, as evidenced by the attention given by both the scientific and lay communities.11,12

The clinical utility of new risk markers such as the polygenic risk score depends on the ability to predict future CHD events, not on the strength of the associations with prevalent CHD. The objective of this study was to evaluate the performance of a polygenic risk score for prediction of incident CHD events compared with risk prediction using a guideline-recommended clinical risk equation.13,14

Methods
Study Population

Data came from 2 population-based cohort studies, the Atherosclerosis Risk in Communities (ARIC) study and the Multi-Ethnic Study of Atherosclerosis (MESA) (Table 1). The ARIC study comprised genotyped adult participants aged between 45 and 64 years old and followed up from 1986 through 2015.13 Publicly available ARIC data were obtained from dB Gap (phs000280). Use of ARIC data was approved by the institutional review board of Vanderbilt University Medical Center.

The MESA cohort comprised genotyped individuals from 45 through 84 years old, recruited from 2000 through 2002, and followed up through 2015.14 Data were obtained through dB Gap (phs000209), and analyses were approved by the institutional review boards of Cedars-Sinai Medical Center, LA BioMed at Harbor UCLA, University of Washington (MESA DCC), and affiliated MESA field centers. All ARIC and MESA participants provided written informed consent.

Because the existing polygenic risk score was derived from a majority of persons (77%) with white European ancestry via genome-wide association study analysis15 and calibrated for use in this population, analyses were restricted to participants with European ancestry. In the ARIC cohort, genetic ancestry was determined using the STRUCTURE program.16,17 In the MESA cohort, individuals of European ancestry were those whose race was reported as white and confirmed by principal components analyses.

Genetic Data

Single-nucleotide polymorphism genotype data for both cohorts were acquired on the Affymetrix 6.0 SNP array. Quality control for the ARIC data set followed the guidelines accompanying the dB Gap release and used PLINK version 1.07.18 For both data sets, SNPs were imputed using the 1000 Genomes cosmopolitan phase 3 version 5 reference haplotypes. Closely related individuals were excluded by randomly removing one of each pair of individuals with pi-hat genetic relatedness that was greater than 0.05 for the ARIC study or pi-hat that was greater than 0.2 for the MESA cohort. Principal components used to control for population stratification19 were generated from the underlying SNP genotypes using the packages from SNPRelate for ARIC or EIGENSOFT for MESA.20

Phenotypes

In ARIC, the primary analyses examined age, smoking status (current vs other), systolic blood pressure, antihypertensive medication use, total cholesterol, high-density lipoprotein cholesterol, and type 2 diabetes status ascertained at the visit 4 examination (1996-1998). The analogous variables in the MESA cohort were ascertained at the baseline examination (visit 1, 2000-2002). In the ARIC cohort, a binary family history variable, which was not used in predictive models, was defined as positive if either the mother or father had CHD or negative if otherwise.

The ARIC study incident CHD cases were defined as having incident myocardial infarction (MI), fatal coronary event, or silent infarction or having undergone a revascularization procedure by December 31, 2015. The ARIC study prevalent CHD cases were participants with a reported history of MI, heart or arterial surgery, coronary artery bypass graft surgery, or angioplasty; or evidence of having had an MI based on electrocardiogram taken at their visit 1 examination. The MESA cohort incident CHD cases were defined as MI, resuscitated cardiac arrest, definite or probable angina if followed by a revascularization, and CHD death occurring by visit 5 (December 31, 2015). For each individual, 10-year risk based on the 2013 ACC/AHA pooled cohort equations was calculated using the race- and sex-specific formulas provided in the guidelines.2 Individuals were also grouped into low risk (10-year risk ≤7.5%) or high risk (>7.5%) groups based on the pooled risk equations.2 Individuals missing any measurement required to compute their pooled cohort equations–based risk were excluded from analyses.

CHD Polygenic Risk Score

These analyses used the CHD polygenic risk score previously developed by Khera et al7 and based on the summary statistics from the Coronary Artery Disease Genome Wide Replication and Meta-analysis plus the Coronary Artery Disease Genetics (CARDIOGRAMplusC4D) consortium GWAS analysis. The Khera study empirically evaluated a large number of polygenic risk scores that were created using differing SNP-selection methods. The study found that the best performing polygenic risk score was based on the linkage-disequilibrium SNP-reweighting approach encoded in the LDpred21 software package, which incorporated the majority of common SNPs analyzed in the GWAS. The best-performing score comprised 6 630 149 million SNPs and was the one used in our analyses. Single-nucleotide polymorphism weightings were downloaded from http://www.broadcvdi.org/informational/data.

The 6-million SNP score included a large number of SNPs below the genome-wide significance threshold for association with CHD, so it is likely that many of those SNPs did not contribute to the explanatory power of the score. In secondary analyses, the performance of 5 additional polygenic risk scores that used smaller numbers of SNPs, from 652 down to 44 corresponding to increasingly stringent thresholds for significance for association with CHD, was also evaluated. Each polygenic risk score was computed for each individual by summing the product of the allele weighting and the allele dosage across the selected SNPs.

Analysis

Although the primary focus was the prediction of incident CHD events, analyses were conducted that used the combination of prevalent and incident CHD events. Prevalent CHD was incorporated into the initial analyses so that the results presented herein could be compared with those of Khera et al, who examined prevalent cases in the UK Biobank data set. Khera et al showed that the polygenic risk score was significantly associated with CHD case status in the UK Biobank in logistic regression analyses.7 To demonstrate that the polygenic risk score in these analyses retained this feature, similar logistic regression analyses were used to measure the association of the polygenic risk score with CHD using prevalent and incident (prior to visit 4) CHD cases and controls from the ARIC study. The analyses adjusted for age, sex, and the first 5 principal components. Because adjusting for principal components is essential to ensure that any associations with a polygenic risk score are not attributable to population stratification, all statistical models incorporated principal components.

Subsequent analyses focused only on incident CHD events. Cox proportional hazards modeling was used to estimate hazard ratios and to compute 10-year CHD event probabilities. All analyses were adjusted for age, the top 5 principal components, and sex. Model fit was evaluated by examining Schoenfeld residuals to evaluate the proportional hazards assumptions for the covariates, Martingale residuals to assess nonlinearity, and deviance results to identify influential outliers.

Harrell C statistics were based on a 10-year follow-up window, as previously described.22 The C statistics were computed using a Cox model that included the pooled cohort equations estimated risk modeled as a continuous variable (range, 0-1).2 The primary analyses examined the difference in C statistics when the polygenic risk score was added to the pooled cohort risk model. We computed 95% confidence intervals for C statistics and for the difference in C statistic values between models by bootstrapping. Model calibration was assessed by comparing observed vs expected event probabilities using the Greenwood-Nam-D'Agostino χ2 test.23

The net reclassification improvement (NRI) assesses the correct reassignment among risk categories.24,25 Risk probabilities are determined by Cox modeling and a base model is compared with an alternate model that includes the additional classifier being evaluated.26 For the primary analyses, the base model included the pooled cohort risk classifier defined as low risk (≤7.5% 10-year risk of incident events) or high risk (>7.5%) based on the pooled risk equations. The alternate model included the CHD polygenic risk score. We used bootstrapping to determine 95% confidence intervals.

All statistical tests were 2-sided and a P < .05 was considered significant. We also considered significant 95% confidence intervals derived from bootstrapping that did not cross 0.

Separate models that included either only men or only women were also run.27 The R v3.5.0 package was used in conjunction with the survival, survminer, nricens, boot, and DescTools packages.

Results

The overall ARIC study sample consisted of 13 113 participants, of whom 7480 participants of European ancestry had complete data at their baseline visit 1 examination (Figure 1; and eTable 1 in the Supplement). Of these, 4847 participants between the ages of 53 and 74 years did not have a prior diagnosis of CHD at their visit 4 examination (43.6% men) (Table 1). The overall MESA sample comprised 6680 participants, of whom 2390 met the inclusion criteria (47.8% men) (Table 1 and Figure 1). The ARIC study reported 696 (14.4%) incident CHD events over a median follow-up of 15.5 years (interquartile range [IQR], 6.3 years) with 448 (64%) occurring in men. The MESA trial reported 227 (9.5%) incident CHD events over 14.2 years (IQR, 2.5 years) with 139 (61%) occurring in men. Demographic characteristics by sex are presented in eTable 2 in the Supplement.

In the ARIC study, 394 participants had prevalent CHD at visit 1 in 1986 and another 611 participants developed incident CHD before visit 4 in 1996. The association of the polygenic risk score with prevalent and incident CHD was assessed in this manner for the age- and sex-adjusted analyses to provide comparable information with prior studies that had used a similar end point. The polygenic risk score was significantly associated with CHD (adjusted odds ratio [OR] per SD increment, 1.89; 95% CI, 1.75-2.03; Table 2). Those in the top decile of the polygenic risk score had an adjusted OR of 3.19 (95%, CI, 2.64-3.84) compared with those in the lower 9 deciles. The odds of CHD for other quantiles of the polygenic risk score are presented in Table 2.

The association of the polygenic risk score with incident CHD events in the ARIC cohort after the fourth visit in 1996 and the MESA cohort after the first visit in 2000 was examined next. The polygenic risk score was significantly associated with incident CHD in the ARIC cohort (adjusted hazards ratio [HR] per SD increment, 1.24; 95% CI, 1.15-1.34) and the MESA cohort (adjusted HR, 1.38; 95% CI, 1.21-1.58; Table 3). Hazard ratios associated with polygenic risk score values in the upper 5th and 10th percentiles are shown in Table 3 and are shown stratified by sex in eTable 3 in the Supplement.

The C statistic associated with the polygenic risk score alone was 0.549 (95% CI, 0.521 to 0.571) for the ARIC cohort and 0.587 (95% CI, 0.532 to 0.623) for the MESA cohort (Table 4). A model that included age and sex in addition to the polygenic risk score had a C statistic of 0.669 (95% CI, 0.644 to 0.691) for the ARIC cohort and 0.672 (95% CI, 0.627 to 0.705) for the MESA cohort. The addition of the polygenic risk score to the pooled equations predictor (modeled as a continuous variable between 0 and 1) did not significantly change the C statistic in the ARIC cohort from 0.701 (difference, −0.001; 95% CI, −0.009 to 0.006). The addition of the polygenic risk score increased the C statistic in the MESA cohort from 0.660 to 0.681 (difference, 0.021; 95% CI, −0.0004 to 0.043; Table 4). In both data sets, the findings were similar when the polygenic risk score was dichotomized at various quartiles (Table 4). Similar findings were observed in sex-stratified analyses, analyses that used alternative risk scores comprising smaller numbers of SNPs, and analyses that excluded participants taking lipid-lowering statin medications (eTable 4, eTable 5, and eTable 6 in the Supplement).

Calibration was assessed by comparing expected and actual event rates for CHD models with and without the polygenic risk score. In the ARIC study, the pooled cohort equations model categorized 39.2% of the sample as low risk (predicted 10-year event rate ≤7.5%) and 60.8% as high risk (predicted 10-year event rate >7.5%) (Figure 2). Actual event rates in these groups were 4.4% and 16.7%, respectively. After adding the polygenic risk score, the model categorized similar proportions of individuals as low risk (42.2%) and high risk (57.8%), with similar event rates as well (4.4% and 17.3%). Calibration analyses suggested better calibration in the model without the polygenic risk score (Greenwood-Nam-D’Agostino χ2, P = .85) than with the polygenic risk score (P = .03) (eFigure 1 in the Supplement).

In the MESA trial, the pooled cohort equations model categorized 54.7% of the sample as low risk and 45.3% as high risk (Figure 2) with actual event rates of 3.4% and 13.4%, respectively. After adding the polygenic risk score, the proportion of individuals categorized as low risk was 59.2% and as high risk was 40.8%, with event rates of 3.8% and 14.0%, respectively. Both models showed good calibration (Greenwood-Nam-D’Agostino χ2, P = .39 and P = .93, respectively, for models without and with the polygenic risk score).

Adding the polygenic risk score to the pooled cohort risk categories did not significantly improve classification accuracy in either the ARIC (NRI, 0.018; 95% CI, −0.012 to 0.036) or the MESA cohorts (NRI, 0.001; 95% CI, −0.038 to 0.076; Table 5). The overall proportions of individuals reclassified to a new category were 4.4% in the ARIC study and 6.9% the MESA study. Among those who subsequently developed a CHD event, these reclassifications were often incorrect (80.0% of reclassifications in the ARIC cohort, and 78.6% of reclassifications in the MESA cohort, Figure 2). There was no significant improvement associated with the polygenic risk score in analyses stratified by sex (eTable 7 in the Supplement).

Discussion

Quiz Ref IDA CHD polygenic risk score offered little to no improvement in CHD risk stratification in middle-aged white populations from 2 well-characterized retrospective studies involving adults of white, European ancestry. The score minimally changed risk discrimination and reclassified fewer than 10% of individuals to a higher or lower CHD risk category. Furthermore, among individuals who subsequently developed CHD, the specific group that screening programs wish to identify, the majority of reclassifications (79%-80%) were incorrect. Neither the proportions of individuals categorized as high risk or low risk nor the observed event rates in each group were substantially altered by the polygenic risk score, suggesting that implementation may have limited effect at the population level.

These analyses used the polygenic risk score developed by Khera and colleagues,7 based on 6 million SNPs from an international GWAS and cross-sectionally validated in the UK Biobank, a middle-aged population of largely European ancestry. Similar results were seen with regard to the robust association of the polygenic risk score with prevalent CHD disease risk, with relative risk estimates in a similar range. These data support the overall applicability of their genetic model to these US-based white populations and suggest that its limited predictive utility was not a result of poor model selection.28 This study also found that polygenic risk scores comprising smaller numbers of SNPs strongly associated with CHD risk did not perform better than the 6-million SNP predictor.

Quiz Ref IDThese findings underscore the frequent discordance between statistical association and predictive performance, a phenomenon that has been observed with other cardiovascular biomarkers.4 Odds ratios greater than 10 are typically required for new risk markers to substantially improve model discrimination.29,30 The odds ratio associated with being in the top 5% of polygenic risk score (≈ 4) is comparable with that observed with other biomarkers such as C-reactive protein and homocysteine that have been shown to have similarly modest predictive utility.31,32

Initial studies characterizing highly polygenic CHD predictors using UK Biobank data reported C statistics of approximately 0.80.7,8 However, these estimates were from models that included age and sex, which are the major determinants of CHD risk.33 These analyses highlight the modest performance of the polygenic risk score when considered alone, consistent with the findings of Inouye et al.8 Another distinction between these analyses and the aforementioned studies is the primary focus on incident events rather than prevalent CHD. A few studies have examined prospective outcomes, and they have observed results similar to those reported herein. For instance, the C statistics for an analysis of incident CHD in French-Canadians were 0.56 to 0.60, despite findings of high risk estimates associated with being in the tails of the polygenic risk score distribution.10 Similarly, a study involving approximately 52 000 white people in a Northern California health care system found a C statistic improvement of 0.008 when a polygenic risk score was added to the Framingham risk score.9

A proposed strength of the polygenic risk score is its ability to identify a subgroup of individuals with a relative risk of CHD comparable with individuals with monogenic traits, such as familial hypercholesterolemia. In the UK Biobank, the polygenic risk score identified 8% of the sample with a relative risk of CHD of 3, similar to the risk associated with some familial hypercholesterolemia mutations.34 Enrichment of high-risk individuals in the tails of the distribution is a feature of many biomarkers of modest prognostic utility.35Quiz Ref ID Furthermore, an important distinction between risk stratification using a mendelian variant or a rare genetic variant vs a polygenic risk score is that the former identifies individuals with a specific mechanism of disease that can be targeted. In contrast, the polygenic risk score does not focus on an underlying mechanism, biology, or behavior that can be intervened upon. Interventions promoting general cardiovascular risk modification in individuals with high genetic risk have yielded mixed results to date.36,37 Thus, the clinical value of the polygenic risk score relies principally on its ability to risk stratify individuals, which, in this case, appears limited.

Another potential advantage of the polygenic risk score, compared with conventional risk markers, is that it can be assessed at an early age. Given the poor discriminative performance of a polygenic risk score observed in these analyses, the clinical implications of finding a high polygenic risk score in a young person with very low absolute risk are unclear, in the absence of an identifiable risk factor such as hyperlipidemia. Screening with a polygenic risk score could provide motivation for lifestyle modification (eg, better diet or increased physical activity), but there may be simpler ways to promote such interventions at the individual or population level.

Limitations

This study has several limitations. First, the SNP weights for the polygenic risk score were derived from the CARDIOGRAMplusC4D GWAS. Although 77% of participants in this GWAS were of European ancestry, inclusion of other ancestries could affect SNP weights and attenuate the performance of the polygenic risk score in a European ancestry population. Similarly, CARDIOGRAMplusC4D captured a heterogenous collection of CHD cases, so while SNP weightings derived from this study may be well-suited to the heterogenous mix of cases that would be expected in the community-based cohorts used in these analyses, the weightings are not optimized to capture specific CHD subtypes such as early-onset CHD. Second, the ARIC study was one of the cohorts in CARDIOGRAMplusC4D, which might lead to an overestimation of the performance of the polygenic risk score. The ARIC study also contributed data used for the pooled cohort equations; however, the ARIC events that contributed to the pooled cohort equations largely occurred prior to the visit 4 time point used in this study. The consistency of the results between the ARIC and MESA studies (the latter of which did not contribute to the derivation of the pooled cohort equations) further supports the study findings. Third, the pooled cohort classifier was calibrated to identify individuals at risk of any atherosclerotic cardiovascular event, not just CHD. This might lead to an overestimation of the performance of the polygenic risk score relative to the pooled cohort equations.

Fourth, a 10-year risk threshold of 7.5% was used to assess reclassification, based on the current ACC/AHA cholesterol guidelines. It is possible that the performance of the predictor could vary using other thresholds. Fifth, the analyses were restricted to participants in epidemiological cohorts, who may not be representative of individuals seen in other settings such as hospitals or clinics. Sixth, these analyses were restricted to individuals of European descent because the polygenic risk score was calibrated to a European-ancestry GWAS using a European linkage-disequilibrium reference panel. The inability to assess polygenic prediction in nonwhite individuals underscores the problem of limited diversity in prior GWAS.38

Conclusions

In this analysis of 2 cohorts of US adults, the polygenic risk score was associated with incident coronary heart disease events but did not significantly improve discrimination, calibration, or risk reclassification compared with conventional predictors. These findings suggest that a polygenic risk score may not enhance risk prediction in a general, white middle-aged population.

Back to top
Article Information

Corresponding Author: Jonathan D. Mosley, MD, PhD, Vanderbilt University School of Medicine, 1285 Medical Research Bldg IV, Nashville, TN 37232 (jonathan.d.mosley@vumc.org).

Accepted for Publication: December 16, 2019.

Author Contributions: Drs Mosley and Guo had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Mosley, Wells, Psaty, Roden, Wang.

Acquisition, analysis, or interpretation of data: Mosley, Gupta, Tan, Yao, Wells, Shaffer, Kundu, Robinson-Cohen, Rich, Post, Guo, Rotter, Gerszten, Wang.

Drafting of the manuscript: Mosley, Tan, Wang.

Critical revision of the manuscript for important intellectual content: Gupta, Yao, Wells, Shaffer, Kundu, Robinson-Cohen, Psaty, Rich, Post, Guo, Rotter, Roden, Gerszten.

Statistical analysis: Mosley, Tan, Yao, Shaffer, Kundu, Robinson-Cohen, Guo, Gerszten.

Obtained funding: Rotter, Gerszten.

Administrative, technical, or material support: Gupta, Shaffer, Robinson-Cohen, Rich, Rotter, Roden, Gerszten, Wang.

Supervision: Guo, Rotter, Roden, Gerszten, Wang.

Conflict of Interest Disclosures: Dr Gupta reported receiving grants from National Institutes of Health (NIH). Dr Psaty reported receiving grants from NIH and serving on the steering committee of the Yale Open Data Access Project funded by Johnson & Johnson. Dr Post reported receiving grants from NIH. Dr Rotter reported receiving grants from NIH. Dr Wang reported receiving personal fees from Novartis. No other disclosures were reported.

Funding/Support: This work was supported by grants 16FTF30130005 from the American Heart Association and R01-HL142856 and R01 HL132320 from the National Heart, Lung, and Blood Institute (NHLBI). ARIC is supported by contracts HHSN268201100005C, HHSN268201100006C, HHSN268201100007C, HHSN268201100008C, HHSN268201100009C, HHSN268201100010C, HHSN268201100011C, and HHSN268201100012C from the NHLBI. Funding for GENEVA was provided by grant U01HG004402 from the National Human Genome Research Institute (NHGRI). MESA and the MESA SHARe project are conducted and supported by the NHLBI in collaboration with MESA investigators. Support for MESA is provided by contracts HHSN268201500003I, N01-HC-95159, N01-HC-95160, N01-HC-95161, N01-HC-95162, N01-HC-95163, N01-HC-95164, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, N01-HC-95169, UL1-TR-000040, UL1-TR-001079, and UL1-TR-001420. The provision of genotyping data was supported in part by the National Center for Advancing Translational Sciences, grants UL1TR001881 from the Clinical and Translational Science Institute (CTSI), and DK063491 from the National Institute of Diabetes and Digestive and Kidney Disease Diabetes Research Center (DRC) to the Southern California Diabetes Endocrinology Research Center. Funding for SHARe genotyping was provided by contract N02-HL-64278 from the NHLBI. This genotyping was performed at Affymetrix (Santa Clara, California) and from the Broad Institute of Harvard and Massachusetts Institute of Technology using the Affymetrix Genome-Wide Human SNP Array 6.0.

Role of the Funder/Sponsor: None of the external funders had a role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the staff and participants of the ARIC and MESA studies for their contributions.

References
1.
Benjamin  EJ, Muntner  P, Alonso  A,  et al; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee.  Heart disease and stroke statistics-2019 update: a report from the American Heart Association.   Circulation. 2019;139(10):e56-e528. doi:10.1161/CIR.0000000000000659PubMedGoogle ScholarCrossref
2.
Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines.   Circulation. 2014;129(25)(suppl 2):S49-S73. doi:10.1161/01.cir.0000437741.48606.98PubMedGoogle ScholarCrossref
3.
Muntner  P, Colantonio  LD, Cushman  M,  et al.  Validation of the atherosclerotic cardiovascular disease pooled cohort risk equations.   JAMA. 2014;311(14):1406-1415. doi:10.1001/jama.2014.2630PubMedGoogle ScholarCrossref
4.
Greenland  P, Hassan  S.  Precision preventive medicine-ready for prime time?   JAMA Intern Med. 2019;179(5):605-606. doi:10.1001/jamainternmed.2019.0142PubMedGoogle ScholarCrossref
5.
Torkamani  A, Wineinger  NE, Topol  EJ.  The personal and clinical utility of polygenic risk scores.   Nat Rev Genet. 2018;19(9):581-590. doi:10.1038/s41576-018-0018-xPubMedGoogle ScholarCrossref
6.
Khera  AV, Chaffin  M, Wade  KH,  et al.  Polygenic prediction of weight and obesity trajectories from birth to adulthood.   Cell. 2019;177(3):587-596.e9. doi:10.1016/j.cell.2019.03.028PubMedGoogle ScholarCrossref
7.
Khera  AV, Chaffin  M, Aragam  KG,  et al.  Genome-wide polygenic scores for common diseases identify individuals with risk equivalent to monogenic mutations.   Nat Genet. 2018;50(9):1219-1224. doi:10.1038/s41588-018-0183-zPubMedGoogle ScholarCrossref
8.
Inouye  M, Abraham  G, Nelson  CP,  et al; UK Biobank CardioMetabolic Consortium CHD Working Group.  Genomic risk prediction of coronary artery disease in 480,000 adults: implications for primary prevention.   J Am Coll Cardiol. 2018;72(16):1883-1893. doi:10.1016/j.jacc.2018.07.079PubMedGoogle ScholarCrossref
9.
Iribarren  C, Lu  M, Jorgenson  E,  et al.  Clinical utility of multimarker genetic risk scores for prediction of incident coronary heart disease: a cohort study among over 51 000 individuals of European ancestry.   Circ Cardiovasc Genet. 2016;9(6):531-540. doi:10.1161/CIRCGENETICS.116.001522PubMedGoogle ScholarCrossref
10.
Wünnemann  F, Sin Lo  K, Langford-Avelar  A,  et al.  Validation of genome-wide polygenic risk scores for coronary artery disease in French Canadians.   Circ Genom Precis Med. 2019;12(6):e002481. doi:10.1161/CIRCGEN.119.002481PubMedGoogle Scholar
11.
Knowles  JW, Ashley  EA.  Cardiovascular disease: the rise of the genetic risk score.   PLoS Med. 2018;15(3):e1002546. doi:10.1371/journal.pmed.1002546PubMedGoogle Scholar
12.
Warren  M.  The approach to predictive medicine that is taking genomics research by storm.   Nature. 2018;562(7726):181-183. doi:10.1038/d41586-018-06956-3PubMedGoogle ScholarCrossref
13.
The ARIC investigators.  The Atherosclerosis Risk in Communities (ARIC) study: design and objectives. The ARIC investigators.   Am J Epidemiol. 1989;129(4):687-702. doi:10.1093/oxfordjournals.aje.a115184PubMedGoogle ScholarCrossref
14.
Bild  DE, Bluemke  DA, Burke  GL,  et al.  Multi-Ethnic Study of Atherosclerosis: objectives and design.   Am J Epidemiol. 2002;156(9):871-881. doi:10.1093/aje/kwf113PubMedGoogle ScholarCrossref
15.
Nikpay  M, Goel  A, Won  H-H,  et al.  A comprehensive 1,000 Genomes-based genome-wide association meta-analysis of coronary artery disease.   Nat Genet. 2015;47(10):1121-1130. doi:10.1038/ng.3396PubMedGoogle ScholarCrossref
16.
Pritchard  JK, Stephens  M, Donnelly  P.  Inference of population structure using multilocus genotype data.   Genetics. 2000;155(2):945-959.PubMedGoogle Scholar
17.
Mosley  JD, van Driest  SL, Wells  QS,  et al.  Defining a contemporary ischemic heart disease genetic risk profile using historical data.   Circ Cardiovasc Genet. 2016;9(6):521-530. doi:10.1161/CIRCGENETICS.116.001530PubMedGoogle ScholarCrossref
18.
Purcell  S, Neale  B, Todd-Brown  K,  et al.  PLINK: a tool set for whole-genome association and population-based linkage analyses.   Am J Hum Genet. 2007;81(3):559-575. doi:10.1086/519795PubMedGoogle ScholarCrossref
19.
Patterson  N, Price  AL, Reich  D.  Population structure and eigenanalysis.   PLoS Genet. 2006;2(12):e190. doi:10.1371/journal.pgen.0020190PubMedGoogle Scholar
20.
Zheng  X, Levine  D, Shen  J, Gogarten  SM, Laurie  C, Weir  BS.  A high-performance computing toolset for relatedness and principal component analysis of SNP data.   Bioinformatics. 2012;28(24):3326-3328. doi:10.1093/bioinformatics/bts606PubMedGoogle ScholarCrossref
21.
Vilhjálmsson  BJ, Yang  J, Finucane  HK,  et al; Schizophrenia Working Group of the Psychiatric Genomics Consortium, Discovery, Biology, and Risk of Inherited Variants in Breast Cancer (DRIVE) study.  Modeling linkage disequilibrium increases accuracy of polygenic risk scores.   Am J Hum Genet. 2015;97(4):576-592. doi:10.1016/j.ajhg.2015.09.001PubMedGoogle ScholarCrossref
22.
Harrell  FE  Jr, Califf  RM, Pryor  DB, Lee  KL, Rosati  RA.  Evaluating the yield of medical tests.   JAMA. 1982;247(18):2543-2546. doi:10.1001/jama.1982.03320430047030PubMedGoogle ScholarCrossref
23.
Demler  OV, Paynter  NP, Cook  NR.  Tests of calibration and goodness-of-fit in the survival setting.   Stat Med. 2015;34(10):1659-1680. doi:10.1002/sim.6428PubMedGoogle ScholarCrossref
24.
Pencina  MJ, D’Agostino  RB  Sr, D’Agostino  RB  Jr, Vasan  RS.  Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond.   Stat Med. 2008;27(2):157-172. doi:10.1002/sim.2929PubMedGoogle ScholarCrossref
25.
Kerr  KF, Wang  Z, Janes  H, McClelland  RL, Psaty  BM, Pepe  MS.  Net reclassification indices for evaluating risk prediction instruments: a critical review.   Epidemiology. 2014;25(1):114-121. doi:10.1097/EDE.0000000000000018PubMedGoogle ScholarCrossref
26.
Baker  SG, Schuit  E, Steyerberg  EW,  et al.  How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through.   Stat Med. 2014;33(22):3946-3959. doi:10.1002/sim.6195PubMedGoogle ScholarCrossref
27.
Hajek  C, Guo  X, Yao  J,  et al.  Coronary heart disease genetic risk score predicts cardiovascular disease risk in men, not women.   Circ Genom Precis Med. 2018;11(10):e002324. doi:10.1161/CIRCGEN.118.002324PubMedGoogle Scholar
28.
Janes  H, Pepe  MS, Gu  W.  Assessing the value of risk predictions by using risk stratification tables.   Ann Intern Med. 2008;149(10):751-760. doi:10.7326/0003-4819-149-10-200811180-00009PubMedGoogle ScholarCrossref
29.
Wald  NJ, Old  R.  The illusion of polygenic disease risk prediction.   Genet Med. 2019;21(8):1705-1707. doi:10.1038/s41436-018-0418-5PubMedGoogle ScholarCrossref
30.
Pepe  MS, Janes  H, Longton  G, Leisenring  W, Newcomb  P.  Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker.   Am J Epidemiol. 2004;159(9):882-890. doi:10.1093/aje/kwh101PubMedGoogle ScholarCrossref
31.
Homocysteine Studies Collaboration.  Homocysteine and risk of ischemic heart disease and stroke: a meta-analysis.   JAMA. 2002;288(16):2015-2022. doi:10.1001/jama.288.16.2015PubMedGoogle ScholarCrossref
32.
Kaptoge  S, Di Angelantonio  E, Pennells  L,  et al; Emerging Risk Factors Collaboration.  C-reactive protein, fibrinogen, and cardiovascular disease prediction.   N Engl J Med. 2012;367(14):1310-1320. doi:10.1056/NEJMoa1107477PubMedGoogle ScholarCrossref
33.
Karmali  KN, Goff  DC  Jr, Ning  H, Lloyd-Jones  DM.  A systematic examination of the 2013 ACC/AHA pooled cohort risk assessment tool for atherosclerotic cardiovascular disease.   J Am Coll Cardiol. 2014;64(10):959-968. doi:10.1016/j.jacc.2014.06.1186PubMedGoogle ScholarCrossref
34.
Perak  AM, Ning  H, de Ferranti  SD, Gooding  HC, Wilkins  JT, Lloyd-Jones  DM.  Long-term risk of atherosclerotic cardiovascular disease in US adults with the familial hypercholesterolemia phenotype.   Circulation. 2016;134(1):9-19. doi:10.1161/CIRCULATIONAHA.116.022335PubMedGoogle ScholarCrossref
35.
Wang  TJ, Gona  P, Larson  MG,  et al.  Multiple biomarkers for the prediction of first major cardiovascular events and death.   N Engl J Med. 2006;355(25):2631-2639. doi:10.1056/NEJMoa055373PubMedGoogle ScholarCrossref
36.
Knowles  JW, Zarafshar  S, Pavlovic  A,  et al.  Impact of a genetic risk score for coronary artery disease on reducing cardiovascular risk: a pilot randomized controlled study.   Front Cardiovasc Med. 2017;4:53. doi:10.3389/fcvm.2017.00053PubMedGoogle ScholarCrossref
37.
Kullo  IJ, Jouni  H, Austin  EE,  et al.  Incorporating a genetic risk score into coronary heart disease risk estimates: effect on low-density lipoprotein cholesterol levels (the MI-GENES Clinical Trial).   Circulation. 2016;133(12):1181-1188. doi:10.1161/CIRCULATIONAHA.115.020109PubMedGoogle ScholarCrossref
38.
De La Vega  FM, Bustamante  CD.  Polygenic risk scores: a biased prediction?   Genome Med. 2018;10(1):100. doi:10.1186/s13073-018-0610-xPubMedGoogle ScholarCrossref
×