[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.159.189.139. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1.
Sample and Statistical Process for Evaluation of the 9-Protein Model
Sample and Statistical Process for Evaluation of the 9-Protein Model

aSamples were sourced from the Heart and Soul study and the HUNT3 study.

bProteins (n = 76) and samples (n = 73) that failed standard interrun and intrarun assay quality control acceptance metrics (section 1 of the Supplement) were deemed unfit for analysis.

Figure 2.
Event-Free Survival for End Points of Myocardial Infarction, Stroke, Heart Failure, and Death, Stratified by Deciles of the 9-Protein 4-Year Risk Score
Event-Free Survival for End Points of Myocardial Infarction, Stroke, Heart Failure, and Death, Stratified by Deciles of the 9-Protein 4-Year Risk Score

The shading in the survival plots indicates 95% CI for the first and 10th deciles. Decile 1 indicates the lowest score; decile 10 indicates the highest score. Data defining the deciles of risk score are presented in Figure 3.

Figure 3.
Agreement Between Observed vs Predicted 4-Year Incidence of Myocardial Infarction, Stroke, Heart Failure, and Death With the 9-Protein Model
Agreement Between Observed vs Predicted 4-Year Incidence of Myocardial Infarction, Stroke, Heart Failure, and Death With the 9-Protein Model

Predicted 4-year incidence for each decile is the mean predicted risk score in each decile. Error bars indicate standard deviations. For additional details, see section 4 in the Supplement.

Figure 4.
Changes in Risk Scores of Myocardial Infarction, Stroke, Heart Failure, and Death in Paired Samples 4.8 Years Apart
Changes in Risk Scores of Myocardial Infarction, Stroke, Heart Failure, and Death in Paired Samples 4.8 Years Apart

A, Predicted cardiovascular risk in paired samples among 139 participants who experienced an event after the second sample (left), and 375 participants who were event free during the entire study (right). Both panels show 4-year risk predictions at baseline and follow-up for the 9-protein and refit Framingham models.

B, Annualized within-person change in 9-protein risk was greater than for refit Framingham for participants who experienced events (P = .002) and similar to change in refit Framingham for participants who were event free (P = .30). Median absolute annualized within-person change in the 9-protein risk score was 1.86% compared with 1.00% for refit Framingham.

Key to symbols: horizontal line indicates the median, top and bottom ends of the boxes indicate the interquartile range (IQR), upper and lower error bars extend to 1.5 × the IQR, and the circles indicate data points beyond 1.5 × the IQR.

Table 1.  
Baseline Characteristics of the Study Cohorts
Baseline Characteristics of the Study Cohorts
Table 2.  
Risk Prediction Models for Primary End Point of Myocardial Infarction, Stroke, Heart Failure, and Deatha
Risk Prediction Models for Primary End Point of Myocardial Infarction, Stroke, Heart Failure, and Deatha
Table 3.  
Comparative Performance Metrics in Derivation and Validation Cohorts for Refit Framingham Model, 9-Protein Model, and Their Combination When Predicting Primary End Points of Myocardial Infarction, Stroke, Heart Failure, and Death
Comparative Performance Metrics in Derivation and Validation Cohorts for Refit Framingham Model, 9-Protein Model, and Their Combination When Predicting Primary End Points of Myocardial Infarction, Stroke, Heart Failure, and Death
1.
Go  AS, Mozaffarian  D, Roger  VL,  et al; American Heart Association Statistics Committee and Stroke Statistics Subcommittee.  Heart disease and stroke statistics—2014 update: a report from the American Heart Association. Circulation. 2014;129(3):e28-e292.
PubMedArticle
2.
Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25)(suppl 2):S49-S73.
PubMedArticle
3.
Ridker  PM, Cook  NR.  Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762-1765.
PubMedArticle
4.
Mega  JL, Stitziel  NO, Smith  JG,  et al.  Genetic risk, coronary heart disease events, and the clinical benefit of statin therapy: an analysis of primary and secondary prevention trials. Lancet. 2015;385(9984):2264-2271.
PubMedArticle
5.
Paynter  NP, Chasman  DI, Paré  G,  et al.  Association between a literature-based genetic risk score and cardiovascular events in women. JAMA. 2010;303(7):631-637.
PubMedArticle
6.
Lindsey  ML, Mayr  M, Gomes  AV,  et al; American Heart Association Council on Functional Genomics and Translational Biology, Council on Cardiovascular Disease in the Young, Council on Clinical Cardiology, Council on Cardiovascular and Stroke Nursing, Council on Hypertension, and Stroke Council.  Transformative impact of proteomics on cardiovascular health and disease: a scientific statement from the American Heart Association. Circulation. 2015;132(9):852-872.
PubMedArticle
7.
Gold  L, Ayers  D, Bertino  J,  et al.  Aptamer-based multiplexed proteomic technology for biomarker discovery. PLoS One. 2010;5(12):e15004.
PubMedArticle
8.
Steyerberg  EW, Vickers  AJ, Cook  NR,  et al.  Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138.
PubMedArticle
9.
McShane  LM, Cavenagh  MM, Lively  TG,  et al.  Criteria for the use of omics-based predictors in clinical trials. Nature. 2013;502(7471):317-320.
PubMedArticle
10.
Beatty  AL, Ku  IA, Bibbins-Domingo  K,  et al.  Traditional Risk Factors Versus Biomarkers for Prediction of Secondary Events in Patients With Stable Coronary Heart Disease: From the Heart and Soul Study. J Am Heart Assoc. 2015;4(7):e001646.
PubMedArticle
11.
Eagle  KA, Ginsburg  GS, Musunuru  K,  et al.  Identifying patients at high risk of a cardiovascular event in the near future: current status and future directions: report of a national heart, lung, and blood institute working group. Circulation. 2010;121(12):1447-1454.
PubMedArticle
12.
Krokstad  S, Langhammer  A, Hveem  K,  et al.  Cohort profile: the HUNT study, Norway. Int J Epidemiol. 2013;42(4):968-977.
PubMedArticle
13.
Marcus  GM, Olgin  JE, Whooley  M,  et al.  Racial differences in atrial fibrillation prevalence and left atrial size. Am J Med. 2010;123(4):375 e1-375 e7.
PubMedArticle
14.
Olson  KA, Beatty  AL, Heidecker  B,  et al.  Association of growth differentiation factor 11/8, putative anti-ageing factor, with cardiovascular outcomes and overall mortality in humans: analysis of the Heart and Soul and HUNT3 cohorts. Eur Heart J. 2015;36(48):3426-3434.
PubMedArticle
15.
Vaught  JD, Bock  C, Carter  J,  et al.  Expanding the chemistry of DNA for in vitro selection. J Am Chem Soc. 2010;132(12):4141-4151.
PubMedArticle
16.
Brody  E, Gold  L, Mehan  M,  et al.  Life’s simple measures: unlocking the proteome. J Mol Biol. 2012;422(5):595-606.
PubMedArticle
17.
Tibshirani  R.  The lasso method for variable selection in the Cox model. Stat Med. 1997;16(4):385-395.
PubMedArticle
18.
McClelland  RL, Jorgensen  NW, Budoff  M,  et al.  10-Year coronary heart disease risk prediction using coronary artery calcium and traditional risk factors: derivation in the MESA (Multi-Ethnic Study of Atherosclerosis) with validation in the HNR (Heinz Nixdorf Recall) study and the DHS (Dallas Heart Study). J Am Coll Cardiol. 2015;66(15):1643-1653.
PubMedArticle
19.
Tibshirani  R.  Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc B. 2011;73(3):273-282. doi:10.1111/j.1467-9868.2011.00771.xArticle
20.
D’Agostino  RB, Russell  MW, Huse  DM,  et al.  Primary and subsequent coronary risk appraisal: new results from the Framingham study. Am Heart J. 2000;139(2 Pt 1):272-281.
PubMedArticle
21.
Heagerty  PJ, Zheng  Y.  Survival model predictive accuracy and ROC curves. Biometrics. 2005;61(1):92-105.
PubMedArticle
22.
Pencina  MJ, D’Agostino  RB  Sr, Steyerberg  EW.  Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11-21.
PubMedArticle
23.
Pencina  MJ, D’Agostino  RB, Pencina  KM, Janssens  AC, Greenland  P.  Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176(6):473-481.
PubMedArticle
24.
Pepe  MS, Janes  H. Methods for evaluating prediction performance of biomarkers and tests, UW Biostatistics Working Paper Series (384). http://biostats.bepress.com/uwbiostat/paper384. Accessed June, 5, 2016.
25.
R Core Team RFfSC. Austria R: A Language and Environment for Statistical Computing. Vienna, Austria: The R Foundation; 2013.
26.
Ripatti  S, Tikkanen  E, Orho-Melander  M,  et al.  A multilocus genetic risk score for coronary heart disease: case-control and prospective cohort analyses. Lancet. 2010;376(9750):1393-1400.
PubMedArticle
27.
Ridker  PM, Danielson  E, Fonseca  FA,  et al; JUPITER Trial Study Group.  Reduction in C-reactive protein and LDL cholesterol and cardiovascular event rates after initiation of rosuvastatin: a prospective study of the JUPITER trial. Lancet. 2009;373(9670):1175-1182.
PubMedArticle
28.
Nissen  SE, Tuzcu  EM, Schoenhagen  P,  et al; Reversal of Atherosclerosis with Aggressive Lipid Lowering (REVERSAL) Investigators.  Statin therapy, LDL cholesterol, C-reactive protein, and coronary artery disease. N Engl J Med. 2005;352(1):29-38.
PubMedArticle
29.
Dadu  RT, Ballantyne  CM.  Lipid lowering with PCSK9 inhibitors. Nat Rev Cardiol. 2014;11(10):563-575.
PubMedArticle
30.
Kraemer  S, Vaught  JD, Bock  C,  et al.  From SOMAmer-based biomarker discovery to diagnostic and clinical applications: a SOMAmer-based, streamlined multiplex proteomic assay. PLoS One. 2011;6(10):e26332.
PubMedArticle
31.
Mehan  MR, Ostroff  R, Wilcox  SK,  et al.  Highly multiplexed proteomic platform for biomarker discovery, diagnostics, and therapeutics. Adv Exp Med Biol. 2013;735:283-300.
PubMed
32.
Pencina  MJ, D’Agostino  RB  Sr, Song  L.  Quantifying discrimination of Framingham risk functions with different survival C statistics. Stat Med. 2012;31(15):1543-1553.
PubMedArticle
33.
Wilson  PW, D'Agostino  R  Sr, ., Bhatt  DL,  et al.  An international model to predict recurrent cardiovascular disease. Am J Med. 2012;125(7):695-703 e1.
PubMedArticle
34.
Whooley  MA, de Jonge  P, Vittinghoff  E,  et al.  Depressive symptoms, health behaviors, and risk of cardiovascular events in patients with coronary heart disease. JAMA. 2008;300(20):2379-2388.
PubMedArticle
Original Investigation
Innovations in Health Care Delivery
June 21, 2016

Development and Validation of a Protein-Based Risk Score for Cardiovascular Outcomes Among Patients With Stable Coronary Heart Disease

Author Affiliations
  • 1Department of Medicine, University of California-San Francisco
  • 2Division of Cardiology, San Francisco General Hospital
  • 3Division of Cardiology, University of Zurich, Zurich, Switzerland
  • 4HUNT Research Center, Department of Public Health, NTNU, Levanger, Norway
  • 5NEC Corporation of America, Irving, Texas
  • 6Department of Epidemiology and Biostatistics, University of California-San Francisco
  • 7SomaLogic, Boulder, Colorado
JAMA. 2016;315(23):2532-2541. doi:10.1001/jama.2016.5951
Abstract

Importance  Precise stratification of cardiovascular risk in patients with coronary heart disease (CHD) is needed to inform treatment decisions.

Objective  To derive and validate a score to predict risk of cardiovascular outcomes among patients with CHD, using large-scale analysis of circulating proteins.

Design, Setting, and Participants  Prospective cohort study of participants with stable CHD. For the derivation cohort (Heart and Soul study), outpatients from San Francisco were enrolled from 2000 through 2002 and followed up through November 2011 (≤11.1 years). For the validation cohort (HUNT3, a Norwegian population-based study), participants were enrolled from 2006 through 2008 and followed up through April 2012 (5.6 years).

Exposures  Using modified aptamers, 1130 proteins were measured in plasma samples.

Main Outcomes and Measures  A 9-protein risk score was derived and validated for 4-year probability of myocardial infarction, stroke, heart failure, and all-cause death. Tests, including the C statistic, were used to assess performance of the 9-protein risk score, which was compared with the Framingham secondary event model, refit to the cohorts in this study. Within-person change in the 9-protein risk score was evaluated in the Heart and Soul study from paired samples collected 4.8 years apart.

Results  From the derivation cohort, 938 samples were analyzed, participants’ median age at enrollment was 67.0 years, and 82% were men. From the validation cohort, 971 samples were analyzed, participants’ median age at enrollment was 70.2 years, and 72% were men. In the derivation cohort, C statistics were 0.66 for refit Framingham, 0.74 for 9-protein, and 0.75 for refit Framingham plus 9-protein models. In the validation cohort, C statistics were 0.64 for refit Framingham, 0.70 for 9-protein, and 0.71 for refit Framingham plus 9-protein models. Adding the 9-protein risk score to the refit Framingham model increased the C statistic by 0.09 (95% CI, 0.06-0.12) in the derivation cohort, and in the validation cohort, the C statistic was increased by 0.05 (95% CI, 0.02-0.09). Compared with the refit Framingham model, the integrated discrimination index for the 9-protein model was 0.12 (95% CI, 0.08-0.16) in the derivation cohort and 0.08 (95% CI, 0.05-0.10) in the validation cohort. In analysis of paired samples among 139 participants with cardiovascular events after the second sample, absolute within-person annualized risk increased more for the 9-protein model (median, 1.86% [95% CI, 1.15%-2.54%]) than for the refit Framingham model (median, 1.00% [95% CI, 0.87%-1.19%]) (P = .002), while among 375 participants without cardiovascular events, both scores changed less and similarly (P = .30).

Conclusions and Relevance  Among patients with stable CHD, a risk score based on 9 proteins performed better than the refit Framingham secondary event risk score in predicting cardiovascular events, but still provided only modest discriminative accuracy. Further research is needed to assess whether the score is more accurate in a lower-risk population.

Introduction

Coronary heart disease (CHD) remains a leading cause of mortality and morbidity.1 Despite the importance of risk assessment,2 considerable room for improvement remains.3 Genetic risk factors4,5 and candidate proteins, such as C-reactive protein, have delivered only modest advances2 and do not adequately enable precision medicine—management based on accurately stratified personal phenotyping.

A recent scientific statement from the American Heart Association predicted that proteomics will be transformative,6 but the proteomic characterization of cardiovascular risk phenotypes in large populations requires a high-throughput technology. In this study, such a technology was applied, based on modified aptamers as binding reagents,7 to quantify 1130 proteins in 2 prospective cohorts of participants with stable CHD. The objectives of this study were the following: (1) to evaluate a broader range of prognostic plasma protein biomarkers than previously possible; (2) to create a multiprotein model of biomarkers for prognostic stratification; (3) to validate the performance of the model in an external cohort8; (4) to assess the robustness of this model and key prognostic proteins within it to typical variations in sample collection and processing9; (5) to determine whether inclusion of this multiprotein panel in a risk score composed of traditional risk factors improves risk prediction; and (6) to determine from analysis of paired samples collected nearly 5 years apart whether the interval change in multiprotein panel risk score is greater among participants who experience a cardiovascular event after the second sample than among participants who do not. This study focused on participants with stable CHD because they have a broad range of risk that is not adequately identified by traditional risk factors.10,11

Methods
Study Populations

Studies in both cohorts were approved by the appropriate institutional review boards, and all participants provided written informed consent. The derivation cohort consisted of 938 baseline plasma samples from the Heart and Soul study—a prospective cohort of patients with stable CHD from 12 clinics in the San Francisco Bay Area (enrollment, September 2000-December 2002; last follow-up, November 2011). The Heart and Soul study included participants with history of myocardial infarction (MI), angiographic evidence of at least 50% stenosis in 1 or more coronary vessels, prior evidence of inducible ischemia by stress testing, or history of coronary revascularization. Participants were excluded if they had an MI within the previous 6 months, were unable to walk 1 block, or were planning to relocate from the local area within 2 years. From this cohort, a prognostic 9-protein model was constructed and then validated on 971 samples from HUNT3, a prospective population-based cohort study from Nord-Trøndelag County in Norway (enrollment, 2006-2008; last follow-up, April 2012).12 HUNT3 participants were included who met Heart and Soul study inclusion criteria and had not had an MI within the previous 6 months. In the Heart and Soul study, race was self-identified in a questionnaire with categories of white, black, Asian, Latino, or other.13 HUNT3 was a racially homogeneous cohort (≥98% white).14 The information about race was used to discern whether the racial composition of the subset of participants with paired samples was similar to that of the overall Heart and Soul population in this study.

In contrast to the more standardized sample collection in the derivation cohort (fasted samples were collected at the same time of day and centrifuged and frozen within 1 hour of collection), sample collection in the validation cohort was more representative of likely clinical practice conditions: participants did not fast, and samples were collected at random times of day and processed (≤24 hours) after blood draw.

Changes in the 9-protein risk score were assessed by using paired samples from 514 participants in the Heart and Soul study in whom second plasma samples were taken a median 4.8 years after the first; participants had no cardiovascular events between these 2 samples. The study evaluated whether the second 9-protein risk score or the change from the baseline risk score could help to differentiate those participants who had a cardiovascular event after the second sample from those who did not. A flowchart of the sample and statistical process is shown in Figure 1 and explained further in section 1 of the Supplement.

Quantification of Proteins in Human Plasma by Modified Aptamers

The method of quantification of proteins by modified aptamers has been previously described.7,15,16 In brief, each of the 1130 individual proteins measured (eTable 1 in the Supplement) has its own binding reagent made of chemically modified DNA, referred to as modified aptamer.7 Each sample of plasma was incubated with the mixture of modified aptamers to generate modified aptamer-protein complexes. Unbound modified aptamers and unbound or nonspecifically bound proteins were eliminated by 2 bead-based immobilization steps. After eluting the modified aptamers from the target protein, the fluorescently labeled modified aptamers were directly quantified on an Agilent hybridization array (Agilent Technologies). Calibrators were included so that the degree of fluorescence was a quantitative reflection of protein concentration. The 1054 proteins that passed quality control (eTable 1 in the Supplement) had median intraassay and interassay coefficient of variation of less than 5%. The key data processing steps, statistical modeling, and specific assessments are summarized in Figure 1.

Statistical Methods

The primary outcome in this study was defined as the first event among MI, stroke/transient ischemic attack (referred to as stroke), heart failure hospitalization, or all-cause death. Cox proportional hazards models were used to estimate the association between levels of individual proteins and risk of primary outcome. In single-variable analysis of an association of individual proteins with the primary outcome, Bonferroni-corrected significance levels were reported, adjusting for 1054 comparisons, resulting in a nominal significance level (P < 4.74 × 10−5). All other statistical tests were 2-sided using a nominal 5% significance level (P < .05). To construct the multiprotein risk model for primary outcome, the least absolute shrinkage and selection operator17 (LASSO) was used for variable (protein) selection with the Cox model. This method penalized the sum of the absolute values of the regression coefficients leading to some coefficients shrinking to zero and thus simultaneously performed variable selection.1719 LASSO regularization level was chosen by cross-validation using the 1 standard error rule (section 3 in the Supplement). LASSO was used for variable selection only, with the fully parametric (Weibull) survival model as the final prognostic model. Stepwise backward elimination, starting from the set of LASSO-selected proteins, was used to remove proteins that were not significant predictors in the absence of the constraint imposed by the LASSO penalty using the Bayesian information criterion stopping criteria.

As a comparative reference for the multiprotein risk model, the variables from the Framingham secondary event risk model20 were refit to the Heart and Soul derivation cohort (referred to as refit Framingham). This model included age, sex, total cholesterol, high-density lipoprotein cholesterol (HDL-C), diabetes, systolic blood pressure, and current smoking status.20 The 4-year time horizon was retained, for which this risk score was originally validated.20

Model performance within each cohort was assessed by discrimination and calibration. For discrimination, both the C statistic21 and discrimination slope8 are reported. The category-free net reclassification index (NRI>0)22 and integrated discrimination index (IDI)8 were used to assess reclassification performance and improvement in discrimination over the refit Framingham model. Calibration performance was assessed with a calibration plot and summarized across the full range of risk scores using the Hosmer-Lemeshow statistic. Calibration-in-the-large is also reported—the difference between the observed 4-year event frequency and the mean predicted risk score. Both the refit Framingham and protein models were recalibrated (Section 4, eTables 2 and 3; eFigures 1 and 2 in the Supplement) for use in the validation cohort to enable an equal comparison and reduce the effect of miscalibration.23,24 Distribution-free (nonparametric) 95% CIs were reported for median values and bootstrap intervals for point estimates of performance metrics when asymptotic intervals were not available.

Changes in risk score in paired samples were assessed using the Wilcoxon rank sum test comparing the within-person change for patients with and without events after their second blood sample. Within-person risk score differences were expressed, relative to the elapsed time between the 2 blood collections, and annualized. A likelihood ratio test was used to compare the fit of the augmented model and combining within-person change with the baseline proteomic risk score. All statistical computing was performed using the R Language for Statistical Computing (version 3.2.1).25

Results
Population Characteristics

The characteristics of the derivation and external validation cohorts are summarized in Table 1. There were fewer events in the validation cohort, primarily because of shorter follow-up.

Proteins Prognostic of Outcomes

At a Bonferroni significance level of 5%, corrected for 1054 comparisons, 200 proteins were associated with the primary outcome (145 positively and 55 negatively). The hazard ratios (HRs) and levels of statistical significance for these 200 prognostic proteins are listed in eTable 4 in the Supplement. In the construction of the risk model, the LASSO process selected 16 prognostic proteins, for which biological functions are listed in section 5.1 of the Supplement and HRs in the derivation and validation cohorts are shown in eFigure 3 in the Supplement. Stepwise backward elimination reduced these to the subset of 9 proteins used in the final prognostic model. The 9 proteins and their HRs are angiopoietin-2 (ANGPT2) (HR, 1.67 [95% CI,1.53-1.82]; P < 1.00 × 10−16), matrix metalloproteinase-12 (MMP12) (HR, 1.65 [95% CI, 1.50-1.80]; P < 1.00 × 10−16), chemokine (C-C motif) ligand 18 (CCL18) (HR, 1.47 [95% CI, 1.34-1.61]; P = 1.11 × 10−16), complement 7 (C7) (HR, 1.47 [95% CI, 1.36-1.59]; P < 1.00 × 10−16), α1-antichymotrypsin complex (SERPINA3) (HR, 1.39 [95% CI, 1.28-1.51]; P = 1.97 × 10−14), angiopoietin-related protein 4 (ANGPTL4) (HR, 1.27 [95% CI, 1.18-1.37]; P = 4.95 × 10−11), troponin I (TNNI3) (HR, 1.27 [95% CI, 1.19-1.35]; P = 1.02 × 10−12), growth differentiation factor 11/8 (GDF8/11) (HR, 0.72 [95% CI, 0.57-0.69]; P = 8.79E × 109), and α2-antiplasmin (SERPINF2) (HR, 0.64 [95% CI, 0.59-0.71]; P < 1.00 × 10−16).

9-Protein Risk Score

The 9-protein risk score reflects the probability of a cardiovascular event occurring within 4-years and is given by risk score (Supplement, section 5.2):

Image description not available.where the prognostic index (PI) combines the measurements of the 9 proteins as follows:

prognostic index = 16.61 − 1.55 × ANGPT2 + 1.22 × GDF8/11 − 2.12 × C7 + 2.64 × SERPINF2 − 0.57 × CCL18 −1.02 × ANGPTL4 − 1.43 × SERPINA3 − 0.72 × MMP12 − 0.59 × TNN13.Table 2 provides the estimated HRs and associated model coefficients for a Cox proportional hazards model based on the refit Framingham variables for the full duration of follow-up, with and without the addition of prognostic index from the 9-protein model. In the presence of the information from 9 proteins, most clinical variables remained as significant risk predictors except for HDL-C. Systolic blood pressure was not a significant risk predictor either in the refit Framingham model or with the addition of the 9 proteins. Adjusting the 9-protein prognostic index for the Framingham variables reduced its HR only modestly (eFigure 4 in the Supplement), suggesting that the 9 proteins contained prognostic information that was at least partly independent of traditional risk factors.

Proteomic Model Performance

Risk stratified survival curves of the 2 study populations are shown in Figure 2, illustrating that in both the derivation and validation cohorts, the participants had 4-year cumulative event rates of 60% to 80% in the 10th deciles and less than 10% in the first deciles. Discrimination performance was assessed using the 4-year time horizon, the same as the original Framingham secondary event model.20Table 3 lists the performance metrics for the refit Framingham model, the 9-protein model, and for the combination of both models. In the derivation cohort, the C statistic increased from 0.66 for the refit Framingham model to 0.74 (Δ C statistic, 0.09 [95% CI, 0.06-0.12]) for the 9-protein model alone and to 0.75 (Δ C statistic, 0.10 [95% CI, 0.08-0.12]) for the 9-protein model combined with the refit Framingham model. The discrimination slope was 0.09 (95% CI, 0.07-0.11) for the refit Framingham model, 0.21 (95% CI, 0.17-0.24) for the 9-protein model, and 0.23 (95% CI, 0.19-0.26) for the refit Framingham combined with the 9-protein model. When compared with refit Framingham, the 9-protein model had an IDI of 0.12 (95% CI, 0.08-0.16), which indicates an absolute increase of 12% in mean risk for participants with events compared with participants without events over the clinical variable model. The 9-protein model had an NRI(>0) of 0.52 (95% CI, 0.40-0.65), with event-specific components of 0.22 (95% CI, 0.11-0.36) and no event-specific components of 0.30 (95% CI, 0.22-0.36). In the validation cohort, inclusion of the 9-protein score with the refit Framingham model generated an NRI(>0) of 0.48 (95% CI, 0.33-0.62) (Table 3). The mean 4-year risk proteomic risk was within 2 percentage points of the observed event rate in the external validation cohort (calibration-in-the-large). Calibration performance across the full range of the 9-protein risk scores is shown in Figure 3 (eFigure 2 [for refit Framingham model] in the Supplement); for the 9-protein model, the observed risk in each decile of the validation cohort was within 5 percentage points of the mean protein risk score. The 9-protein model was developed for the composite end points of MI, heart failure, stroke, and death. For individual end points, median 9-protein risk score in derivation for MI was 33% (95% CI, 25.6%-38.6%); for heart failure, 37% (95% CI, 31.5%-43.7%); for stroke, 24% (95% CI, 19.6%-29.7%); and for death, 30% (95% CI, 27.0%-34.0%). In the absence of any event, the median 4-year 9-protein risk score was 14.2% (95% CI, 13.5%-15.2%). Similar risk score distributions across these event types were observed in the validation cohort (eFigure 5 in the Supplement).

Analysis of Paired Samples

Changes in the 9-protein risk score were evaluated from paired samples from 514 participants (Heart and Soul study) in whom second plasma samples were taken a median of 4.8 years after the first, and participants were event-free between these 2 samples. The baseline characteristics of this subset of participants were similar to all Heart and Soul participants in this study (Table 1) except the time to the first event was longer because of the requisite absence of events prior to the second sample.

Among the participants with paired samples, 139 had an event (MI, heart failure, stroke, or death) after the second sample; the paired samples were taken a median of 2.8 years and 7.7 years prior to that event. The remaining 375 participants had paired samples a median of 4.3 and 9.0 years prior to completing their event-free follow-up. This analysis assessed whether the 9-protein risk score changed to a greater extent for participants approaching an event compared with participants who remained event free.

As Figure 4 shows, 139 participants who experienced an event after the second sample had a median 9-protein risk of 24.6% (95% CI, 22.6%-27.7%) at baseline and 34.0% (95% CI, 29.2%-38.4%) at 4.8 years while median refit Framingham risk was 28.7% (95% CI, 26.6%-30.3%) at baseline and 33.8% (95% CI, 32.5%-36.2%) at 4.8 years. The 375 participants who were event free during the entire study had a median 9-protein risk of 14.4% (95% CI, 13.5%-16.5%) at baseline and 17.4% (95% CI, 16.0%-19.0%) at 4.8 years while median refit Framingham risk was 20.3% (95% CI, 19.0%-21.5%) at baseline and 23.8% (95% CI, 22.2%-25.8%) at 4.8 years. The absolute within-person change in the 9-protein risk was greater than for the refit Framingham model for participants with events (P = .002); median annualized within-person change was 1.86% (95% CI, 1.15%-2.54%) for the 9-protein model compared with 1.00% (95% CI, 0.87%-1.19%) for refit Framingham. Over 5 years, these annualized values represent an absolute change in risk of 9.3% for the 9-protein score and 5.0% for refit Framingham. For both risk models, these within-person changes were greater than for the event-free group (P < .001), in which the median annualized within-person change in the 9-protein risk group was 0.65% (95% CI, 0.45%-0.86%) compared with 0.72% (95% CI, 0.64%-0.80%) for refit Framingham (P = .3). The IDI for the 9-protein risk predictions at baseline, compared with 4.8 years, was 0.07 (95% CI, 0.04-0.10)—an absolute increase in mean risk of 7% for participants with events after the second sample over their baseline risk. Combining the 9-protein prognostic index at 4.8 years with the within-person change from baseline yielded an augmented model that fit slightly better (P = .03) than the 9-protein prognostic index at 4.8 years alone, although the discriminatory power was not meaningfully improved (IDI, 0.009; NRI(>0), 0.26; Δ C statistic, 0.006).

Discussion

Individualized risk assessment in patients diagnosed with apparently stable CHD is necessary because stable CHD appears to be a heterogeneous entity with a broad range of outcomes.10,11 For stratification of cardiovascular risk using the “omics” technologies, genomics has been investigated most extensively, but genomic risk scores do not substantively improve risk discrimination over traditional risk factors.4,5,26 Even if genomic approaches are ultimately successful, they will succeed primarily in predicting risk related to lifelong exposure and will not discern any changes in risk over time.4,5,11,26 Compared with genomics, proteomics offers several advantages: proteins integrate both environmental and genetic influences; proteins are responsive to lifestyle and therapeutic interventions, informing of changes in risk27,28; and proteins are effectors of biological process and thus potential targets of therapies.29 However, limitations in proteomic techniques have to this point hindered the implementation of these advantages.

In this study, levels of 1130 plasma proteins were measured using modified aptamers7,15,30,31 to identify prognostic proteins that improve cardiovascular risk prediction. A prediction time horizon of 4 years was chosen—sufficiently long to implement therapeutic changes11—and yet not so distant that risk becomes deniable, losing its motivation. In the discovery cohort, 200 proteins were prognostic of cardiovascular events (eTable 4 in the Supplement), many of which are newly discovered biomarkers of cardiovascular risk.

An unbiased statistical approach was used to arrive at a 9-protein risk prediction model which, by itself, performed better than traditional risk factors represented by a refit Framingham secondary event model20 and offered fair discrimination based on the C statistic (Table 3). The discrimination slope represents the separation in mean risk between participants with and without events.8,18 The addition of the 9-protein risk score to refit Framingham offered a substantial improvement in this separation (Table 3). Admittedly, the large magnitude of the improvement in discrimination (in C statistic, discrimination slope separation, and IDI) and net reclassification by the 9-protein model (Table 3) was partly reflective of the weak performance of traditional risk factors in predicting the risk of secondary events,32 also observed in the present study.

By including an independent external cohort in this study, best practices for validation were followed,8 reducing the risk of translation to clinical use by verifying the predictive capacity of the key prognostic proteins and their combination in the proteomic model to less-stringent sample collection and processing that are more typical of clinical practice.6,9,12 In applying protein-based risk assessment to patients with stable CHD, this diagnosis was found to be associated with a broad range of cardiovascular and mortality risks (Figure 2, Figure 3), suggesting that stable CHD may not represent a single homogeneous entity.

Paired samples were used to evaluate whether the proteomic risk changed over time as participants approached a cardiovascular event. The 9-protein risk score changed more than the refit Framingham model among participants approaching new events. In addition, the 9-protein risk score generated at the follow-up sample was a stronger predictor of subsequent outcomes than the preceding baseline risk score. The mutability of the proteomic risk score, in relation to future events, offers a potential advantage over genetic risk prediction, which remains unchanged during lifetime. It remains unclear, however, whether the magnitude of changes in the proteomic risk score among participants with future events might lead to a change in management.

Other cardiovascular risk algorithms for stable CHD are available, including a model from the REACH (Reduction of Atherothrombosis for Continued Health) registry, which combines traditional risk factors with information about the extent of diseased vascular beds, heart failure, atrial fibrillation, medical treatments, and geographic location.33 The REACH registry algorithm reported a C statistic for the prediction of a next cardiovascular event of 0.67 (95% CI, 0.66-0.68) and lacked external validation. The present study results could not be directly compared with the REACH model because some of the REACH variables were unavailable in its 2 cohorts.

Another cardiovascular risk prediction model used the best available candidate biomarkers for cardiovascular outcomes in the Heart and Soul cohort,10 including high-sensitivity troponin, NT-proBNP, C-reactive protein, and urine albumin:creatinine ratio. This risk prediction model did not replicate well in external validation.10 Genetic variants have also been associated with the risk of CHD. A recent study tested how well a genetic risk score based on 27 variants could predict recurrent CHD events in the CARE (Cholesterol and Recurrent Events) and PROVE IT-TIMI 22 (Pravastatin or Atorvastatin Evaluation and Infection Therapy-Thrombolysis in Myocardial Infarction 22) trial populations.4 The adjusted quintile HR was 1.81 (95% CI, 1.22-2.67), a risk prediction that is appreciably smaller than proteomics yielded in the present study, with an adjusted quintile HR of 7.63 (95% CI, 4.53- 12.85) in the validation set (Table 3).

Study Strengths

This study conducted a large-scale proteomic analysis of cardiovascular risk, using a high-throughput proteomic platform.7,16,30,31 The study was conducted in 2 large well-characterized cohorts with standardized adjudication of outcome events12,34 across 2 continents and included cross-sectional and longitudinal assessments. Specimen quality has been noted as an important reason why omics findings reported from one laboratory may not replicate in others.9 Accordingly, the analyses in the present study were conducted across a range of specimen qualities, representative of standardized (derivation) and clinical practice conditions (validation). The findings were consistent across this range of specimen quality.

Limitations

This initial analysis of circulating proteins focused on a population of relatively high-risk individuals with established CHD. There is additional need for accurate cardiovascular risk prediction in the lower-risk general population or in even higher-risk individuals with CHD. Another limitation is that this study investigated only the sensitivity to increasing risk as represented by an approaching event; it will be important to evaluate individual medical interventions that alter risk to learn how well proteins can discern changes in risk in specific settings.

Conclusions

Among patients with stable CHD, a risk score based on 9 proteins performed better than the refit Framingham secondary event risk score in predicting cardiovascular events but still only provided modest discriminative accuracy. Further research is needed to assess whether the score is more accurate in a lower-risk population.

Back to top
Article Information

Corresponding Author: Peter Ganz, MD, Department of Medicine, University of California-San Francisco, 1001 Potrero Ave, Room 5G1, San Francisco, CA 94110 (peter.ganz@ucsf.edu).

Author Contributions: Dr Ganz had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Ganz, Hveem, Jonasson, Kato, Williams.

Acquisition, analysis, or interpretation of data: Ganz, Heidecker, Hveem, Kato, Segal, Sterling, Williams.

Drafting of the manuscript: Ganz, Kato, Segal, Sterling, Williams.

Critical revision of the manuscript for important intellectual content: Ganz, Heidecker, Hveem, Jonasson, Kato, Segal, Williams.

Statistical analysis: Ganz, Kato, Segal, Sterling, Williams.

Administrative, technical, or material support: Ganz, Hveem, Jonasson, Williams.

Study supervision: Ganz, Segal, Williams.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Segal reports receipt of payment from SomaLogic to support statistical analyses costs. Drs Hveem and Jonasson report affiliation with the HUNT study, which received a payment from SomaLogic for providing plasma specimens and database information. Drs Sterling and Williams are employees of SomaLogic. Dr Williams reports serving on the board of Venaxis Inc. Mr Kato is an employee of the NEC Corporation of America, which has a research contract with SomaLogic. No other disclosures are reported.

Funding/Support: The Heart and Soul cohort was supported by the Department of Veterans Affairs; the National Heart, Lung, and Blood Institute (R01 HL079235); the American Federation for Aging Research; the Robert Wood Johnson Foundation; and the Ischemia Research and Education Foundation. The HUNT3 cohort was funded by the Norwegian Ministry of Health, Norges Teknisk-Naturvitenskapelige University, the Norwegian Research Council, Central Norway Regional Health Authority, the Nord-Trøndelag County Council, and the Norwegian Institute of Public Health. Measurement of all proteins in both cohorts was funded by SomaLogic.

Role of the Funder/Sponsor: SomaLogic provided the funding for and execution of the protein assays. SomaLogic had no veto rights regarding the content of the study or the decision to submit the manuscript for publication.

Additional Contributions: The authors thank Christopher J. Bock, BA, Joy L. Boyd, BS, Kaitlin Gargiulo, BS, Amanda L. Grisco, BA, Tracy R. Keeney, BA, and Elyn E. Lytton, BS, all current or former compensated employees of SomaLogic, for performing the protein assays; Rosalynn Gill, PhD, a former compensated employee of SomaLogic, and Christina Lee, PhD, MBA, a current compensated employee of SomaLogic, for providing logistical support; Edward N. Brody, MD, PhD, Trudi Foreman, BA, Richard M. Lawn, PhD, Robert E. Mehler, MD, Rachel Ostroff, PhD, Britta Singer, PhD, all current or former compensated employees of SomaLogic, for assisting with the design of the study, sample management and interpretation of analytical data; Stephan Kraemer, PhD, a compensated employee of SomaLogic, for overseeing the execution of protein assays; Hilde Kjeldstad Berg, MSc, a compensated employee of the HUNT cohort research group, for providing logistical support; and Bjørnar Klykken, MD, a compensated employee of the HUNT cohort research group, for adjudicating outcomes.

References
1.
Go  AS, Mozaffarian  D, Roger  VL,  et al; American Heart Association Statistics Committee and Stroke Statistics Subcommittee.  Heart disease and stroke statistics—2014 update: a report from the American Heart Association. Circulation. 2014;129(3):e28-e292.
PubMedArticle
2.
Goff  DC  Jr, Lloyd-Jones  DM, Bennett  G,  et al; American College of Cardiology/American Heart Association Task Force on Practice Guidelines.  2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation. 2014;129(25)(suppl 2):S49-S73.
PubMedArticle
3.
Ridker  PM, Cook  NR.  Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762-1765.
PubMedArticle
4.
Mega  JL, Stitziel  NO, Smith  JG,  et al.  Genetic risk, coronary heart disease events, and the clinical benefit of statin therapy: an analysis of primary and secondary prevention trials. Lancet. 2015;385(9984):2264-2271.
PubMedArticle
5.
Paynter  NP, Chasman  DI, Paré  G,  et al.  Association between a literature-based genetic risk score and cardiovascular events in women. JAMA. 2010;303(7):631-637.
PubMedArticle
6.
Lindsey  ML, Mayr  M, Gomes  AV,  et al; American Heart Association Council on Functional Genomics and Translational Biology, Council on Cardiovascular Disease in the Young, Council on Clinical Cardiology, Council on Cardiovascular and Stroke Nursing, Council on Hypertension, and Stroke Council.  Transformative impact of proteomics on cardiovascular health and disease: a scientific statement from the American Heart Association. Circulation. 2015;132(9):852-872.
PubMedArticle
7.
Gold  L, Ayers  D, Bertino  J,  et al.  Aptamer-based multiplexed proteomic technology for biomarker discovery. PLoS One. 2010;5(12):e15004.
PubMedArticle
8.
Steyerberg  EW, Vickers  AJ, Cook  NR,  et al.  Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138.
PubMedArticle
9.
McShane  LM, Cavenagh  MM, Lively  TG,  et al.  Criteria for the use of omics-based predictors in clinical trials. Nature. 2013;502(7471):317-320.
PubMedArticle
10.
Beatty  AL, Ku  IA, Bibbins-Domingo  K,  et al.  Traditional Risk Factors Versus Biomarkers for Prediction of Secondary Events in Patients With Stable Coronary Heart Disease: From the Heart and Soul Study. J Am Heart Assoc. 2015;4(7):e001646.
PubMedArticle
11.
Eagle  KA, Ginsburg  GS, Musunuru  K,  et al.  Identifying patients at high risk of a cardiovascular event in the near future: current status and future directions: report of a national heart, lung, and blood institute working group. Circulation. 2010;121(12):1447-1454.
PubMedArticle
12.
Krokstad  S, Langhammer  A, Hveem  K,  et al.  Cohort profile: the HUNT study, Norway. Int J Epidemiol. 2013;42(4):968-977.
PubMedArticle
13.
Marcus  GM, Olgin  JE, Whooley  M,  et al.  Racial differences in atrial fibrillation prevalence and left atrial size. Am J Med. 2010;123(4):375 e1-375 e7.
PubMedArticle
14.
Olson  KA, Beatty  AL, Heidecker  B,  et al.  Association of growth differentiation factor 11/8, putative anti-ageing factor, with cardiovascular outcomes and overall mortality in humans: analysis of the Heart and Soul and HUNT3 cohorts. Eur Heart J. 2015;36(48):3426-3434.
PubMedArticle
15.
Vaught  JD, Bock  C, Carter  J,  et al.  Expanding the chemistry of DNA for in vitro selection. J Am Chem Soc. 2010;132(12):4141-4151.
PubMedArticle
16.
Brody  E, Gold  L, Mehan  M,  et al.  Life’s simple measures: unlocking the proteome. J Mol Biol. 2012;422(5):595-606.
PubMedArticle
17.
Tibshirani  R.  The lasso method for variable selection in the Cox model. Stat Med. 1997;16(4):385-395.
PubMedArticle
18.
McClelland  RL, Jorgensen  NW, Budoff  M,  et al.  10-Year coronary heart disease risk prediction using coronary artery calcium and traditional risk factors: derivation in the MESA (Multi-Ethnic Study of Atherosclerosis) with validation in the HNR (Heinz Nixdorf Recall) study and the DHS (Dallas Heart Study). J Am Coll Cardiol. 2015;66(15):1643-1653.
PubMedArticle
19.
Tibshirani  R.  Regression shrinkage and selection via the lasso: a retrospective. J R Stat Soc B. 2011;73(3):273-282. doi:10.1111/j.1467-9868.2011.00771.xArticle
20.
D’Agostino  RB, Russell  MW, Huse  DM,  et al.  Primary and subsequent coronary risk appraisal: new results from the Framingham study. Am Heart J. 2000;139(2 Pt 1):272-281.
PubMedArticle
21.
Heagerty  PJ, Zheng  Y.  Survival model predictive accuracy and ROC curves. Biometrics. 2005;61(1):92-105.
PubMedArticle
22.
Pencina  MJ, D’Agostino  RB  Sr, Steyerberg  EW.  Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11-21.
PubMedArticle
23.
Pencina  MJ, D’Agostino  RB, Pencina  KM, Janssens  AC, Greenland  P.  Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176(6):473-481.
PubMedArticle
24.
Pepe  MS, Janes  H. Methods for evaluating prediction performance of biomarkers and tests, UW Biostatistics Working Paper Series (384). http://biostats.bepress.com/uwbiostat/paper384. Accessed June, 5, 2016.
25.
R Core Team RFfSC. Austria R: A Language and Environment for Statistical Computing. Vienna, Austria: The R Foundation; 2013.
26.
Ripatti  S, Tikkanen  E, Orho-Melander  M,  et al.  A multilocus genetic risk score for coronary heart disease: case-control and prospective cohort analyses. Lancet. 2010;376(9750):1393-1400.
PubMedArticle
27.
Ridker  PM, Danielson  E, Fonseca  FA,  et al; JUPITER Trial Study Group.  Reduction in C-reactive protein and LDL cholesterol and cardiovascular event rates after initiation of rosuvastatin: a prospective study of the JUPITER trial. Lancet. 2009;373(9670):1175-1182.
PubMedArticle
28.
Nissen  SE, Tuzcu  EM, Schoenhagen  P,  et al; Reversal of Atherosclerosis with Aggressive Lipid Lowering (REVERSAL) Investigators.  Statin therapy, LDL cholesterol, C-reactive protein, and coronary artery disease. N Engl J Med. 2005;352(1):29-38.
PubMedArticle
29.
Dadu  RT, Ballantyne  CM.  Lipid lowering with PCSK9 inhibitors. Nat Rev Cardiol. 2014;11(10):563-575.
PubMedArticle
30.
Kraemer  S, Vaught  JD, Bock  C,  et al.  From SOMAmer-based biomarker discovery to diagnostic and clinical applications: a SOMAmer-based, streamlined multiplex proteomic assay. PLoS One. 2011;6(10):e26332.
PubMedArticle
31.
Mehan  MR, Ostroff  R, Wilcox  SK,  et al.  Highly multiplexed proteomic platform for biomarker discovery, diagnostics, and therapeutics. Adv Exp Med Biol. 2013;735:283-300.
PubMed
32.
Pencina  MJ, D’Agostino  RB  Sr, Song  L.  Quantifying discrimination of Framingham risk functions with different survival C statistics. Stat Med. 2012;31(15):1543-1553.
PubMedArticle
33.
Wilson  PW, D'Agostino  R  Sr, ., Bhatt  DL,  et al.  An international model to predict recurrent cardiovascular disease. Am J Med. 2012;125(7):695-703 e1.
PubMedArticle
34.
Whooley  MA, de Jonge  P, Vittinghoff  E,  et al.  Depressive symptoms, health behaviors, and risk of cardiovascular events in patients with coronary heart disease. JAMA. 2008;300(20):2379-2388.
PubMedArticle
×