Predicted 5-year risk of an incident coronary heart disease event, comparing the basic model with the basic model plus log C-reactive protein (CRP), for men (A) and women (B) in the Atherosclerosis Risk in Communities Study.
Predicted 5-year risk of an incident coronary heart disease event, comparing the basic model with the basic model plus lipoprotein-associated phospholipase A2 (LpPLA2), for men (A) and women (B) in the Atherosclerosis Risk in Communities Study.
Aaron R. Folsom, Lloyd E. Chambless, Christie M. Ballantyne, Josef Coresh, Gerardo Heiss, Kenneth K. Wu, Eric Boerwinkle, Thomas H. Mosley, Paul Sorlie, Guoqing Diao, A. Richey Sharrett. An Assessment of Incremental Coronary Risk Prediction Using C-Reactive Protein and Other Novel Risk MarkersThe Atherosclerosis Risk in Communities Study. Arch Intern Med. 2006;166(13):1368–1373. doi:10.1001/archinte.166.13.1368
There has been interest in recent years in whether additional, and in particular novel, risk factors or blood markers, such as C-reactive protein, can enhance existing coronary heart disease (CHD) prediction models.
Using a series of case-cohort studies, the prospective Atherosclerosis Risk in Communities (ARIC) Study assessed the association of 19 novel risk markers with incident CHD in 15 792 adults followed up since 1987-1989. Novel markers included measures of inflammation, endothelial function, fibrin formation, fibrinolysis, B vitamins, and antibodies to infectious agents. Change in the area under the receiver operating characteristic curve (AUC) was used to assess the additional contribution of novel risk markers to CHD prediction beyond that of traditional risk factors.
The basic risk factor model, which included traditional risk factors (age, race, sex, total and high-density lipoprotein cholesterol levels, systolic blood pressure, antihypertensive medication use, smoking status, and diabetes), predicted CHD well, as evidenced by an AUC of approximately 0.8. The C-reactive protein level did not add significantly to the AUC (increase in AUC of 0.003), and neither did most other novel risk factors. Of the 19 markers studied, lipoprotein-associated phospholipase A2, vitamin B6, interleukin 6, and soluble thrombomodulin added the most to the AUC (range, 0.006-0.011).
Our findings suggest that routine measurement of these novel markers is not warranted for risk assessment. On the other hand, our findings reinforce the utility of major, modifiable risk factor assessment to identify individuals at risk for CHD for preventive action.
Epidemiologic research has identified many risk factors associated with increased incidence of coronary heart disease (CHD). The Framingham Heart Study pioneered the use of the major risk factors (eg, age, high blood pressure, cigarette smoking, elevated total cholesterol level, low high-density lipoprotein cholesterol [HDL-C] level, and diabetes mellitus) in regression models to predict an individual's risk of CHD reasonably well.1 The Framingham CHD prediction equations apply generally to other US populations,2 including the middle-aged Atherosclerosis Risk in Communities (ARIC) Study cohort.3 Clinical professionals have adapted prediction models for use in preventive cardiology; for example, via the National Cholesterol Education Panel's adult treatment guidelines.4 An advantage of current prediction models is that most of the major risk factors included are modifiable, with known clinical benefit to their modification.
There has been interest in recent years in whether novel risk factors or markers, such as C-reactive protein (CRP), can enhance existing CHD prediction models.5- 8 Risk markers often show a statistically significant association with CHD, but receiver operating characteristic (ROC) curve analysis indicates that they add little to CHD prediction. Using ROC analysis, we recently showed in the ARIC Study that several novel risk factors or markers did not individually improve CHD prediction much beyond established risk factors.9 A combination of additional risk markers (body mass index, waist-hip ratio, Keys dietary score, serum albumin level, forced expiratory volume in 1 second, lipoprotein (a) level, heart rate, fibrinogen level, factor VIII, von Willebrand factor, pack-years of smoking, sport activity, and carotid intima media thickness) moderately increased CHD prediction.9 However, adding a large battery of tests to improve prediction of CHD would have little use in the typical clinical setting.
The ARIC ROC analysis9 did not include CRP or several other novel risk markers because they were measured in nested case-cohort studies on subsets of the ARIC Study cohort, not the entire cohort. We subsequently developed statistical methods for ROC analysis of incident disease using nested case-cohort data. In this study, we examined whether CRP or other individual novel risk factors or markers contribute to the prediction of incident CHD in the ARIC Study beyond that achieved using the traditional major risk factors alone.
The ARIC Study is a prospective study of cardiovascular disease incidence in a cohort of 15 792 persons, initially aged 45 to 74 years and sampled from 4 US communities in 1987-1989.3 A baseline examination and 3 subsequent triennial examinations were conducted. Follow-up is ongoing and complete for incident CHD (ie, myocardial infarction, fatal CHD, or coronary revascularization).9,10
At baseline, cigarette smoking, blood pressure, antihypertensive medication use, total cholesterol and HDL-C levels, and diabetes were measured using standardized methods.3 Serum, plasma, and DNA samples were collected and stored for nested case-cohort studies at 4 times during follow-up. The design involved all available incident CHD cases at the time, compared with a stratified random sample of the ARIC Study cohort. Sampling excluded patients with CHD, stroke, or transient ischemic attacks at baseline and was restricted to white and African American individuals. We stratified the sampling by age, sex, and ultrasonographically assessed carotid intima media thickness. Cohort members were randomly selected within these strata, and the analysis was weighted to provide ARIC Study population estimates. Methods have been published for measuring levels of CRP and lipoprotein-associated phospholipase A2 (LpPLA2)11; antibodies to Chlamydia pneumoniae, cytomegalovirus, and herpes simplex virus 112,13; serum vitamin B6, folate, and homocysteine14; D-dimer, plasminogen, plasminogen activator inhibitor 1 antigen, and tissue plasminogen activator antigen15; soluble thrombomodulin16; intracellular adhesion molecule 1; and E-selectin.17 In addition, using commercial assays, we measured plasma leptin (Linco Research Inc, St Charles, Mo) and interleukin 6 (IL-6), matrix metalloproteinase 1, and tissue inhibitor of metalloproteinase 1 (R&D Systems Inc, Minneapolis, Minn). Reliability coefficients from split samples (n = 20-96) sent blinded to the laboratory were 0.81 for leptin, 0.40 for IL-6, 0.74 for matrix metalloproteinase 1, and 0.62 for tissue inhibitor of metalloproteinase 1.
The data were analyzed by weighted proportional hazards regression, accounting for the stratified random sampling and the case-cohort design using the method by Barlow.18 To initially assess associations with incident CHD, hazard rate ratios were calculated for each categorical risk factor or for a 1-SD increment of each continuous risk factor. From the coefficients of the proportional hazards models, a risk score was calculated for each person by multiplying each model coefficient by the person's measured level for the risk variable associated with that coefficient, then summing all these products. Our measure of individual risk predictivity of a model is the area under the ROC curve (AUC),19 which is the probability that a person who had an incident event within a specified time (in this case 5 years) had a higher risk score than a person who did not have an event by that time. We used Kaplan-Meier–like methods to calculate the relevant probabilities of an event by 5 years in the face of censoring,20 modified for this analysis by weighting for inverse sampling fractions. When the variables in the model are unrelated to the event of interest, the expected AUC would be 0.5. Thus, the AUC has a range of 0.5 to 1. A test of the hypothesis “risk score A yields a higher AUC than risk score B” has been presented19 but is not directly applicable to use with the curves derived from censored data through a proportional hazards model. Our approach to this test used the bootstrap method.19,21
As an alternative to ROC curves, to visually display the increase in predictivity due to additional variables, we present graphs of the predicted probability of an incident CHD event within the first 5 years of follow-up by decile of risk score. Improved prediction would be indicated by moving more of the predicted events out of the lower deciles of risk score into the upper deciles.
As indicated in Table 1, many of the novel risk factors showed a statistically significant age-adjusted association with incident CHD. For example, the hazard rate ratio of CHD per SD of log CRP was 1.28 (P<.001). Several of these hazard rate ratios, including those for log CRP (1.19), LpPLA2 (1.17), log IL-6 (1.28), intracellular adhesion molecule 1 (1.40), log D-dimer (1.36), soluble thrombomodulin (0.65), and log vitamin B6 (0.73), persisted after adjustment for established risk factors. For comparison, the adjusted hazard rate ratios per SD for traditional continuous risk factors were as follows: total cholesterol, 1.28; HDL-C, 0.53; and systolic blood pressure, 1.56. The adjusted hazard rate ratios per SD for categorical risk factors were as follows: antihypertensive medication use, 1.96; current smoking, 2.44; and diabetes, 1.78.
Table 2 gives the sample sizes (n = 203-666 incident CHD events) and ROC analysis results in the ARIC nested case-cohort studies. The basic risk factor model included age, race, sex, total cholesterol and HDL-C levels, systolic blood pressure, antihypertensive medication use, smoking status, and diabetes. The basic model yielded good AUC values of approximately 0.8, varying somewhat among the 4 case-cohort sample groups. The AUC was approximately 0.82 for sample group 1, 0.81 for sample group 2, and 0.77 for sample groups 3 and 4, with variability due to missing data.
When CRP was added to the basic model, the 0.003 increase in AUC, from 0.767 to 0.770, was small and not statistically significant (P>.05) (Table 2). The incremental change for most other analytes was also small to nonexistent. The largest AUC increases were for log vitamin B6 (0.011), IL-6 (0.010), soluble thrombomodulin (0.005), and LpPLA2 (0.006). Because the sample size was largest for LpPLA2 (equivalent to CRP), the AUC increment for LpPLA2 was statistically significant (P<.05), whereas the larger AUC increments for vitamin B6, IL-6, and soluble thrombomodulin were not statistically significant.
To provide a perspective on how little prediction of CHD incidence was affected by adding log CRP to the basic risk equation, Figure 1 shows the predicted sex-specific probability of incident CHD within 5 years plotted by decile of risk. The distributions of event probabilities were virtually identical for the 2 models (with vs without log CRP). The corresponding, virtually identical, curves for LpPLA2 are shown in Figure 2.
We assessed in a prospective study of middle-aged men and women whether adding CRP or 18 other novel risk factors individually to a basic risk model would improve prediction of incident CHD. The basic risk factor model predicted CHD well, as evidenced by an AUC of approximately 0.8. The CRP level did not add significantly to the AUC (increase in AUC of 0.003), and neither did most other novel risk factors. This is consistent with the finding from our previous report using the whole ARIC Study cohort that adding single novel risk factors to a basic CHD risk prediction model only modestly improved risk assessment, despite the fact that those novel risk factors were statistically significantly associated with CHD.9
Six previous prospective studies have also tended to show little or no gain by adding CRP to risk prediction models. For example, Ridker et al5 reported an AUC of 0.81 for a CHD incidence model in American women that included age, smoking, diabetes, blood pressure, hormone replacement therapy, and low-density lipoprotein cholesterol level; adding CRP to the model apparently did not change the AUC of 0.81. Koenig et al6 reported that the Framingham model1 yielded an ROC of 0.735 for CHD incidence prediction in German men; this AUC increased statistically significantly to 0.750 with the addition of CRP level. Danesh et al7 reported that a model that contained age, sex, total cholesterol level, smoking, and systolic blood pressure as CHD predictors yielded an AUC of 0.64 in Icelandic men; this AUC rose slightly to 0.65 with the addition of CRP. The Framingham Offspring Study reported that a model that contained age, sex, and metabolic syndrome as cardiovascular disease predictors yielded an AUC of 0.74; when CRP was added, this AUC remained 0.74.22 The Rotterdam Study reported an AUC of 0.773 for a model predicting myocardial infarction with traditional risk factors and 0.777 after adding CRP to the model.23 Similarly, the Quebec Cardiovascular Study reported an AUC of 0.705 with traditional risk factors predicting CHD and 0.706 after adding CRP.24
Several aspects of our analysis and the interpretation of ROC curves warrant discussion. Various methods can be used to assess the value of a new risk factor to risk prediction, and each method answers different questions regarding the impact of that new risk factor. An adjusted relative risk describes the impact of the risk factor when all others in the model are held constant. If this adjusted relative risk is more than trivially different from 1.0, the prediction could vary in a clinically significant way if this risk factor is present or absent. Thus, an adjusted relative risk that is both statistically and clinically significant is of value in predicting an individual's risk and is relevant to the physician. The ROC analysis and approaches that assess the impact of a risk equation in a population depend on sensitivity and specificity as well as on the magnitude of the relative risk estimates, the covariation among the risk factors, and their combinations. If the risk factors are highly correlated, few in the population would be discordant for these factors, and thus their contribution to excess cases could not be large even though such risk factors may have statistically significant risk estimates.
Furthermore, the prior inclusion of several major risk factors in a prediction model constrains the ability of a single additional risk factor to contribute to the AUC. This is especially the case in our study, given that our starting AUC using the basic risk factor model already was approximately 0.80. Since a rationale underlying the Framingham risk prediction equation is optimization based on established modifiable risk factors that are conveniently and inexpensively accessible to clinical professionals, addition of a candidate risk marker must improve prediction beyond that of a model optimized throughout 2 decades.
Although the significant and independent association of a novel risk factor with CHD often does not equate to improved prediction of CHD beyond that of basic risk factors, this does not imply that the novel risk factor is pathophysiologically unimportant or unsuitable as a target for intervention. In the ARIC Study, plasma cholesterol level adds only approximately 0.02 to the AUC of a prediction model that already includes systolic blood pressure, antihypertensive medication use, current smoking, diabetes, and HDL-C level.9 However, an elevated cholesterol level is unquestionably an important modifiable cause of CHD, and clinical trials demonstrate unequivocally that lowering cholesterol levels reduces CHD risk. Similarly, small reductions in average blood cholesterol levels in populations translate into reductions of CHD risk burden in communities that are of considerable public health importance. The systemic marker of inflammation, CRP, is another good example. The CRP level is consistently associated with CHD risk, in the context of inflammation playing an important role in atherogenesis and atherothrombotic events, and reduction of inflammation in the context of low-density lipoprotein cholesterol lowering (as indexed by CRP reduction) may reduce risk of plaque rupture.5,25- 27 Some have also argued that CRP may be useful in both identifying and monitoring those who may benefit from statin therapy.25,26 Based on the totality of evidence, however, CRP level does not emerge as a clinically useful addition to basic risk factor assessment for identifying patients at risk of a first CHD event.
Strengths of this ARIC Study analysis are the large and diverse cohort, careful risk factor assessment, complete follow-up, and wide variety of novel factors assessed on samples obtained before CHD onset. The ARIC Study involved middle-aged individuals and can best be generalized to a similar population. A limitation was that the series of nested case-cohort studies resulted in sets of analytes being measured in different sets of CHD cases and controls. However, this should not have caused bias in the hazard rate ratios or AUC estimates. As is typical of nested case-cohort studies, the blood samples were stored at −70°C for up to a decade before analysis. It is possible that certain novel analytes degraded over time, which would have tended to weaken their predictive power. Also, having only a single assessment of less reliable analytes would tend to weaken their apparent predictive power. Nevertheless, the hazard rate ratios for many of the novel analytes, including CRP and LpPLA2, were similar to those in the literature, suggesting that error in novel analytes had no greater impact in the ARIC Study than in published studies.
One of the aims of the ARIC Study is to identify new risk factors or markers. Several novel risk markers examined were found to be associated with CHD risk, but few proved useful in improving the AUC of the basic risk factor prediction model. Therefore, we advise against their routine measurement for risk assessment. On the other hand, our findings reinforce the utility of major, modifiable risk factor assessment to identify individuals at risk for CHD for preventive action.
Correspondence: Aaron R. Folsom, MD, MPH, Division of Epidemiology and Community Health, School of Public Health, University of Minnesota, Suite 300, 1300 S Second St, Minneapolis, MN 55454-1015 (firstname.lastname@example.org).
Accepted for Publication: March 28, 2006.
Author Contributions: Dr Folsom had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Financial Disclosure: None reported.
Funding/Support: The ARIC Study is a collaborative study supported by contracts N01-HC-55015, N01-HC-55016, N01-HC-55018, N01-HC-55019, N01-HC-55020, N01-HC-55021, and N01-HC-55022 from the National Heart, Lung, and Blood Institute.
Acknowledgment: We thank Laura Kemmis for technical assistance and the participants and staff of the ARIC Study for important contributions for many years.