Cardiovascular Risk Prediction in Diabetic Men and Women Using Hemoglobin A_1c vs Diabetes as a High-Risk Equivalent

Background It is unclear whether models that include hemoglobin A_1c (HbA_1c) levels only for diabetic patients improve the ability to predict cardiovascular disease (CVD) risk compared with the currently recommended classification of diabetes as a cardiovascular risk equivalent.

Methods A total of 24 674 women (including 685 diabetic participants at baseline) and 11 280 men (including 563 diabetic participants at baseline) were followed up prospectively for cardiovascular disease (CVD). One hundred twenty-five CVD events occurred in diabetic women (666 in nondiabetic women), and 170 events occurred in diabetic men (1382 in nondiabetic men). Models for CVD risk were generated separately for men and women using the traditional CVD risk factors with the addition of a term for HbA_1c levels only for diabetic individuals. In diabetic participants, the resulting predicted risks were compared with classification of diabetes as a cardiovascular risk equivalent (10-year CVD risk of at least 20%).

Results In women, the models including HbA_1c levels in diabetic participants improved the C statistic by 0.177 (P < .001) over the risk equivalence model and showed improved reclassification (net reclassification improvement [NRI] of 26.7% [P = .001]). In men, the improvements were more modest but still statistically significant (C statistic change of 0.039 [P = .02]; NRI of 9.2% [P = .04]). Including HbA_1c levels also improved prediction over a dichotomous term for diabetes in women (NRI of 11.8% [P = .03]) but not in men.

Conclusions In both women and men with diabetes at baseline, we observed significant improvements in predictive ability of CVD risk using models incorporating HbA_1c levels compared with classification of diabetes as a cardiovascular risk equivalent.

Diabetes is a well-established risk factor for future cardiovascular disease (CVD), and current treatment guidelines for assessment of CVD risk recommend designation of diabetes as a risk equivalent to prevalent CVD, thus placing all diabetic patients in a high-risk classification.^1,2However, more recent studies suggest that diabetes alone has a consistently lower relative risk of future CVD than a prior myocardial infarction (MI).^3,4 In addition, substantial variability in CVD risk among diabetic patients has been demonstrated, with higher risks seen in those with additional CVD risk factors such as high blood pressure and high cholesterol level.^5,6 Simulated cost-benefit analyses have suggested that this variability in CVD risk could provide an opportunity for tailored preventive therapy in diabetic patients.⁷

In numerous prospective epidemiologic studies, hemoglobin A_1c (HbA_1c) level has been shown to predict CVD risk in addition to traditional CVD risk factors among individuals with diabetes.⁸ However, whether allowing CVD risk associated with diabetes to vary based on HbA_1c levels and other CVD risk factors compared with classification of all diabetic patients as high risk would improve CVD risk prediction is uncertain, especially in populations with differing levels of overall CVD risk. This is of clinical relevance because, if superior, a single CVD risk model combining traditional factors with an additional HbA_1c term for diabetic patients, for whom this information is already clinically available, could be used irrespective of diabetes status.

To address this question, we used large prospective cohorts of men and women to generate CVD risk models that included HbA_1c levels for the diabetic participants. We then compared the predictive ability of the resulting models with classification based on current guidelines for the diabetic participants. In secondary analyses, we also examined the effect of a simple dichotomous term for diabetes in place of HbA_1c levels.

Female study participants were members of the Women's Health Study (WHS). The WHS was designed as a trial of vitamin E and aspirin for the prevention of CVD and cancer.⁹ Participants included 39 896 US female health professionals, older than 45 years, and free of major chronic disease at the time of enrollment. All participants of the WHS provided written informed consent, and the study was approved by the institutional review board of the Brigham and Women's Hospital, Boston, Massachusetts. After excluding women 80 years or older and those without complete data or a baseline blood sample, 685 women who were diabetic at baseline were available for model comparisons. For the model generation, 24 674 women were available (including the 685 diabetic women), with median follow-up of 10.2 years (interquartile range, 9.7-10.6).

Male study participants were members of the Physician's Health Study II (PHS II), a randomized trial of beta carotene, ascorbic acid, and vitamin E, and multivitamins in 14 641 male US physicians for the prevention of CVD and cancer.¹⁰ All PHS II participants provided written informed consent, and the study was approved by the institutional review board of the Brigham and Women's Hospital. After excluding men 80 years or older and those without complete data, 563 men who were diabetic at baseline were available for the model comparisons. For the model generation, 11 280 men (including the 563 diabetic men) were included, with median follow-up of 11.8 years (interquartile range, 8.7-12.2).

In both WHS and PHS II, information on race, age, parental history of premature MI, smoking, blood pressure, and medication use was collected at study baseline by questionnaire. High-density lipoprotein and total cholesterol, high-sensitivity C-reactive protein, and HbA_1c levels were measured using the baseline blood sample, which had been stored in liquid nitrogen until laboratory analysis. Hemoglobin A_1c was measured using the Tina-Quant turbidimetric inhibition immunoassay (Roche Diagnostics, Indianapolis, Indiana) standardized to the Diabetes Control and Complications Trial. The coefficient of variation from blinded, simultaneously analyzed, quality controls was 7.2%. Participants were classified as having diabetes at baseline if they reported ever having a diagnosis of diabetes. Participants were followed up for incident CVD, which included MI, ischemic stroke, coronary revascularization, or cardiovascular death, and events were adjudicated by medical record review using standardized criteria.^9,10

All analysis was performed separately in the WHS and PHS II cohorts using the structure outlined below. Baseline characteristics were compared using the Kruskal-Wallis test for continuous measures and the χ² test for categorical measures.

The cardiovascular risk equivalent model was generated to simulate application of current risk scores and to allow for variation above the 20% 10-year risk cutoff. To achieve that, a model using a base set of known CVD risk factors was fit in the nondiabetic participants and applied to the diabetic participants. The nondiabetic participants were used both to generate stable estimates of risk factor effects and to ensure that risk factor effect estimates were not affected by correlations between traditional risk factors and diabetes status. Then the diabetic participants were assigned the higher of 20% 10-year risk or their risk-equivalent model–predicted 10-year risk. The comparison model was fit in the whole population and included the same base set of known CVD risk factors as well as a linear term for HbA_1c only in the diabetic participants. Hemoglobin A_1c levels were set to zero in the nondiabetic participants, and linearity was assessed using penalized spline models. Predicted risks were generated in Cox proportional hazards models and compared at 8 years, a time point at which follow-up was more than 90% complete in both cohorts, for increased stability, and all results were extrapolated to 10-year predicted risks for presentation by assuming an exponential survival distribution.

Two base sets of cardiovascular risk factors were used. For the primary base model, we refit the variables (natural log of age, natural log of systolic blood pressure, natural log of total and high-density lipoprotein cholesterol, and smoking) from the Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (ATP III) risk score, which is based on the Framingham risk score, to our data to obtain new coefficients.¹ A second base model, including the additional Reynolds risk score (RRS) variables natural log of C-reactive protein and parental history of premature MI, was examined using the same structure.¹¹ Calibration, or match between absolute predicted and observed rates, was assessed on the total study population, using the Hosmer-Lemeshow goodness-of-fit test, which divides the predicted risk into deciles and compares the average predicted risk in each decile to the observed risk for the decile.¹²

The estimated predicted risks from the risk equivalent model and the HbA_1c model were then compared for discrimination, or ability to rank cases higher then noncases, using the C statistic for participants with complete 8-year follow-up.¹³ Reclassification was assessed by comparing the predicted 10-year risk for each pair of models across 4 categories (<5%, 5% to <10%, 10% to <20%, and ≥20%).¹⁴ From the resulting table, we computed the net reclassification improvement (NRI) and the integrated discrimination improvement (IDI) for the participants with complete 8-year follow-up.¹⁵ We also calculated the reclassification calibration statistic for the table, which assesses the match between predicted and observed event rates for each model in each division of the reclassification table, with lower values and higher P values suggesting better fit.¹⁶

To compare the 2 cohorts as directly as possible, the model comparisons were repeated in a subgroup of diabetic WHS and PHS III participants aged 55 to 70 years, for whom the distribution of ages was matched by 5-year age category. We also assessed an intermediate model for diabetes with a dichotomous term for diagnosed diabetes at baseline added to the ATP III or RRS covariates, to see if the linear term for HbA_1c added predictive value beyond the simpler dichotomous term. We also repeated the entire analysis using 10-fold cross-validation.

All statistical analysis was done using R version 2.10.0 (R Foundation for Statistical Computing, Vienna, Austria).

Over the follow-up period, 125 cardiovascular events occurred in the 685 diabetic women (103 by 8 years), with an additional 666 events in the nondiabetic women. The diabetic subjects were older with a less favorable risk factor profile compared with the nondiabetic subjects, with the exception of similar rates of smoking and parental history of premature MI (Table 1).

Adding a linear term for HbA_1c in the diabetic women to the ATP III or RRS covariates was independently predictive (hazard ratio [HR], 1.17 [P < .001] for ATP III; HR, 1.16 [P < .001] for RRS) and the coefficients for all models are given in the eTable. There was no evidence of nonlinearity. All of the models, including the risk equivalent model, were calibrated in the total participants. The distributions of the 10-year predicted risks in the diabetic women from the ATP III model including HbA_1c are shown in Figure, A, with similar results from the RRS model (data not shown). As shown, 71.9% of the diabetic WHS participants had a predicted 10-year risk less than 20%.

When compared with classification of baseline diabetes as a cardiovascular risk equivalent (in which all diabetic participants are placed in the ≥20% 10-year risk of CVD category), 19% to 20% of diabetic women moved into the lowest predicted 10-year risk category using the HbA_1c models (Table 2). The lowest predicted risk categories for the ATP III and RRS models (<5% 10-year risk) had observed Kaplan-Meier 10-year incident rates of 2.8% and 1.9%, respectively, which was within the predicted range (lower than 5%). The intermediate risk categories (5% to <10% and 10% to <20%) had slightly higher than expected risks (15.2% and 17.8%, respectively, for ATP III and 14.0% and 20.5%, respectively, for RRS), while the observed rate in the high-risk category was higher than 30% for both groups. Overall, the models with an HbA_1c term showed substantial improvement in the diabetic participants (Table 3). The models including HbA_1c improved the C statistic (P < .001 for both ATP III and RRS), had a significant NRI of 26.7% for ATP III (P = .001) and 23.6% for RRS (P = .003) and had an IDI of 0.072, with P < .001 for both.

When a dichotomous diabetes term was used instead of a linear HbA_1c term, the pattern of improvement over classification of diabetes as a cardiovascular risk equivalent was similar. However, the HbA_1c term showed significant improvement in prediction over models with the diabetes term, with an NRI of 11.8% (P = .03) in ATP III and an NRI of 10.5% (P = .04) for RRS.

In the 563 diabetic men, 170 cardiovascular events occurred over follow-up (147 by 8 years), with an additional 1382 events in the nondiabetic men. As in the women, the diabetic participants were older with a less favorable risk factor profile than the nondiabetic participants, with the exception smoking and parental history of premature MI (Table 1). However, the PHS II participants were older overall than the WHS participants (median age, 67.8 years vs 55.0 years for the diabetic participants).

Similar to the women, adding a linear term for HbA_1c in the diabetic participants to the ATP III or RRS covariates was independently predictive of CVD risk for men (HR, 1.10 [P < .001] for ATP III, and HR, 1.10 [P < .001] for RRS; additional model information is given in the eTable). As with the women, all models were calibrated in the total participants, and there was no evidence of nonlinearity. The distributions of the 10-year predicted risks in the diabetic men from the ATP III model including HbA_1c are shown in Figure, B with similar results from the RRS models (not shown). In contrast to the WHS, only 24.5% of the diabetic PHS II participants had a predicted 10-year CVD risk less than 20%.

As shown in the Figure, few men were predicted to have low CVD risk, and consequently only 0.4% of the men were reclassified to the lowest-risk categories for the ATP III model, with 0.5% reclassified for the RRS model (Table 2). All of the risk categories had higher observed event rates than expected. The models with an HbA_1c term showed modest improvement in prediction overall for the diabetic participants compared with the cardiovascular risk equivalent classification (Table 3). There was a significant improvement in the C statistic only in the ATP III model (P = .02), significant NRIs of 9.2% (P = .04) and 12.4% (P = .004) for ATP III and RRS, respectively, and an IDI of 0.04 (P < .001) for both.

When a dichotomous diabetes term was used instead of a linear HbA_1c term, a similar pattern of improvementover classification of diabetes as a cardiovascular risk equivalent was seen. There was no significant improvement in the C statistic for either model, with an NRI of 11.1% (P = .009) and 9.4% (P = .034) for ATP III and RRS, respectively, and an IDI of 0.03 for ATP III and 0.04 for RRS (P < .001 for both). Unlike in women, there was no consistent improvement in prediction for models using HbA_1c compared with models using a dichotomous diabetes term in men (NRI of −2.3% [P = .37] for ATP III and 2.8% [P = .04] for RRS).

Limiting to a subset of 255 of the diabetic PHS II and WHS participants matched by 5-year age groups (55-59 years, 60-64 years, and 65-70 years) yielded similar results to the overall analysis. The distribution of predicted risk for the age-matched subset is shown in the eFigure. While the distributions in the subset are more similar than those shown overall (Figure), the percentage of PHS II participants with predicted 10-year CVD risk at least 20% remains higher than that in the WHS. Ten-fold cross-validations also produced similar results to the primary analysis (eg, NRI of 8.2% for ATP III with HbA_1c compared with diabetes as a high-risk equivalent for men vs 26.2% for women). Using direct estimates of 10-year risk also produced similar results (data not shown).

We found that in these large population-based cohorts of both men and women, presence of diabetes alone did not confer a 10-year risk of CVD higher than 20%, and measurement of HbA_1c level in diabetic subjects improved risk prediction compared with classification as cardiovascular risk equivalent. We found that the improvement in prediction was stronger in the WHS, where the use of HbA_1c levels improved prediction when compared with classification as a cardiovascular risk equivalent and when compared with models that included a dichotomous term for diabetes.

These results are consistent with previously published studies suggesting that not all diabetic patients are at high risk of future vascular events.^5,6 One response to the range of risk in diabetic patients has been the development of specialized risk models, such as the UK Prospective Diabetes Study (UKPDS) model,¹⁷ or the use of models with traditional cardiovascular risk factors and a dichotomous term for diabetes diagnosis such as the Atherosclerosis Risk In Communities (ARIC) model.¹⁸ We chose to use a hybrid approach that incorporates an HbA_1c term only in the diabetic individuals to generate a single model that can be used in both diabetic and nondiabetic populations. We were also able to show that in the WHS, the model with the HbA_1c term improved prediction over the model with the dichotomous term. However, further research in populations with both a larger range of HbA_1c values and additional clinical variables such as duration of diabetes and age at onset will be necessary to derive and test an optimal model for CVD prediction in individuals with and without diabetes. The other limitation of a single model as opposed to separate models for diabetic and nondiabetic populations is calibration. We saw a lack of calibration in the middle-risk categories in both the WHS and PHS II. This is consistent with previous reports of underestimation of absolute risk using nonspecialized models.¹⁹ However, as might be expected, the lowest-risk group of women appears to be reasonably calibrated.

The differences in results between the 2 cohorts are worthy of note and are likely attributable at least in part to the increase in CVD risk with age as well as the delayed risk in women. In the cohorts included in this study, the men were older at baseline with higher rates of CVD, so we were unable to compare populations of men and women with similar overall CVD risk. Even in the age-matched subset, the differences in prediction persisted. There were also differences in the estimated effects of HbA_1c between WHS and PHS II, with a larger effect observed in the WHS. This is consistent with prior studies showing a stronger relative risk of CVD for diabetes in women than in men.^20,21 Given these differences, further replication is needed before the results can be generalized. However, our findings suggest that the improvement in CVD risk prediction, and possibly calibration, obtained with adding HbA_1c levels is highest in lower-risk populations. The current high-risk equivalent classification confines the potential for reclassification to movement into lower-risk categories. Thus, the improved performance in low-risk populations is due to the reclassification of people to those lower risk categories, while retaining the high-risk classification for the subset at high risk. Our analysis demonstrates the potential for improvement in prediction by incorporating the full range of predicted risks.

This study draws on a large, prospective sample of both men and women for generation of the models with extensive follow-up. While the group of diabetic participants may be viewed as small, there were a substantial number of events in the both the diabetic men and women during follow-up. Diabetes information was collected by self-report but has been shown to be highly valid in similar populations of health professionals.^22,23 Treating diabetic individuals earlier in life may improve later-life risk through reducing cumulative exposure to risk factors, resulting in effects that may not be evident in studies of 10-year risk. Other timescales of risk may also be important in evaluating treatment decisions and were not explored in this analysis, including lifetime risk of CVD, which has been shown to be higher for the diabetic population.²⁴ Lifetime risk might also provide additional comparability across populations.

In conclusion, we found improvements in prediction in both men and women with the use of HbA_1c level in diabetic subjects compared with classification of diabetes as a cardiovascular risk equivalent. These results may be particularly helpful in light of current discussion around treatment choices for diabetic patients for prevention of CVD, including use of statins²⁵ and aspirin.²⁶ Our results suggest that the use of HbA_1c levels as part of overall CVD risk scores may improve predictive ability in diabetic patients, whose HbA_1c levels are routinely measured in clinical practice.

Correspondence: Nina P. Paynter, PhD, Brigham and Women's Hospital, Division of Preventive Medicine, 900 Commonwealth Ave, Boston, MA 02215 (npaynter@partners.org).

Accepted for Publication: April 8, 2011.

Published Online: July 25, 2011. doi:10.1001/archinternmed.2011.351

Author Contributions: Dr Paynter had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Paynter, Mazer, Ridker, and Cook. Acquisition of data: Paynter, Gaziano, Ridker, and Cook. Analysis and interpretation of data: Paynter, Mazer, Pradhan, Gaziano, and Cook. Drafting of the manuscript: Paynter. Critical revision of the manuscript for important intellectual content: Paynter, Mazer, Pradhan, Gaziano, Ridker, and Cook. Statistical analysis: Paynter and Cook. Obtained funding: Paynter, Mazer, and Ridker. Administrative, technical, and material support: Gaziano, Ridker, and Cook. Study supervision: Gaziano, Ridker, and Cook.

Funding/Support: This work was carried out with support from F. Hoffmann-La Roche Ltd; grants HL 043851, HL 080467, and CA 047988 for the Women's Health Study and grants CA 97193, CA 34944, CA 40360, HL 26490, and HL 34595 for the Physician's Health Study II from the National Heart Lung and Blood Institute and National Cancer Institute, Bethesda, Maryland; and the Donald W. Reynolds Foundation, Las Vegas, Nevada.

Role of the Sponsors: The funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation or review of the manuscript.