Receiver operating characteristic curves for the Framingham Risk Score (fifths) and number of metabolic abnormalities for prediction of coronary heart disease (CHD) events and type 2 diabetes mellitus over 10 and 20 years in men with no history of CHD, stroke, or diabetes.
Wannamethee SG, Shaper AG, Lennon L, Morris RW. Metabolic Syndrome vs Framingham Risk Score for Prediction of Coronary Heart Disease, Stroke, and Type 2 Diabetes Mellitus. Arch Intern Med. 2005;165(22):2644-2650. doi:10.1001/archinte.165.22.2644
Copyright 2005 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2005
We sought to compare metabolic syndrome (MetS) with the Framingham Risk Score (FRS) as predictors of coronary heart disease (CHD), stroke, and type 2 diabetes mellitus (DM2) in middle-aged men.
A prospective study of 5128 men aged 40 to 59 years with no history of cardiovascular disease (CVD) (CHD or stroke) or DM2 drawn from general practices in 24 British towns and observed for 20 years. Metabolic syndrome was defined as the presence of 3 or more metabolic abnormalities based on modified National Cholesterol Education Program criteria.
Men with MetS at baseline (26%) showed significantly higher relative risk (RR) than men without MetS of developing CHD (RR, 1.64; 95% confidence interval [CI], 1.41-1.90), stroke (RR, 1.61 95% CI, 1.26-2.06), and DM2 (RR, 3.57; 95% CI, 2.83-4.50). The probability of developing CVD or DM2 over 20 years increased from 11.9% in those with no abnormalities to 31.2% in those with 3 abnormalities to 40.8% in those with 4 or 5 abnormalities. The FRS was a better predictor of CHD and stroke than MetS but was less predictive of DM2. Areas under the receiver-operating characteristic curves for FRS vs the number of metabolic abnormalities were 0.68 vs 0.59 for CHD, 0.60 vs 0.70 for DM2, and 0.66 vs 0.55 for stroke (P<.001 for all).
Presence of MetS is a significant predictor of CVD and DM2 but is a stronger predictor of DM2 than of CHD. Although MetS does not predict CHD as well as the FRS, it serves well as a simple clinical tool for identifying high-risk subjects predisposed to CVD or DM2.
Metabolic syndrome (MetS), defined as a cluster of risk factors, including obesity, hypertension, hyperglycemia, and dyslipidemia,1 is becoming increasingly common because of the increasing prevalence of obesity.2 The World Health Organization (WHO) Consultation in 1998 provided a provisional definition for MetS3 and revised it in 1999.4 The Third Report of the National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III)5 presented a simpler definition of MetS intended for the identification of people with the syndrome in clinical practice.6 Whether defined on the basis of NCEP criteria7- 21 or WHO criteria,8- 10,13,18,22,23 MetS is associated with significantly increased risk of developing type 2 diabetes mellitus (DM2) and cardiovascular disease (CVD) and has been widely promoted as a means of identifying patients for lifestyle intervention to reduce risk factors and incident disease, in particular CVD.24,25
While it has not been established whether MetS is a better predictor of coronary heart disease (CHD) risk than other risk assessments such as the Framingham Risk Score (FRS), 3 recent US studies indicate that MetS is inferior to the FRS in predicting CHD.19,21,24 Stern et al19 conclude that MetS is inferior to established rules for the prediction of either DM2 or CVD, and the use of MetS as a focus of screening for metabolic risk modification has become questionable. Nonetheless, MetS predicts both CVD and DM2, and since previous studies have not shown whether the FRS is a good predictor of DM2,25 MetS may be a good all-purpose predictor compared with the FRS. Furthermore, there have been few prospective studies on the relationship between MetS and risk of stroke. We have assessed the prospective relationship between MetS and risk of CHD, stroke, and DM2 using a modified NCEP definition and have compared MetS with the FRS in predicting risk over 20 years follow-up.
The British Regional Heart Study26 is a large prospective study of CVD comprising 7735 men aged 40 to 59 years drawn from general practices in each of 24 towns in England, Wales, and Scotland. The criteria for selecting the towns, the general practices, and the subjects as well as the methods of data collection have been reported.26 Research nurses administered a standard questionnaire that included questions on lifestyle and medical history. The men were asked to recall a physician’s diagnosis of CHD (angina or myocardial infarction), stroke, and DM2. Physical measurements including height and weight were made, and venous nonfasting blood samples were obtained for measurement of biochemical and hematologic characteristics. Triglyceride measurements were only available for men in the seventh to 24th towns visited (n = 5663).
Details of smoking history, alcohol use, physical activity, social class, blood pressure (BP), and blood lipid measurements have been reported.26- 28 Heavy drinking was defined as 6 alcoholic drinks daily or on each of most days in the week (1 drink = 10 g of alcohol). A physical activity (exercise) score was derived for each man, and the men were initially grouped into 6 broad categories based on their total score (none, occasional, light, moderate, moderately vigorous, or vigorous).27 Men who reported no or occasional physical activity were defined as “inactive.” Adjustments were made for the marked diurnal variation in serum triglyceride concentration.29 Men with recall of a physician diagnosis of CHD or stroke, known diabetes at screening, and/or asymptomatic hyperglycemia (glucose level, ≥200 mg/dL [≥11.1 mmol/L]) were excluded (n = 441). Data on MetS were not available in 94 men. After these exclusions, data were available for a group of 5128 men, the subjects of this study.
Metabolic syndrome as defined by NCEP5 comprises 3 or more of the following: (1) fasting plasma glucose level of at least 110 mg/dL (6.1 mmol/L); (2) serum triglyceride level of at least 150 mg/dL (1.7 mmol/l); (3) serum high-density lipoprotein cholesterol level lower than 40 mg/dL (1.04 mmol/L); (4) BP of at least 130/85 mm Hg or controlled with antihypertensive treatment; and/or (5) waist circumference of more than 102 cm.5
Waist circumference was not measured in the present study. However, in a subset of 3000 of these men whose waist circumference and body mass index (BMI) (calculated as weight in kilograms divided by the square of height in meters) were measured 20 years after initial screening, regression analysis showed a BMI of 28.9 to be equivalent to a waist circumference of 102 cm, similar to analysis using data from a large US survey.30 We have therefore used a BMI of 28.8 or higher as a proxy for abdominal adiposity in conformity with other studies that have used BMI to replace waist circumference.7
Since triglyceride levels were based on nonfasting measurements, which are on average 20% to 30% higher than fasting levels,31 we have used a higher threshold and defined high triglyceride level as greater than or equal to 200 mg/dL (≥2.28 mmol/L). We retained the cutoff level for blood glucose because this represents 20% of the men, and previous reports in this cohort have shown such levels to be associated with significantly increased risk of CVD and DM2.32 Thus, in this study, MetS includes any 3 of the following: (1) hypertension (systolic BP ≥130 mm Hg, diastolic BP ≥85 mm Hg, or BP controlled with antihypertensive treatment); (2) high blood glucose level (≥110 mg/dL [≥6.1 mmol/L]); (3) hypertriglyceridemia (triglyceride level, ≥200 mg/dL [≥2.28 mmol/L]); (4) low level of high-density lipoprotein cholesterol (<40 mg/dL [<1.04 mmol/L]); and/or (5) elevated BMI (≥28.8). However, in a subsidiary analysis, we used higher thresholds for the criteria of high blood glucose level: greater than or equal to 126 mg/dL (>7.0 mmol/L) and greater than or equal to 140 mg/dL (≥7.8 mmol/L).
The FRS for CHD over 10 years was calculated for each man,33,34 and the men were categorized according to quintiles of risk score.
All men were evaluated on follow-up for all-cause mortality, cardiovascular morbidity, and development of DM2 from the initial screening in January 1978–July 1980 to June 2000, a mean period of 21.3 years; follow-up was achieved for 99% of the cohort.35 This analysis is based on a 20-year follow-up for each man. Criteria for accepting a diagnosis of nonfatal myocardial infarction, stroke, and death have been reported, as has the method for ascertaining new cases of DM2.27,36 A combined end point was also used, defined as the confirmed development of a heart attack, stroke, or DM2.
The Cox proportional hazards model was used to assess the adjusted relative risks (RRs). Receiver-operating characteristic (ROC) curves and their respective areas under the curve (AUCs) were used to compare the ability of the FRS and the number of metabolic abnormalities to predict CHD and DM2.37 Tests for differences between the curves were performed using Stata software, version 9.0 (Stata Corp, College Station, Tex).38
During the 20-year follow-up, there were 763 major CHD events (fatal CHD and nonfatal myocardial infarction), 291 major stroke events (fatal and nonfatal), and 299 cases of DM2 in the 5128 men with no history of CHD, stroke, or diabetes. Table 1 lists the baseline characteristics in these men. Metabolic syndrome was present in over a quarter of the men (26.0%); 53.9% of men who developed DM2 had MetS at baseline compared with 35.5% of the men developing CHD and 34.7% of the men who developed stroke. Metabolic syndrome was associated with a significant increase in CHD, stroke, and particularly DM2 after adjustment for age, social class, smoking status, physical activity level, and alcohol use (Table 2). Risk of CHD and DM2 increased significantly with increasing number of components of MetS (Table 3). The relationship with stroke was less consistent.
When the higher thresholds for nonfasting glucose were used for the definition of high blood glucose level in MetS (ie, ≥126 mg/dL (≥7.0 mmol/L) and ≥140 mg/dL (≥7.8 mmol/L), this slightly attenuated the RR of CHD associated with MetS (RR, 1.57; 95% confidence interval [CI], 1.34-1.83 and RR, 1.54; 95% CI, 1.31-1.80, respectively), but absolute cumulative incidence rates in men with MetS remained unchanged. Similar attenuation was seen for the DM2 RR, but risk was still over 3-fold (RR, 3.22; 95% CI, 2.55-4.06 using ≥126 mg/dL [≥7.0 mmol/L] and RR, 3.05; 95% CI, 2.41-3.85 using ≥140 mg/dL [≥7.8 mmol/L]), with little change in absolute cumulative incidence rates.
We compared the predictive power of number of metabolic abnormalities with the FRS over 10 and 20 years. The subjects in the top quintile of the FRS showed a higher probability of developing CHD than even those with 4 or more abnormalities for both 10- and 20-year follow-up (Table 4). The ROC curves for the FRS and the number of metabolic abnormalities in predicting CHD and DM2 are shown in the Figure. The FRS was a significantly better predictor of CHD than was the number of metabolic abnormalities, as indicated by the AUC of the ROC at both 10 and 20 years, but it was less predictive of DM2 (Table 4) (P<.001 for all differences). Metabolic syndrome provided no additional predictive value for CHD when the FRS was included in the multivariate model but remained strongly associated with DM2 (adjusted RR, 1.14; 95% CI, 0.96-1.35 for CHD and adjusted RR, 2.98; 95% CI, 2.28-3.88 for DM2). Metabolic syndrome had a higher sensitivity for DM2 than for CHD at both 10 and 20 years’ follow-up (Table 4). For a given specificity (fixed at the specificity levels for MetS) the FRS was more sensitive than MetS in identifying CHD cases for both 10- and 20-year events but less sensitive than MetS in identifying DM2. Further analysis indicated that if obesity and high glucose levels, which are not included in the FRS, were excluded from the syndrome score, the sensitivity for DM2 would be reduced to 32.1%.
The FRS was also a significantly better discriminator of stroke than the number of metabolic abnormalities (AUC, 0.71; 95% CI, 0.65-0.77 vs AUC, 0.54; 95% CI, 0.48-0.60 for 10 years and AUC, 0.66; 95% CI, 0.62-0.70 vs AUC, 0.55; 95% CI, 0.51-0.59 for 20 years; P<.001 for all).
We also repeated the ROC analysis using different thresholds for blood glucose level. The AUC hardly altered when we used glucose level thresholds of 126 mg/dL (7.0 mmol/L) and 140 mg/dL (7.8 mmol/L) compared with 110 mg/dL (6.1 mmol/L) in our definition of MetS (data not shown).
Although the FRS was a better discriminator of the combined cases (CHD, stroke, or diabetes) over 20 years than MetS (AUC, 0.67; 95% CI, 0.65-0.69 vs AUC, 0.61; 95% CI, 0.59-0.63; P<.001; sensitivity 44.1% vs 37.3% for a fixed specificity of 77.6%), the probability of developing heart attacks, stroke, or DM2 over 20 years increased markedly with increasing number of components from 11.9% in those with none to 19.0% in those with 1; 24.9% in those with 2; 31.2% in those with 3; and 40.8% in those with 4 or more abnormalities. The absolute probability was comparable to the probability associated with quintiles of the FRS: 10.0%, 14.9%, 22.9%, 30.8%, and 40.8% for the 5 FRS groups.
In this study of middle-aged British men with no history of CVD or diabetes, the risk of CHD and DM2 increased significantly and progressively with increasing number of metabolic abnormalities, as has been reported in other studies7,15,17,20; less consistent relationships were seen with stroke risk. Metabolic syndrome, as defined by the modified NCEP criteria,5 was present in a quarter of the men (26%) and was associated with significantly increased risk of CHD, stroke, and DM2, in agreement with results from other studies.7- 19 There have been few prospective studies on the relationship between MetS using NCEP criteria and stroke. A recent report from the Atherosclerosis Risk in Communities (ARIC) study21 showed a nonsignificant increased risk of stroke in men with MetS. However, in this study, MetS was shown to be associated with significant increased risk of stroke, consistent with the evidence that insulin resistance is associated with increased risk of stroke in subjects without diabetes.39,40
Metabolic syndrome was less effective at predicting CHD than the FRS, which is consistent with the 2 recent US reports showing the syndrome to be less predictive of CHD then the FRS.19,21 In the recent report of the National Heart, Lung, and Blood Institute and American Heart Association conference proceedings, analysis of the Framingham data indicated that no advantage is gained in risk assessment by adding the components of MetS to the FRS.24
It is not surprising that the FRS is a better predictor of CHD. Prediction criteria based on MetS alone do not include several well-established risk factors for CHD such as serum total cholesterol level and smoking status. However, MetS was a far stronger predictor of DM2 than of CHD, as observed in previous studies,7,11 and was a better predictor of DM2 than was FRS. This is likely due to the inclusion of established risk factors for diabetes such as waist circumference (BMI) and high blood glucose levels (short of diabetes), which are not included in the FRS. Exclusion of obesity and high glucose level from the syndrome score reduced the sensitivity of MetS for DM2 from over 50% to just over 30%.
Since CVD is more common than DM2, the FRS was more sensitive in identifying overall events (CHD, stroke, or diabetes), but absolute risk in those with 3 or more metabolic abnormalities was very high (30%-40% chance of developing CVD or DM2, comparable to the absolute incidence rates in the top 2 quintiles of the FRS distribution). Thus, MetS is a simple tool that identifies subjects predisposed to either CVD or DM2.
Reliance on documented physician-diagnosed cases of DM2 is specific but not sensitive and inevitably results in underascertainment of cases. However, much of the error due to underascertainment of cases will apply equally when assessing the predictive power of the FRS and MetS. Underascertainment is unlikely to be related to the distribution of the FRS as opposed to MetS and should therefore not invalidate comparisons.
Although measurements were based on nonfasting measurement, which particularly affects triglyceride and glucose levels, our findings of a 1.64-fold increase in risk of CHD and a more than 3-fold increase in risk of DM2 are comparable to those found in the West of Scotland Coronary Prevention study7 (1.76 vs 3.51 for CHD and diabetes). Although we used the same cutoff point for fasting glucose measurement (≥110 mg/dL [≥6.1 mmol/L]), which may result in a higher proportion of men classified as having high glucose level, our modified definition yielded a prevalence of MetS similar to that found in the Scottish study (26%).7 When higher thresholds for nonfasting glucose measurement are used, this slightly attenuates the RR of CHD and DM2 associated with MetS, but absolute cumulative incidence rates in men with MetS remain unchanged. Moreover, the AUC hardly altered when we used threshold glucose levels of 126 mg/dL (7.0 mmol/L) and 140 mg/dL (7.8 mmol/L) compared with 110 mg/dL (6.1 mmol/L) in our definition of MetS. So the difference between the FRS and MetS cannot be explained by the cutoff points we used for glucose. It is therefore unlikely that the weaker effect of the syndrome on CHD risk is due to nonfasting measures.
Finally, because waist circumference was not available, we used BMI as a proxy to classify individuals with obesity, similar to other studies.7,17,20 Despite these limitations our findings are consistent with those of previous reports showing MetS to be a stronger predictor of diabetes than CHD7,11 and with recent reports from US studies showing the FRS to be a better predictor of CHD than MetS.19,21
In conclusion, MetS (NCEP definition5) is associated with a significant increase in risk of CHD, stroke, and DM2 and is a far stronger predictor of DM2 than of CHD and stroke. Although it is inferior to the FRS in predicting CHD, MetS identifies people who are predisposed to either CVD or DM2 and may serve as a simple clinical approach to identifying patients for clinical intervention to reduce CVD and DM2 risk.
Correspondence: S. Goya Wannamethee, PhD, Department of Primary Care and Population Sciences, Royal Free and University College Medical School, Rowland Hill Street, London NW3 2PF, England (email@example.com).
Accepted for Publication: August 19, 2005.
Financial Disclosure: None.
Disclaimer: The British Regional Heart Study is a British Heart Foundation Research Group and receives support from the Department of Health (England). However, the views expressed in this publication are those of the authors and not necessarily those of the Department of Health (England).