Assessing and Refining Myocardial Infarction Risk Estimation Among Patients With Human Immunodeficiency Virus: A Study by the Centers for AIDS Research Network of Integrated Clinical Systems

Key Points

Question  How well do the American College of Cardiology/American Heart Association pooled cohort equations and new models incorporating human immunodeficiency virus (HIV)–specific variables predict myocardial infarction risk for HIV-infected persons living in the United States?

Findings  In a large multicenter HIV cohort study with good myocardial infarction adjudication, the pooled cohort equations discriminated myocardial infarction risk adequately and exhibited moderate calibration, particularly among white men. Two data-derived models incorporating HIV-specific factors exhibited similar discrimination as the PCEs but worse calibration.

Meaning  The pooled cohort equations were moderately calibrated among HIV-infected white men in a multicenter HIV clinical cohort and adding HIV-specific factors did not improve their ability to predict myocardial infarction.

Importance  Persons with human immunodeficiency virus (HIV) that is treated with antiretroviral therapy have improved longevity but face an elevated risk of myocardial infarction (MI) due to common MI risk factors and HIV-specific factors. Despite these elevated MI rates, optimal methods to predict MI risks for HIV-infected persons remain unclear.

Objective  To determine the extent to which existing and de novo estimation tools predict MI in a multicenter HIV cohort with rigorous MI adjudication.

Design, Setting, and Participants  We evaluated the performance of standard of care and 2 new data-derived MI risk estimation models in 5 Centers for AIDS Research Network of Integrated Clinical Systems sites across the United States where a multicenter clinical prospective cohort of 19 829 HIV-infected adults received care in inpatient and outpatient settings since 1995. The new risk estimation models were validated in a separate cohort from the derivation cohort.

Exposures  Traditional cardiovascular risk factors, HIV viral load, CD4 lymphocyte count, statin use, antihypertensive use, and antiretroviral medication use were used to calculate predicted event rates.

Main Outcomes and Measures  We observed MI rates over the course of follow-up that were scaled to 10 years using the Greenwood-Nam-D’Agostino Kaplan-Meier approach to account for dropout and loss to follow-up before 10 years.

Results  Of the 11 288 patients with complete baseline data, 6904 were white and 9250 were men. Myocardial infarction rates were higher among black men (6.9 per 1000 person-years) and black women (7.2 per 1000 person-years) than white men (4.4 per 1000 person-years) and white women (3.3 per 1000 person-years), older participants (7.5 vs 2.2 MI per 1000 person-years for adults 40 years and older vs < 40 years old at study entry, respectively), and participants who were not virally suppressed (6.3 vs 4.7 per 1000 person-years for participants with and without detectable viral load, respectively). The 2013 Pooled Cohort Equations, which predict composite rates of MI and stroke, adequately discriminated MI risk (Harrell C statistic = 0.75; 95% CI, 0.71-0.78). Two data-derived models incorporating HIV-specific covariates exhibited weak calibration in a validation sample and did not discriminate risk any better (Harrell C statistic = 0.72; 95% CI, 0.67-0.78 and 0.73; 95% CI, 0.68-0.79) than the Pooled Cohort Equations. The Pooled Cohort Equations were moderately calibrated in the Centers for AIDS Research Network of Clinical Systems but predicted consistently lower MI rates.

Conclusions and Relevance  The Pooled Cohort Equations discriminated MI risk and were moderately calibrated in this multicenter HIV cohort. Adding HIV-specific factors did not improve model performance. As HIV-infected cohorts capture and assess MI and stroke outcomes, researchers should revisit the performance of risk estimation tools.

Longevity among persons with human immunodeficiency virus (HIV) has increased dramatically owing to effective antiretroviral therapy (ART).^1-4 There are more than 1.2 million and 35 million HIV-infected (HIV⁺) adults in the United States and worldwide, respectively. These numbers are projected to grow and the aging HIV⁺ population is increasingly at risk for noncommunicable disease–related morbidity and mortality.^5-7 Human immunodeficiency virus–positive persons have nearly twice the risk for myocardial infarction (MI) and greater risks for heart failure and sudden death compared with uninfected persons.^8-19

The paradigm for preventing atherosclerotic cardiovascular disease (ASCVD)—which consists of nonfatal MI, coronary heart disease death, and stroke—is based on the principle that the intensity of prevention efforts should match patients’ absolute risks. This requires accurate prediction of ASCVD risks. Although some studies have evaluated ASCVD risk estimation models in HIV⁺ patients, these studies have generally been small and suboptimally assessed end points.^20-23 One of the larger studies to date used a predominantly white cohort to derive the Data Collection on Adverse Events of Anti-HIV Drugs risk score, which incorporates specific antiretroviral agents but not race/ethnicity.²³

The American College of Cardiology (ACC) and American Heart Association (AHA) 2013 Guideline on the Assessment of Cardiovascular Risk developed the Pooled Cohort Equations (PCEs) for 10-year ASCVD risk prediction to assist with ASCVD prevention decision making.^24,25 These equations were derived from more diverse population-based cohorts than prior ASCVD risk prediction tools.²⁶ The PCEs have not been evaluated in a large multicenter HIV cohort. Thus, the objectives of this study were to (1) evaluate PCE performance; (2) create data-derived, HIV-specific MI risk estimation models in a diverse, multicenter HIV cohort with rigorous MI adjudication; and (3) externally validate these models in a separate sample.

We used the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) cohort for all analyses.²⁷ The CNICS is a multicenter clinical cohort composed of HIV⁺ adults aged 18 years or older receiving HIV care at 1 of 8 Centers for AIDS Research clinics in the United States. Institutional review board approval was granted by Northwestern University and patient consent was waived because the study was a retrospective analysis of previously collected data. Participants’ initial enrollment was 1995 or later depending on the site, enrollment and follow-up are continuous (the most recently adjudicated MIs are from July 2015), and there is a detailed description of the rigorous central adjudication for MI within CNICS.²⁷

For analyses evaluating PCE model performance, we included all CNICS participants from the 5 sites participating in MI adjudication at the time of analysis who were free of MI at baseline and had baseline data for ASCVD risk calculator variables (age, sex, race/ethnicity, total cholesterol level, high-density lipoprotein cholesterol level, smoking status, diabetes, systolic blood pressure, and antihypertensive treatment), HIV viral load, and CD4 T-cell count. We calculated predicted ASCVD risk by inserting individual-level baseline data into the PCEs. To account for variable follow-up time, we used a Kaplan-Meier–based approach with the Greenwood-Nam-D’Agostino (GND) goodness-of-fit tests to assess calibration²⁸ instead of older approaches less appropriate for a clinical cohort.²⁶ Study follow-up started 6 months after the initial CNICS study visit or the initial MI surveillance date. The outcome of interest was time to MI, and study participants were censored at 10 years if they died or were lost to follow-up. We performed sensitivity analyses of the PCEs in which we excluded all participants younger than 40 years at baseline (because the PCEs were originally intended for ASCVD risk assessment in those aged 40 to 79 years) and all participants with histories of cocaine use, given the association of cocaine with MI independent of cardiovascular disease risk factors.²⁹

We evaluated PCE discrimination using the Harrell C statistic, and less than 0.70, 0.70 to 0.80, and greater than 0.80 were considered inadequate, acceptable, and excellent discrimination levels, respectively.³⁰ Model calibration was evaluated by the GND approach.^28,31 We analyzed our model’s calibration on 4 levels using a hierarchy of increasing strictness (mean, weak, moderate, or strong calibration).²⁸ The first level (mean calibration or calibration in the large) measures whether the overall average predicted risk equals the observed event rate.²⁸ The second level (weak calibration) evaluates the slope and intercept of the models. There is better calibration between the model and the observed date the closer the slope is to 1 and the closer the intercept is to 0 because these 2 measures should map onto each other.²⁸ The third level evaluates whether observed and predicted event rates are equal for patient groups within the same general predicted risk strata; if so, the model is considered moderately calibrated.²⁸ The fourth level (strong calibration) requires predicted and observed event rates to correspond for every covariate pattern and is not possible when continuous predictors are used, or in sample sizes like we have because of the number of possible combinations.²⁸ We used the GND method to assess model calibration for the overall cohort and separately by race and sex.

After analyzing the PCEs, we created 2 risk scores using data-derived coefficients, aiming to improve on the ASCVD risk score. We included all variables from the ASCVD risk score, statin use, and HIV-specific variables (HIV viral load, CD4 lymphocyte count, antiretroviral therapy, and protease inhibitor use) as possible covariates. Single imputation was used for baseline variables (all with <50% missingness). We removed 1 site, the University of Alabama–Birmingham, from model derivation analyses to use it as a validation sample because of its demographic similarities to the US HIV⁺ population(eMethods in the Supplement).³² Data from the other 4 CNICS sites were randomly split into a training sample (to fit the models) and a holdout sample (to evaluate the models). The first new risk score (HIVMI-1) used lasso and ridge regression, while the second (HIVMI-2) used an average of Cox models to select variables and determine coefficients. Scores were evaluated in the validation cohort by Harrell C statistic and the aforementioned calibration hierarchy.²⁸

We also considered simpler methods than creating new data-derived risk scores. One supplementary analysis added 10 years to each participant’s baseline PCE risk estimates to account for earlier MI occurrence in HIV, while the other multiplied MI rates by a data-derived multiplier.

Although the differentiation between type 1 MIs, which are caused by plaque rupture and thrombosis, and type 2 MIs, which result from supply-demand mismatch, is a strength of CNICS, we did not differentiate between MI types for this analysis because existing cardiovascular risk prediction models (including the PCEs) do not.³³

Characteristics of participants with complete baseline covariate measurements (n=11 288; 247 persons with MI over a mean follow-up of 4.1 years) are shown in Table 1. The cohort was relatively young at the baseline and most (70%) were taking antiretroviral therapy. Mean 10-year ASCVD risk estimates based on the PCEs were the highest for black men and the lowest for white women. Observed MI rates across demographic and clinical strata are shown in the eTable in the Supplement. Myocardial infarction rates were higher among black men and women, participants 40 years or older at baseline, and participants with greater HIV viral loads and lower CD4 lymphocyte counts. Racial differences in MI rates were largely driven by more type 2 MIs occurring among black participants; type 1 MI rates were identical for black and white men (3.1/1000 person-years). Participants with worse HIV viral control and immunologic suppression had elevated total and type 1 MI rates.

The PCEs discriminated ASCVD risk adequately overall (Harrell C statistic = 0.75; 95% CI, 0.71-0.78) and in most race and sex combinations (Table 2). The PCEs were relatively well fitted in the overall cohort (Table 2 and Figure 1; slope = 0.815; intercept = 0.0015; GND test statistic = 13.1; P = .16), particularly for white men (Table 2 and Figure 2A; slope = 0.857; intercept = 0.009; GND test statistic = 6.4; P = .50), meeting GND framework criteria for moderate calibration. However, the PCEs were not as well fitted for black men, black women, or white women (Table 2 and Figure 2B-D). Observed MI rates consistently exceeded predicted rates across the low-to-moderate risk spectrum (<10% predicted 10-year ASCVD risk), and this underprediction was particularly marked among black men (Figure 2B) and women (Figure 2D). In a sensitivity analysis excluding the 987 participants with histories of cocaine use, PCE performance was unchanged (data not shown). In separate supplementary analyses in which we added 10 years to all study participants’ baseline PCE inputs and multiplied MI rates by a multiplier, discrimination was similar to the PCEs.

To create and derive new HIV-specific risk scores, we expanded the cohort from the other analyses to include all study participants (n = 19 829 [derivation sample that was split randomly into training and holdout samples, 15 849; validation sample, 3980]). Of the 15 849 participants not in the validation sample, 353 experienced MIs during a mean follow-up of 4.8 years.

The HIVMI-1 score included several HIV-specific variables and traditional ASCVD risk factors and adequately discriminated (Harrell C statistic = 0.72; 95% CI, 0.67-0.78) in the validation sample (Figure 3A). The HIVMI-2 score also incorporated several HIV-specific variables in addition to traditional ASCVD risk factors and adequately discriminated in the validation sample (Harrell C statistic = 0.73; 95% CI, 0.68-0.79) (Figure 3B). Both HIVMI-1 (slope = 0.40; intercept = 0.01; GND test statistic = 102.5; P < .001) and HIVMI-2 (slope = 0.50; intercept = 0.01; GND test statistic = 84.2; P < .001) exhibited substantially worse calibration than the PCEs. In a sensitivity analysis in the validation cohort only including participants with complete covariates, there was little change in HIVMI-1 and HIVMI-2 performance.

The ACC/AHA PCEs exhibited acceptable discrimination and moderate calibration for predicting MIs among HIV⁺ individuals in a multisite US HIV cohort with adjudicated MIs. Neither of the 2 data-derived models incorporating HIV-specific variables (HIVMI-I and HIVMI-2 scores) improved MI risk prediction in this population for several potential reasons. Traditional ASCVD risk factors are essential for risk estimation in general and remain important in an HIV setting; thus, the bar is high for new risk prediction tools incorporating new factors to predict ASCVD events. Although we know HIV-related factors (eg, HIV viral load and CD4 T-cell count) are associated with ASCVD, we know little about how strongly these factors or other HIV-associated biomarkers are associated with ASCVD. Researchers are interested in risk estimation models incorporating novel HIV-specific factors and biomarkers that reflect HIV-related ASCVD risks, and these may provide a future use for biorepository samples from HIV cohorts.

Pooled Cohort Equation calibration was generally acceptable for the whole cohort. The whole cohort showed acceptable calibration-in-the-large, a slope near 1 between observed and expected, and an intercept near 0, and the GND goodness-of-fit test suggested moderate calibration. However, this high calibration level primarily applied to white men, as other sexes and races showed worse evidence of calibration. The slope between observed and expected events was particularly poor for women. The mismatch between the event rates predicted by the PCEs and observed MIs in CNICS was apparent for groups with risks at or near thresholds at which statins are often recommended in the general population (5%-10% 10-year predicted ASCVD risk). Participants for whom the PCEs predicted less than 10% 10-year ASCVD risk had consistently higher-than-predicted MI rates, while participants with greater than or equal to 10% predicted risk generally had lower-than-predicted MI rates.

The consistent underprediction of the PCEs over the low-to-moderate risk spectrum may have direct clinical implications. Current ACC/AHA guidelines recommend consideration of statin therapy for patients at greater than or equal to 7.5% (and in some cases, ≥5%) predicted 10-year ASCVD risk. Thus, model calibration is more clinically meaningful in the low-to-moderate risk spectrum—near decision thresholds for statin use—than in the highest-risk groups, in which statin therapy’s benefits most clearly outweigh the risks. Given our finding that HIV⁺ patients at less than 10% predicted 10-year risk (by the PCEs) had consistently greater-than-expected event rates, it may be reasonable for clinicians to use the PCEs as a baseline gauge of what their patients’ lowest predicted ASCVD risks are. For instance, if an HIV⁺ patient’s predicted 10-year ASCVD risk is 8%, our findings suggest that the patient’s “true” risk is at least 8% and perhaps greater. This risk of at least 8% could guide clinician-patient consideration of cardiovascular disease–preventive therapy, keeping in mind potential interactions of these therapies (particularly statins) with antiretrovial therapy medications.³⁴

There are several potential reasons for the overprediction of MI risk in the highest-risk groups. Patients with poorly controlled HIV may have high predicted MI risk and competing risks for noncardiovascular causes of death. Thus, they may have contributed many person-years at high predicted risk but died of noncardiovascular causes before theoretical MIs would have occurred, leading to overprediction of higher levels of risk. Survival bias may also contribute to this mismatch. Age is a strong contributor to MI risk, and older HIV-infected persons may have an elevated burden of traditional and HIV-related MI risk factors. While these factors would contribute to elevated predicted MI risk for older HIV-infected persons, they also would be likely to have survived longer with HIV and may therefore be likely to have greater care access, social support, or other unmeasured factors that could prevent incident MI. Finally, more patients at higher MI risk were taking statins, which decrease incident MI rates and may have decreased observed MI rates disproportionately in these higher-risk groups.

A caveat to the clinical applicability of these risk estimates is the relatively high number of type 2 MIs among HIV⁺ people in CNICS. Type 2 MIs occur during systemic disease rather than inciting atherothrombotic events and may not be as effectively prevented by atheroprotective therapies such as statins. It is likely that type 2 MIs are more common for HIV⁺ persons than uninfected persons because of their systemic disease (HIV) and susceptibility to states that trigger type 2 MIs, including infection and sepsis. Furthermore, the CNICS MI screening protocol incorporated cardiac biomarkers, which tend toward correctly identifying a high number of type 2 MIs that may not have been identified otherwise. Observed MI rates exceeded PCE-predicted rates most markedly for black men and women. Further, the proportion of MIs that were type 2 was also greater for black men and women than their white counterparts, and it is possible that this excess in type 2 MIs among black people drove much of this apparent underprediction of MI risk. However, distinguishing between types of MIs was beyond the scope of this study because previous risk estimation tools have not differentiated between MI types, and we wanted to assess the performance of the score on “any MI.” Future analyses may compare risk prediction models incorporating all MIs vs only type 1 MIs.

Our findings should be interpreted in the context of their limitations. We were unable to assess stroke because adjudicated stroke data were not yet available in CNICS (or many other large HIV cohorts). We accepted this limitation for several reasons. Statins result in greater absolute risk reductions for MI than stroke in men, who represent most of the HIV⁺ population in the United States. Thus, while stroke risk is incorporated into global ASCVD risk estimates in the PCEs, the primary purpose of statin therapy for men is to prevent MIs.³⁵ It is noteworthy that strokes are a greater proportion of overall ASCVD for women, particularly black women. Thus, the MI rates we observed for women in CNICS may be lower than their overall ASCVD rates. Additionally, CNICS incorporated biomarkers into its MI screening protocol and likely captured more MIs than less-sensitive adjudication protocols in the cohorts from which the PCEs were derived. It is possible that this relative overassessment of MIs and definite underassessment of strokes resulted in ASCVD rates that would have been similar had the methods from these previous cohort studies been used to adjudicate MI and stroke in CNICS.

Another limitation of this study was the relatively short mean follow-up, although many participants did have 10 years or greater of follow-up. We sought to address the relatively low number of people with follow-up to 10 years by using the GND approach.³¹ Another study limitation was that only 5 of 8 CNICS sites were analyzed because of the availability of adjudicated MI data. These limitations were unavoidable and acceptable given the strengths of CNICS as a modern HIV clinical cohort with rigorous MI adjudication. Another complicating factor was the young age of the US HIV⁺ population compared with cohorts used to develop the PCEs. Atherosclerotic cardiovascular disease risks increase with age and separation between people with higher and lower lifetime risks for ASCVD becomes more apparent; thus, discriminating between risk strata may be difficult in younger populations. Finally, few analyzed patients included in our analyses were taking statins. Statin use was more common for those with greater ASCVD risk and may have decreased observed MI rates for higher-risk deciles more than for lower-risk deciles.

Despite these limitations, this study suggests that the 2013 ACC/AHA Risk Estimator’s PCEs perform better than data-derived models incorporating HIV-specific variables at predicting MI risks for HIV⁺ people. As longer-term follow-up data with more ASCVD events become available for HIV⁺ people, studies should reassess whether data-derived, HIV-specific risk estimation models can improve ASCVD risk prediction over the PCEs in this population.

Corresponding Author: Donald M. Lloyd-Jones, MD, ScM, Northwestern University Feinberg School of Medicine, 680 N Lake Shore Dr, Ste 1400, Chicago, IL 60611 (dlj@northwestern.edu).

Accepted for Publication: September 23, 2016.

Published Online: December 21, 2016. doi:10.1001/jamacardio.2016.4494

Author Contributions: Drs Feinstein and Lloyd-Jones had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Feinstein, Drozd, Achenbach, Lloyd-Jones, Crane.

Acquisition, analysis, or interpretation of data: Feinstein, Nance, Drozd, Ning, Delaney, Heckbert, Budoff, Mathews, Kitahata, Saag, Eron, Moore, Achenbach, Crane.

Drafting of the manuscript: Feinstein, Nance.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Feinstein, Nance, Ning, Delaney, Crane.

Administrative, technical, or material support: Feinstein, Drozd, Saag, Eron, Moore, Achenbach, Crane.

Obtained funding: Heckbert, Saag, Moore.

Study supervision: Drozd, Budoff, Achenbach, Lloyd-Jones.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for the Disclosure of Potential Conflict of Interest. Dr Delaney reports receiving grant R01HL126538 from the National Institutes of Health (NIH)/National Heart, Lung, and Blood Institute during the conduct of the study. Drs Heckbert, Matthews, and Moore report grant support from the NIH during the conduct of the study. Dr Budoff reports grant support from the NIH during the conduct of the study and from General Electric outside of the submitted work. Dr Saag reports grant support from the NIH/National Institute of Allergy and Infectious Diseases (NIAID)/Centers for AIDS Research Network of Integrated Clinical Systems (CNICS) (R24) during the conduct of the study; grants from Gilead Sciences, Merck, Janssen, Bristol-Myers Squibb, and ViiV Healthcare paid to his institution for other work; and consulting fees from Gilead Sciences, Merck, and Bristol-Myers Squibb for services as a scientific advisor. Dr Enron reports grant support from the NIH during the conduct of the study and grants and personal fees from Gilead Sciences, Janssen, Bristol-Myers Squibb, and Viiv Healthcare and personal fees from Merck outside of the submitted work. Dr Crane reports grant support from the NIH during the conduct of the study and a grant from the NIH and the Patient-Centered Outcomes Research Center outside of the submitted work. No other disclosures were reported.

Funding/Support: This study was supported by several grants from the NIH (CNICS R24 AI067039, CNICS MI supplement R24S AI067039, National Heart, Lung, and Blood Institute R01 HL126538, University of Washington Center for AIDS Research NIAID grant P30 AI027757, and Third Coast Center for AIDS Research NIAID grant P30AI117943). The study was also supported by grant 16FTF31200010 from the American Heart Association.

Role of the Funder/Sponsor: The NIH and American Heart Association had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We acknowledge all CNICS study personnel and participants for their contributions to this work.