Prognostic Models Derived in PARADIGM-HF and Validated in ATMOSPHERE and the Swedish Heart Failure Registry to Predict Mortality and Morbidity in Chronic Heart Failure | Cardiology | JAMA Cardiology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 1 Year for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Scores (C)
Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 1 Year for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Scores (C)

A, Predicted vs observed risk of primary composite end point at 1 year. B, Predicted vs observed risk of cardiovascular death at 1 year. C, Predicted vs observed risk of all-cause mortality at 1 year.

Figure 2.  Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 2 Years for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Score (C)
Observed and Predicted Risk in Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) at 2 Years for Primary Composite End Point (A), Cardiovascular Death (B), and All-Cause Mortality for Patients Categorized by Quintile of Risk Score (C)

A, Predicted vs observed risk of primary composite end point at 2 years. B, Predicted vs observed risk of cardiovascular death at 2 years. C. Predicted vs observed risk of all-cause mortality at 2 years.

Table 1.  Final Model for Primary Composite End Point Based on Backward Stepwise Cox Proportional Hazard Regressiona
Final Model for Primary Composite End Point Based on Backward Stepwise Cox Proportional Hazard Regressiona
Table 2.  Final Model for CV Death Based on Backward Stepwise Cox Proportional Hazard Regressiona
Final Model for CV Death Based on Backward Stepwise Cox Proportional Hazard Regressiona
Table 3.  Final Model for All-Cause Mortality Based on Backward Stepwise Cox Proportional Hazard Regression
Final Model for All-Cause Mortality Based on Backward Stepwise Cox Proportional Hazard Regression
1.
Alba  AC, Agoritsas  T, Jankowski  M,  et al.  Risk prediction models for mortality in ambulatory patients with heart failure: a systematic review.   Circ Heart Fail. 2013;6(5):881-889. doi:10.1161/CIRCHEARTFAILURE.112.000043PubMedGoogle ScholarCrossref
2.
Ouwerkerk  W, Voors  AA, Zwinderman  AH.  Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure.   JACC Heart Fail. 2014;2(5):429-436. doi:10.1016/j.jchf.2014.04.006PubMedGoogle ScholarCrossref
3.
Rahimi  K, Bennett  D, Conrad  N,  et al.  Risk prediction in patients with heart failure: a systematic review and analysis.   JACC Heart Fail. 2014;2(5):440-446. doi:10.1016/j.jchf.2014.04.008PubMedGoogle ScholarCrossref
4.
Ferrero  P, Iacovoni  A, D’Elia  E, Vaduganathan  M, Gavazzi  A, Senni  M.  Prognostic scores in heart failure: critical appraisal and practical use.   Int J Cardiol. 2015;188:1-9. doi:10.1016/j.ijcard.2015.03.154PubMedGoogle ScholarCrossref
5.
Simpson  J, Jhund  PS, Silva Cardoso  J,  et al; PARADIGM-HF Investigators and Committees.  Comparing LCZ696 with enalapril according to baseline risk using the MAGGIC and EMPHASIS-HF risk scores: an analysis of mortality and morbidity in PARADIGM-HF.   J Am Coll Cardiol. 2015;66(19):2059-2071. doi:10.1016/j.jacc.2015.08.878PubMedGoogle ScholarCrossref
6.
Levy  WC, Mozaffarian  D, Linker  DT,  et al.  The Seattle Heart Failure Model: prediction of survival in heart failure.   Circulation. 2006;113(11):1424-1433. doi:10.1161/CIRCULATIONAHA.105.584102PubMedGoogle ScholarCrossref
7.
McMurray  JJ, Packer  M, Desai  AS,  et al; PARADIGM-HF Committees and Investigators.  Dual angiotensin receptor and neprilysin inhibition as an alternative to angiotensin-converting enzyme inhibition in patients with chronic systolic heart failure: rationale for and design of the Prospective comparison of ARNI with ACEI to Determine Impact on Global Mortality and morbidity in Heart Failure trial (PARADIGM-HF).   Eur J Heart Fail. 2013;15(9):1062-1073. doi:10.1093/eurjhf/hft052PubMedGoogle ScholarCrossref
8.
McMurray  JJ, Packer  M, Desai  AS,  et al; PARADIGM-HF Investigators and Committees.  Angiotensin-neprilysin inhibition versus enalapril in heart failure.   N Engl J Med. 2014;371(11):993-1004. doi:10.1056/NEJMoa1409077PubMedGoogle ScholarCrossref
9.
Krum  H, Massie  B, Abraham  WT,  et al; ATMOSPHERE Investigators.  Direct renin inhibition in addition to or as an alternative to angiotensin converting enzyme inhibition in patients with chronic systolic heart failure: rationale and design of the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure (ATMOSPHERE) study.   Eur J Heart Fail. 2011;13(1):107-114. doi:10.1093/eurjhf/hfq212PubMedGoogle ScholarCrossref
10.
McMurray  JJ, Krum  H, Abraham  WT,  et al; ATMOSPHERE Committees Investigators.  Aliskiren, enalapril, or aliskiren and enalapril in heart failure.   N Engl J Med. 2016;374(16):1521-1532. doi:10.1056/NEJMoa1514859PubMedGoogle ScholarCrossref
11.
Jonsson  A, Edner  M, Alehagen  U, Dahlström  U.  Heart failure registry: a valuable tool for improving the management of patients with heart failure.   Eur J Heart Fail. 2010;12(1):25-31. doi:10.1093/eurjhf/hfp175PubMedGoogle ScholarCrossref
12.
Voors  AA, Ouwerkerk  W, Zannad  F,  et al.  Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure.   Eur J Heart Fail. 2017;19(5):627-634. doi:10.1002/ejhf.785PubMedGoogle ScholarCrossref
13.
Cannon  JA, Collier  TJ, Shen  L,  et al.  Clinical outcomes according to QRS duration and morphology in the Eplerenone in Mild Patients: Hospitalization and SurvIval Study in Heart Failure (EMPHASIS-HF).   Eur J Heart Fail. 2015;17(7):707-716. doi:10.1002/ejhf.303PubMedGoogle ScholarCrossref
14.
Inglis  SC, Hermis  A, Shehab  S, Newton  PJ, Lal  S, Davidson  PM.  Peripheral arterial disease and chronic heart failure: a dangerous mix.   Heart Fail Rev. 2013;18(4):457-464. doi:10.1007/s10741-012-9331-1PubMedGoogle ScholarCrossref
15.
Pocock  SJ, Ariti  CA, McMurray  JJ,  et al; Meta-Analysis Global Group in Chronic Heart Failure.  Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies.   Eur Heart J. 2013;34(19):1404-1413. doi:10.1093/eurheartj/ehs337PubMedGoogle ScholarCrossref
16.
Pocock  SJ, Wang  D, Pfeffer  MA,  et al.  Predictors of mortality and morbidity in patients with chronic heart failure.   Eur Heart J. 2006;27(1):65-75. doi:10.1093/eurheartj/ehi555PubMedGoogle ScholarCrossref
17.
Senni  M, Parrella  P, De Maria  R,  et al.  Predicting heart failure outcome from cardiac and comorbid conditions: the 3C-HF score.   Int J Cardiol. 2013;163(2):206-211. doi:10.1016/j.ijcard.2011.10.071PubMedGoogle ScholarCrossref
18.
Barlera  S, Tavazzi  L, Franzosi  MG,  et al; GISSI-HF Investigators.  Predictors of mortality in 6975 patients with chronic heart failure in the Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico-Heart Failure trial: proposal for a nomogram.   Circ Heart Fail. 2013;6(1):31-39. doi:10.1161/CIRCHEARTFAILURE.112.967828PubMedGoogle ScholarCrossref
19.
Sartipy  U, Goda  A, Yuzefpolskaya  M, Mancini  DM, Lund  LH.  Utility of the Seattle Heart Failure Model in patients with cardiac resynchronization therapy and implantable cardioverter defibrillator referred for heart transplantation.   Am Heart J. 2014;168(3):325-331. doi:10.1016/j.ahj.2014.03.025PubMedGoogle ScholarCrossref
20.
Cleland  JG, Chiswell  K, Teerlink  JR,  et al.  Predictors of postdischarge outcomes from information acquired shortly after admission for acute heart failure: a report from the Placebo-Controlled Randomized Study of the Selective A1 Adenosine Receptor Antagonist Rolofylline for Patients Hospitalized With Acute Decompensated Heart Failure and Volume Overload to Assess Treatment Effect on Congestion and Renal Function (PROTECT) Study.   Circ Heart Fail. 2014;7(1):76-87. doi:10.1161/CIRCHEARTFAILURE.113.000284PubMedGoogle ScholarCrossref
Original Investigation
January 29, 2020

Prognostic Models Derived in PARADIGM-HF and Validated in ATMOSPHERE and the Swedish Heart Failure Registry to Predict Mortality and Morbidity in Chronic Heart Failure

Author Affiliations
  • 1BHF Cardiovascular Research Centre, University of Glasgow, Glasgow, Scotland
  • 2Department of Medicine, Karolinska Institutet, Stockholm, Sweden
  • 3Cardiovascular Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
  • 4Davis Heart and Lung Research Institute, Division of Cardiovascular Medicine, Ohio State University, Columbus
  • 5Stavanger University Hospital, University of Bergen, Stavanger, Norway
  • 6Rigshospitalet Copenhagen University Hospital, Copenhagen, Denmark
  • 7Baylor Heart and Vascular Institute, Baylor University Medical Center, Dallas, Texas
  • 8Institut de Cardiologie de Montréal, Université de Montréal, Montréal, Quebec, Canada
  • 9Novartis Pharmaceutical Corporation, Basel, Switzerland
  • 10Department of Molecular and Clinical Medicine, University of Gothenburg, Gothenburg, Sweden
  • 11National Heart and Lung Institute, Imperial College, London, England
  • 12Medical University of South Carolina, Charleston
  • 13Ralph H. Johnson Veterans Administration Medical Center, Charleston, South Carolina
JAMA Cardiol. 2020;5(4):432-441. doi:10.1001/jamacardio.2019.5850
Key Points

Question  Can the Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) trial predict outcome in patients with heart failure (HF)?

Findings  In this prognostic study, there were 1546 deaths and 2031 cardiovascular death or HF hospitalization events of the 8399 patients in the PARADIGM-HF trial, and models were derived for a composite of cardiovascular death or HF hospitalization, cardiovascular death, and all-cause death. The models were externally validated using data from the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure and the Swedish Heart Failure registry.

Meaning  The PREDICT-HF models can be used to accurately predict outcome of patients with chronic HF.

Abstract

Importance  Accurate prediction of risk of death or hospitalizations in patients with heart failure (HF) may allow physicians to explore how more accurate decisions regarding appropriateness and timing of disease-modifying treatments, advanced therapies, or the need for end-of-life care can be made.

Objective  To develop and validate a prognostic model for patients with HF.

Design, Setting, and Participants  Multivariable analyses were performed in a stepwise fashion. Harrell C statistic was used to assess the discriminative ability. The derivation cohort was Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF) participants. The models were validated using the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure Trial (ATMOSPHERE) study and in the Swedish Heart Failure Registry (SwedeHF). A total of 8399 participants enrolled in PARADIGM-HF. Data were analyzed between June 2016 and June 2018.

Main Outcomes and Measures  Cardiovascular death, all-cause mortality, and the composite of cardiovascular death or HF hospitalization at both 1 and 2 years.

Results  Complete baseline clinical data were available for 8011 patients in PARADIGM-HF. The mean (SD) age of participants was 64 (11.4) years, 78.2% were men (n = 6567 of 8399), and 70.6% were New York Heart Association class II (n = 5919 of 8399). During a mean follow-up of 27 months, 1546 patients died, and 2031 had a cardiovascular death or HF hospitalization. The common variables were: male sex, race/ethnicity (black or Asian), region (Central Europe or Latin America), HF duration of more than 5 years, New York Heart Association class III/ IV, left ventricular ejection fraction, diabetes mellitus, β-blocker use at baseline, and allocation to sacubitril/valsartan. Ranked by χ2, N-terminal pro brain natriuretic peptide was the single most powerful independent predictor of each outcome. The C statistic at 1 and 2 years was 0.74 (95% CI, 0.71-0.76) and 0.71 (95% CI, 0.70-0.73) for the primary composite end point, 0.73 (95% CI, 0.71-0.75) and 0.71 (95% CI, 0.69-0.73) for cardiovascular death, and 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.67-0.74) for all-cause death, respectively. When validated in ATMOSPHERE, the C statistic at 1 and 2 years was 0.71 (95% CI, 0.69-0.72) and 0.70 (95% CI, 0.68-0.71) for the primary composite end point, 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.69-0.72) for cardiovascular death, and 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.68-0.72) for all-cause death, respectively. An online calculator was created to allow calculation of an individual’s risk (http://www.predict-hf.com).

Conclusions and Relevance  Predictive models performed well and were developed and externally validated in large cohorts of geographically representative patients, comprehensively characterized with clinical and laboratory data including natriuretic peptides, who were receiving contemporary evidence-based treatment.

Introduction

Rates of morbidity and mortality are high in patients with heart failure (HF), but predicting prognosis is difficult.1-4 Predicting risk of death or hospitalizations in patients with HF can allow physicians and patients to make important decisions regarding appropriateness and timing of treatments or the need for end-of-life care.1-4 Risk estimation can be used to alter care, with closer follow-up and intensified therapy. It may also aid physicians in assisting patients to make fully informed decisions about their care. Furthermore, identifying factors common to patients at high risk of death or hospitalization may reveal potential targets for interventions to improve prognosis.1-4

Individual measures, such as left ventricular ejection fraction (LVEF) or New York Heart Association (NYHA) classification, are associated with morbidity and mortality but are not sensitive enough to be used to predict prognosis. For example, in the Prospective Comparison of ARNI With ACEI to Determine Impact on Global Mortality and Morbidity in Heart Failure trial (PARADIGM-HF), although 70% of patients were deemed to have mild symptoms (NYHA class II), when stratified into quintiles using a predictive risk score, the rate of cardiovascular (CV) death or HF hospitalization ranged from 7.6 to 20.6 per 100 patient-years of follow-up.5 Using multiple variables and calculating risk by regression models more accurately predicts risk.1-4

Several multivariable models exist for predicting adverse outcomes in patients with HF.1-4 The Seattle Heart Failure Model (SHFM) remains the most extensively validated of these6 but was derived from (and validated in) a historical cohort not receiving contemporary evidence based guideline-recommended therapies for HF. Furthermore, the SHFM (and most other models) did not include biomarkers in the derivation model and, in particular, did not take into account levels of natriuretic peptides, which are strongly associated with prognosis in HF.1-4

We used patient-level data (including natriuretic peptides) from the PARADIGM-HF trial to develop prognostic models for morbidity and mortality in contemporary patients with HF treated with evidence-based therapies for HF.7,8 The models were validated using data from the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure Trial (ATMOSPHERE) study and in the Swedish Heart Failure Registry (SwedeHF) containing data on an unselected nationwide cohort.9-11

Methods

The PARADIGM-HF patient cohort (n = 8399) was used for model derivation. The ATMOSPHERE (n = 7016) and SwedeHF cohorts were used for model validation.

Design of Trials

The study design and main results of both PARADIGM-HF and ATMOSPHERE are published.7-10 The inclusion and exclusion criteria of the 2 trials were almost identical and are described in eAppendix 1 in the Supplement. Briefly, patients were eligible at screening if they had NYHA class II through IV, LVEF of 35% or less (changed from ≤40% initially in PARADIGM-HF by amendment), had an elevated natriuretic peptide level (see eAppendix 1 in the Supplement for levels) and were receiving an angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB), along with a β-blocker (unless contraindicated or not tolerated) and a mineralocorticoid receptor antagonist (MRA), if indicated. The trial was approved by ethics committees at all participating centers in PARADIGM-HF and ATMOSPHERE, and all patients provided written informed consent. In PARADIGM-HF, patients were randomized to sacubitril/valsartan or enalapril and in ATMOSPHERE to enalapril, aliskiren, or both following a run in period in both trials (eAppendix 1 in the Supplement).

The primary outcome in each trial was the composite of death from CV causes or a first hospitalization for HF. The median duration of follow-up was 27 months in PARADIGM-HF and 36.6 months in ATMOSPHERE.

Swedish Heart Failure Registry

Patients with HF included in SwedeHF between July 2005 and December 2012 were studied.11 This nationwide internet-based voluntary registry has been described in detail.11 Patients are included in the registry based on a physician-based clinical diagnosis of HF and registered at discharge from hospital or following an outpatient visit. For the purposes of externally validating the PARADIGM Risk of Events and Death in the Contemporary Treatment of Heart Failure (PREDICT-HF) model for all-cause mortality, only patients registered following an outpatient encounter (rather than at hospital discharge) were included, and those with an LVEF greater than or equal to 50% were excluded. The LVEF is reported in the registry as less than 30%, 30% to 39%, and 40% to 49%, and each group were considered to have an LVEF of 25%, 35%, and 40%, respectively. Race/ethnicity was not available in SwedeHF, so all patients were classified as white. Heart failure duration was coded in SwedeHF as less than or greater than 6 months; this was treated as a patient with HF duration of greater than 1 year for the purposes of model validation (PARADIGM-HF categories for duration of HF: 1-2 years and >2 years). The remaining 10 baseline covariates in the PREDICT-HF model were available in SwedeHF. Serum potassium, hemoglobin, and N-terminal pro brain natriuretic peptide (NTproBNP) levels were available in SwedeHF. For the remaining laboratory values, the mean or median values from the PARADIGM-HF cohort were used because this is the default procedure for the online calculator. The variables included albumin, uric acid, percentage monocytes, total bilirubin, absolute neutrophils, chloride, low-density lipoprotein cholesterol, aspartate aminotransferase, urea, and triglycerides. Only patients with complete data across the covariates for the model for all-cause mortality were included in the primary analyses. Establishment of SwedeHF and this analysis assessing prognostic scores were approved by a multisite ethics committee. In SwedeHF, individual patient consent was not required, but patients are informed of registration in national registries and allowed to opt out.

Statistical Analysis

The main analysis includes only patients who had complete baseline data. The outcomes studied were the primary composite end point, CV death, and all-cause mortality.

The multivariable analyses were performed in a stepwise fashion. In step one, 63 baseline candidate variables potentially associated with outcomes were examined. Continuous variables were inspected using restricted cubic splines for nonlinear effects. Left ventricular ejection fraction and heart rate were included as continuous variables. Age, systolic blood pressure, and body mass index were included with a cutoff value, at which point the effect of the variable had negligible effect on the outcome. A backward stepwise Cox proportional hazards model was built for each outcome, with a 2-sided P value less than .01 as the initial significance level and selected variables were then taken forward to step 2.

In step 2, all initial candidate variables were evaluated for significant interactions for each outcome. For the interaction terms, a 2-sided P value of less than .001 was considered statistically significant. Significant interaction terms were added and selected again using a backward stepwise procedure, and the final model taken forward to step 3.

In step 3 laboratory test results (excluding natriuretic peptides) were added. Laboratory test results were evaluated by visual inspection of restricted cubic splines to identify nonlinearity. Cutoff values were selected for each variable, at which point the value did not alter the risk of the outcome. Laboratory test results that were statistically significant in the model were added and taken forward to step 4.

In step 4, the association between natriuretic peptides and outcome was considered in 5 categories (1, NTproBNP <400 pg/mL; 2, 400 pg/mL-799 pg/mL; 3, 800 pg/mL-1599 pg/mL; 4, 1600 pg/mL-3199 pg/mL; 5, ≥3200 pg/mL). The NTproBNP and BNP measurements were available in 8385 and 8340 patients, respectively.

The Cox model was used to obtain baseline survival at 1 and 2 years. The predicted vs actual outcome at 1 and 2 years was compared in quintiles. Kaplan-Meier survival curves for each outcome by quintile of risk were estimated and compared directly with baseline survival function for each outcome using Cox proportional hazards method.

The C statistic was used to assess the discriminative ability of the model, and predicted scores from the PARADIGM-HF cohort were applied to the ATMOSPHERE cohort to estimate survival at 1 and 2 years. Predicted vs observed outcomes in ATMOPSHERE at 1 and 2 years were examined in quintiles and deciles (shown in eAppendix 8C and D in the Supplement). Sensitivity analyses were also carried out for missing data using the multiple imputation technique chained method for missing data in SwedeHF. All analyses were conducted using Stata, version 14.0 (StataCorp) and SAS, version 9.4 (SAS Institute Inc).

Results

Complete baseline data were available for 8011 of 8399 patients in PARADIGM-HF. During a mean follow-up of 27 months, 1546 patients died and 2031 had a CV death or HF hospitalization.

Baseline characteristics of candidate variables for the prognostic models are shown in eAppendix 2 in the Supplement according to outcome (no outcome of interest, primary composite end point, CV death, or all-cause death). Patients experiencing an adverse outcome were older, more likely to be male, had higher resting heart rates, evidence of more advanced HF, more comorbidity, worse renal function, and higher natriuretic peptide levels than those not experiencing one of these outcomes.

Predictive Models
Primary Composite End Point

A total of 7683 patients had complete data for all variables in the final multivariable model, and a total of 1853 experienced the primary composite end point of CV death or hospitalization for HF. Twenty-three characteristics were found to be independently associated with the primary composite end point on multivariate analysis (eAppendix 3A in the Supplement). The final prognostic model for the primary composite end point, based on backward stepwise Cox proportional hazard regression, is shown in Table 1.

Cardiovascular Death

A total of 7804 patients had complete data across all variables in the final multivariable model, and 1176 patients died. Twenty-one predictors of CV death were identified on multivariate analysis (eAppendix 3B in the Supplement). The final prognostic model for CV death based on backward stepwise Cox proportional hazard regression is shown in Table 2.

All-Cause Death

A total of 7876 patients had complete data for all variables in the final multivariable model, and 1453 patients died. Twenty variables were independent predictors of all-cause death on multivariate analysis (eAppendix 3C in the Supplement). The final prognostic model for all-cause death based on backward stepwise Cox proportional hazard regression is shown in Table 3.

Comparison of Predictors

Most of the clinical/demographic predictive variables were the same in each of the 3 models. The common variables were: sex, race/ethnicity, region, duration of HF, NYHA class, LVEF, diabetes mellitus, prior myocardial infarction, peripheral arterial disease, β-blocker use at baseline, and allocation to sacubitril/valsartan.

Prior HF hospitalization was only predictive of the primary composite outcome as was valvular heart disease. Bundle-branch block predicted the primary end point and CV death but not all-cause death. Age predicted CV death and all-cause mortality, and prior percutaneous coronary intervention predicted CV and all-cause death but not the primary composite outcome. Body mass index was predictive of all-cause death only.

Among the routine laboratory variables, the top 3 predictors for all 3 outcomes were bilirubin, uric acid, and albumin. Ranked by χ2, NTproBNP was the most powerful predictor of each of the 3 outcomes.

Missing Data

The models for all 3 end points were developed on complete cases. The extent of missing data for baseline characteristics, laboratory values, and natriuretic peptides are shown in eAppendix 4 in the Supplement. Data were complete for 50 of 63 candidate variables. The proportion of missing data for the other 13 variables ranged from 0.01% to 0.2%. There were also few missing data for laboratory variables (ranging from 1.8% to 5.6%).

Predicting an Individual’s Risk

Model calibration for each end point examined was explored by comparing the predicted and observed probabilities across patient risk quintiles at 1 and 2 years (Figure 1 and Figure 2). For the primary composite end point there was a strong gradient in risk, with patients in the highest risk quintile having 9 times the risk of HF hospitalization or CV death than patients in the lowest-risk quintile (Figure 1A). This was similar for CV death and all-cause death, with patients in the fifth quintile having 9 and 8 times the risk of the outcome in question when compared with patients in the first quintile (Figures 1B and C). A similar trend was seen across the quintiles for predicted and observed risk at 2 years with gradient risk between the first and fifth quintile of 6 for all outcomes (Figure 2). Kaplan-Meier event-free survival curves over 3 years for the primary composite end point, CV death, and all-cause mortality by quintile of risk were estimated and compared directly with baseline survival function using Cox proportional hazards method (eAppendix 5 and 6 in the Supplement).

The C statistic at 1 and 2 years for the primary composite end point was 0.74 (95% CI, 0.71-0.76) and 0.71 (95% CI, 0.70-0.75), respectively. For cardiovascular death, the C statistic for 1 and 2 years was 0.73 (95% CI, 0.71-0.75) and 0.71 (95% CI, 0.69-073), respectively, and for all-cause death it was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.67-0.72), respectively.

The models performed similarly well without the inclusion of natriuretic peptides, although performance was better with their inclusion and was substantially better than natriuretic peptides on their own (eAppendix 7 in the Supplement). The model performed better than currently available scores (eAppendix 8 in the Supplement).

External Validation of the Models

The PARADIGM-HF prognostic model was prospectively validated in ATMOSPHERE. Baseline characteristics for patients in the ATMOPSHERE cohort according to outcome are shown in the eAppendix 2B in the Supplement. The C statistic for the model applied to the ATMOSPHERE cohort for the primary composite end point at 1 and 2 years was 0.71 (95% CI, 0.69-0.72) and 0.70 (95% CI, 0.68-0.71), respectively. For CV death, the Harrell C statistic for 1 and 2 years was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.69-0.72), respectively, and for all-cause death was 0.71 (95% CI, 0.69-0.74) and 0.70 (95% CI, 0.68-0.72), respectively.

The PARADIGM-HF model was also validated in SwedeHF for all-cause death. Baseline characteristics for patients in the SwedeHF cohort by outcome are shown in the eAppendix 2C in the Supplement. The C statistic for the model applied to SwedeHF for all-cause death at 1 and 2 years was 0.79 (95% CI, 0.75-0.81) and 0.78 (95% CI, 0.75-0.80) (see eAppendix 9B in the Supplement).

Sensitivity Analyses

The models were developed using BNP instead of NTproBNP. The C statistic for the model using BNP for the primary end point, CV death, and all-cause death were 0.72, 0.73, and 0.72 and 0.70, 0.72, and 0.70 at 1 and 2 years, respectively. The final models including BNP are shown in eAppendix 10 in the Supplement (eAppendix 10 in the Supplement).

Calculation of Individual Estimated Survival

An online calculator (http://www.predict-hf.com) was created for each outcome to allow calculation of an individual’s risk in the outpatient setting. These tools allow calculation of risk at each model step (demographics/clinical characteristics, routine laboratory variables, and natriuretic peptide level) and, if natriuretic peptide levels are available, either NTproBNP or BNP.

Discussion

Our prognostic models for patients with chronic ambulatory HF reduced ejection fraction (HF-REF) were developed in the PARADIGM-HF cohort and validated in the ATMOSPHERE cohort (and for all-cause death in SwedeHF). The models for the composite outcome of CV death or HF hospitalization (1853 derivation events and 1988 validation events), CV death alone (1176 of 1371), and all-cause death (1453 of 1627) demonstrated good calibration and discrimination. To our knowledge, this is one of the few studies of prognostic models in HF to report external validation, is one of a minority that included natriuretic peptides, and is by far the largest to include either of these features.1-4 The cohorts used had more extensive and complete collection of clinical and laboratory data at baseline, were better treated with contemporary evidence-based treatment (100% treated with ACE inhibitor/ARB, 93% β-blockers, and 56% MRA in PARADIGM-HF) than in any prior study of a similar nature, and were more geographically representative than in previous reports. Because PARADIGM-HF was a global clinical trial, enrolling patients from 1043 centers in 47 countries (and ATMOSPHERE from 789 centers in 43 countries), we could examine both geographical region and race/ethnicity in our models, which has rarely been possible previously.1-4 One model, derived in the Biology Study to Tailored Treatment in Chronic Heart Failure cohort (n = 2516) did incorporate natriuretic peptides and was externally validated12 but only enrolled European patients and those not treated with (or with ≤50% of the target dose of) an ACE inhibitor/ARB and/or a β-blocker at baseline. Most patients were enrolled during a hospital admission with worsening heart failure, and, although a higher risk cohort, only 657 deaths were reported (compared with 1546 in PARADIGM-HF), presumably because of the relatively small sample size.

Most of the predictors identified in our models are established prognostic markers in HF, although some are less so because of incomplete collection in prior studies. These less-well-recognized predictors include peripheral arterial disease, β-blocker use, bundle-branch block, prior percutaneous coronary intervention, race/ethnicity, and region.

The presence of peripheral arterial disease and electrocardiogram QRS configuration has not been documented in many HF trials, but where they have been, they have consistently been predictors of poor outcome.13,14 β-Blockers (and even ACE inhibitors/ARBs and MRAs) were much less frequently taken by patients included in some of the studies, giving rise to established predictive models. In the derivation cohort used to generate the widely used SHFM, only 3% of patients were taking an MRA and none were prescribed a β-blocker.6 Overall, in Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC), only 67% of patients were treated with an ACE inhibitor/ARB, 34% with a β-blocker, and 21% with an MRA.15 In the CHARM and 3C-HF data sets, 55% and 67% of patients, respectively, were prescribed a β-blocker, but only 17% of CHARM patients were prescribed an MRA (not reported in 3C-HF).16,17 Although pharmacologic therapy has been more comprehensive in some models, and a few of these included biomarkers, the patients enrolled came from a handful of countries or even a single country.18 Each of these limitations make the generalizability of these prior models to contemporary patients, especially those not living in Western Europe or North America, uncertain. This is especially important because both region and race/ethnicity featured prominently among the predictive variables in each of our 3 models.

Routine laboratory tests added valuable prognostic information in our models. Few studies have measured each of these biomarkers together, and many prior cohorts had a high proportion of patients with missing data; eg, in validation of the SHFM, 65% of uric acid levels and up to 100% of lymphocyte values were missing (compared with more than 90% completeness in our derivation and validation cohorts).6 Consequently, we describe, to our knowledge, the most comprehensive and complete simultaneous evaluation of the relative predictive value of routine biomarkers in patients with HF-REF. Additionally, the SHFM is known to underestimate risk in high-risk HF, when risk prediction is particularly important for selecting patients for scarce and/or expensive therapy, such as heart transplantation or durable mechanical circulatory support.19 In contrast, our model discriminated equally well regardless of risk, although the actual rate of the primary composite end point was minimally greater than the predicted in the highest-risk quintile. Creatinine and estimated glomerular filtration rate (eGFR) were predictors of outcome in our univariable models but notably missing from the multivariable models, although blood urea nitrogen (BUN) was retained. Surprisingly, although widely considered an important predictor of outcome in heart failure, eGFR has not been included in many prior models and usually not simultaneously with BUN (although it has been used with creatinine).1-4,20 The Biology Study to Tailored Treatment in Chronic Heart Failure did and found, as we did, that BUN displaced eGFR/creatinine.12

A particular strength of our model was the inclusion of both NTproBNP and BNP. Despite the inclusion of a wide range of demographic and clinical variables, and as alluded to, a comprehensive routine laboratory panel, NTproBNP and BNP had the highest predictive value for each outcome. Our main model was developed using NTproBNP because this value was available in 99.8% of patients in the derivation cohort (n = 8385) and 90.5% of patients in the validation cohort (n = 6352). However, BNP performed similarly.

Another strength was external validation of our models in a similarly contemporaneous population (the ATMOSPHERE trial). Our models performed well in the ATMOSPHERE data set. We were also able to validate our all-cause mortality model in a third, real-world data set, the SwedeHF registry. Again, our models performed well, indeed, at least as well as in the ATMOSPHERE data set. Therefore our model, using contemporary patients from around the world, receiving contemporary levels of guideline-recommended therapies and using natriuretic peptides, and that has been externally validated in a real-world cohort, may be considered the strongest of currently available models. To make our models clinically applicable, we have created an online calculator.

Limitations

Our study also had limitations. Both the derivation and main validation data sets were obtained from clinical trials and therefore included selected patients (for example, both trials excluded individuals with markedly reduced renal function); however, this limitation was overcome to some extent by additional validation of one of the outcomes in SwedeHF, where prescribing rates and doses are different to the 2 clinical trials. Both trial data sets included only patients with HF-REF, and we do not know the value of our models in patients with HF with preserved ejection fraction. Implantable cardioverter defibrillators and cardiac resynchronization therapy were used relatively infrequently in PARADIGM-HF and ATMOSPHERE, and we do not know how well our models would perform in patients extensively treated with such devices. Additional prognostic variables (such as functional capacity and measures of frailty), and biomarkers were not available in PARADIGM-HF. The accessibility and clinical application of our models might be easier if we created an integer score, but this would have resulted in loss of some predictive accuracy.

Conclusions

In summary, the PARADIGM-HF models can accurately predict morbidity and mortality in ambulatory patients with chronic HF-REF at 1 and 2 years. The models are based on readily available clinical and laboratory characteristics and were derived from a robust, complete, data set where most patients were receiving evidence-based therapy for HF. External validation in a similarly large cohort confirmed the model’s discriminative ability. Clinically, the model can be used to predict patient outcomes and has potential wider uses in the organization and targeting of services and therapies in patients with HF-REF.

Back to top
Article Information

Corresponding Author: John J. V. McMurray, MD, British Heart Foundation Cardiovascular Research Centre, University of Glasgow, 126 University Pl, Glasgow G12 8TA, Scotland (john.mcmurray@glasgow.ac.uk).

Accepted for Publication: December 2, 2019.

Correction: This article was corrected on April 15, 2020, to fix an error in the Abstract.

Published Online: January 29, 2020. doi:10.1001/jamacardio.2019.5850

Author Contributions: Dr Simpson had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Simpson, Jhund, Abraham, Køber, Mueller-Velten, Solomon, Zile, McMurray.

Acquisition, analysis, or interpretation of data: Simpson, Jhund, Lund, Padmanabhan, Claggett, Shen, Petrie, Abraham, Desai, Dickstein, Packer, Rouleau, Mueller-Velten, Swedberg, Zile, McMurray.

Drafting of the manuscript: Simpson, Petrie.

Critical revision of the manuscript for important intellectual content: Jhund, Lund, Padmanabhan, Claggett, Shen, Petrie, Abraham, Desai, Dickstein, Køber, Packer, Rouleau, Mueller-Velten, Solomon, Swedberg, Zile, McMurray.

Statistical analysis: Simpson, Jhund, Shen, Mueller-Velten.

Administrative, technical, or material support: Lund, Padmanabhan, Petrie, Packer, McMurray.

Supervision: Jhund, Petrie, Abraham, Dickstein, Køber, Packer, Solomon, McMurray.

Conflict of Interest Disclosures: Dr Jhund reported personal fees from Novartis during the conduct of the study; grants from Boehringer Ingelheim, other support from AstraZeneca, and personal fees from Cytokinetics outside the submitted work. Dr Lund reported grants and personal fees from Novartis, AstraZeneca, Boehringer Ingelheim, and Vifor Pharma; personal fees from Bayer, Sanofi, Merck, Abbott, Medscape, and Pharmacosmos; and grants from Boston Scientific outside the submitted work. Dr Petrie reported grants and personal fees from Astra Zeneca and Boehringer Ingelheim and personal fees from Novo Nordisk, Novartis, Alnylam, Cardiorentis, and Resverlogix outside the submitted work. Dr Abraham reported personal fees from Novartis during the conduct of the study and personal fees from Abbott, Boehringer Ingelheim, Edwards Lifesciences, Respicardia, and Sensible Medical outside the submitted work. Dr Desai reported grants from Novartis during the conduct of the study; grants and personal fees from Abbott, Alnylam, Biofourmis, and AstraZeneca; and personal fees from Amgen, Boston Scientific, Boehringer Ingelheim, DalCor Pharma, Novartis, Relypsa, and Regeneron outside the submitted work. Dr Køber reported other support from Rigshospitalet, University of Copenhagen, Copenhagen, Denmark, during the conduct of the study and personal fees from speakers honorarium from Novartis and AstraZeneca outside the submitted work. Dr Packer reported personal fees from AbbVie, Akcea, Amgen, AstraZeneca, Actavis, Boehringer Ingelheim, Cardiorentis, Sankyo, Johnson & Johnson, Novo Nordisk, Pfizer, Sanofi, Synthetic Biologics, and Theravance outside the submitted work. Dr Rouleau reported personal fees from Novartis during the conduct of the study and from AstraZeneca outside the submitted work. Dr Solomon reported grants from Novartis during the conduct of the study; grants from Alnylam, Amgen, AstraZeneca, Bellerophon, Bayer, BMS, Celladon, Cytokinetics, Eidos, Gilead, GSK, Ionis, Lone Star Heart, Mesoblast, MyoKardia, National Institutes of Health/National Heart, Lung, and Blood Institute, Novartis, Sanofi Pasteur, and Theracos; and personal fees from Akros, Alnylam, Amgen, Arena, AstraZeneca, Bayer, BMS, Cardior, Corvia, Cytokinetics, Daiichi-Sankyo, Gilead, GSK, Ironwood, Merck, Myokardia, Novartis, Roche, Takeda, Theracos, Quantum Genetics, Cardurion, AoBiome, Janssen, Cardiac Dimensions, and Tenaya outside the submitted work. Dr Swedberg reported personal fees from Novartis during the conduct of the study and from AstraZeneca, Pfizer, and Novartis outside the submitted work. Dr Zile reported grants and personal fees from Novartis during the conduct of the study. Dr McMurray reported other support from Novartis during the conduct of the study and from Novartis, Amgen, GSK, Theracos, DalCor Pharmaceuticals, AstraZeneca, BMS, and Cardurion. No other disclosures were reported.

References
1.
Alba  AC, Agoritsas  T, Jankowski  M,  et al.  Risk prediction models for mortality in ambulatory patients with heart failure: a systematic review.   Circ Heart Fail. 2013;6(5):881-889. doi:10.1161/CIRCHEARTFAILURE.112.000043PubMedGoogle ScholarCrossref
2.
Ouwerkerk  W, Voors  AA, Zwinderman  AH.  Factors influencing the predictive power of models for predicting mortality and/or heart failure hospitalization in patients with heart failure.   JACC Heart Fail. 2014;2(5):429-436. doi:10.1016/j.jchf.2014.04.006PubMedGoogle ScholarCrossref
3.
Rahimi  K, Bennett  D, Conrad  N,  et al.  Risk prediction in patients with heart failure: a systematic review and analysis.   JACC Heart Fail. 2014;2(5):440-446. doi:10.1016/j.jchf.2014.04.008PubMedGoogle ScholarCrossref
4.
Ferrero  P, Iacovoni  A, D’Elia  E, Vaduganathan  M, Gavazzi  A, Senni  M.  Prognostic scores in heart failure: critical appraisal and practical use.   Int J Cardiol. 2015;188:1-9. doi:10.1016/j.ijcard.2015.03.154PubMedGoogle ScholarCrossref
5.
Simpson  J, Jhund  PS, Silva Cardoso  J,  et al; PARADIGM-HF Investigators and Committees.  Comparing LCZ696 with enalapril according to baseline risk using the MAGGIC and EMPHASIS-HF risk scores: an analysis of mortality and morbidity in PARADIGM-HF.   J Am Coll Cardiol. 2015;66(19):2059-2071. doi:10.1016/j.jacc.2015.08.878PubMedGoogle ScholarCrossref
6.
Levy  WC, Mozaffarian  D, Linker  DT,  et al.  The Seattle Heart Failure Model: prediction of survival in heart failure.   Circulation. 2006;113(11):1424-1433. doi:10.1161/CIRCULATIONAHA.105.584102PubMedGoogle ScholarCrossref
7.
McMurray  JJ, Packer  M, Desai  AS,  et al; PARADIGM-HF Committees and Investigators.  Dual angiotensin receptor and neprilysin inhibition as an alternative to angiotensin-converting enzyme inhibition in patients with chronic systolic heart failure: rationale for and design of the Prospective comparison of ARNI with ACEI to Determine Impact on Global Mortality and morbidity in Heart Failure trial (PARADIGM-HF).   Eur J Heart Fail. 2013;15(9):1062-1073. doi:10.1093/eurjhf/hft052PubMedGoogle ScholarCrossref
8.
McMurray  JJ, Packer  M, Desai  AS,  et al; PARADIGM-HF Investigators and Committees.  Angiotensin-neprilysin inhibition versus enalapril in heart failure.   N Engl J Med. 2014;371(11):993-1004. doi:10.1056/NEJMoa1409077PubMedGoogle ScholarCrossref
9.
Krum  H, Massie  B, Abraham  WT,  et al; ATMOSPHERE Investigators.  Direct renin inhibition in addition to or as an alternative to angiotensin converting enzyme inhibition in patients with chronic systolic heart failure: rationale and design of the Aliskiren Trial to Minimize Outcomes in Patients with Heart Failure (ATMOSPHERE) study.   Eur J Heart Fail. 2011;13(1):107-114. doi:10.1093/eurjhf/hfq212PubMedGoogle ScholarCrossref
10.
McMurray  JJ, Krum  H, Abraham  WT,  et al; ATMOSPHERE Committees Investigators.  Aliskiren, enalapril, or aliskiren and enalapril in heart failure.   N Engl J Med. 2016;374(16):1521-1532. doi:10.1056/NEJMoa1514859PubMedGoogle ScholarCrossref
11.
Jonsson  A, Edner  M, Alehagen  U, Dahlström  U.  Heart failure registry: a valuable tool for improving the management of patients with heart failure.   Eur J Heart Fail. 2010;12(1):25-31. doi:10.1093/eurjhf/hfp175PubMedGoogle ScholarCrossref
12.
Voors  AA, Ouwerkerk  W, Zannad  F,  et al.  Development and validation of multivariable models to predict mortality and hospitalization in patients with heart failure.   Eur J Heart Fail. 2017;19(5):627-634. doi:10.1002/ejhf.785PubMedGoogle ScholarCrossref
13.
Cannon  JA, Collier  TJ, Shen  L,  et al.  Clinical outcomes according to QRS duration and morphology in the Eplerenone in Mild Patients: Hospitalization and SurvIval Study in Heart Failure (EMPHASIS-HF).   Eur J Heart Fail. 2015;17(7):707-716. doi:10.1002/ejhf.303PubMedGoogle ScholarCrossref
14.
Inglis  SC, Hermis  A, Shehab  S, Newton  PJ, Lal  S, Davidson  PM.  Peripheral arterial disease and chronic heart failure: a dangerous mix.   Heart Fail Rev. 2013;18(4):457-464. doi:10.1007/s10741-012-9331-1PubMedGoogle ScholarCrossref
15.
Pocock  SJ, Ariti  CA, McMurray  JJ,  et al; Meta-Analysis Global Group in Chronic Heart Failure.  Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies.   Eur Heart J. 2013;34(19):1404-1413. doi:10.1093/eurheartj/ehs337PubMedGoogle ScholarCrossref
16.
Pocock  SJ, Wang  D, Pfeffer  MA,  et al.  Predictors of mortality and morbidity in patients with chronic heart failure.   Eur Heart J. 2006;27(1):65-75. doi:10.1093/eurheartj/ehi555PubMedGoogle ScholarCrossref
17.
Senni  M, Parrella  P, De Maria  R,  et al.  Predicting heart failure outcome from cardiac and comorbid conditions: the 3C-HF score.   Int J Cardiol. 2013;163(2):206-211. doi:10.1016/j.ijcard.2011.10.071PubMedGoogle ScholarCrossref
18.
Barlera  S, Tavazzi  L, Franzosi  MG,  et al; GISSI-HF Investigators.  Predictors of mortality in 6975 patients with chronic heart failure in the Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico-Heart Failure trial: proposal for a nomogram.   Circ Heart Fail. 2013;6(1):31-39. doi:10.1161/CIRCHEARTFAILURE.112.967828PubMedGoogle ScholarCrossref
19.
Sartipy  U, Goda  A, Yuzefpolskaya  M, Mancini  DM, Lund  LH.  Utility of the Seattle Heart Failure Model in patients with cardiac resynchronization therapy and implantable cardioverter defibrillator referred for heart transplantation.   Am Heart J. 2014;168(3):325-331. doi:10.1016/j.ahj.2014.03.025PubMedGoogle ScholarCrossref
20.
Cleland  JG, Chiswell  K, Teerlink  JR,  et al.  Predictors of postdischarge outcomes from information acquired shortly after admission for acute heart failure: a report from the Placebo-Controlled Randomized Study of the Selective A1 Adenosine Receptor Antagonist Rolofylline for Patients Hospitalized With Acute Decompensated Heart Failure and Volume Overload to Assess Treatment Effect on Congestion and Renal Function (PROTECT) Study.   Circ Heart Fail. 2014;7(1):76-87. doi:10.1161/CIRCHEARTFAILURE.113.000284PubMedGoogle ScholarCrossref
×