Eagle KA, Lim MJ, Dabbous OH, Pieper KS, Goldberg RJ, Van de Werf F, Goodman SG, Granger CB, Steg PG, Gore JM, Budaj A, Avezum Á, Flather MD, Fox KAA, for the GRACE Investigators . A Validated Prediction Model for All Forms of Acute Coronary SyndromeEstimating the Risk of 6-Month Postdischarge Death in an International Registry. JAMA. 2004;291(22):2727–2733. doi:10.1001/jama.291.22.2727
Author Affiliations: Department of Internal Medicine, University of Michigan Health System, Ann Arbor (Drs Eagle and Lim); Department of Medicine, University of Massachusetts Medical School, Worcester (Drs Dabbous, Goldberg, and Gore); Department of Medicine, Duke University Medical Center, Durham, NC (Dr Granger and Ms Pieper); Department of Cardiology, Universitair Ziekenhuis Gasthuisberg, Leuven, Belgium (Dr Van de Werf); Canadian Heart Research Centre and Terrence Donnelly Heart Centre, and Division of Cardiology, St Michael's Hospital, University of Toronto, Toronto, Ontario (Dr Goodman); Cardiology, Hôpital Bichat, Paris, France (Dr Steg); Postgraduate Medical School, Grochowski Hospital, Warsaw, Poland (Dr Budaj); Clinical Research Center, Education, and Research Institute Hospital Albert Einstein, São Paulo, Brazil (Dr Avezum); Royal Brompton and Harefield NHS Trust, London, England (Dr Flather); and The University and The Royal Infirmary of Edinburgh, Edinburgh, Scotland (Dr Fox).
Context Accurate estimation of risk for untoward outcomes after patients have
been hospitalized for an acute coronary syndrome (ACS) may help clinicians
guide the type and intensity of therapy.
Objective To develop a simple decision tool for bedside risk estimation of 6-month
mortality in patients surviving admission for an ACS.
Design, Setting, and Patients A multinational registry, involving 94 hospitals in 14 countries, that
used data from the Global Registry of Acute Coronary Events (GRACE) to develop
and validate a multivariable stepwise regression model for death during 6
months postdischarge. From 17 142 patients presenting with an ACS from
April 1, 1999, to March 31, 2002, and discharged alive, 15 007 (87.5%)
had complete 6-month follow-up and represented the development cohort for
a model that was subsequently tested on a validation cohort of 7638 patients
admitted from April 1, 2002, to December 31, 2003.
Main Outcome Measure All-cause mortality during 6 months postdischarge after admission for
Results The 6-month mortality rates were similar in the development (n = 717;
4.8%) and validation cohorts (n = 331; 4.7%). The risk-prediction tool for
all forms of ACS identified 9 variables predictive of 6-month mortality: older
age, history of myocardial infarction, history of heart failure, increased
pulse rate at presentation, lower systolic blood pressure at presentation,
elevated initial serum creatinine level, elevated initial serum cardiac biomarker
levels, ST-segment depression on presenting electrocardiogram, and not having
a percutaneous coronary intervention performed in hospital. The c statistics for the development and validation cohorts were 0.81 and
Conclusions The GRACE 6-month postdischarge prediction model is a simple, robust
tool for predicting mortality in patients with ACS. Clinicians may find it
simple to use and applicable to clinical practice.
Clinical prediction models may be helpful for medical decision making1 as patients judged to be at higher risk may receive
more aggressive surveillance and/or earlier treatment, while patients estimated
to be at lower risk may be reassured and managed less aggressively.2,3 By using simple yet valid risk calculations,
clinicians can accurately advise patients about their likelihood of an event,
and how this likelihood translates into treatment decisions.
The acute coronary syndrome (ACS) encompasses a continuum of conditions
ranging from ST-segment elevation myocardial infarction (STEMI) to non–ST-segment
elevation myocardial infarction (NSTEMI) and unstable angina.1,4- 8 Numerous
risk-prediction models for differing outcomes exist for the different types
of ACS.5,9 Most models have been
developed from large randomized clinical trial populations in which the generalizability
to risk prediction in the average clinician's experience is questionable.6- 10
We were interested in developing a simple risk-prediction tool that
would be applicable to all types of ACS, would focus on an important clinical
end point of all-cause mortality, and be developed in patients who are similar
to those encountered in routine clinical practice.
The Global Registry of Acute Coronary Events (GRACE) is a multinational
cooperative effort involving 94 hospitals in 14 countries that has been designed
to reflect an unbiased, representative population of patients with ACS. Full
details of the GRACE methods have been previously published.11,12 For
this analysis, patients had to be at least 18 years old and alive at the time
of discharge, be admitted for ACS as a presumptive diagnosis (ie, have symptoms
consistent with acute myocardial ischemia), and have at least 1 of the following:
electrocardiographic changes consistent with ACS, serial increases in serum
cardiac biomarkers, and/or documentation of coronary artery disease. The qualifying
ACS must not have been precipitated by significant noncardiovascular comorbidity,
such as acute anemia or hyperthyroidism. At approximately 6 months after hospital
discharge, patients were followed up to ascertain vital status. At each enrolling
hospital, study investigators worked with their ethics or institutional review
board to obtain appropriate approval to participate.
To ensure enrollment of an unbiased population, the first 10 to 20 consecutive
eligible patients were recruited from each site per month. Data were collected
by trained coordinators using a standardized case report form. Demographic
characteristics, medical history, presenting symptoms, biochemical and electrocardiographic
findings, treatment practices, and a variety of hospital outcome data were
collected. Standardized definitions for all patient-related variables and
clinical diagnoses were used.11,12 At
discharge, all cases were assigned to STEMI, NSTEMI, or unstable angina categories.11,12 Standardized definitions were used
for selected hospital complications and outcomes.11
The primary end point was all-cause mortality occurring within 6 months
of discharge from hospital. The model was developed by using a multivariable
Cox proportional hazards regression backward elimination technique.13 The following variables were included in the stepwise
Cox proportional hazards regression: baseline characteristics, symptoms and
signs at presentation, in-hospital treatments and procedures, and in-hospital
complications. A backward stepwise technique evaluated all potential univariate
correlates (P<.25) to create a multivariable model
containing variables with P<.05. No imputation
was performed on the final model. Imputation was tested but did not influence
the identification of multivariate predictors or the discriminative power
of the model for predicting death.14 All variables
in the final model met the assumptions for proportional hazards.
The model accuracy was tested in several ways. First, we developed the
model in all patients with ACS enrolled in GRACE between April 1, 1999, and
March 31, 2002. The c statistic was extended to evaluate
the discrimination of survival analytic techniques.15 We
checked for any nonlinear relationship between death and each variable in
the final model and found none. Selected testing was performed for interactions
using the significant prognostic variables from the final model based on interactions
that have been reported from other published models. Second, we tested the
model in a validation cohort of consecutive patients enrolled in GRACE between
April 1, 2002, and December 31, 2003. Statistical analyses were performed
using SAS version 8.2 (SAS Institute Inc, Cary, NC) and S-Plus version 3.4
(MathSoft Inc, Seattle, Wash).
The overall follow-up rate in our development cohort was 87.5% for death.
A comparison of the patients with and those patients without available follow-up
data demonstrated no significant clinical differences in terms of baseline
characteristics, symptoms at presentation, in-hospital treatments and procedures,
in-hospital outcomes, and postdischarge outcomes.
We developed a bedside prediction tool that can be applied to either
a pocket card or a handheld device. Clinical prediction variables were given
weighted scores based on the model's variable coefficients. The prediction
tool considers the variables taken from the final model assigning a point
total to each variable, which allows a total point score for each patient
to be calculated. This then is applied to a reference plot nomogram, which
shows the corresponding risk of death. Alternatively, the risk of death could
be calculated using a handheld device. The clinical application can be found
Baseline characteristics, in-hospital treatments, and outcomes of the
15 007 patients used to develop the model (development cohort), 7638
patients used to test the model (validation cohort), and 5116 patients for
whom 6-month vital status was unknown are shown in Table 1 and Table 2.
The mean age was 65 years and 67% of the cohort were men. Prior or current
smoking and hypertension were present as risk factors in more than half of
the patients. Approximately 45% of patients had a diagnosis of hyperlipidemia
and less than 25% had diabetes mellitus.
The 6-month mortality risk was similar in the development (4.8%, n =
717) and validation cohorts (4.7%, n = 331). Nine multivariate predictors
of mortality were identified (Table 3).
The model calibrations were retained when testing the model in an independent
validation cohort (Table 3 and Figure 1). The model performed well in all
forms of ACS with a c statistic of at least 0.70
Figure 2 illustrates a method
to estimate a patient's risk depending on the total score obtained by summing
the individual scores for each of the 9 variables in the model. A similar
nomogram can be programmed into a handheld device to make the risk calculation
automatic once the individual variables have been entered.
We have developed and validated a simple bedside prediction tool that
can be used to estimate a patient's postdischarge risk of 6-month mortality
in all forms of ACS, regardless of their initial electrocardiogram or biomarker
results. By using the power of the GRACE registry (>25 000 patients studied)
and focusing on the clinically relevant and wholly unbiased end point of all-cause
mortality, we believe that clinicians may find this tool usable and reliable
as they attempt to make key diagnostic and treatment decisions among patients
hospitalized with ACS.
Previous risk tools have been proposed in estimating in-hospital risk
after admission for ACS,1,4,6- 8 including
risk models developed from large clinical trials or registry data by the Thrombolysis
in Myocardial Infarction (TIMI) clinical trials group.4,9 These
models are robust in predicting risk for specific end points and in the population
in which they have been studied. However, comparison between our methods and
those of the TIMI group elucidate the issue of using prediction models in
practice. For example, many ACS trials, including the TIMI trial, have used
a combined end point that includes recurrent coronary ischemia with a secondary
end point of ischemia requiring coronary revascularization. This particular
end point is potentially troublesome because it is influenced by local practice
style in which the availability of a catheterization laboratory may have more
to do with a decision to revascularize than with patient characteristics.16,17 It may also lead to inaccuracies
in overall prediction. For instance, prior coronary artery bypass graft surgery
predicts an increased risk of death after percutaneous coronary intervention
but a decreased likelihood of surgical coronary revascularization in patients
undergoing coronary angioplasty.18 This is
because prior coronary artery bypass graft surgery is associated with a worse
complication rate after reoperative coronary artery bypass graft surgery,
so surgeons may try to avoid it where possible. In ACS, studies suggest that
at the extremes of older age or serum creatinine clearance, interventionalists
reduce their use of percutaneous coronary intervention for fear of complications,
while at the same time the risk of death steadily increases.19 Finally,
it is not clear that revascularization is an event that should be avoided.
Given the benefits of an early invasive strategy, revascularization may be
considered a good event, and in our analysis was a predictor of improved survival
at 6-months postdischarge.
Another limitation of prior ACS prediction models relates to the arbitrary
separation into STEMI and non–ST-segment elevation ACS populations.
Because few models incorporate the entire spectrum of patients with ACS, physicians
wishing to apply these models in routine practice need to be aware of multiple
risk scores for different types of patients with ACS. In the GRACE model,
we were able to demonstrate that diagnostic prediction for in-hospital mortality
is similar whether the ST-segment deviation is elevation or depression.14 In this analysis, we extended that observation to
show that one can robustly predict 6-month mortality, regardless of whether
the patient presents with STEMI, NSTEMI, or unstable angina. The overall c statistic is high (0.81) compared with other risk models
for combined end points such as the TIMI risk model (0.65).9
How generalizable are prediction tools? They are as generalizable as
the population from which they are derived. In this way, we believe that the
GRACE model is unique. In so far as we have studied consecutive patients,
observed in care at 94 hospitals, representing 14 countries, and have used
a population-based enrollment wherever possible, the GRACE registry model
may well be closer to real-world practice than previous studies limited to
clinical trial populations or single-region registries. As such, we believe
that clinicians will find greater confidence in its applicability in their
practices. Because it has been validated on patients enrolled in 2002 and
2003, clinicians can also feel confident that it is current.
GRACE is the largest multinational registry study to include the entire
spectrum of patients with ACS. It is designed to be representative of regional
communities and uses standardized criteria for defining ACS and hospital outcomes
and quality control and audit measures. Given the overall number of deaths
in the development (n = 717) and validation (n = 331) cohorts, one has increased
confidence of the robustness of the model.
Risk-prediction tools are developed in populations, not individuals.
Although the risk-prediction tools may inform practitioners regarding the
estimated likelihood of complications in a patient, each individual patient
is unique and subject to many influences not measured or ascertained in clinical
practice. Local practice referral patterns or practice style may influence
average risks of patients being cared for by an individual clinician. To the
extent that GRACE studies patients cared for all over the world by hundreds
of clinicians, it has an excellent chance to adjust for this variation to
create a risk-prediction instrument that is robust for most situations. However,
like individual patients, each practice has its set of unmeasured variation
that may influence risks, such as socioeconomic status, which was not considered
in the current model. The model is applicable to patients hospitalized with
ACS and discharged alive, not patients being observed in an emergency department.
In summary, our GRACE 6-month postdischarge prediction model is a simple
robust tool for predicting death in patients with ACS and has very good discriminative
ability. We believe that clinicians will find it simple to use and applicable
to clinical practice.