[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.211.168.204. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1. Comparison of Predicted vs Observed Mortality Rate for the Validation Cohort
Image description not available.
Each data marker represents a decile of risk.
Figure 2. GRACE Prediction Score Card and Nomogram for All-Cause Mortality From Discharge to 6 Months
Image description not available.
Table 1. Baseline Characteristics of Patients in the GRACE Model*
Image description not available.
Table 2. In-Hospital Treatments and Outcomes of Patients in the GRACE Model*
Image description not available.
Table 3. Multivariate Predictors for the 6-Month Postdischarge Mortality Model
Image description not available.
Table 4. Model Performance in the STEMI, NSTEMI, and Unstable Angina Groups
Image description not available.
1.
Ohman EM, Granger CB, Harrington RA, Lee KL. Risk stratification and therapeutic decision making in acute coronary syndromes.  JAMA.2000;284:876-878.PubMed
2.
Bertrand ME, Simoons ML, Fox KA.  et al.  Management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: the Task Force on the Management of Acute Coronary Syndromes of the European Society of Cardiology.  Eur Heart J.2002;23:1809-1840.PubMed
3.
Braunwald E, Antman EM, Beasley JW.  et al.  ACC/AHA 2002 guideline update for the management of patients with unstable angina and non-ST-segment elevation myocardial infarction—summary article: a report of the American College of Cardiology/American Heart Association task force on practice guidelines (Committee on the Management of Patients With Unstable Angina).  J Am Coll Cardiol.2002;40:1366-1374.PubMed
4.
Morrow DA, Antman EM, Charlesworth A.  et al.  TIMI risk score for ST-elevation myocardial infarction: a convenient, bedside, clinical score for risk assessment at presentation.  Circulation.2000;102:2031-2037.PubMed
5.
Krumholz HM, Chen J, Wang Y.  et al.  Comparing AMI mortality among hospitals in patients 65 years of age and older: evaluating methods of risk adjustment.  Circulation.1999;99:2986-2992.PubMed
6.
Boersma E, Pieper KS, Steyerberg EW.  et al.  Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation: results from an international trial of 9461 patients.  Circulation.2000;101:2557-2567.PubMed
7.
Morrow DA, Antman EM, Giugliano RP.  et al.  A simple risk index for rapid initial triage of patients with ST-elevation myocardial infarction: an InTIME II substudy.  Lancet.2001;358:1571-1575.PubMed
8.
Lee KL, Woodlief LH, Topol EJ.  et al.  Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction: results from an international trial of 41,021 patients.  Circulation.1995;91:1659-1668.PubMed
9.
Antman EM, Cohen M, Bernink PJ.  et al.  The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making.  JAMA.2000;284:835-842.PubMed
10.
Rathore SS, Weinfurt KP, Gross CP, Krumholz HM. Validity of a simple ST-elevation acute myocardial infarction risk index: are randomized trial prognostic estimates generalizable to elderly patients?  Circulation.2003;107:811-816.PubMed
11.
The GRACE Investigators.  Rationale and design of the GRACE (Global Registry of Acute Coronary Events) Project: a multinational registry of patients hospitalized with acute coronary syndromes.  Am Heart J.2001;141:190-199.PubMed
12.
Eagle KA, Goodman SG, Avezum A.  et al.  Practice variation and missed opportunities for reperfusion in ST-segment-elevation myocardial infarction: findings from the Global Registry of Acute Coronary Events (GRACE).  Lancet.2002;359:373-377.PubMed
13.
 SAS STAT User's Guide, Version 8 . Cary, NC: SAS Institute Inc; 1999:2571-2657.
14.
Granger CB, Goldberg RJ, Dabbous O.  et al.  Predictors of hospital mortality in the global registry of acute coronary events.  Arch Intern Med.2003;163:2345-2353.PubMed
15.
Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests.  JAMA.1982;247:2543-2546.PubMed
16.
Budaj A, Brieger D, Steg PG.  et al.  Global patterns of use of antithrombotic and antiplatelet therapy in patients with acute coronary events: insights from the Global Registry of Acute Coronary Events (GRACE).  Am Heart J.2003;146:999-1006.PubMed
17.
Brieger D, Nussbacher A, Knobel M.  et al. for the GRACE Investigators.  Predictors of in-hospital coronary artery bypass grafting across the spectrum of acute coronary syndromes: an analysis from the Global Registry of Acute Coronary Events.  Eur Heart J.In press.
18.
Moscucci M, Malenka DJ, Wennberg DE.  et al.  Potential bias in the use of combined end points in quality assessment of percutaneous coronary interventions: an analysis of the relationship beween history of prior CABG and MACE.  J Am Coll Cardiol.2001;37A:502A.
19.
Moscucci M, Kline-Rogers E, Share D.  et al.  Simple bedside additive tool for prediction of in-hospital mortality after percutaneous coronary interventions.  Circulation.2001;104:263-268.PubMed
Original Contribution
June 9, 2004

A Validated Prediction Model for All Forms of Acute Coronary SyndromeEstimating the Risk of 6-Month Postdischarge Death in an International Registry

Author Affiliations

Author Affiliations: Department of Internal Medicine, University of Michigan Health System, Ann Arbor (Drs Eagle and Lim); Department of Medicine, University of Massachusetts Medical School, Worcester (Drs Dabbous, Goldberg, and Gore); Department of Medicine, Duke University Medical Center, Durham, NC (Dr Granger and Ms Pieper); Department of Cardiology, Universitair Ziekenhuis Gasthuisberg, Leuven, Belgium (Dr Van de Werf); Canadian Heart Research Centre and Terrence Donnelly Heart Centre, and Division of Cardiology, St Michael's Hospital, University of Toronto, Toronto, Ontario (Dr Goodman); Cardiology, Hôpital Bichat, Paris, France (Dr Steg); Postgraduate Medical School, Grochowski Hospital, Warsaw, Poland (Dr Budaj); Clinical Research Center, Education, and Research Institute Hospital Albert Einstein, São Paulo, Brazil (Dr Avezum); Royal Brompton and Harefield NHS Trust, London, England (Dr Flather); and The University and The Royal Infirmary of Edinburgh, Edinburgh, Scotland (Dr Fox).

JAMA. 2004;291(22):2727-2733. doi:10.1001/jama.291.22.2727
Context

Context Accurate estimation of risk for untoward outcomes after patients have been hospitalized for an acute coronary syndrome (ACS) may help clinicians guide the type and intensity of therapy.

Objective To develop a simple decision tool for bedside risk estimation of 6-month mortality in patients surviving admission for an ACS.

Design, Setting, and Patients A multinational registry, involving 94 hospitals in 14 countries, that used data from the Global Registry of Acute Coronary Events (GRACE) to develop and validate a multivariable stepwise regression model for death during 6 months postdischarge. From 17 142 patients presenting with an ACS from April 1, 1999, to March 31, 2002, and discharged alive, 15 007 (87.5%) had complete 6-month follow-up and represented the development cohort for a model that was subsequently tested on a validation cohort of 7638 patients admitted from April 1, 2002, to December 31, 2003.

Main Outcome Measure All-cause mortality during 6 months postdischarge after admission for an ACS.

Results The 6-month mortality rates were similar in the development (n = 717; 4.8%) and validation cohorts (n = 331; 4.7%). The risk-prediction tool for all forms of ACS identified 9 variables predictive of 6-month mortality: older age, history of myocardial infarction, history of heart failure, increased pulse rate at presentation, lower systolic blood pressure at presentation, elevated initial serum creatinine level, elevated initial serum cardiac biomarker levels, ST-segment depression on presenting electrocardiogram, and not having a percutaneous coronary intervention performed in hospital. The c statistics for the development and validation cohorts were 0.81 and 0.75, respectively.

Conclusions The GRACE 6-month postdischarge prediction model is a simple, robust tool for predicting mortality in patients with ACS. Clinicians may find it simple to use and applicable to clinical practice.

Clinical prediction models may be helpful for medical decision making1 as patients judged to be at higher risk may receive more aggressive surveillance and/or earlier treatment, while patients estimated to be at lower risk may be reassured and managed less aggressively.2,3 By using simple yet valid risk calculations, clinicians can accurately advise patients about their likelihood of an event, and how this likelihood translates into treatment decisions.

The acute coronary syndrome (ACS) encompasses a continuum of conditions ranging from ST-segment elevation myocardial infarction (STEMI) to non–ST-segment elevation myocardial infarction (NSTEMI) and unstable angina.1,48 Numerous risk-prediction models for differing outcomes exist for the different types of ACS.5,9 Most models have been developed from large randomized clinical trial populations in which the generalizability to risk prediction in the average clinician's experience is questionable.610

We were interested in developing a simple risk-prediction tool that would be applicable to all types of ACS, would focus on an important clinical end point of all-cause mortality, and be developed in patients who are similar to those encountered in routine clinical practice.

METHODS

The Global Registry of Acute Coronary Events (GRACE) is a multinational cooperative effort involving 94 hospitals in 14 countries that has been designed to reflect an unbiased, representative population of patients with ACS. Full details of the GRACE methods have been previously published.11,12 For this analysis, patients had to be at least 18 years old and alive at the time of discharge, be admitted for ACS as a presumptive diagnosis (ie, have symptoms consistent with acute myocardial ischemia), and have at least 1 of the following: electrocardiographic changes consistent with ACS, serial increases in serum cardiac biomarkers, and/or documentation of coronary artery disease. The qualifying ACS must not have been precipitated by significant noncardiovascular comorbidity, such as acute anemia or hyperthyroidism. At approximately 6 months after hospital discharge, patients were followed up to ascertain vital status. At each enrolling hospital, study investigators worked with their ethics or institutional review board to obtain appropriate approval to participate.

To ensure enrollment of an unbiased population, the first 10 to 20 consecutive eligible patients were recruited from each site per month. Data were collected by trained coordinators using a standardized case report form. Demographic characteristics, medical history, presenting symptoms, biochemical and electrocardiographic findings, treatment practices, and a variety of hospital outcome data were collected. Standardized definitions for all patient-related variables and clinical diagnoses were used.11,12 At discharge, all cases were assigned to STEMI, NSTEMI, or unstable angina categories.11,12 Standardized definitions were used for selected hospital complications and outcomes.11

Model Development

The primary end point was all-cause mortality occurring within 6 months of discharge from hospital. The model was developed by using a multivariable Cox proportional hazards regression backward elimination technique.13 The following variables were included in the stepwise Cox proportional hazards regression: baseline characteristics, symptoms and signs at presentation, in-hospital treatments and procedures, and in-hospital complications. A backward stepwise technique evaluated all potential univariate correlates (P<.25) to create a multivariable model containing variables with P<.05. No imputation was performed on the final model. Imputation was tested but did not influence the identification of multivariate predictors or the discriminative power of the model for predicting death.14 All variables in the final model met the assumptions for proportional hazards.

The model accuracy was tested in several ways. First, we developed the model in all patients with ACS enrolled in GRACE between April 1, 1999, and March 31, 2002. The c statistic was extended to evaluate the discrimination of survival analytic techniques.15 We checked for any nonlinear relationship between death and each variable in the final model and found none. Selected testing was performed for interactions using the significant prognostic variables from the final model based on interactions that have been reported from other published models. Second, we tested the model in a validation cohort of consecutive patients enrolled in GRACE between April 1, 2002, and December 31, 2003. Statistical analyses were performed using SAS version 8.2 (SAS Institute Inc, Cary, NC) and S-Plus version 3.4 (MathSoft Inc, Seattle, Wash).

The overall follow-up rate in our development cohort was 87.5% for death. A comparison of the patients with and those patients without available follow-up data demonstrated no significant clinical differences in terms of baseline characteristics, symptoms at presentation, in-hospital treatments and procedures, in-hospital outcomes, and postdischarge outcomes.

Making the Model Usable

We developed a bedside prediction tool that can be applied to either a pocket card or a handheld device. Clinical prediction variables were given weighted scores based on the model's variable coefficients. The prediction tool considers the variables taken from the final model assigning a point total to each variable, which allows a total point score for each patient to be calculated. This then is applied to a reference plot nomogram, which shows the corresponding risk of death. Alternatively, the risk of death could be calculated using a handheld device. The clinical application can be found at http://www.outcomes-umassmed.org/grace.

RESULTS

Baseline characteristics, in-hospital treatments, and outcomes of the 15 007 patients used to develop the model (development cohort), 7638 patients used to test the model (validation cohort), and 5116 patients for whom 6-month vital status was unknown are shown in Table 1 and Table 2. The mean age was 65 years and 67% of the cohort were men. Prior or current smoking and hypertension were present as risk factors in more than half of the patients. Approximately 45% of patients had a diagnosis of hyperlipidemia and less than 25% had diabetes mellitus.

The 6-month mortality risk was similar in the development (4.8%, n = 717) and validation cohorts (4.7%, n = 331). Nine multivariate predictors of mortality were identified (Table 3). The model calibrations were retained when testing the model in an independent validation cohort (Table 3 and Figure 1). The model performed well in all forms of ACS with a c statistic of at least 0.70 (Table 4).

Figure 2 illustrates a method to estimate a patient's risk depending on the total score obtained by summing the individual scores for each of the 9 variables in the model. A similar nomogram can be programmed into a handheld device to make the risk calculation automatic once the individual variables have been entered.

COMMENT

We have developed and validated a simple bedside prediction tool that can be used to estimate a patient's postdischarge risk of 6-month mortality in all forms of ACS, regardless of their initial electrocardiogram or biomarker results. By using the power of the GRACE registry (>25 000 patients studied) and focusing on the clinically relevant and wholly unbiased end point of all-cause mortality, we believe that clinicians may find this tool usable and reliable as they attempt to make key diagnostic and treatment decisions among patients hospitalized with ACS.

Previous risk tools have been proposed in estimating in-hospital risk after admission for ACS,1,4,68 including risk models developed from large clinical trials or registry data by the Thrombolysis in Myocardial Infarction (TIMI) clinical trials group.4,9 These models are robust in predicting risk for specific end points and in the population in which they have been studied. However, comparison between our methods and those of the TIMI group elucidate the issue of using prediction models in practice. For example, many ACS trials, including the TIMI trial, have used a combined end point that includes recurrent coronary ischemia with a secondary end point of ischemia requiring coronary revascularization. This particular end point is potentially troublesome because it is influenced by local practice style in which the availability of a catheterization laboratory may have more to do with a decision to revascularize than with patient characteristics.16,17 It may also lead to inaccuracies in overall prediction. For instance, prior coronary artery bypass graft surgery predicts an increased risk of death after percutaneous coronary intervention but a decreased likelihood of surgical coronary revascularization in patients undergoing coronary angioplasty.18 This is because prior coronary artery bypass graft surgery is associated with a worse complication rate after reoperative coronary artery bypass graft surgery, so surgeons may try to avoid it where possible. In ACS, studies suggest that at the extremes of older age or serum creatinine clearance, interventionalists reduce their use of percutaneous coronary intervention for fear of complications, while at the same time the risk of death steadily increases.19 Finally, it is not clear that revascularization is an event that should be avoided. Given the benefits of an early invasive strategy, revascularization may be considered a good event, and in our analysis was a predictor of improved survival at 6-months postdischarge.

Another limitation of prior ACS prediction models relates to the arbitrary separation into STEMI and non–ST-segment elevation ACS populations. Because few models incorporate the entire spectrum of patients with ACS, physicians wishing to apply these models in routine practice need to be aware of multiple risk scores for different types of patients with ACS. In the GRACE model, we were able to demonstrate that diagnostic prediction for in-hospital mortality is similar whether the ST-segment deviation is elevation or depression.14 In this analysis, we extended that observation to show that one can robustly predict 6-month mortality, regardless of whether the patient presents with STEMI, NSTEMI, or unstable angina. The overall c statistic is high (0.81) compared with other risk models for combined end points such as the TIMI risk model (0.65).9

How generalizable are prediction tools? They are as generalizable as the population from which they are derived. In this way, we believe that the GRACE model is unique. In so far as we have studied consecutive patients, observed in care at 94 hospitals, representing 14 countries, and have used a population-based enrollment wherever possible, the GRACE registry model may well be closer to real-world practice than previous studies limited to clinical trial populations or single-region registries. As such, we believe that clinicians will find greater confidence in its applicability in their practices. Because it has been validated on patients enrolled in 2002 and 2003, clinicians can also feel confident that it is current.

GRACE is the largest multinational registry study to include the entire spectrum of patients with ACS. It is designed to be representative of regional communities and uses standardized criteria for defining ACS and hospital outcomes and quality control and audit measures. Given the overall number of deaths in the development (n = 717) and validation (n = 331) cohorts, one has increased confidence of the robustness of the model.

Risk-prediction tools are developed in populations, not individuals. Although the risk-prediction tools may inform practitioners regarding the estimated likelihood of complications in a patient, each individual patient is unique and subject to many influences not measured or ascertained in clinical practice. Local practice referral patterns or practice style may influence average risks of patients being cared for by an individual clinician. To the extent that GRACE studies patients cared for all over the world by hundreds of clinicians, it has an excellent chance to adjust for this variation to create a risk-prediction instrument that is robust for most situations. However, like individual patients, each practice has its set of unmeasured variation that may influence risks, such as socioeconomic status, which was not considered in the current model. The model is applicable to patients hospitalized with ACS and discharged alive, not patients being observed in an emergency department.

In summary, our GRACE 6-month postdischarge prediction model is a simple robust tool for predicting death in patients with ACS and has very good discriminative ability. We believe that clinicians will find it simple to use and applicable to clinical practice.

References
1.
Ohman EM, Granger CB, Harrington RA, Lee KL. Risk stratification and therapeutic decision making in acute coronary syndromes.  JAMA.2000;284:876-878.PubMed
2.
Bertrand ME, Simoons ML, Fox KA.  et al.  Management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: the Task Force on the Management of Acute Coronary Syndromes of the European Society of Cardiology.  Eur Heart J.2002;23:1809-1840.PubMed
3.
Braunwald E, Antman EM, Beasley JW.  et al.  ACC/AHA 2002 guideline update for the management of patients with unstable angina and non-ST-segment elevation myocardial infarction—summary article: a report of the American College of Cardiology/American Heart Association task force on practice guidelines (Committee on the Management of Patients With Unstable Angina).  J Am Coll Cardiol.2002;40:1366-1374.PubMed
4.
Morrow DA, Antman EM, Charlesworth A.  et al.  TIMI risk score for ST-elevation myocardial infarction: a convenient, bedside, clinical score for risk assessment at presentation.  Circulation.2000;102:2031-2037.PubMed
5.
Krumholz HM, Chen J, Wang Y.  et al.  Comparing AMI mortality among hospitals in patients 65 years of age and older: evaluating methods of risk adjustment.  Circulation.1999;99:2986-2992.PubMed
6.
Boersma E, Pieper KS, Steyerberg EW.  et al.  Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation: results from an international trial of 9461 patients.  Circulation.2000;101:2557-2567.PubMed
7.
Morrow DA, Antman EM, Giugliano RP.  et al.  A simple risk index for rapid initial triage of patients with ST-elevation myocardial infarction: an InTIME II substudy.  Lancet.2001;358:1571-1575.PubMed
8.
Lee KL, Woodlief LH, Topol EJ.  et al.  Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction: results from an international trial of 41,021 patients.  Circulation.1995;91:1659-1668.PubMed
9.
Antman EM, Cohen M, Bernink PJ.  et al.  The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making.  JAMA.2000;284:835-842.PubMed
10.
Rathore SS, Weinfurt KP, Gross CP, Krumholz HM. Validity of a simple ST-elevation acute myocardial infarction risk index: are randomized trial prognostic estimates generalizable to elderly patients?  Circulation.2003;107:811-816.PubMed
11.
The GRACE Investigators.  Rationale and design of the GRACE (Global Registry of Acute Coronary Events) Project: a multinational registry of patients hospitalized with acute coronary syndromes.  Am Heart J.2001;141:190-199.PubMed
12.
Eagle KA, Goodman SG, Avezum A.  et al.  Practice variation and missed opportunities for reperfusion in ST-segment-elevation myocardial infarction: findings from the Global Registry of Acute Coronary Events (GRACE).  Lancet.2002;359:373-377.PubMed
13.
 SAS STAT User's Guide, Version 8 . Cary, NC: SAS Institute Inc; 1999:2571-2657.
14.
Granger CB, Goldberg RJ, Dabbous O.  et al.  Predictors of hospital mortality in the global registry of acute coronary events.  Arch Intern Med.2003;163:2345-2353.PubMed
15.
Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests.  JAMA.1982;247:2543-2546.PubMed
16.
Budaj A, Brieger D, Steg PG.  et al.  Global patterns of use of antithrombotic and antiplatelet therapy in patients with acute coronary events: insights from the Global Registry of Acute Coronary Events (GRACE).  Am Heart J.2003;146:999-1006.PubMed
17.
Brieger D, Nussbacher A, Knobel M.  et al. for the GRACE Investigators.  Predictors of in-hospital coronary artery bypass grafting across the spectrum of acute coronary syndromes: an analysis from the Global Registry of Acute Coronary Events.  Eur Heart J.In press.
18.
Moscucci M, Malenka DJ, Wennberg DE.  et al.  Potential bias in the use of combined end points in quality assessment of percutaneous coronary interventions: an analysis of the relationship beween history of prior CABG and MACE.  J Am Coll Cardiol.2001;37A:502A.
19.
Moscucci M, Kline-Rogers E, Share D.  et al.  Simple bedside additive tool for prediction of in-hospital mortality after percutaneous coronary interventions.  Circulation.2001;104:263-268.PubMed
×