Reilly BM, Evans AT, Schaider JJ, Das K, Calvin JE, Moran LA, Roberts RR, Martinez E. Impact of a Clinical Decision Rule on Hospital Triage of Patients With Suspected Acute Cardiac Ischemia in the Emergency Department. JAMA. 2002;288(3):342-350. doi:10.1001/jama.288.3.342
Author Affiliations: Department of Medicine (Drs Reilly, Evans, Das, Calvin, and Martinez and Ms Moran) and Department of Emergency Medicine (Drs Schaider and Roberts), Cook County Hospital and Rush Medical College, Chicago, Ill.
Context Emergency department (ED) physicians often are uncertain about where
in the hospital to triage patients with suspected acute cardiac ischemia.
Many patients are triaged unnecessarily to intensive or intermediate cardiac
Objective To determine whether use of a clinical decision rule improves physicians'
hospital triage decisions for patients with suspected acute cardiac ischemia.
Design and Setting Prospective before-after impact analysis conducted at a large, urban,
US public hospital.
Participants Consecutive patients admitted from the ED with suspected acute cardiac
ischemia during 2 periods: preintervention group (n = 207 patients enrolled
in March 1997) and intervention group (n = 1008 patients enrolled in August-November
Intervention An adaptation of a previously validated clinical decision rule was adopted
as the standard of care in the ED after a 3-month period of pilot testing
and training. The rule predicts major cardiac complications within 72 hours
after evaluation in the ED and stratifies patients' risk of major complications
into 4 groups—high, moderate, low, and very low—according to electrocardiographic
findings and presence or absence of 3 clinical predictors in the ED.
Main Outcome Measures Safety of physicians' triage decisions, defined as the proportion of
patients with major cardiac complications who were admitted to inpatient cardiac
care beds (coronary care unit or inpatient telemetry unit); efficiency of
decisions, defined as the proportion of patients without major complications
who were triaged to an ED observation unit or an unmonitored ward.
Results By intention-to-treat analysis, efficiency was higher in the intervention
group (36%) than the preintervention group (21%) (difference, 15%; 95% confidence
interval [CI], 8%-21%; P<.001). Safety was not
significantly different (94% in the intervention group vs 89%; difference,
5%; 95% CI, −11% to 39%; P = .57). Subgroup
analysis of intervention-group patients showed higher efficiency when physicians
actually used the decision rule (38% vs 27%; difference, 11%; 95% CI, 3%-18%; P = .01). Improved efficiency was explained solely by different
triage decisions for very low-risk patients. Most surveyed physicians (16/19
[84%]) believed that the decision rule improved patient care.
Conclusions Use of the clinical decision rule had a favorable impact on physicians'
hospital triage decisions. Efficiency improved without compromising safety.
Among patients admitted from the emergency department (ED) with possible
acute cardiac ischemia, only one quarter are diagnosed with unstable angina
or acute myocardial infarction (MI), and less than 5% experience a life-threatening
complication.1,2 Many of these
patients are admitted unnecessarily to cardiac care units. At the same time,
2% to 5% of patients with acute cardiac ischemia are improperly diagnosed
in the ED and not triaged to cardiac care units.3- 8
Some of these patients experience life-threatening complications, raising
concerns about the safety of physicians' decisions.
In response to these concerns, researchers have developed prediction
rules to risk-stratify patients in the ED, and hospitals have established
various levels of care: coronary care units, telemetry units, and, more recently,
short-stay observation units.1,2,9- 17
However, no study has addressed whether accurate risk stratification can improve
physicians' decisions in triaging patients to these different levels of care.
In 1996, Goldman and colleagues1 published
their prediction rule for complications in ED patients with suspected acute
cardiac ischemia. Although these investigators derived and validated their
prediction rule in more than 15 000 ED patients, they did not measure
its actual impact on physicians' decisions or patients' outcomes. In this
study, we performed a prospective impact analysis of their prediction rule,
as recommended by the Evidence-Based Medicine Working Group.18
Using previous studies (Table 118- 21),
we hypothesized that the rule's use would reduce unnecessary admissions to
the coronary care or telemetry unit without increasing complications in patients
triaged instead to observation units or unmonitored wards.
Our prospective study compared physicians' triage decisions before and
after the clinical decision rule was established as the standard of care in
our ED. We studied 2 cohorts of patients admitted consecutively from the ED
with suspected acute cardiac ischemia: one cohort before (preintervention
group) and one cohort after introduction of the decision rule (intervention
group). This study design is consistent with the Evidence-Based Medicine Working
Group conclusion that "randomization of individual patients is unlikely to
be appropriate" in the impact analysis of clinical decision rules.18
The primary impact of the decision rule was assessed by measuring the
safety and efficiency of ED physicians' triage decisions. Safety was defined
as the proportion of all patients who experienced major cardiac complications
within 72 hours who were triaged to a coronary care or telemetry unit after
evaluation in the ED. Efficiency was defined as the proportion of all patients
who did not experience major cardiac complications who were triaged to an
ED observation unit or unmonitored ward.
To test for possible temporal confounding, we used identical methods
to study a third cohort (postintervention group) 1 year after the decision
rule was discontinued as standard practice in the ED.
The study was performed at Cook County Hospital, a 700-bed urban teaching
hospital whose ED cares for 120 000 adults annually. The ED is staffed
by full-time emergency medicine attending physicians and residents in the
departments of emergency medicine or medicine. Usual options for hospital
triage of patients with suspected acute cardiac ischemia include the coronary
care unit, the inpatient telemetry unit, and the ED observation unit. When
the observation unit is full, physicians might also admit patients to an unmonitored
ward. Unlike the 12-bed inpatient telemetry unit, the 11-bed ED observation
unit does not have telemetry monitoring, house staff coverage, or nurses with
special training in cardiac care.20,22
Patients triaged to the observation unit remain for fewer than 24 hours and
receive serial electrocardiograms and cardiac enzyme measurements according
to protocols administered by ED nurses and physician assistants, with backup
from attending physicians. Patients triaged to the telemetry unit have an
average hospital stay of 3 days.
Emergency medicine attending physicians make all admitting decisions,
but cardiology consultation and approval is required for admitting patients
to the coronary care unit. During the 4 years spent studying the decision
rule, there were no significant changes in our hospital's ED patient volume,
number of medical admissions, number of admissions to the ED observation unit,
number of admissions for suspected acute cardiac ischemia, or number of beds
in the coronary care unit, inpatient telemetry unit, or ED observation unit.
Patients were eligible for study if they were triaged from the ED to
the hospital or the ED observation unit with suspected acute cardiac ischemia.
Inclusion criteria included an admitting diagnosis of acute MI, rule out MI,
unstable angina, acute cardiac ischemia, or coronary artery disease if cardiac
enzyme tests were ordered in the ED. Patients in cardiac or respiratory arrest
on initial presentation to the ED were ineligible for study, and we excluded
patients who experienced cardiac or respiratory arrest after presentation
to the ED but before triage could be performed.
The preintervention group was studied prospectively during 4 consecutive
weeks in March 1997.19 After 3 months of pilot
testing the decision rule in the ED, we enrolled the intervention group during
14 consecutive weeks from August 1999 through November 1999. We then discontinued
use of the decision rule during the data analysis phase and 12 months later
enrolled the postintervention group during 4 consecutive weeks in November
Our eligibility criteria differed from those in the study by Goldman
and colleagues1 in 2 ways: Goldman et al enrolled
only patients presenting with chest pain and included patients discharged
home directly from the ED. We enrolled patients with and without chest pain
because a minority of patients presenting with acute coronary syndromes do
not have chest pain,23 and the decision rule
appears to perform well for these patients.20
We excluded patients discharged home directly from the ED because in our ED,
patients are discharged home only if the ED physician has excluded the possibility
of acute cardiac ischemia and identified a convincing alternative diagnosis.
Thus, our study eligibility criteria included all patients in whom the diagnosis
of acute cardiac ischemia remained possible after evaluation by the ED physician.
However, to investigate whether use of the decision rule might have
an unintended effect on decisions to discharge patients home, we studied a
separate cohort of patients discharged home directly from the ED to monitor
the safety of those decisions. During 5 consecutive weeks of the 14-week intervention
period, we identified and followed up all patients discharged home from the
ED after presenting with complaints of chest pain, epigastric pain, or dyspnea
and in whom a 12-lead electrocardiogram was performed to evaluate possible
acute cardiac ischemia. These patients were followed up for occurrence of
major cardiac complications within 72 hours after discharge from the ED.
The rule of Goldman and colleagues1 predicts
major cardiac complications within 72 hours after evaluation in the ED. These
complications include ventricular fibrillation, cardiac arrest, new complete
heart block, insertion of a temporary pacemaker, emergency cardioversion,
cardiogenic shock, use of an intra-aortic balloon pump, intubation, and recurrent
ischemic chest pain requiring urgent coronary revascularization (coronary
artery bypass grafting or percutaneous transluminal angioplasty before discharge
from the hospital). The definition of complications in this study was identical
to that used in the original study.1
The prediction rule stratifies patients' risk of major complications
into 4 groups—high, moderate, low, and very low—according to electrocardiographic
findings (Q waves or ST-segment elevations suggesting acute MI; ST-segment
depressions or T-wave inversions suggesting acute ischemia) and the presence
or absence of 3 clinical predictors in the ED: systolic blood pressure less
than 100 mm Hg, rales heard above both lung bases, and known unstable ischemic
heart disease. Risk is upgraded if complications (eg, cardiogenic shock) occur
in the ED.1
We created a 1-page written decision rule (available on request) for
physicians' use in the ED; it incorporates the prediction rule's risk-stratification
algorithm and Goldman's subsequently published recommendations24
about how to use the algorithm for triage decisions (Figure 1). During the process of creating the written decision rule,
our ED physicians insisted on slight modifications to Goldman's triage recommendations
because, unlike his risk-stratification algorithm, his triage recommendations
had never been evaluated prospectively. However, only 1 of these modifications
had the potential to affect our primary study outcomes: the written decision
rule recommended triage to the telemetry unit, not the observation unit, for
very low-risk patients with left bundle-branch blocks not known to be old.
On the 1-page depiction of the decision rule, there were prompts for
ED physicians to provide the clinical data necessary to apply the rule accurately
and space for an explanation when physicians made a triage decision different
from the decision rule recommendation. Physicians' completion of the written
decision rule form in the ED determined whether the physician had used the
decision rule when making hospital triage decisions.
After approval by the institutional review board, trained research assistants
enrolled all eligible patients who met inclusion criteria. The research assistants,
blinded to the risk stratification process and not involved in subsequent
data collection, identified patients' actual site of triage and personally
interviewed patients to corroborate demographic data needed for posthospital
follow-up (telephone numbers, addresses, next of kin, etc) after obtaining
oral informed consent as specified by the hospital's institutional review
For patients discharged before 72 hours, we determined whether complications
occurred outside the hospital within the 72-hour follow-up period by using
telephone interviews. We visited the residence of patients we were unable
to contact by telephone for face-to-face interviews. Interviews that identified
patient contact with a clinician during the follow-up period prompted a review
of all medical records. Any deaths within 72 hours were attributed to a major
cardiac complication. For patients lost to follow-up, we reviewed clinic and
hospital records and searched county, state, and national death records for
6 to 12 months following enrollment.
After patients were discharged from the hospital, their medical records
were reviewed for possible cardiac complications by trained physician chart
reviewers blinded to the risk stratification process and the clinical decision
rule forms. At least 2 members of the outcomes adjudication panel then independently
reviewed the charts of all patients with possible complications; if the 2
panel members disagreed about the occurrence, type, location, or timing of
a complication, a third panel member reviewed the records and helped resolve
the disagreement after further discussion.
At the conclusion of the study (after all patients were enrolled but
before results were analyzed), we surveyed ED physicians about their perceptions
of the usefulness of the clinical decision rule, its impact on their work
in the ED, its impact on their triage decisions, and their opinions about
continuing or discontinuing its use.
We planned a sample size of 1000 intervention-group patients so that
the width of the 95% confidence interval (CI) for each of our 2 primary outcome
measures—safety and efficiency—would be no greater than 10 percentage
points, assuming a 3% overall risk of major complications. This sample size
provided more than 80% power to demonstrate a 10% difference in efficiency
compared with that of the preintervention group, assuming a 2-sided α
To test for differences between groups, we used Wilcoxon rank-sum tests
for ordinal data and Fisher exact or χ2 tests for categorical
data. For the primary study outcomes—triage decisions and their safety
and efficiency—we constructed CIs for the differences between proportions
in comparison groups by using the method of Miettinen and Nurminen.26
We assessed whether the effect of the clinical decision rule on triage
decisions was homogeneous across the 4 risk strata by using stratified analysis
and logistic regression (with design variables for the interaction between
risk strata and comparison group). We chose to reject the conclusion of a
homogeneous effect if the test for interaction was significant at the 10%
We stratified on baseline risk to control for differences between comparison
groups. In subgroup analysis, we used logistic regression to examine whether
ED physician differences confounded the relationship between use of the clinical
decision rule and triage site (using an indicator variable for each attending
ED physician). To assess for temporal trend in outcomes during the preintervention,
intervention, and postintervention periods, we used χ2 tests
of linear trend. All P values are 2-sided, and because
of multiple comparisons, we considered P<.005
as statistically significant. Data analysis was conducted with Stata, versions
6 and 7 (Stata Corp, College Station, Tex) and Arcus QuickStat Biomedical
(Research Solutions, Cambridge, England).
Patients in the preintervention group have been described.19
Among 1011 eligible patients in the intervention group, 3 were excluded because
of respiratory arrest before triage. Of the remaining 1008 patients (Table 2), follow-up was complete in 994
(98.6%); no deaths were documented in the remaining 14. Overall, 35 patients
(3.5%) experienced major cardiac complications during the initial 72 hours
(Table 3). The clinical decision
rule effectively stratified patients according to risk (Table 4).
Intervention-group and preintervention-group patients were comparable
in distribution across risk strata, overall risk of major complications, and
stratum-specific risk of major complications (Table 4).
Among 973 intervention-group patients who did not experience major complications,
350 were triaged to an observation unit or unmonitored ward. Thus, efficiency
during the intervention period was 36% (350/973), significantly higher than
that of the preintervention group (21% [42/198]; difference, 15%; 95% CI,
Among the 35 intervention-group patients who experienced major cardiac
complications, 33 were triaged to the coronary care unit (n = 18) or telemetry
unit (n = 15), and 2 were triaged to the observation unit. Thus, safety in
the intervention group (94% [33/35]) was not significantly different from
that of the preintervention group (89% [8/9]; difference, 5%; 95% CI: −11%
to 39%; P = .57).
Overall, fewer intervention-group patients were triaged to inpatient
monitored beds (coronary care or telemetry unit), reflecting a significant
increase in triage to the observation unit, a decrease in triage to the telemetry
unit, but no change in triage to the coronary care unit. These changes in
triage decisions were explained by differences involving only very low-risk
patients (P<.001); there were no significant differences
in decisions involving high- (P = .71),
moderate- (P = .73),
or low-risk (P = .43) patients. As a result, patients in the intervention group who were triaged
to the telemetry unit were more likely to be from risk strata higher than
very low risk.
Physicians used the decision rule in 832 (83%) of the 1008 intervention-group
patients. When this subgroup was compared with the subgroup of 176 (17%) intervention-group
patients in whom the decision rule was not used, all comparisons exactly paralleled
the comparisons described above between the entire intervention group and
the preintervention group. The 2 subgroups were comparable in the distribution
across risk strata, stratum-specific risk, and overall risk of major complications.
Efficiency was greater in the subgroup in which the decision rule was actually
used (38% vs 27%; difference, 11%; 95% CI, 3%-18%; P
= .01). In this subgroup, fewer very low-risk patients were triaged to inpatient
monitored beds (P<.001), and a smaller proportion
of admissions to the telemetry unit were patients at very low risk (P = .02). These subgroup differences persisted after patients'
baseline risk and the identity of ED physicians were controlled for in the
logistic regression model.
Figure 2 depicts temporal
trends in physicians' triage decisions for the preintervention (1997), intervention
(1999), and postintervention (2000) periods. Patient characteristics were
comparable during these 3 periods, and tests for linear trend were not significant
(P>.14 for all patients, P
= .22 for very low-risk patients, and P = .64 for
telemetry patients) for each of the 3 outcome measures. Thus, there was no
evidence supporting temporal trend as an alternative explanation for the observed
differences in triage decisions during the intervention period.
The physician survey was completed by 19 (73%) of the 26 attending ED
physicians. Respondents evaluated the prediction rule favorably: 68% (13/19)
reported that they thought it helped them make their triage decisions, and
84% (16/19) reported that they thought it improved patient care. Only 1 respondent
favored discontinuing use of the prediction rule.
During the intervention period, we followed up 326 consecutive patients
in whom possible acute cardiac ischemia was initially suspected and who were
discharged home directly from the ED (and therefore were not included in the
intervention group) after the attending physician excluded the possibility
of acute cardiac ischemia and made an alternative diagnosis. There were no
complications or deaths among the 300 patients with complete follow-up. Among
the 26 patients who were lost to follow-up, there were no recorded deaths
(according to hospital, county, state, and national databases) during the
6 months following enrollment.
Use of the decision rule improved our physicians' decisions. It reduced
unnecessary admissions to inpatient monitored beds without increasing complications
in patients triaged instead to a short-stay observation unit. This improvement
was achieved primarily by identifying very low-risk patients and not admitting
them to inpatient telemetry beds. These results provide strong evidence that
the decision rule can "inform physicians' judgments . . . with beneficial
These findings are important for 3 reasons. First, no other decision
rule or practice guideline applicable to all ED patients with suspected acute
cardiac ischemia has achieved the same high level of evidentiary support.
Goldman and colleagues' original study1 met
all methodologic standards of the Evidence-Based Medicine Working Group for
the derivation and validation of a decision rule.18
Subsequent studies in other settings19,20
provided broader validation of the rule (Table 1). This study extends those findings and documents the beneficial
impact of the decision rule when it is used prospectively in clinical practice
(level 1 evidence).18
Second, our study expands the clinical scope of the decision rule. Goldman
and colleagues1 included in their study only
patients presenting with a chief complaint of chest pain. In this study, we
demonstrated benefit of the decision rule when it is applied to all ED patients
with suspected acute cardiac ischemia, not just those with chest pain, which
is important because one third of all patients with acute MI may present without
Third, our study's primary impact measures—safety and efficiency—link
the decision physicians must make in the ED (triage to an inpatient monitored
bed or not) with the most telling outcome of that decision (occurrence of
a life-threatening complication within the next few days). Although these
impact measures seem obvious, they have not been well described or studied
before. It is important not to confuse safety and efficiency (as we define
them) with the sensitivity and specificity of the decision rule itself. These
2 sets of measures may differ for a variety of reasons: physicians may apply
the decision rule inaccurately or unreliably, they may overrule its recommendations
in specific cases even when using it as a general rule, or they may be unable
to implement its recommendations because of practical constraints (for example,
bed availability). For these reasons, measures of the decision rule's impact
(safety and efficiency) must be distinguished from measures of its predictive
accuracy (sensitivity and specificity).
Although our study is not a randomized controlled trial, our analyses
strongly support its internal validity. Baseline risk stratification and complication
rates were similar in all patient groups studied. Improvement in physicians'
efficiency could not be explained by temporal trends, and subgroup analysis
during the intervention period suggested that actual use of the decision rule
best explained the differences.
However, the external validity of our findings must be questioned. When
used in other settings, the decision rule's impact may vary for several reasons.
First, not all hospitals have observation units.27,28
Hospitals without observation units will need alternative low-intensity-care
triage options for very low-risk patients to benefit from use of the decision
rule. Second, we undertook this study only after extensive baseline data collection,
pilot testing, and simulated impact analyses of the decision rule.19- 21 Similar painstaking
groundwork—essential for physician "buy in" to quality improvement—may
be needed in other hospitals, too. We cannot exclude the possibility of a
Hawthorne effect in the intervention group, because the 1-page decision rule
might have served as a daily reminder of ongoing investigation, whereas no
such reminder occurred during the preintervention period. Finally, physicians'
diagnosis of unstable angina and interpretation of electrocardiograms vary
Research designed to improve and standardize performance in these areas—critical
to the accurate, reliable use of the decision rule—deserves wide attention.2,14
Precedent for caution about clinical impact can be found in studies
of the Acute Cardiac Ischemia Time-Insensitive Predictive Instrument (ACI-TIPI),2 which provides probability estimates of the diagnosis
of unstable angina or acute MI. In the pretelemetry era (before 1983), Pozen
and colleagues9 found that use of this instrument
modestly reduced admission to the coronary care unit for patients without
acute cardiac ischemia (from 24% to 17%; P = .003).
However, in a much larger study (N = 10 016) involving different hospitals
in 1993,2 the ACI-TIPI instrument had no significant
impact on attending physicians' admitting decisions for patients with or without
acute cardiac ischemia. Thus, we believe that broad verification of the decision
rule's impact is no less important than broad validation of its accuracy.
Patients discharged home directly from the ED were not included in our
measures of the decision rule's safety and efficiency. If they were, the calculated
estimate of efficiency would rise markedly. The absence of life-threatening
complications in such patients is reassuring; however, uncomplicated MIs may
have been missed.3,5- 7
The constant distribution of risk strata across the 3 periods—before,
during, and after use of the decision rule—suggests that the decision
rule did not change physicians' threshold for hospital triage, another potential
It is noteworthy that much larger sample sizes will be needed to achieve
narrow confidence limits for the decision rule's safety. For example, given
a 4% major complication rate,1,19
14 500 patients would be needed in each group to have 90% power to demonstrate
a statistically significant increase in safety from 89% to 94%, the point
estimates in our intention-to-treat analysis. Meta-analysis of more trials
like this study, performed in varied settings, could provide the necessary
statistical power, which is important from a medicolegal perspective because
litigants may demand perfect safety (100%), an unrealistic goal. There have
been no published reports documenting safety greater than that observed in
this study when physicians actually used the decision rule (97%).
Future studies should also measure the impact of the decision rule on
other outcomes: longer-term morbidity and mortality, resource use and cost-effectiveness,
and physicians' timely use of complementary decision aids for treating subsets
of triaged patients.30- 35
It is also important to learn whether information not used to derive and validate
the original prediction rule—serum markers,36- 40
for example, or physicians' clinical judgment—can further improve the
decision rule's accuracy or its safe efficient use. But, pending future research,
we believe the decision rule provides the best available evidence-based foundation
for physicians' decision making in this challenging clinical area.