Ferguson, Jr TB, Peterson ED, Coombs LP, Eiken MC, Carey ML, Grover FL, DeLong ER, for the Society of Thoracic Surgeons and the National Cardiac Database . Use of Continuous Quality Improvement to Increase Use of Process Measures in Patients Undergoing Coronary Artery Bypass Graft SurgeryA Randomized Controlled Trial. JAMA. 2003;290(1):49-56. doi:10.1001/jama.290.1.49
Author Affiliations: Department of Surgery and Cardiovascular Outcomes Research Group, Louisiana State University Health Sciences Center, New Orleans (Dr Ferguson); Duke Clinical Research Institute, Duke University, Durham, NC (Drs Peterson, Coombs, and DeLong); Society of Thoracic Surgeons, Chicago, Ill (Mss Eiken and Carey); and Department of Surgery, University of Colorado Health Sciences Center, Denver (Dr Grover).
Context A rigorous evaluation of continuous quality improvement (CQI) in medical
practice has not been carried out on a national scale.
Objective To test whether low-intensity CQI interventions can be used to speed
the national adoption of 2 coronary artery bypass graft (CABG) surgery process-of-care
measures: preoperative β-blockade therapy and internal mammary artery
(IMA) grafting in patients 75 years or older.
Design, Setting, and Participants Three hundred fifty-nine academic and nonacademic hospitals (treating
267 917 patients using CABG surgery) participating in the Society of
Thoracic Surgeons National Cardiac Database between January 2000 and July
2002 were randomized to a control arm or to 1 of 2 groups that used CQI interventions
designed to increase use of the process-of-care measures.
Intervention Each intervention group received measure-specific information, including
a call to action to a physician leader; educational products; and periodic
longitudinal, nationally benchmarked, site-specific feedback.
Main Outcome Measure Differential incorporation of the targeted care processes into practice
at the intervention sites vs the control sites, assessed by measuring preintervention
(January-December 2000)/postintervention (January 2001-July 2002) site differences
and by using a hierarchical patient-level analysis.
Results From January 2000 to July 2002, use of both process measures increased
nationally (β-blockade, 60.0%-65.6%; IMA grafting, 76.2%-82.8%). Use
of β-blockade increased significantly more at β-blockade intervention
sites (7.3% [SD, 12.8%]) vs control sites (3.6% [SD, 11.5%]) in the preintervention/postintervention
(P = .04) and hierarchical analyses (P<.001). Use of IMA grafting also tended to increase at IMA intervention
sites (8.7% [SD, 17.5%]) vs control sites (5.4% [SD,15.8%]) (P = .20 and P = .11 for preintervention/postintervention
and hierarchical analyses, respectively). Both interventions tended to have
more impact at lower-volume CABG sites (for interaction: P = .04 for β-blockade; P = .02 for IMA
Conclusions A multifaceted, physician-led, low-intensity CQI effort can improve
the adoption of care processes into national practice within the context of
a medical specialty society infrastructure.
In 1917 Ernest Codman, MD, a Massachusetts surgeon, described his view
of a quality evaluation system for medicine, in which clinicians assessed
outcomes and the processes that led to those outcomes.1 The
modern iteration of Codman's quality system for medicine is continuous quality
improvement (CQI).2,3 Adapted
from industrial manufacturing principles in Japan and the United States,4 CQI in medicine is the repetitive cycle of process
and outcomes measurement, design and implementation of interventions to improve
the processes of care, and remeasurement to determine the effect on quality
of care. Successful CQI programs in medicine have been difficult to achieve,
in part due to a lack of appropriate information technology5 and
Randomized trials testing the effectiveness of CQI as an approach to
quality improvement in medicine to date have yielded mixed results. A report
by Soumerai et al7 documenting the impact of
local opinion leaders in 37 centers was mostly positive, but subsequent studies8,9 did not show a benefit from CQI efforts.
These trials involved small numbers of sites, making difficult the scalability
of the finding to the nation as a whole.
The report herein documents the results of a randomized trial of CQI
in medicine undertaken on a national scale. We used the infrastructure of
the Society of Thoracic Surgeons (STS) National Cardiac Database (NCD) and
a partnership between the STS and the Duke Clinical Research Institute (DCRI)
to administer the trial under a grant from the Agency for Healthcare Research
and Quality to the STS. The setting of the trial was coronary artery bypass
graft (CABG) surgery from January 2000 until July 2002, with the first round
of intervention taking place in January 2001.
The objective of this trial was to test whether a low-intensity CQI
intervention could be used to speed the national adoption of 2 CABG surgery
process-of-care measures: β-blockade and internal mammary artery (IMA)
The voluntary NCD was begun in 1989,10 and
in collaboration with the DCRI outcomes group has developed mortality, morbidity,
and length-of-stay risk models for CABG surgery and other major cardiac procedures
for adults.11 Changes in the NCD to facilitate
quality improvement efforts have been documented elsewhere.12,13 As
of December 2002 there were more than 2.1 million adult patient records in
the NCD collected from over 400 sites nationwide.14
The quality of this voluntary dataset has been assessed in a regional
independent chart abstraction study, which documented a 96.2% correlation
between submitted and reabstracted data elements.15 Completeness
of NCD data has also been compared with data from a Centers for Medicare and
Medicaid Services diagnosis–related group dataset (1994-1999) for CABG
surgery, and no evidence for underreporting of cases or postoperative events
was found in the STS data.15
Preoperative β-Blockade Therapy in CABG. This measure was selected because of the protective effects and underuse
of β-blockade reported in patients with cardiovascular disease16- 18 and in the noncardiac
surgical literature.19,20 Nonetheless,
some surgeons have expressed concerns regarding use of β-blockade in
CABG surgery due to its negative inotropic effects.21 Additionally,
incorporation of this measure requires significant collaboration of the surgical
team with cardiologists and anesthesiologists. Thus this β-blockade measure
was felt to be a complex process measure that assesses the incorporation of
new evidence into practice under circumstances in which cooperation among
clinicians is necessary.
IMA Grafting in Elderly Patients. The acute22,23 and long-term24,25 survival benefit conveyed by IMA
grafting to the left anterior descending coronary artery in patients undergoing
CABG surgery overall has been well documented. However, use of IMA grafting
in elderly (≥75 years) patients was controversial at the trial inception
because of age comorbidity (associated with increased procedural risk)26 and to increased technical complexity and potential
for morbidity of IMA grafting (prolonged ventilation, increased perioperative
bleeding),27 and to absence of a long-term
mortality benefit. Because its use was solely at the discretion of the operating
surgeon, IMA grafting was considered a simple measure. This measure thus assessed
the expansion of existing evidence into a new patient population.
Prior to the trial, all STS participant sites were surveyed regarding
CABG care processes. Of 477 (93%) sites responding, 348 (73%) estimated they
used preoperative β-blockade therapy in their patients; however, 86 (18%)
strongly disagreed with its use. Likewise, 372 (78%) estimated they used IMA
grafting in elderly patients, but 105 (22%) strongly disagreed with its use
in this setting.
Because sites were the unit of randomization, the power for these analyses
was conservatively calculated using sites as the unit of analysis. The mean
(SD) percentage of β-blockade use in the 1997 STS database retrieval
of data was 55% (17%) across sites. Assuming an increase of 5 percentage points
in use over time in the control arm, sample sizes of 110 sites in the control
arm and 110 in each intervention group would yield 86% power to detect an
improvement of 7 percentage points from baseline to end-of-study use of β-blockade
(ie, 67% use of β-blockade in the intervention group vs 60% in the control
arm at study end, or a difference of 7 percentage points in improvement between
groups), with a 2-sample t test and a conservative
estimate of the variance in the difference. The SD across sites for use of
IMA grafting was much lower; assuming the same level of difference, fewer
sites were needed for the IMA analysis to detect the same magnitude of difference.
A total of 399 sites performing cardiac surgery and geographically distributed
within the United States were considered for inclusion in this study based
on ongoing participation in the NCD over the prior 3 years. As a component
of the study, 40 sites in Alabama, Colorado, Iowa, Minnesota, New Mexico,
and Wyoming participated in a regional consortium group and were not randomized
or included in this analysis. The remaining 359 sites were randomized into
1 of 3 arms (Figure 1). Sites randomized
to the intervention arm received either material on preoperative β-blockade
(n = 124) or on use of IMA grafting in elderly patients (n = 120), and those
randomized to the control arm (n = 115) received no intervention material.
Some sites (n = 40) did not provide data during the trial because of
economic and/or software issues. There were no systematic differences between
intervention groups among these sites, so a resulting selection bias was unlikely.
Participant sites were stratified by yearly volume of CABG surgery prior
to randomization by the DCRI because CABG processes and outcomes may be significantly
influenced by the procedural volume at that site.
All NCD sites were informed that they may periodically receive supplemental
educational reports in addition to the standard site-specific semiannual reports.
Participants were specifically not told of the current study design, nor that
other NCD participants might be receiving interventions different than the
one they received. The Duke University institutional review board served as
the multiple projects assurance entity for this study for the STS, and determined
that informed consent was not required.
From January through December 2000, baseline measurement data were collected
from all 359 sites. In January 2001, a sequence of 3 intervention rounds was
begun by distribution of material to randomized sites. This low-intensity
CQI intervention consisted of (1) identifying a local opinion leader (quality
champion) at each site; (2) a call-to-action letter; (3) process-measure data
(scientific evidence for use of data; site-specific measure performance data
illustrated against regional, national, and national "best practice" benchmarks);
(4) action plan for CQI; (5) CQI slide set and reprints of evidence-based
data for process measure and CQI; and (6) contact information.
In round 1, primary recipients were the CQI leader and the data manager
at the site, and abbreviated electronic versions were sent to secondary surgeons
at the sites. Each site documented if the material was used and by whom.
Rounds 2 (July 2001) and 3 (November 2001) were similar in content.
Round 2 additionally included a CQI newsletter specific for the appropriate
measure. For round 3, a measure-specific CQI Web site was created, with a
tracking system for collecting the number of hits with site identification.
Demographic patient-level and hospital-level characteristics between
the 3 arms were compared to evaluate the adequacy of the randomization scheme.
When preoperative β-blockade therapy was considered, all patients
were included in the analyses; where use of IMA grafting was considered, patients
younger than 75 years and reoperative cases were excluded (Figure 1). First, we performed a preintervention/postintervention
analysis for which the outcomes of interest were the site differences between
preintervention (January 2000-December 2000) and postintervention (July 2001-July
2002) percentage for use of the measures. Mean differences in use of the measures
among intervention sites were compared with mean differences in the control
arm using 2-sample t tests.
We then performed a risk-adjusted patient-level analysis to account
for differences in patient mixes at the different sites. The outcome measure
for this analysis was a dichotomous variable indicating whether the patient
received the measure of interest. A hierarchical logistic regression analysis
included 28 preoperative patient characteristics used in the current STS model
for CABG mortality risk along with random site effects to account for within-site
clustering.28 To allow for changes in the use
of measures over time, a time trend (linear on the logit scale) was included.
To determine whether the time trend for intervention groups differed from
that of the control arm, the statistical test of interest was the interaction
between time and intervention group.
We also performed a subgroup analysis on the β-blockade and IMA
change in the percentage of use, examining the interaction between intervention
group and site of CABG surgery, academic vs nonacademic center site characteristic,
and use of process measure at baseline. For volume and measure performance,
sites were categorized as low, medium, or high according to whether they were
in the first, second, or third tertile for the variable of interest.
Table 1 shows demographic
patient- and hospital-level characteristics for the 3 randomized arms. There
were 309 sites that provided data during the intervention: 101 sites each
for the control arm and for the IMA grafting group and 107 for the β-blockade
group. Mean annual case volumes were between 305 and 348. The proportion of
patients older than 75 years and the baseline (mean for year 2000) use of β-blockade
and IMA grafting were equivalent across the 3 arms.
Compared with baseline year 2000 data, the percentage of control sites
that improved (increased use of measures in last year) was 66% for both measures;
in the β-blockade group, 81% had an increase in use of β-blockade
and 63% had an increase in use of IMA grafting; in the IMA group, 75% had
an increase in use of β-blockade while 71% had an increase in use of
Table 2 demonstrates the
change in measure use over the study interval in the targeted intervention
group compared with the control arm (no intervention) and with the other,
nontargeted, intervention group. From January 2000 to July 2002, use of both
process measures increased nationally (β-blocked, 60.0% 65.6%; IMA grafting,
76.2%82.8%). While all groups showed an increase in the measure being evaluated
over time, patients in the β-blockade intervention group were more likely
to receive β-blockade (7.3% [SD, 12.8%]) compared with those in the control
arm (3.6% [SD, 11.5%]) (P = .04 in the preintervention/postintervention
analyses and P<.001 in the hierarchical analyses). Use of β-blockade
also increased more in the nontargeted IMA grafting group than in the control
arm (5.4% vs 3.6%, P = .33 and P = .02, respectively, for the 2 analyses).
Use of IMA grafting also tended to increase at IMA intervention sites
(8.7% [SD, 17.5%]) vs control sites (5.4% [SD, 15.8%]), although the comparisons
were not statistically significant (P = .20 or P = .11 for preintervention/postintervention and hierarchical
When the targeted intervention group was compared with both other groups
combined, both targeted interventions achieved significance (for hierarchical
analysis, P = .03 for IMA grafting vs control plus β-blockade; P = .01 for β-blockade vs control plus IMA grafting).
Figure 2 shows the results
of the preintervention/postintervention trends for β-blockade and IMA
grafting over the 5 report intervals from spring 2000 to spring 2002, comparing
the intervention groups with the control arm for purposes of clarity. With
the January 2001 round 1, both the β-blockade and IMA grafting groups
increased the use of the process measure and continued to increase use throughout
the remainder of the trial interval.
Table 3 indicates that for
both the β-blockade and IMA grafting groups the difference in percentage
measure use compared with the control percentage difference was significant
for site case volumes stratified into low (<228 cases/y), medium (229-449
cases/y), and high (≥450 cases/y) strata. Low-volume sites had significantly
greater incorporation of both measures compared with the corresponding medium-
or high-volume sites (for interaction: P = .04 for β-blockade; P = .02 for IMA grafting). Compared with nonacademic centers,
academic centers showed a trend toward a higher incorporation for both measures.
Also, compared with sites with a higher use of measures at baseline, sites
with a lower use showed a trend toward improvement for both measures.
This study reports the first randomized trial of CQI accomplished on
a national scale. We found that a provider-driven, low-intensity CQI intervention
could have demonstrable impact on local CABG care practices within a 2-year
period. Although the clinical impact of the trial was modest, we believe the
results demonstrate the potential for medical specialty societies to have
an impact on the national adoption of important care processes into clinical
This trial tested whether a medical specialty society could leverage
clinician motivation in a call to action to CQI at the local level. Bradley
et al29 identified factors found to predict
adoption of care processes, including defined goals and empowerment of local
site leadership; the other 2 factors of CQI infrastructure support and high-quality
performance feedback were provided by the platform of a nationally representative
clinical database in this trial. The NCD assessed baseline care patterns,
gaps between actual and recommended care, and variability among site practices.30,31 In the intervention phase, the NCD
provided a clinically integrated mechanism for ongoing measurement of quality
process and outcomes measures and their evaluation in the context of adjusted
patient risk. This site-specific feedback, along with national and best-practice
benchmarks,32 provided contemporaneous data
for motivation, goal setting, and documentation of local success.
We designed a low-intensity CQI intervention because this approach might
have greater effects on health care and policy than a more intensive intervention
that might be difficult to duplicate in other health care settings. Moreover,
a low-intensity intervention matched the scope of leadership and resources
that could be provided by national society sponsorship. Specifically, this
initiative did not mandate the use of specific CQI tools but rather allowed
individual sites to determine how best to implement changes in practice at
their own sites.
A successful change in site behavior due to this low-intensity, multi-faceted33 intervention was suggested by the percentage of sites
that improved in use of the measures during the trial and by the percentage
increases in measure use (Table 2).
In addition, the variance of practice surrounding these measures generally
narrowed; this was more apparent in the IMA grafting group ("simple measure")
than in the β-blockade group ("complex measure").
While the β-blockade CQI intervention had significant overall effect
and the IMA grafting intervention showed positive trends, the overall clinical
magnitude of the results was quite modest. In part, this result may have been
due to the process measures examined and the trial context. The 2 measures
were selected to test the implementation of new information into clinical
practice through this CQI platform, and thus had limited scientific support
at the start of the trial. Additionally, their link to outcomes was not established,
particularly in context with the underlying ongoing decline in CABG mortality
observed over the past decade.34 Subsequent
observational analyses from the NCD documented the positive independent impact
of IMA grafting30 and preoperative β-blockade
therapy31 on acute (30-day) CABG mortality
and morbidity. Both published in May 2002, these reports had little impact
on the trial results.
The improvement over time in the control group for both measures (Figure 2) suggests that the context of the
quality improvement infrastructure and the goals of the NCD may have influenced
the trial results as well. This improvement in the control arm may reflect
use by these sites of intervention measure data included in the routine NCD
site reports, but also may reflect cross-contamination of educational and
targeted interventional materials as sites (blinded to the study design) compared
clinical practices. However, despite these process-measure considerations
and general quality improvement trends, this low-intensity CQI effort resulted
in a significant improvement in the β-blockade intervention (overall
difference in the increase in β-blockade, round 3: 8% between the targeted β-blockade
group and the control arm), both in the trend analysis (Figure 2A, P = .04) and in the hierarchical
analysis (Table 2).
The lack of significance in the IMA grafting intervention may relate
to a number of factors. First, the differential between intervention and control
sites was less than anticipated. Although the IMA preintervention/postintervention
analysis difference was comparable with the β-blockade difference, the
variability of the site differences was about 50% greater for IMA grafting
than for β-blockade, and greater than anticipated (Table 2). The patient-level analysis was more sensitive (P = .11), again despite results similar to those of the β-blockade
intervention. Other factors included more room for improvement nationwide
in the β-blockade group (lower percentage use at the start of the trial),
the possibility of a ceiling effect at the highest-performing sites in the
IMA grafting group, and the possibility that the intervention was too low-intensity
for the nonresponding surgeons (22% in the survey) to change their minds.
This trial was able to test the efficacy of the CQI process in a controlled
trial format on a national scale. The trial groups (Table 1) each reflected a national representation. This national
scalability distinguishes this trial from prior important observational quality
improvement efforts in cardiovascular disease in cardiac surgery,34- 39 and
to a more limited extent, in cardiovascular medicine.40- 42
In addition, the rigorous evaluation of CQI using the randomized controlled
trial format distinguishes this trial from prior studies. Many of these prior
successes consisted of either high-intensity or site-specific interventions,
while in others it was difficult to account for baseline changes in quality
of care. For example, the decline in mortality for CABG surgery observed in
New York state37 was paralleled in the Northern
New England Consortium group,35 in Massachusetts
Medicare-aged patients,43 and more recently
was documented to have occurred nationally in both the Veterans Affairs39 and STS34 analyses.
Low-volume sites had a higher incorporation of both measures than their
high-volume counterparts (Table 3),
and this relationship was inversely related to volume. This response in low-volume
sites may reflect a limitation of resources for CQI at low-volume sites that
were now being addressed through the STS CQI platform. Given the interest
in use of volume criteria alone as a marker of clinician quality,44,45 this more aggressive adoption of
CQI for local improvement in care may well be an important finding.
It is noteworthy that the results of the trial, while modestly successful
in terms of improvement in the use of the measures, were accomplished on a
national scale and within a time interval that was rapid compared with many
other CQI efforts.46 The database and the medical
specialty society sponsorship provided mechanisms for national organization,
provider engagement, and site-specific performance feedback on quality-of-care
metrics. Because improvement and adoption of best practices is primarily a
local issue,47,48 we speculate
that the accomplishments of the trial rest with the local sites and the clinician-led,
voluntary focus on CABG outcomes and quality manifested by site participation
in the NCD. This platform could provide a template for other areas of medicine.
First, these modest results might have been greater with a more intensive
intervention, a more static control group, and/or a lesser degree of cross-contamination;
however, the positive impact at a national level with this low-level intervention
suggests that both the infrastructure and the intervention contributed to
the trial outcome. Second, the outcomes assessed were processes of care with
unestablished links to end point outcomes. Other factors that impact on dissemination
of information in health care, including organizational and contextual factors,
will be important in this study as well.46 Third,
we did not assess the CQI infrastructure costs; however, the trial costs and
the cost of NCD participation at the site level compare favorably with effective
regional voluntary and mandatory quality improvement efforts in CABG surgery.
Finally, while nationally representative, the trial was not comprehensive
for all CABG surgical procedures in the United States. While this study documents
that this platform can achieve improvements in the quality of care, it remains
for government and third-party entities to recognize this type of effort to
facilitate expansion of this approach to all cardiac surgical centers and
to other areas of medicine.
This trial of CQI in CABG surgery is the first rigorous successful randomized
trial of CQI in medicine achieved on a national scale. As such, these results
have important implications for medical specialty clinicians, because the
CQI infrastructure used here documents a mechanism for active and effective
clinician involvement in the CQI process. Further refinements in this infrastructure
and the CQI process are suggested by these modest results; however, the overall
scope and success of this trial suggest a model that can be adopted by other
clinicians for translating research into everyday practice.