Individual hospital 30-day rates of stroke or death after carotid endarterectomy by indication, with the number of operations noted (represented by the horizontal bars) and the recommended threshold rates for complications (6% for symptomatic indications [A] and 3% for asymptomatic indications [B]) shown in the dashed lines.
Feasby TE, Kennedy J, Quan H, Girard L, Ghali WA. Real-World Replication of Randomized Controlled Trial Results for Carotid Endarterectomy. Arch Neurol. 2007;64(10):1496-1500. doi:10.1001/archneur.64.10.1496
The efficacy of carotid endarterectomy (CE) has been shown in randomized clinical trials (RCTs), but doubts remain about whether the results can be replicated in routine clinical practice, especially in asymptomatic patients for whom the absolute risk reduction shown in the trials is small. In particular, a low rate of short-term adverse events is required for the long-term benefits of CE to accrue over time.
To determine whether the incidence of short-term adverse events after CE met the standards established by the major RCTs and those recommended by major clinical practice guidelines.
Design, Setting, and Patients
We used clinically detailed data derived from a comprehensive medical record review to measure the short-term adverse outcomes of CE, focusing on in-hospital death and stroke, in 3283 cases in western Canada in 2000 and 2001, and compared the results with those from the RCTs.
For symptomatic patients, the in-hospital ischemic stroke or death rate was 3.9%; for asymptomatic patients, the rate was 2.6%. Extrapolating our in-hospital results to the 30-day post-CE results of the RCTs gave stroke or death rates of 4.9% for cases involving symptomatic patients and 4.1% for cases involving asymptomatic patients. These results are comparable to or better than those of the major RCTs for symptomatic patients but slightly worse for asymptomatic patients. About 45% of hospitals had adverse event rates higher than those recommended by authoritative clinical practice guidelines.
This large population-based study shows that the RCT results for CE can be achieved in the real world. However, the finding that some hospitals exceeded the maximum suggested rates of adverse events highlights the need for continuous outcome monitoring and associated quality improvement efforts to ensure that all providers and institutions involved achieve desired outcomes.
Carotid endarterectomy (CE) is a widely performed operation for stroke prevention.1 Its efficacy has been demonstrated in several major international randomized controlled trials2- 5 (RCTs), and its rate of use in the United States has increased in response to publication of these results.6,7 However, the risk reduction demonstrated in the clinical trials depends on low rates of perioperative stroke or death due to the procedure. This is especially true for CE performed in asymptomatic patients, for whom the absolute risk reduction demonstrated in the RCTs is small. For these patients, published guidelines have recommended that CE not be performed unless the stroke or death rate is less than 3%.8 Reluctance by some physicians to endorse asymptomatic carotid stenosis as an appropriate indication for CE has been largely based on the concern that this low level of perioperative mortality and morbidity is often not achieved in routine clinical practice.9,10 A meta-analysis of 46 case series of CE published between 1990 and 2000,11 inclusive, reported a death rate of 1.1% (95% confidence interval [CI], 0.9%-1.3%) for cases involving asymptomatic patients, compared with the death rate of 0.14% in the Asymptomatic Carotid Atherosclerosis Study (ACAS).4 When the combined end point of stroke or death was analyzed in the 8 studies with neurologist follow-up,11 the real-world rate was 4.30% (95% CI, 3.5%-5.2%) compared with 1.5% (95% CI, 0.6%-2.4%) in the ACAS.4 The recently completed Asymptomatic Carotid Stenosis Trial5 reported results similar to those of the ACAS, although it counted strokes in any arterial territory, did not exclude cardioembolic strokes, and has not demonstrated ipsilateral benefit. Furthermore, in the wake of the ACAS, it has not dispelled skepticism about whether these results can be replicated in clinical practice.12,13
Results achieved in clinical practice may be worse than those in RCTs because of less restrictive selection criteria (eg, age and comorbidities); less adequate control of blood pressure, lipid levels, and other modifiable risk factors; and less qualified surgeons. Surgeons in the North American Symptomatic Carotid Endarterectomy Trial (NASCET)2 and ACAS4 were required to have met rigorous performance standards. To address questions about whether the results of the clinical trials of CE can be achieved in typical clinical settings, we sought to examine the practice of CE in a large population from a broad geographic area. We studied the practice of CE in the 4 western Canadian provinces for 2 years to determine whether the incidence of short-term adverse events after surgery met the standards established by the major RCTs and those recommended by the major clinical practice guidelines.8,14,15
We identified all hospitals in the provinces of Manitoba, Saskatchewan, Alberta, and British Columbia in which CE was performed in 2000 or 2001 and identified all CE cases from hospital discharge administrative data. Two trained medical chart reviewers reviewed all CE charts from the calendar years 2000 and 2001 and entered clinically detailed information directly into a computerized database. Both reviewers extracted data on age, sex, symptom status, indication for CE, ischemic stroke, death, and other complications (details are discussed in Kennedy et al16,17). Symptom status was categorized as symptomatic if the patient had had a preceding stroke or transient ischemic attack in the territory of the carotid artery undergoing operation, or as asymptomatic if the patient had not.
The primary outcome variable was in-hospital ischemic stroke or death. Statistical analysis involved only simple descriptive statistics. The variables were described by means and proportions. The χ2 test was used to test the statistical significance of outcomes (stroke or death and complications) across the categories of variables.
The major RCTs2- 5 presented adverse events occurring within 30 days of surgery. However, in this observational study of real-world CE, we report only complications occurring in the hospital. Because of this discrepancy, we needed to develop a methodological approach to extrapolate our observed in-hospital event rates to estimated 30-day event rates. To do this, we used a careful breakdown of the timing of postoperative stroke and death in the NASCET (cases of symptomatic patients) and ACAS (cases of asymptomatic patients) and derived estimates that are the best possible extrapolations from observed in-hospital event rates. In the NASCET, 79.4% of the adverse events occurred by the third post-CE day18; in our cases, the median and the mode length of stay after CE were both 3 days. Thus, we extrapolated our results to those of the NASCET by multiplying our results by 100 and then by dividing by 79.4 to derive an adverse event estimate for 30 days after the operation, which is comparable to the observed event rates reported in the RCTs. Similarly, for asymptomatic case, of the ACAS patients who were randomized to surgery and received surgery (n = 721), 63.6% (7 of 11) of the strokes or death during the postoperative period occurred by the third day (James F. Toole, MD, and Virginia J. Howard, PhD; verbal communication for ACAS; 2006).19 Therefore, we multiplied our in-hospital stroke or death rate for cases involving asymptomatic patients by 100 and then divided by 63.6 to get a 30-day rate, which is comparable to the rates reported in the ACAS.
A total of 3360 CE cases were identified in 23 hospitals. We located and reviewed 3309 medical charts, but eliminated 26 because of missing data, leaving 3283 cases for analysis. The general characteristics of the case patients are shown in Table 1. The mean age of the patients was 70.9 years, and 66.2% were male. Most procedures were performed by vascular surgeons (78.8%) rather than neurosurgeons (12.3%) or other surgeons (8.9%). The only exception was Alberta, where 40% of CEs were performed by neurosurgeons. Overall, 38.0% of cases involved asymptomatic patients and 62.0% involved symptomatic patients. The case mix varied by surgeon, with neurosurgeons operating on 85.2% of symptomatic patients; vascular surgeons, on 61.1%; and other surgeons, on 47.3%.
The overall rate for in-hospital stroke or death was 3.4% (Table 2). The rate was 3.9% for symptomatic patients and 2.6% for asymptomatic patients. Extrapolating the rates to 30 days after CE, the rates were 4.9% for symptomatic patients and 4.1% for asymptomatic patients. After extrapolating the rates, 8 of 22 hospitals had a postoperative in-hospital rate of stroke or death greater than 6% (range, 6.7%-21.9%) for cases involving symptomatic patients (Figure). For cases involving asymptomatic patients, 10 of 22 hospitals had a rate greater than 3% (range, 3.3%-57.2%).
Complications of surgery, other than stroke and death, occurred in relatively small numbers of cases (Table 3). Myocardial infarction was slightly more common after CE for asymptomatic stenosis (1.8% vs 0.9%; P = .02), but there was no difference for unstable angina, pulmonary embolus, renal failure, wound hematoma, or nerve injury.
The overall rate of complications due to CE in western Canada compares favorably with the results from the recent RCTs. We found an extrapolated rate of in-hospital stroke or death of 4.9% for patients with symptomatic carotid stenosis in western Canada, whereas the 30-day rate in the pooled results of the major trials of symptomatic stenosis, the NASCET and European Carotid Surgery Trial, was 7.0% (95% CI, 6.2%-8.0%).20 The rate in our cases is also much less than the 6.9% 30-day rate found for symptomatic stenosis in a recent large, multistate US medical chart review study of CE.21 The rate is also less than the 6.0% rate recommended by US, Canadian, and European clinical practice guidelines.8,14,15
For cases involving asymptomatic patients, the western Canadian rate of stroke or death extrapolated to 30 days was 4.1%. The multistate US CE study found a 30-day rate of stroke or death of 4.6% for those with no symptoms or nonspecific symptoms,21 and the meta-analysis of the 8 case series published in 1990 to 2000 using neurologist follow-up found a rate for cases involving asymptomatic patients of 4.3%.11 The CEs in asymptomatic patients who were followed up by neurologists in the Asymptomatic Carotid Endarterectomy (ACE) trial had a rate of stroke and death of 4.4%.22 In contrast, the 2 large RCTs of asymptomatic stenosis found rates of 1.5% (ACAS)4 and 3.1% (Asymptomatic Carotid Stenosis Trial).5 The American Heart Association's clinical practice guidelines recommended that complication rates of 3.0% or less be achieved to justify the performance of CE in asymptomatic patients.8 This standard has not been met overall in our study of usual care, although most hospitals had acceptable rates.
Eight of 22 hospitals had complication rates greater than the recommended 6% for symptomatic patients and 10 hospitals exceeded 3% for asymptomatic patients. When we performed a sensitivity analysis on the cases involving asymptomatic patients that more conservatively extrapolated 30-day event rates using data from the symptomatic NASCET, 7 of 22 hospitals still had complication rates above the 3% threshold (range, 3.3%-45.8%). These findings question whether these hospitals should continue to provide CE. Several authors have recommended that performance of CE by hospitals and by surgeons be audited and the results used to determine suitability to practice.12,23 Recently, the Leapfrog Group, a large group of US employers and purchasers of medical care, made surgical case volumes and mortality rates key criteria for referral for cardiac surgery and other high-risk procedures.24 This might seem reasonable for CE at first glance, but a recent study on the use of mortality as a surgical quality indicator for hospitals showed that a very large caseload is necessary to provide a valid measure of quality.25 For instance, for coronary artery bypass surgery, a minimum caseload of 744 was necessary to detect a 1.5-fold increase in mortality (based on an average mortality rate of 3.5%, a rate close to the complication rate of CE for asymptomatic stenosis). This suggests that case numbers far greater than those found in the hospitals we studied (asymptomatic caseload range for 2 years, 0-170 cases; median, 32 cases) are needed to use this approach to determine whether CE should be performed for asymptomatic stenosis in individual hospitals. It would be even more difficult to assess the performance of individual surgeons. In this regard, novel statistical approaches, such as cumulative sum charts, may be more appropriate26 because these methods permit detection of outcome trends shortly after the occurrence of adverse events.
Our study showed a marked difference in practice between Canadian and US surgeons. Only 38% of the patients operated on by our western Canadian surgeons were asymptomatic, and a recent study in Ontario, Canada, found that 30.6% of patients undergoing surgery were asymptomatic.27 In contrast, 2 recent large US studies, the multistate study21 and a New York State registry study,28 found CE rates for asymptomatic patients of 81% and 67%, respectively. Clearly, this reflects the tendency of US surgeons to consider surgery for asymptomatic stenosis of at least 60% as an appropriate indication for CE,29,30 whereas Canadian surgeons are more likely to consider the benefit of CE in these cases to be uncertain.14 This is reflected in clinical practice guidelines from both countries. Those from the Canadian Stroke Consortium10 and the Canadian Neurosurgical Society14 considered CE to be of uncertain benefit in asymptomatic stenosis, but those from the American Heart Association8 recommended CE for asymptomatic stenosis. In a recent study by our group of the appropriateness of CE, a Canadian expert panel rated CE for most asymptomatic scenarios as being of uncertain benefit.16,17 European guidelines are even more conservative, recommending against CE for those with asymptomatic stenosis.15 The real-world rates we found in western Canada point to a legitimate concern about widespread use of CE for asymptomatic disease if the complication rates are above the 3% threshold in so many hospitals. This argues against the trend of increasing numbers of CEs for asymptomatic patients or widespread screening for asymptomatic carotid disease,31 especially in view of the modest absolute risk reduction afforded to asymptomatic patients in the best of circumstances.32
Our study has several limitations. Because it is a regional study, it may be that hospitals in other areas are more successful in meeting the recommended mortality/morbidity targets recommended by clinical practice guidelines. Changes in practice may also result in improvements in outcome. Also, the trend to carotid stenting, a procedure with as yet unproved efficacy, may change the case mix for CE, producing different results. Finally, extrapolation of our in-hospital results to 30 days is an estimate around which there may be some error. Nevertheless, it is an extrapolation based on available evidence and has reasonable face validity.
We have shown that the short-term adverse event results for CE in RCTs can be replicated in a large real-world setting in most hospitals, especially for symptomatic carotid stenosis. The excess adverse event rate in some hospitals, however, is of continuing concern and underlines the need for continued quality improvement efforts and new statistical approaches to provide valid and timely measures of outcome.
Correspondence: Thomas E. Feasby, MD, Department of Clinical Neurosciences, University of Calgary, Seventh Floor, TRW Building, 3280 Hospital Dr NW, Calgary, AB T2N 4Z6, Canada (email@example.com).
Accepted for Publication: March 15, 2007.
Author Contributions: Dr Feasby had access to all the data and takes full responsibility for the integrity of the data and the analysis. Study concept and design: Feasby, Quan, and Ghali. Acquisition of data: Feasby, Kennedy, and Ghali. Analysis and interpretation of data: Feasby, Kennedy, Quan, Girard, and Ghali. Drafting of the manuscript: Feasby and Kennedy. Critical revision of the manuscript for important intellectual content: Feasby, Kennedy, Quan, Girard, and Ghali. Statistical analysis: Kennedy, Quan, and Ghali. Obtained funding: Feasby and Ghali. Administrative, technical, and material support: Quan, Girard, and Ghali.
Funding/Support: This study was supported by an operating grant from the Canadian Institutes of Health Research, by a Government of Canada Research Chair in Health Services Research and a Health Scholar Award from the Alberta Heritage Foundation for Medical Research (Dr Ghali), and by a Population Health Investigator Award from the Alberta Heritage Foundation for Medical Research and a New Investigator Award from the Canadian Institutes for Health Research (Dr Quan).
Additional Contributions: Marie McLelland, RN, and Caroline Stephenson, RN, conducted the chart review, and Janet Bennett provided administrative support.
Financial Disclosure: None reported.