Predicting Medical and Surgical Complications of Carotid Endarterectomy: Comparing the Risk Indexes

Background  Several generic cardiac risk assessment tools predict perioperative cardiac complications, but their ability to predict a broader range of medical, neurologic, and surgical complications is unknown.

Methods  A multicenter retrospective observational cohort study of 1998 patients undergoing carotid endarterectomy (CEA). Complications within 30 days of surgery were assessed by medical record review, including death or nonfatal stroke and cardiac, noncardiac medical, minor neurologic, and wound complications. Logistic regression and receiver operating characteristic curve analyses assessed the predictive abilities of the Goldman, Detsky, Revised Cardiac Risk, and American Society of Anesthesiologists indexes and of 2 CEA-specific risk models (the Halm and Tu scores).

Results  Death or stroke occurred in 3.2% of patients, cardiac complications in 4.0%, noncardiac medical complications in 3.2%, minor neurologic complications in 6.9%, and wound complications in 6.0%. All risk models (except the Tu score) significantly predicted cardiac complications equally well (P<.05). All 6 risk models were equivalent in predicting noncardiac medical complications. Only the Revised Cardiac Risk Index and the 2 CEA-specific risk models (Halm and Tu scores) predicted death or stroke and minor neurologic and wound complications. The Halm score was superior in predicting death or stroke compared with the Tu score and the Revised Cardiac Risk Index (area under the receiver operating characteristic curve, 0.72 vs 0.62 and 0.61, respectively; P<.05). Patients with cardiac, noncardiac medical, minor neurologic, or wound complications had 3- to 16-fold increased odds of death or stroke.

Conclusion  The Halm score CEA-specific risk model and the generic Revised Cardiac Risk Index predicted a broad range of medical, neurologic, and surgical complications following CEA.

Carotid endarterectomy (CEA) is one of the most common vascular surgical procedures in the United States. From 1993 to 2002, the number of CEAs performed per year increased from 91 000 to 134 000.^1,2 Assessing the appropriateness of CEA involves balancing the potential benefits and harms. The expected benefits of CEA to reduce the risk of stroke depend on the presence of neurologic symptoms and the degree of carotid artery stenosis. The short-term harms are quantified by estimating the risk of perioperative death, stroke, and myocardial infarction (MI). Assessing the risk of complications also helps guide decision making about the need for additional preoperative cardiac testing, revascularization, medication therapy (eg, β-blockade), or intraoperative monitoring.³

The following 4 risk indexes have been widely used to aid the assessment of cardiac risk for patients undergoing noncardiac surgery: the American Society of Anesthesiologists⁴ (ASA) index, the index by Goldman et al,⁵ the index by Detsky et al,⁶ and the Revised Cardiac Risk (RCR) Index.⁷ Each of these cardiac risk indexes was designed to predict the chance of death and major cardiac events across a broad range of patients and types of surgery. Data are scant about the ability of these risk indexes to predict other important perioperative adverse events such as neurologic, noncardiac medical, and surgical complications.

Although patients undergoing CEA were included in the analyses of these generic cardiac risk assessment indexes, these cases represent a small proportion of patients. To our knowledge, no study has evaluated the relative performance of thesegeneric cardiac risk assessment indexes in a large cohort of CEA patients or analyzed their ability to predict noncardiac medical, neurologic, and surgical complications that are important to patients undergoing vascular surgery.

The objectives of this study were (1) to describe the incidence of a broad range of complications following CEA, including death or stroke and cardiac, noncardiac medical, minor neurologic, and wound complications; (2) to evaluate and compare the ability of the standard cardiac risk assessment indexes to predict this broad spectrum of operative complications; and (3) to compare the prognostic performance of the standard indexes with that of 2 CEA-specific risk models.

Using administrative databases as described previously,⁸ we identified a cohort of patients in 6 hospitals who underwent CEA between January 1, 1997, and December 31, 1998. Five hospitals were located in the New York City metropolitan area, and 1 hospital was in upstate New York; 4 sites were university teaching hospitals, and 2 were community teaching hospitals. Each site's institutional review board approved the study. Because 2 surgeons each performed more than 250 CEAs, we randomly sampled 50% of the cases of these 2 high-volume surgeons and 100% of all other physicians' cases.

To confirm patient eligibility, we reviewed the medical records of 2365 (99.0%) of 2390 patients identified by administrative records. For this article, we focus on 1998 patients who underwent a confirmed single CEA procedure. Reasons for exclusion were CEA combined with coronary artery bypass graft surgery (149 patients), same-side reoperations (91 patients), surgery combined with another major procedure (47 patients), missing data (43 patients), or no CEA performed (37 patients). Each hospital contributed between 130 and 583 patients.

Trained research nurses abstracted detailed clinical information from inpatient and outpatient medical records. This included sociodemographic characteristics, indication for surgery, cardiac risk factors, ASA index classification, preoperative physical examination and electrocardiographic findings, laboratory values, medication use, and medical, neurologic, and surgical history.

Patients were categorized according to the following 4 standard perioperative risk indexes: the ASA index,⁴ the index by Goldman et al⁵ (the Goldman index), the index by Detsky et al⁶ (the Detsky index), and the RCR Index.⁷ We also examined the predictive ability of 2 CEA-specific risk models that were designed to predict death or nonfatal stroke within 30 days of surgery, one developed by Tu et al⁹ (the Tu score) and another that was previously developed from the present data set by Halm et al¹⁰ (the Halm score). Details of each of these risk indexes are available from the author. The Halm score includes 3 factors that increase the risk of complications (active coronary artery disease, stroke as the indication for surgery, and contralateral stenosis >50%) and 1 protective factor (use of local anesthesia). Active coronary artery disease indicates unstable angina or Canadian Cardiovascular Society class III or IV angina.¹¹

We collected data about complications within 30 days of surgery from the inpatients' medical records and from the surgeons' postdischarge office records. We reviewed records for 96% of patients and for 97% of surgeons. To identify complications, we also reviewed all readmissions to the index hospital within 30 days of surgery.

Complications were grouped into the following 5 categories: combined death or nonfatal stroke and cardiac, noncardiac medical, minor neurologic, and wound complications. All-cause death and nonfatal stroke were considered together because this is the combined adverse outcome that is used in major randomized controlled trials of CEA^12,13 and in national practice guidelines about CEA.¹⁴ Cardiac complications included MI, unstable angina, congestive heart failure, and ventricular tachycardia. Noncardiac medical complications included mechanical ventilatory assistance, postoperative pneumonia, sepsis, renal failure, deep venous thrombosis or pulmonary embolism, and gastrointestinal tract bleeding. Minor neurologic complications included transient ischemic attack (TIA), cranial nerve palsy, and seizure. Wound complications included bleeding or hematoma and infection. Ascertainment of complications was based on a combination of a documented physician diagnosis and objective evidence (physical examination findings and laboratory, radiology, and electrocardiographic test results). Individual patients may have had more than 1 occurrence of the same complication within 30 days, but that complication was counted only once in the analysis.

All deaths, strokes, MIs, and TIAs were confirmed by physician overreading. Strokes and TIAs were independently reviewed by a neurologist (S.T.) and 1 of 2 physician investigators (M.R.C. or E.A.H.). Initial agreement was 95%. Disagreements were resolved by consensus and by the use of a third physician reviewer as needed.

The relative ability of the 6 risk models to predict each category of complications was evaluated by calculating the areas under the receiver operating characteristic (ROC) curves using SAS statistical software version 9.0 (SAS Institute, Cary, NC) and by using χ² and Cochran-Mantel-Haenszel tests for trend. An area under the ROC curve of 0.5 represents chance performance, and 1.0 indicates perfect predictive ability. Pairwise comparisons between risk models were calculated by the technique of Hanley and McNeil¹⁵ using STATA software version 8.0 (StataCorp LP, College Station, Tex). Differences in rates of each complication category between symptomatic and asymptomatic patients were examined using χ² tests. Associations between different types of complications were identified using χ² tests. For all analyses, 2-sided P<.05 was indicative of statistical significance.

Characteristics of the 1998 patients in the cohort are given in Table 1. The mean ± SD patient age was 72.4 ± 8.7 years; the average patient had 1.8 ± 1.5 comorbid conditions. Most patients (57.8%) had coronary artery disease. The clinical indications for surgery were as follows: 69.3% of cases were for asymptomatic carotid stenosis, 17.2% were for carotid TIA, 11.4% were for stroke, 1.5% were for vertebrobasilar TIA, 0.4% were for crescendo TIA, and 0.4% were for stroke-in-evolution. The classification of patients by the 6 risk models is given in Table 2. Between 1% and 8% of cases fell into the highest-risk subgroups for each risk model.

Complications within 30 days of surgery are given in Table 3. Death occurred in 13 patients, and nonfatal stroke occurred in 51 patients, resulting in a combined death or stroke rate of 3.2%. Cardiac complications occurred in 4.0%, with an MI rate of 1.2%. Noncardiac medical complications occurred in 3.2% of patients, the most common of which was prolonged mechanical ventilatory assistance, followed by postoperative pneumonia and renal failure. The rate of minor neurologic complications was 6.9%, largely because of a 6.0% rate of cranial nerve palsies. Wound complications, predominantly wound bleeding or hematoma, occurred in 6.0% of cases. There were no significant differences in rates of death, stroke, or noncardiac medical complications by hospital. There were small but significant differences in the unadjusted rates of cardiac, minor neurologic, and wound complications by hospital.

Patients undergoing CEA for symptomatic carotid stenosis (compared with those who were asymptomatic) had higher rates of combined death or stroke (5.5% vs 2.3%, P<.001) and minor neurologic complications (8.9% vs 6.1%, P = .02). There were no significant differences between symptomatic and asymptomatic patients for rates of cardiac (3.1% vs 4.4%, P = .17), noncardiac medical (3.2% vs 3.1%, P = .88), or wound (7.2% vs 5.4%, P = .14) complications.

Overall, 18.5% of CEA patients had at least 1 complication: 14.8% had 1 complication, 2.7% had 2 complications, 0.8% had 3 complications, and 0.2% had 4 complications. Among the 3.7% of patients who had more than 1 category of complication, adverse outcomes tended to cluster. For example, the odds of combined death or stroke increased 16-fold (odds ratio, 16.4; 95% confidence interval, 8.8-30.5) in patients with noncardiac medical complications. The odds of death or stroke increased 4-fold (odds ratio, 4.3; 95% confidence interval, 2.0-9.0) in patients with cardiac complications and increased 3-fold (odds ratio, 3.0; 95% confidence interval, 1.5-5.8; and odds ratio, 3.1; 95% confidence interval, 1.5-6.2) in patients with minor neurologic complications and with wound complications, respectively. Among the subset of patients with multiple complications, the timing of cardiac complications was evenly distributed: 35.7% of cardiac complications occurred at least 1 day before the other complications, 33.3% occurred on the same day, and 30.9% occurred 1 day or more after the other complications.

Table 4 quantifies the prognostic ability of each of the risk models in predicting the rates of each category of complications. Two of the generic models (the Detsky and RCR indexes) and both CEA-specific models (the Halm and Tu scores) predicted combined death or stroke significantly better than chance (P<.05). As expected, given the objective of its development and the fact that it was derived from the present data set, the Halm score (area under the ROC curve, 0.72) was superior to the Tu score and to all of the generic risk models in predicting death or stroke (P<.01).

All models except the Tu score predicted cardiac complications, and all predicted noncardiac medical complications moderately well (area under the ROC curve, 0.58-0.68) (Table 4). In pairwise comparisons, the ASA, Goldman, RCR, and Halm models predicted cardiac complications equally well; all performed similarly in predicting noncardiac medical complications. Only the RCR Index and the Halm and Tu scores predicted minor neurologic and wound complications. All 3 models were similar and performed modestly in predicting these complications (area under the ROC curve, 0.54-0.61).

In this study of 1998 CEAs performed at 6 hospitals by 64 surgeons, we evaluated the rates of a broad range of complications within 30 days of surgery. Overall, 18.5% of patients had some type of perioperative complication, with the most serious adverse events of death or stroke and MI occurring in 4.4% of patients. Death or stroke and minor neurologic complications, the outcomes most closely related to the underlying severity of cerebrovascular disease, were more common among patients with symptomatic carotid disease (compared with those without symptoms). Symptomatic and asymptomatic patients were equally at risk for the more general types of postoperative complications such as cardiac, noncardiac medical, and wound complications.

The good news for medical consultants, cardiologists, neurologists, anesthesiologists, and surgeons is that 2 straightforward 4-item risk models (the Halm score and the RCR Index) predicted the full spectrum of major and minor complications of CEA. The RCR Index is a validated risk model developed to predict cardiac events among patients undergoing noncardiac surgery. An additional strength is that this model predicts neurologic events (including stroke and TIA) and noncardiac medical complications. To our knowledge, no prior studies have evaluated the ability of the standard cardiac risk indexes to predict noncardiac complications.

The RCR Index has been previously recommended to guide decision making in the perioperative management of patients undergoing noncardiac surgery, includingCEA.^3,17 In conjunction with other clinical information (eg, functional capacity), the RCR Index helps to identify patients who would benefit from perioperative β-blockade or from preoperative noninvasive cardiac testing.

For clinicians who are interested in predicting the combined risk of death or stroke, which is the most important determinant of who will or who will not ultimately benefit from CEA, the Halm score performed the best overall. The superior prediction of death or stroke by this previously developed CEA-specific model was expected.¹⁰ The model was derived from the present data set for this purpose. However, the Halm score also predicted cardiac, noncardiac medical, minor neurologic, and wound complications, which enhances its potential usefulness to those who frequently evaluate and treat patients undergoing CEA. Comparing the 2 CEA-specific models, the Halm score was better than the Tu score at predicting death or stroke and cardiac complications. Differences in the performance of the CEA-specific models are likely because of differences between derivation and validation populations and because of underlying differences between US¹⁰ and Canadian⁹ CEA patients and complication rates. Canadian patients were more likely to have symptomatic disease than the US patients presented herein (69.4% vs 30.7%), less likely to have coronary disease (35.7% vs 58.0%), and more likely to experience death or stroke (6.0% vs 3.2%).

From a practical clinical standpoint, these risk models may be useful in 3 major ways. First, they may help physicians identify patients at low risk of any major perioperative complications from CEA, who can be referred for surgery without additional testing. Second, the information can assist physicians in recognizing patients whose risk of perioperative death or stroke from CEA is very high and for whom nonsurgical alternatives such as endovascular stenting or maximal medical therapy alone may be more appropriate. Third, patients assessed as being at high risk or at intermediate risk for cardiac complications would need to be evaluated and treated according to national guidelines regarding additional cardiac testing, revascularization, or β-blockade.³

Our findings confirm prior reports that the RCR Index performed better than the other generic cardiac risk models^7,18 and extend this evidence to one of the most common types of vascular surgery, CEA, and to noncardiac complications. The overall prognostic performance of the RCR in the present CEA cohort (area under the ROC curve, 0.56-0.66) was somewhat more modest than that in the original derivation studies^6,7,16 (area under the ROC curve, 0.69-0.81). This is likely because the generic risk indexes were developed to predict cardiac complications among a broad range of patients undergoing low-, intermediate-, and high-risk noncardiac surgery, not just intermediate-risk CEA. Still, the model performances that we report closely match the results of other nonderivation or validation studies^18,19 of noncardiac surgery that compared the ASA, Goldman, and Detsky indexes (area under the ROC curve, 0.60-0.64).

Rates of death and stroke in this cohort are consistent with those in the CEA literature. The 5.5% rate of combined death or stroke among symptomatic patients in our study is similar to the 5.1% rate reported in a meta-analysis²⁰ of 95 studies but is lower than the 6.5% and 7.1% rates reported in the North American Symptomatic Carotid Endarterectomy Trial²¹ and the European Carotid Surgery Trial,²² respectively. Among asymptomatic patients in our study, the combined death or stroke rate was 2.3%. This is comparable to the rate of 2.8% in a meta-analysis²⁰ of 60 studies and to the rate of 2.3% in the Asymptomatic Carotid Atherosclerosis Study.¹³Table 5 summarizes how the rates of cardiac and noncardiac medical complications that we observed are largely comparable to those in other studies and are close to the medians of the various rates that are reported in the literature.

Our finding that the RCR Index and the Halm and Tu scores predict noncardiac medical complications, minor neurologic complications, and wound complications is novel. No prior studies to our knowledge have analyzed the model characteristics for these types of outcomes. The prediction of noncardiac medical complications, which occurred in 3.2% of patients, is important for 2 reasons. First, these complications result in patient morbidity and increase their length of hospital stay.²⁹ Second, patients with noncardiac medical complications are at very high risk of death or stroke.

Our study has several strengths. First, we examined a large number of patients of all ages with varied comorbid illness burden undergoing surgery by 64 surgeons in 6 hospitals. Second, our review of subsequent readmissions and outpatient records minimized the chance of missing strokes and MIs that occurred after initial hospital discharge. Most studies of real-world practice have only reported complications during the index admission. The randomized controlled trials of CEA reported 30-day outcomes, but these trials excluded patients who were older or who had significant comorbidities,^12,13 subgroups who represent a considerable proportion of patients undergoing CEA in the community. Third, we collected detailed clinical data about preoperative risk factors and about a broad range of major and minor complications that affect mortality, morbidity, and length of hospital stay.

Our study has several limitations. First, we focused on patients undergoing 1 procedure, albeit a common one. Second, the overall prognostic performances of the risk models were modest although similar to those reported in the literature.^{5-10,16,18,19} Third, the standard cardiac risk indexes were not explicitly designed to predict several of the outcomes that we examined. However, our intention was to determine if these commonly used generic risk assessment tools could be more broadly useful in predicting other important perioperative outcomes. Fourth, the performance of the Halm score should be confirmed in an independent data set.

Although all the standard cardiac risk models predicted cardiac complications equally well, the RCR seemed to be best in predicting the broad range of major and minor medical and surgical complications. As such, the validated 5-item RCR Index seems to be the most useful generic risk assessment tool for patients undergoing CEA. Overall, the Halm score seemed to have the best all-around performance: it was superior for predicting the most important complications of death and stroke and was a significant but modest predictor of all other postoperative outcomes. The modest performance of all the risk models indicates that further research is needed to develop more accurate tools to assess operative risk.

Correspondence: Ethan A. Halm, MD, MPH, Division of General Internal Medicine, Mount Sinai School of Medicine, Box 1087, One Gustave L. Levy Place, New York, NY 10029 (ethan.halm@mountsinai.org).

Accepted for Publication: November 7, 2005.

Financial Disclosure: None.

Funding/Support: This study was supported by investigator-initiated research grant R01 HS09754-01 from the Agency for Healthcare Research and Quality.

Disclaimer: The views expressed are solely those of the authors.

Acknowledgment: We thank Patricia Formisano, MPH, for her management of the overall project, and Mary Rojas, PhD, Hugh Dai, MD, and Virginia Chan for their data management support.