Hospital and Surgeon Determinants of Carotid Endarterectomy Outcomes

Background  Prior studies have found associations between surgeon and hospital case volumes and outcomes after carotid endarterectomy (CEA), but they have not simultaneously assessed the importance of a number of surgeon and hospital characteristics.

Objective  To simultaneously assess associations between hospital case volume, teaching status, clinical trial participation, and surgeon specialty and case volume and the outcome after CEA.

Design  Analysis of a large administrative data-base using logistic regression to correlate adverse outcomes after CEA with surgeon and hospital characteristics.

Setting and Patients  A Canadian administrative hospital discharge database of all patients undergoing CEA in fiscal years 1994 through 1997.

Main Outcome Measures  In-hospital stroke and/or death.

Results  We found an inverse relationship between both hospital and surgeon case volumes and adverse outcomes. Teaching status had no association with outcome, but previous clinical trial participation predicted a better outcome. General surgeons fared worse than other specialists. Low-volume surgeons in low-volume hospitals had a relative risk of 3.5 for adverse outcomes compared with high-volume surgeons in high-volume hospitals.

Conclusions  Several physician and hospital characteristics are determinants of outcome after CEA, but the negative effects of low hospital and surgeon case volumes, in particular, suggest that regionalization should be considered for CEA and that surgeons with low case volumes should not be performing CEA.

CAROTID endarterectomy (CEA) is an operation to remove atherosclerotic narrowing from the cervical internal carotid artery to prevent stroke. The procedure has been used increasingly in the 1990s,¹ perhaps in response to the publication of several large randomized controlled trials (RCTs).^2,3 These RCTs have demonstrated the efficacy of the procedure in both symptomatic^4-6 and asymptomatic patients.⁷

The outcome of CEA is influenced by many factors. Most important, patients with symptomatic carotid disease are at higher risk of perioperative death and stroke than those with asymptomatic stenosis.^5-7 Surgeons and hospitals with high volumes of cases have generally shown lower complication rates than those with lower volumes for both CEA and many other procedures.^8,9 Surgeons in at least 4 different specialties (general, vascular, cardiovascular, and neurosurgery) perform CEA, but studies^10-12 have failed to find consistent outcome differences among these specialists. Other hospital and physician characteristics, such as teaching status and participation in RCTs of CEA, may also influence outcomes, although the relationship between such variables and outcomes is less clear.

To build on earlier research evaluating surgeon and hospital characteristics and their association with outcomes, we used a national data source for all CEAs performed in Canada to assess the association between several hospital and surgeon characteristics and rates of postoperative adverse events. The specific hospital characteristics evaluated in this study were case volume, teaching status, and participation in RCTs of CEA. The surgeon characteristics assessed were specialty and case volume.

Hospital discharge data on all CEAs were obtained from a national database compiled by the Canadian Institute for Health Information (CIHI) (Toronto, Ontario). This database includes information on all hospital discharges in Canada except for the province of Quebec, which does not report data to the CIHI. Patients undergoing a CEA in the other 9 provinces and 2 territories for the fiscal years 1994 through 1997 were identified by searching the database for those years using the code 50.12 of the Canadian Classification of Procedures.¹³ We excluded those patients who underwent coronary artery bypass grafting (CABG) and CEA at the same admission.

The outcomes of interest were the observed and risk-adjusted rates of in-hospital stroke and/or death. Stroke was defined using International Classification of Diseases, Ninth Revision¹⁴ (ICD-9) codes 433, 434, 436, or 438 when the diagnostic-type indicators available in Canadian administrative data indicated that the stroke had occurred during hospitalization.¹⁵ In addition, a postoperative stroke was considered to have occurred when ICD-9 code 997.0 was present.

The sociodemographic variables studied were age and sex. To define comorbidity variables that may predict outcome, we used the Deyo coding system,¹⁶ which is adapted for use with the International Classification of Diseases, Ninth Revision, Clinical Modification¹⁷ (ICD-9-CM) coding manual to identify the 17 comorbidities that comprise the Charlson Comorbidity Index.¹⁸ In addition to the Charlson Comorbidity Index, we also identified prior CABG (ICD-9-CM code V458.1), admission through the emergency department (as an indicator or urgency), and unstable angina (ICD-9-CM code 411.1).

Hospital-related variables included hospital CEA volume, teaching status, and clinical trial status. We categorized hospital CEA volume in the 4 fiscal years into 2 categories: low (<150) and high (≥150). The cut point of 150 was selected because it is close to the median of 126 and because it clearly demonstrated the association between hospital volume and outcome. We contacted each hospital in which CEA was performed and obtained written permission to release hospital identifiers to this study from the CIHI, which allowed us to identify individual hospitals in the administrative data. The teaching status of hospitals was identified using the Guide to Canadian Healthcare Facilities.¹⁹ Hospitals were categorized by trial status as having participated in either the North American Symptomatic Carotid Endarterectomy Trial (NASCET) of CEA for symptomatic carotid stenosis or the Asymptomatic Carotid Atherosclerosis Study (ACAS) of asymptomatic carotid stenosis. They were identified from the list of participant hospitals in the trial publications.^4,5,7

Surgeon-related variables included the surgical specialty and the surgeon case volume during the 4 years. The surgeons were grouped into 4 categories: general surgeons, vascular surgeons, neurosurgeons, and cardiovascular/thoracic surgeons. Surgeon case volumes were divided into 4 categories: 1 through 14, 15 through 29, 30 through 59, and 60 or more. For surgeon volumes, we explored a variety of cut points, and the cut points we selected best represent the stepped increases in adverse outcome rates as volumes decrease.

We initially performed χ² analyses and unpaired t tests to explore associations between outcomes and age, sex, admission through the emergency department, comorbidities, and hospital- and physician-related variables. Then, logistic regression was used to identify multivariate predictors of outcome and to calculate adjusted rates of stroke or mortality. Four variables (mild liver disease, moderate-to-severe liver disease, dementia, and human immunodeficiency virus) were excluded from multivariate analysis because they were rarely present and because no outcome events occurred in patients with these variables. The final model included patient age and sex, admission through the emergency department, chronic lung disease, diabetes, diabetes with complications, neoplastic disease, hemiplegia, myocardial infarction (old and recent), rheumatologic disease, peptic ulcer disease, chronic renal failure, peripheral vascular disease, metastatic disease, congestive heart failure, unstable angina, and prior CABG.

We used this risk adjustment model to calculate the predicted probability of in-hospital stroke or mortality for each patient who underwent CEA. We then averaged these predicted probabilities in a given hospital- or physician-related variable to calculate an expected rate of stroke or mortality (E). We then divided the observed rate of stroke or mortality (O) by the expected rate (E) to generate an O/E ratio. The adjusted rate of stroke or mortality was calculated by multiplying the O/E ratio by the overall in-hospital stroke or mortality rate. To statistically compare the adjusted rates among categories of a given hospital- and physician-related variable, we added that variable to the risk adjustment model to obtain a risk-adjusted P value for that variable using the log likelihood test. To simultaneously assess the association between outcomes and the hospital and physician characteristics examined in this study, we also created a logistic regression model that included all of these variables along with the clinical characteristics.

From the CIHI database, we identified 14 268 patients who underwent CEA and who did not undergo CABG during the same admission during 1994 through 1997. We had previously shown that this represents an age- and sex-adjusted rate of 31.7 cases per 100 000 population per year in 1994, which increased to 40.5 in 1997 for those 40 years and older.²⁰Table 1 presents the patients' sociodemographic and clinical characteristics. Table 2 presents the hospital and surgeon characteristics by percentage of patients. The overall national in-hospital death rate in this study was 1.3%. The in-hospital stroke rate was 3.4%, and the combined in-hospital stroke and/or death rate was 4.1%.

The relationships between hospital characteristics and outcomes are shown in Figure 1, which illustrates the effect of hospital volumes of CEA on outcomes and shows that low volume predicted a significantly worse adjusted outcome. Hospitals in which fewer than 150 patients underwent CEA (median, 126 patients) throughout 4 years had a combined adjusted outcome rate of 5.1%, whereas those with higher volume had a rate of 4.0% (P = .02). Hospitals that participated in the 2 large, North American CEA RCTs in the 1990s had a significantly lower adjusted adverse outcome rate (3.7%) than did nontrial hospitals (4.5%, P = .03). In contrast, no significant difference was found in adjusted outcome rates between teaching (4.1%) and nonteaching (4.2%) hospitals (P = .70).

Five surgical specialists were recorded as having performed CEAs in 1994 through 1997. Vascular surgeons and neurosurgeons each performed approximately one third of the cases. Because of the small number of cases attributed to thoracic surgeons (378), they were included with cardiovascular surgeons under the heading cardiovascular/thoracic (Figure 2). Using the combined outcome of stroke and/or death, there was a trend toward outcome differences across the 4 surgical divisions (P = .08, df = 3). When general surgeons were compared with the other 3 surgical specialists combined, a statistically significant difference was found (5.0% vs 4.0%, P = .03, df = 1).

A total of 367 surgeons performed at least 1 CEA in 1994 through 1997. Many surgeons performed only a small number of procedures. The number of procedures per surgeon ranged from 1 to 401 throughout the 4 years. The mean number of procedures per surgeon was 49.4 and the median was 23. One third of the surgeons performed fewer than 1 procedure per year. Figure 2 also shows the results of surgeons, categorized by case volume, on the combined stroke and/or death outcome measure and indicates a strong inverse relationship between surgeon case volume and adverse events, especially for those who performed fewer than 15 procedures during 4 years.

We examined the interaction between the hospital and surgeon case volumes. The percentage of adverse outcomes (95% confidence intervals) were as follows: for low-volume surgeons in low-volume hospitals, 13.6% (7.6%-21.8%); for low-volume surgeons in high-volume hospitals, 5.9% (3.5%-9.1%); for high-volume surgeons in low-volume hospitals, 4.7% (3.8%-5.6%); and for high-volume surgeons in high-volume hospitals, 3.9% (3.5%-4.3%). The relative risk (RR) for low-volume surgeons in high-volume hospitals is only slightly higher than that for high-volume surgeons (RR, 1.5). However, low-volume surgeons in low-volume hospitals have a much higher relative risk (RR, 3.5). In the analysis that simultaneously included all hospital and surgeon characteristics along with clinical characteristics and sociodemographic factors, RCT participation status of the hospital and surgeon volume were the characteristics that remained independently significant.

The measurement of outcomes is critical to the improvement of quality of health care. It allows monitoring of performance over time to detect trends and facilitates comparisons among hospitals and physicians. It also allows comparisons with performance goals that are based on the efficacy results from RCTs. Studies to explain the outcomes in terms of hospital and physician characteristics may point the way toward steps for quality improvement. The strength of our study is that we have used a single data source with a large number of cases to assess several hospital and physician characteristics as potential predictors of outcome and have demonstrated several important associations.

We found that hospital characteristics played a major role in predicting outcome after CEA (Figure 1). Hospitals in which fewer than 150 procedures were performed throughout 4 years had higher patient mortality and morbidity. Hospitals that had participated in the NASCET or ACAS performed better than those that did not. This likely reflects a variety of surgeon factors, including volume, since NASCET and ACAS surgeons had to meet strict volume and performance criteria to be admitted to the studies.^4,21 It may also reflect a special interest in the procedure on the part of surgeons. The teaching status of the hospitals did not correlate with a difference in performance. This was also observed in a large American study in 1993.²²

At least 5 different surgical specialists perform CEAs. We found that in Canada vascular surgeons (36.7%) and neurosurgeons (31.1%) perform most of the procedures, whereas in the United States, vascular surgeons predominate.¹⁰ We found that 14.9% of cases were performed by general surgeons, and this group had a worse outcome compared with all the other surgeons. In a study in Pennsylvania of 12 725 CEA cases from 1994 through 1995, general surgeons also had the highest mortality and adverse outcome rate, but the difference was not statistically significant.¹⁰ We were not able to separate general surgeons into those who might have had vascular surgical training but not certification from those who did not have vascular surgical training. The latter group might be responsible for the higher adverse outcome rate.

We found that low surgical volume predicted a worse outcome. Surgeons who performed 14 or fewer CEAs throughout 4 years had an adjusted stroke or death rate of 6.3% compared with a rate of 4.0% (P = .008; odds ratio, 1.7; 95% confidence interval, 1.1-2.5) for those with higher volumes (Figure 2). Kucey et al²³ found that Toronto surgeons with a case volume of fewer than 6 cases per year had a higher adverse outcome rate (odds ratio, 3.98). O'Neill et al¹⁰ found that surgeons who performed only 1 or 2 CEAs in 1994 through 1995 had a statistically significant increased adverse outcome rate. An older Pennsylvania study (cases from 1989-1992) showed a poorer outcome for surgeons performing fewer than 15 procedures per year.²⁴ An inverse relationship between surgeons' volumes and negative outcomes also has been documented for other surgical procedures.⁸ There has been debate about whether the volume-outcome relationship reflects a practice effect (practice makes perfect²⁵) or a selective referral pattern to more accomplished surgeons. We suspect that it may be a combination of both factors but predominantly a lack of practice effect on the part of the surgeons and the rest of the health care team. Looking at the relationship between hospital and surgeon volumes in terms of adverse outcomes, we found that the combination of low-volume surgeons with low-volume hospitals was especially risky (RR, 3.5) compared with high-volume surgeons in high-volume hospitals.

One goal of this study was to identify quality-of-care issues related to the performance of CEA. Overall, the results presented herein suggest that the outcome of CEA in 1994 through 1997 in Canada, in terms of death rates, was comparable to outcomes of the major North American RCTs^5,7 and the outcome of a large American medical chart review study²⁶ of CEA. However, our results likely underestimated the negative outcomes, since we measured stroke and/or death only until hospital discharge, not for 30 postoperative days as in the RCTs. Also, the recording of stroke as an outcome in administrative data is likely to be incomplete. In addition, our definition of postoperative stroke is open to debate. We used a combination of specific stroke codes and a nonspecific code (997) for neurologic complications. The latter may not always represent stroke, but we opted to use it because we anticipated that many hospitals code postoperative strokes this way. Finally, our ability to compare our results with the RCTs of CEA is limited because we were unable to separate cases into those that were symptomatic vs asymptomatic in our administrative database.

There clearly is room for improvement in the provision of CEA. The concerns about surgeon volumes and, to a lesser extent, hospital volumes stand out in this and other studies as important issues that need to be addressed. Low-volume hospitals and surgeons, as a group, have worse outcomes. The difficulty is in determining a meaningful cutoff volume below which a surgeon should not perform CEA. Studies in Connecticut and Pennsylvania showed that 43% and 37% of surgeons, respectively, who were performing CEA did only 1 procedure per year. Similarly, one third of Canadian surgeons performed fewer than 1 procedure per year. Hertzer,²⁷ in his presidential address to the Society of Vascular Surgery in 1994, drew attention to this issue but came out against the establishment of minimum volume limits for surgeons. However, the evidence is so convincing that low volumes predict bad outcomes that, surely, surgeons must not continue to perform only a few CEAs per year and consideration must be given to setting a reasonable target level for case volumes.²⁸ Likewise, the evidence of worse performance by low-volume hospitals suggests that regionalization should be considered for highly technical procedures to ensure that adequate volumes of cases are performed to produce good outcomes.

Predictors of outcome can be measured in studies like ours that involve large group comparisons. Factors such as hospital and surgeon volumes, clinical trial participation, and specialty qualification are clearly important determinants of outcome. Identification of outcome predictors is a necessary step on the way to quality improvement that must now be followed by corrective measures to enhance quality of care.

Accepted for publication June 6, 2002.

Author contributions: Study concept and design (Drs Feasby, Quan, and Ghali); acquisition of data (Dr Feasby); analysis and interpretation of data (Drs Feasby and Ghali); drafting of the manuscript (Dr Feasby); critical revision of the manuscript for important intellectual content (Drs Feasby, Quan, and Ghali); statistical expertise (Drs Quan and Ghali); obtained funding (Drs Feasby and Ghali); administrative, technical, and material support (Drs Feasby and Ghali).

This study was supported by a research grant from the Heart and Stroke Foundation of Alberta and the Northwest Territories (Calgary). Dr Ghali is supported by a Population Health Investigator Award from the Alberta Heritage Foundation (Edmonton) for Medical Research and holds a Canada Research Chair.

Corresponding author and reprints: Thomas E. Feasby, MD, Department of Clinical Neurosciences, Foothills Hospital, 1403 29th St NW, Calgary, Alberta, Canada T2N 2T9 (e-mail: feasby@ucalgary.com).