Flow diagram depicting the elimination process of articles retrieved from initial literature search. CEA indicates carotid endarterectomy and SSEPs, somatosensory evoked potentials.
Bivariate normal linear mixed model accounting for unadjusted and adjusted pooled estimate for the reversibility variable.
The HSROC curve represents a global summary of test performance and shows the trade-off between sensitivity and specificity, with pooled area under the ROC curve estimated to be 0.67.
Pooled estimate from random effects model.
eMethods. Data Extraction.
eFigure. Deek Funnel Plot for log Diagnostic Odds Ratio (log-DOR).
Customize your JAMA Network experience by selecting one or more topics from the list below.
Nwachuku EL, Balzer JR, Yabes JG, Habeych ME, Crammond DJ, Thirumala PD. Diagnostic Value of Somatosensory Evoked Potential Changes During Carotid Endarterectomy: A Systematic Review and Meta-analysis. JAMA Neurol. 2015;72(1):73–80. doi:10.1001/jamaneurol.2014.3071
Perioperative stroke is a persistent complication of carotid endarterectomy (CEA) for patients with symptomatic carotid stenosis (CS).
To evaluate whether changes in somatosensory evoked potential (SSEP) during CEA are diagnostic of perioperative stroke in patients with symptomatic CS.
Design, Setting, and Participants
We searched PubMed and the World Science Database for reference lists of retrieved studies and/or experiments on SSEP use in postoperative outcomes following CEA in patients with symptomatic CS from January 1, 1950, through January 1, 2013. We independently screened all titles and abstracts to identify studies that met the inclusion criteria and extracted relevant articles in a uniform manner. Inclusion criteria included randomized clinical trials, prospective studies, or retrospective cohort reviews; population of symptomatic CS; use of intraoperative SSEP monitoring during CEA; immediate postoperative assessment and/or as long as a 3-month follow-up; a total sample size of 50 or more patients; studies with adult humans 18 years or older; and studies published in English.
Main Outcome and Measure
Whether intraoperative SSEP changes were diagnostic of perioperative stroke indicated by postoperative neurological examination.
Four-hundred sixty-four articles were retrieved, and 15 prospective and retrospective cohort studies were included in the data analysis. A 4557-patient cohort composed the total sample population for all the studies, 3899 of whom had symptomatic CS. A change in SSEP exhibited a strong pooled mean specificity of 91% (95% CI, 86-94) but a weaker pooled mean sensitivity of 58% (95% CI, 49-68). A pooled diagnostic odds ratio for individual studies of patients with neurological deficit with changes in SSEPs was 14.39 (95% CI, 8.34-24.82), indicating that the odds of observing an SSEP change among those with neurologic deficits were 14 times higher than in individuals without neurologic deficit.
Conclusions and Relevance
Intraoperative SSEP is a highly specific test in predicting neurological outcome following CEA. Patients with perioperative neurological deficits are 14 times more likely to have had changes in SSEPs during the procedure. The use of SSEPs to design prevention strategies is valuable in reducing perioperative cerebral infarctions during CEA.
Carotid endarterectomy (CEA) is the gold-standard treatment for patients with symptomatic carotid stenosis (CS) to reduce the risk of stroke.1-4 There is a clear benefit for CEA in symptomatic patients (usually with CS ≥70%) when compared with medical management alone.5-12 At the 5-year mark, 85% to 90% of patients treated with CEA are stroke free in comparison with medical treatment, which has a rate of 75% to 80%. However, studies have shown 2% to 3% of CEA cases result in ischemic insult to the patient. These are mainly thought to be a result of thromboembolic events13,14 from the operative and/or distal site as a result of atherosclerotic plaque disruption15,16 or from cerebral hypoperfusion11,17 owing to cross-clamping of the internal carotid artery in conjunction with inadequate collateral circulation.18 Stroke after CEA is associated with significant morbidity and a nearly 3-fold increase in future mortality in these patients.19
Intraoperative intraluminal shunting has been used during CEA to prevent cerebral hypoperfusion and impending ischemia13,20-22 during the cross-clamping of the carotid artery. In conjunction, intraoperative neurophysiological monitoring with somatosensory evoked potential (SSEP) has been shown to be a valuable technique for determining the need for selective intraoperative shunting in place of routine shunting in all patients.1,20,23-27 Somatosensory evoked potential monitoring allows for adequate assessment of collateral circulation and the need for shunt insertion.
A shunt is generally inserted if there is a significant change in response to the SSEP waveform, which is usually defined as a decrease greater than 50% in the amplitude of the cortical N20-P25 complex and/or a 10% increase in latency of cortical wave after the cross-clamping of the carotid artery.15-17,24-26,28-34 However, strokes still occur despite these preventive measures. Moreover, the predictive value of the neurologic outcome of SSEP changes that persist following the placement shunt remains unclear. Such information could offer assistance during the CEA procedure to detect shunt malfunction, as well other events of hypoperfusion, which can lead to perioperative stroke. Furthermore, the predictive values of SSEP changes can help effectively test newer mechanical and pharmacological targets of neuroprotective35strategies during CEA to reduce stroke. Additionally, SSEP intraoperative monitoring holds great potential in its ability to allow for proper decision making and stratification to permit aggressive perioperative management for at-risk patients in efforts to prevent neurological deficits. To our knowledge, such level of neurosurgical care is currently unavailable for patients undergoing CEA. Somatosensory evoked potential can be the vital tool to rectify such a dilemma.
Our primary aim was to perform a systematic review of the scientific literature to evaluate whether changes in SSEPs during CEA are diagnostic for perioperative strokes (neurological deficits). The goal of this review was to ascertain the sensitivity, specificity, diagnostic odds ratio, and area under receiver operating characteristic (ROC) curves of intraoperative SSEP changes in relation to neurological outcome in patients undergoing CEA for symptomatic CS.
We searched PubMed and the World Science Database for reference lists of retrieved reports and/or experiments from January 1, 1950, through January 1, 2013, for studies on SSEP use for postoperative outcome after CEA in patients with symptomatic CS. To execute the search, the following terms were used: cerebrovascular diseases, carotid stenosis, carotid artery diseases, transient ischemic attack, intraoperative monitoring, somatosensory evoked potentials, N20 cortical potential, carotid endarterectomy, postoperative complications, neurologic deficits, stroke, paralysis, and paresis.
The inclusion criteria included the following: (1) randomized clinical trials and prospective or retrospective cohort reviews; (2) population of symptomatic CS; (3) use of intraoperative SSEP-monitoring during CEA; (4) immediate postoperative assessment and/or as long as a 3-month follow-up; (5) a total sample size of 50 or more patients; (6) studies with adult humans 18 years or older; and (7) studies published in English.
The authors (E.L.N. and P.D.T.) independently screened all titles and abstracts to identify studies that met the inclusion criteria and extracted relevant articles in a uniform manner. Subsequently, each author constructed an Excel spreadsheet outlining the articles to be eliminated; a reason for the elimination was dictated by a number corresponding to 1 of the inclusion criteria. After elimination disagreements were reconciled, a final list of articles was assembled. The number of true-positives, false-negatives, false-positives, and true-negatives in patients with symptomatic CS who underwent CEA were extracted and tabulated for each study in a 2 × 2 table (see eMethods in the Supplement for more detail).
The data synthesis used a bivariate normal model for the logit-transformed pairs of sensitivities and false-positive rates prior to the fitting of a linear mixed model.36 This model preserved the bivariate nature of the data by taking into account any correlation between sensitivity and specificity. In the absence of covariates, it was equivalent to the hierarchical summary ROC model.37 The mean logit sensitivity, specificity, and covariance were estimated from this model. Forest plots and a summary ROC with a 95% contour ellipsoid were constructed. As a secondary analysis, we also fitted fixed and random effects models for log-diagnostic odds ratios. Unlike the bivariate normal model, this converted the bivariate nature of the data to a univariate problem.
We assessed heterogeneity in sensitivities and specificities via univariate I2 or the estimated variance coefficients. Heterogeneity was also examined visually through forest and summary ROC plots. Potential sources of heterogeneity were examined by meta-regression. Covariates that were considered included reversibility rate or shunt rate. As a sensitivity analysis, models were fitted with potential outlying studies deleted. To investigate publication bias, we constructed funnel plots of log-diagnostic odds ratios vs the inverse square roots of the effective sample size and tested for funnel plot asymmetry using the Deeks et al38 method, an approach that has been recommended in the literature.37 Additionally, the QUADAS 2 checklist was used to evaluate for bias and applicability of the studies.39 All statistical analyses were carried out in R (version 13.1) using the MADA package.40
There were 464 articles retrieved based on title and abstract from a literary database search (Figure 1). However, 47 articles remained after full assessment by the authors. Further evaluation based on study inclusion criteria led to 15 studies that were included for study analyses. These 15 studies consisted of prospective and retrospective cohort studies.
All of the studies used SSEP as a mode of intraoperative neuromonitoring during CEA. Baseline SSEPs were recorded in all patients either before or after induction of anesthesia. Each study constructed alarm criteria needed for intraluminal shunt placement during significant SSEP change, which were classified in all studies as complete or 50% decrease in N20-P25 cortical waveforms. Furthermore, in these studies neurological clinical evaluation ranged from immediately to 3 months postoperatively (Table 1).
A 4557-patient cohort composed the total sample population for all the studies, of which 3899 had symptomatic CS characterized by recurrent transient ischemic attacks or cerebral vascular accident. The mean age for the patient population from all the studies was approximately 61 years.
Of the total patient population, 514 (11.3%) exhibited significant SSEP change during surgical manipulation, indicating an ischemic event. Furthermore, the significant SSEP change group was subdivided into reversible and irreversible change. Of this subgroup, 480 patients (93.4%) were reversible, with 34 patients (6.6%) who were irreversible. A moderate proportion, 1194 (26.2%) of the patient population, were shunted either routinely as a result of significant SSEP change or at the surgeon’s discretion. In patients with SSEP changes, the incidence of neurological deficit was 65 of 514 (12.6%). In patients without SSEP changes the incidence of neurological deficit was 43 of 4043 (1.1%). There were 2 studies in which most patients underwent shunting irrespective of changes in SSEPs.
Postoperative neurological deficit was observed in 108 patients (2.4%) in the patient sample. Of the minor subgroup of patients with neurological deficit, 65 (60.2%) had intraoperative SSEP change while 43 (39.8%) did not. Additionally, the incidence of neurological deficit in the reversible and irreversible SSEP subgroup was 6.6% and 97%, respectively. In the entire study sample, there was a mortality rate of 0.22% (Table 2).
Figure 2 shows a forest plot of sensitivities and specificities of the ability of SSEP change to predict postoperative neurological outcomes for each study. Study sensitivities ranged from 25% to 93%, while specificities ranged from 67% to 98%. Combining data from all studies without accounting for possible covariates explaining heterogeneity and using the bivariate normal mixed effects model, SSEP change exhibited strong specificity (mean, 91%; 95% CI, 86%-94%) but weaker sensitivity (mean, 58%; 95% CI, 49-68). The model-based pooled area under the ROC curve was estimated to be 67%. The hierarchical summary ROC curve presented a global summary of test performance and showed the trade-off between sensitivity and specificity. A graph of the estimated hierarchical summary ROC along with the summary point, 95% confidence ellipse, and prediction ellipse for SSEP is shown in Figure 3.
A pooled random effects estimate of the diagnostic odds ratios for individual studies of patients with neurological deficit with changes in SSEPs was 14.39 (95% CI, 8.34-24.82). This indicated that the odds of observing an SSEP change among those with neurologic deficit were 14 times higher than those without neurologic deficit. To account for the wide variation in the confidence interval in individual studies, a forest plot of log-diagnostic odds ratios for the individual studies was obtained (Figure 4). Log-diagnostic odds ratios ranged from 0.52 to 5.0. A significant pooled average estimate of 2.67 (95% CI, 2.12-3.21) was based on a random effects model.
Univariate heterogeneity analysis indicated homogeneity in sensitivities (I2<0.01%) but substantial heterogeneity in specificities (I2 = 92%). Subsequently, SSEP change reversibility rate or shunt rate was used as a covariate in bivariate normal mixed effects meta-regression models. The reversibility rate was found to be significant in explaining heterogeneity in specificity (P < .001). Logit specificity tended to decrease with an increasing reversibility rate (estimate, −0.08; 95% CI, −0.09 to −0.06). From this model, a slightly more precise adjusted pooled specificity (mean, 90%; 95% CI, 89%-92%) was achieved compared with the model without covariates (Figure 2). The shunt rate was not found to be a significant source of heterogeneity in either specificity (P = .15) or sensitivity (P = .87). Reanalysis of data excluding 2 studies,13,33 which appear to have outlying specificities, yielded similar estimates as the original analysis.
Our results indicated that SSEP changes possess a strong specificity of 0.91 but a weak sensitivity of 0.58 to detect neurological deficits during CEA. The higher specificity of neurological deficits in patients with SSEP changes indicates that, in some patients, reversal of SSEPs is desirable but not always achievable. We believe that low sensitivity is a result of clinical interventions, such as intraluminal shunts and mean arterial pressure elevation, implemented once ischemia is detected with SSEPs following the cross-clamping of the carotid artery. In our study, this resulted in reversible increase in the cerebral blood flow (CBF), thus preventing infarction. Cerebral blood flow is normally 50 mL/100 g/min, with higher flow to the metabolically active cortical regions compared with subcortical regions. Animal studies have indicated that a drop in CBF below 16 to 20 mL/100 g/min causes a reversible decrease in the amplitude of cortical SSEP responses.41,42 Moreover, CBF values between 12 and 15 mL/100 mg/min result in complete disappearance of SSEP responses resulting in electrical failure. Further animal studies have shown that loss of SSEP responses is a precursor of ion pump failure at the cellular level.43 In addition, human studies have shown that when CBF decreases below 14 mL/100 g/min, persistent reduction in SSEP amplitude by 50% is usually observed.44 Infarction of the tissue or ion pump failure occurs at CBF of 10 to 12 mL/100 mg/min. Hence, in this narrow hemodynamic window where loss of cortical SSEP responses does not imply loss of neuronal viability, reversing CBF changes quickly changes the course of iatrogenic cellular ionic changes.45
As previously shown in the literature, our study supports evidence that perioperative stroke is a major complication of surgery for CS. The result from our data revealed an incidence of 2.4%, which is congruent with rates in the current literature. Additionally, diagnostic odds ratios estimated that when individuals experienced a significant ischemic event perioperatively, SSEPs were approximately 14 times more likely to have changed during CEA. More importantly, patients who had irreversible changes had a 97% chance of an ischemic perioperative insult. These perioperative events are likely owing to hypoperfusion, thromboemboli, shunt malfunction, and inadequate blood pressure control.11,13-17 A significant challenge in identifying perioperative neurological deficits was determining the exact timing of the event and the type of clinical evaluation used to identify stroke. The international carotid stenting study that performed preoperative and postoperative diffusion-weighted magnetic resonance imaging in these patients identified a new lesion in 17% of the patients.35 Intraoperative neuromonitoring with the use of SSEPs based on the current study appears to be a specific indicator of a perioperative neurological deficit.
Based on the literature, false-negative SSEP results appear to be a common problem, with ranges of 0% to 3.5%.26 Our study demonstrated a false negative rate of 0.96%. Again, false-negatives were seen in patients who experienced postoperative neurological deficit but in whom there were no significant SSEP changes. One particular hypothesis is that these neurological deficits could have occurred postoperatively from thromboemboli26 and, in that case, it would be unrealistic for the SSEP to recognize them intraoperatively. Additionally, technical errors could also be a factor that leads to false-negative SSEP results in particular patients.16
Although the current study showed strengths based on comprehensive literature review, with quality assessment using the QUADAS-2, there are several limitations that must be addressed. Most importantly, it is crucial to acknowledge the fact that a search bias may have existed owing to the difficulty associated with acquiring every possible study assessing use of SSEP during CEA. Also, owing to the study design, our analysis is at risk of publication bias because of the dependence on currently published data on the topic of investigation; however, our analysis via funnel plot (eFigure in the Supplement) provides no evidence of such bias in the current study. Statistically significant heterogeneity was observed among the averaged specificity of the studies. Owing to the design of this study, it was difficult to assess every possible factor for such a result because of data pooling from diverse sources. However, we assessed 2 critical variables (rate of shunt and SSEP reversibility) for the study in a meta-regression. The reversibility of SSEP accounted for some of the heterogeneity seen in the averaged specificity.
Somatosensory evoked potentials are not the only modality used during CEA to determine the need for intraluminal shunt placement. Electroencephalography, transcranial Doppler, stump pressure, and cerebral oximetry21 have been used to identify hypoperfusion during cross-clamping of the carotid artery. Our study did not evaluate the diagnostic accuracy of such studies; it is unclear if these tests provide similar prognostic information.12
Intraoperative SSEP is a highly specific test in predicting neurological outcome following CEA. Low sensitivity could be related to interventions performed after a change in SSEPs. Patients with postoperative neurological deficits are 14 times more likely to have had changes in SSEPs during the procedure. Perioperative stroke is almost guaranteed in a patient who demonstrates irreversible SSEP during CEA. Thus, these results are clinically relevant both for surgical approach and patient education in regards to postoperative expectations. Ultimately, understanding the etiologies of perioperative strokes and further using SSEPs to design prevention strategies can prove valuable in reducing perioperative cerebral infarctions during CEA.
Corresponding Author: Parthasarathy D. Thirumala, MD, MS, Department of Neurosurgery, University of Pittsburgh Medical Center, 200 Lothrop St, Ste B-400, Pittsburgh, PA 15213 (email@example.com).
Accepted for Publication: September 2, 2014.
Published Online: November 10, 2014. doi:10.1001/jamaneurol.2014.3071.
Author Contributions: Dr Thirumala had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Balzer.
Acquisition, analysis, or interpretation of data: Nwachuku, Yabes, Habeych, Crammond, Thirumala.
Drafting of the manuscript: Nwachuku, Yabes.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Yabes.
Obtained funding: Nwachuku.
Administrative, technical, or material support: Nwachuku.
Study supervision: Balzer, Habeych, Thirumala.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was funded by grant 5UL1TR000005 from the National Institutes of Health.
Role of the Funder/Sponsor: The National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.