ICTRP indicates International Clinical Trials Registry Platform; and WHO, World Health Organization.
M-H indicates Mantel-Haenszel.
aHeterogeneity: Tau2 = 0.14; χ28 = 13.31 (P = .10); I2 = 40%; test for overall effect: z = 2.96 (P = .003). After sensitivity analysis, heterogeneity: Tau2 = 0.08; χ26 = 8.86 (P = .18); I2 = 32%; test for overall effect: z = 2.85 (P = .004).
eAppendix 1. MOOSE Guidelines Checklist
eAppendix 2. Complete Search Strategies
eAppendix 3. Modified Newcastle Ottawa Scale for Included Studies
eAppendix 4. Sensitivity Analyses by Low Risk of Bias and Time Based
eFigure 1. Funnel Plot of Effect Size Versus Sample Size Among Studies Reporting Stroke
eFigure 2. Funnel Plot of Effect Size Versus Sample Size Among Studies Reporting Major Bleeding
Customize your JAMA Network experience by selecting one or more topics from the list below.
Newhall KA, Saunders EC, Larson RJ, Stone DH, Goodney PP. Use of Protamine for Anticoagulation During Carotid Endarterectomy: A Meta-analysis. JAMA Surg. 2016;151(3):247–255. doi:10.1001/jamasurg.2015.3592
Protamine sulfate can be administered at the conclusion of carotid endarterectomy (CEA) to reverse the anticoagulant effects of heparin and to limit the risk for postoperative bleeding. Protamine use remains controversial owing to concern for increased thrombotic complications with its use.
To review the evidence for and against protamine use, both in its association with increased thrombotic complications and with decreased bleeding.
We searched Medline (1946-2014), EMBASE (1966-2014), Cochrane Library (1972-2014), clinical trial registries (World Health Organization International Clinical Trials Registry and clinicaltrials.gov), and abstracts from conferences of the Society of Vascular Surgery (2002-2014) and American Heart Association Scientific Sessions (1980-2014) in November 2014. No language restrictions were applied.
We included clinical trials and observational studies comparing reversal of heparin with protamine sulfate vs no reversal in patients undergoing carotid revascularization and reporting stroke during hospitalization. Of 360 records screened, 12 studies (3%) of CEA were eligible for data pooling.
Data Extraction and Synthesis
Two reviewers extracted data and assessed quality. Random-effects models were used to summarize relative risks (RRs).
Main Outcome and Measure
Stroke after CEA.
We included 12 observational studies involving 10 621 patients in the meta-analysis. Event rates did not differ significantly between patients who received protamine vs those who did not for the following outcomes: stroke (RR, 0.84; 95% CI, 0.55-1.29; I2 = 15%; 9 studies), myocardial infarction (RR, 0.89; 95% CI, 0.53-1.51; I2 = 0%; 3 studies), or mortality (RR, 0.9, 95% CI, 0.62-1.29; I2 = 0%; 7 studies). The use of protamine was associated with a significant decrease in major bleeding complications requiring reoperation (RR, 0.57; 95% CI, 0.39-0.84; I2 = 32%; 10 studies).
Conclusions and Relevance
Based on available evidence, the use of protamine following CEA is associated with a reduction in bleeding complications, without increasing major thrombotic outcomes, including stroke, myocardial infarction, or death.
While carotid revascularization reduces the long-term risk for stroke when compared with medical treatment alone, the procedure carries with it an increased risk for stroke in the perioperative period, estimated at 3% to 7%.1-5 Standardization of processes of care has helped to make carotid surgery safer in recent years.6 For example, heparin is routinely administered to reduce thrombus formation during artery clamping. However, while heparin use is essentially universal, reversal of heparin with protamine is not.7-9 Some surgeons reverse heparin with protamine at the conclusion of the procedure to limit bleeding; other surgeons do not, citing concern that protamine reversal increases the risk for thrombotic events, namely stroke.
Uncertainty about the risks of reversing heparin with protamine is reflected in ongoing variation of practice patterns. Studies have described up to 5-fold variation in the use of protamine, ranging from less than 20% of procedures to nearly 100% by surgeons.8,10 Concern for thrombotic complications with protamine use stems from early trials that showed increased risk for stroke with protamine.11-13 However, several observational studies have failed to replicate this increase in stroke rates or other thrombotic complications among patients receiving protamine following carotid endarterectomy (CEA).14-16 These studies showed a lower risk for bleeding when protamine was used, suggesting that protamine is effective in limiting the risk for bleeding after carotid surgery. Reported risks for bleeding during CEA vary as well, from 1.2% to 12%, owing in part to a lack of standardized definitions.14,17,18 Determining whether protamine is associated with lower rates of bleeding after CEA is an important goal because bleeding carries significant risk for airway compromise or reoperation—both independent risk factors for perioperative mortality.19
Therefore, we performed a meta-analysis of evidence relating both stroke and bleeding with protamine use in carotid artery revascularization. A better understanding of the relationships between protamine use, stroke, and bleeding across existing randomized and observational studies will help determine whether protamine use is justified in clinical practice. Because of limited literature regarding protamine in carotid stenting and the different nature of bleeding complications between stenting (at the access site) and surgery (in the neck), our quantitative analysis was limited to CEA.
Prior to conducting the review, we outlined our planned approach to the identification and selection of the studies. Meta-Analyses of Observational Studies in Epidemiology guidelines were used to report methods and findings (eAppendix 1 in the Supplement).20 The original protocol is available on request.
We aimed to include all studies that addressed the use of protamine in carotid revascularization. The following eligibility criteria were specified: (1) the design was a randomized clinical trial, retrospective or prospective cohort, nested case-control, before-and-after study, or secondary analysis of a randomized trial; (2) the population included adult patients undergoing any carotid procedures who were therapeutically anticoagulated with heparin; (3) the intervention/exposure was protamine sulfate to reverse heparin at the end of the procedure; (4) the comparison was no reversal of heparin; and (5) the study reported incidence of stroke during hospitalization. Our analysis focused on CEA, although we included the relevant stenting studies in the Table.
Stroke was the primary outcome of interest. Any type or severity of stroke was included in the analysis as long as it was diagnosed using objective criteria. Secondary outcomes included thromboembolic complications (death, myocardial infarction [MI], and transient ischemic attack). To capture the potential benefits of protamine use, postoperative bleeding events were examined as a secondary outcome. Bleeding was defined as reoperation for bleeding.
With consultation from reference librarians, we searched electronic databases including Medline (1946-2014), EMBASE (1966-2014), and the Cochrane Library (1972-2014) during November 2014. By using MeSH terms and key words, we created sets for the various carotid procedures (CEA or carotid artery stenting) and the exposure (protamine). To find studies including both of these components, we used the Boolean term AND to combine the 2 sets. We used no limits or language restrictions (search strategies are in eAppendix 2 in the Supplement).
Several strategies were used to identify unpublished, incomplete, or ongoing clinical trials. Two electronic trial registries were searched in October 2014: clinicaltrials.gov and International Clinical Trials Registry. Additionally, we hand searched the annual proceedings from the annual meetings of the Society for Vascular Surgery (2002-2014) and the American Heart Association Scientific Sessions (1980-2014), as well as references of included articles in November 2014.
Two reviewers independently screened the titles and abstracts of all records. Obviously irrelevant studies were excluded. Two reviewers independently reviewed the remaining full-text articles and selected relevant studies based on our inclusion criteria. At least 2 reviewers independently extracted data from each eligible study. Additional information from principal investigators was sought as needed. All discrepancies were resolved by consensus. Reviewers were health researchers and physicians, in regular consultation with senior vascular surgeons. For articles and abstracts not in English, we relied on translation by native speaker colleagues at The Dartmouth Institute for Health Policy and Clinical Practice (Lebanon, New Hampshire).
Two reviewers independently assessed the risk for bias using a modified version of the Newcastle-Ottawa Scale, which accommodates both observational studies and randomized trials.27,28 This scale consists of 3 categories: selection, comparability, and outcome, with questions in each domain corresponding to study quality. A study received a star in each category when it met the definition for high quality. Discrepancies were resolved by consensus. The results of the risk for bias assessment informed the sensitivity analyses (eAppendix 3 in the Supplement).
Relative risks (RRs) were used to summarize those studies amenable to quantitative pooling. Three otherwise eligible studies were excluded from the meta-analysis of stroke.17,18,21 These studies did not specify whether the patients who experienced stroke had received protamine, and adequate data could not be obtained. As they otherwise met inclusion criteria, these studies were included when possible in the analysis of secondary outcomes.
Review Manager version 5.3 (Cochrane Collaboration) was used to calculate summary estimates using both fixed- and random-effects models. Findings were reported based on random-effects models to account for heterogeneity among treatment effects across studies.
Heterogeneity among the study findings contributing to each summary was estimated based on Review Manager output. Excess heterogeneity was considered present if either the I2 (inconsistency) was greater than 50% or the P value was less than .10 and identification of responsible studies was attempted. Both method and clinical characteristics of studies were examined for possible explanations. Summary estimates were recalculated based on the largest group of homogeneous studies that could be combined.
To assess for evidence of publication bias, funnel plots of stroke and bleeding risk were created in Review Manager. Authors examined plots for an inappropriate correlation between sample size and effect size.
We had planned subgroup analysis by routine protamine use and use of dual antiplatelet agents prior to starting the analysis. However, we were unable to perform these analyses owing to a lack of usable data. A post hoc analysis of patch angioplasty and shunt use was performed after it was noted that this characteristic differed significantly among groups receiving and not receiving protamine in most studies.
Three analyses were prespecified to evaluate the impact of method quality on the overall summary estimates. First, because the studies were published over 2 decades, studies were stratified by year of publication (prior to 2000 and after 2000). Second, because there were several very large studies, studies were stratified by size (<1000 patients and >1000 patients). Last, components of the modified Newcastle-Ottawa Scale and summary estimates were recalculated and restricted to studies considered at low risk for bias. We compared whether the direction, magnitude, or statistical significance of the restricted summary estimates meaningfully differed from the overall estimates.
As shown in Figure 1, our search yielded 360 potentially relevant records. We excluded 217 based on title and abstract screening, and 129 based on full-text review. This left 14 studies that met all inclusion criteria: 12 related to CEA and 2 related to carotid stenting.11-18,21-26
The Table shows the characteristics of the 14 studies that were eligible for the review. Among the 12 studies evaluating patients undergoing endarterectomy, designs included a randomized clinical trial,11 a secondary analysis of a randomized trial,16 8 observational cohorts,12-14,21-24 a nested case-cohort,17 and a nested case-control.18 Sample sizes ranged from 42 to 4587, and baseline characteristics, including age, sex, and race/ethnicity, were similar between the protamine and no protamine groups. When mentioned, goal activated clotting time ranged from 250 to 350 seconds. Between the 2 studies evaluating patients undergoing carotid stenting, one was a pooled analysis of data from 4 randomized trials,25 while the other was a nested case-cohort study.26 Sample sizes were 2104 and 1110, respectively. Baseline characteristics including age, sex, and race/ethnicity were again similar among the protamine and no protamine groups. As a whole, the patients in the carotid stenting studies were older than those in the endarterectomy studies (age range, 70-80 years vs 60-80 years, respectively) and less likely to receive protamine (8%-8.3% vs 10%-60%, respectively).
We applied the modified Newcastle-Ottawa Scale to assess for the risk for bias among both randomized and nonrandomized trials. The most common risk for bias was related to the selection of experimental and control groups. In 5 studies, there was a risk that patients receiving protamine systematically differed from patients not receiving protamine.11,15,18,21,24 Of these, 4 failed to describe the derivation of the protamine group,11,15,18,21,24 while 1 selectively used protamine in patients with surgical wounds deemed excessively hemorrhagic,21 a term that could not be used to categorize the other procedures.
Nine studies involving 9932 patients undergoing CEA provided usable data on stroke during hospitalization.11-16,22-24 Pooling the study findings, the rate of perioperative stroke was 62 of 3907 (1.59%) among patients who received protamine and 122 of 6025 (2.02%) among those who did not receive protamine (Figure 2A). The weighted summary estimate demonstrated no significant differences between groups (RR, 0.84; 95% CI, 0.55-1.29). Heterogeneity among the trials was low (P = .31; I2 = 15%), suggesting a consistent lack of difference in stroke risk between patients who received and did not receive protamine. These findings remained unchanged after sensitivity analyses to account for high risk for bias, older studies, or larger sample size.
Stroke definition varied across studies. Five studies defined stroke by clinical presentation (symptoms persisting >24 hours),12,13,22-24 2 by independent assessment from a neurologist,15,16 1 by direct thrombus visualization,11 and 1 by database-defined criteria.14 No studies used imaging alone to define stroke.
Ten studies involving 8553 patients undergoing CEA reported data on major bleeding, defined as bleeding events that required reoperation during hospitalization.11-18,21,24 Three studies significantly favored protamine,12,14,15 6 found no significant difference between groups,11,13,16-18,24 and 4 had no events in 1 or both arms.15,18,21,24 Pooling the findings across studies, the risk for major bleeding was 66 of 3887 (1.7%) among patients who received protamine and 217 of 6225 (3.5%) among patients who did not. The weighted pooled estimate demonstrated a statistically significant difference favoring the use of protamine among all 10 studies (RR, 0.52; 95% CI, 0.34-0.80; I2 = 40%; P = .10). Owing to a borderline heterogeneity test, we performed a sensitivity analysis by study design. Two excluded studies using a single-surgeon study design11,15 created a statistically homogenous group, which showed similar bleeding risk reduction for patients given protamine (RR, 0.57; 95% CI, 0.39-0.84; I2 = 32%) (Figure 3).
Eleven studies provided data on the risk for any bleeding events, which included wound hematomas and requirement of transfusion.11-18,21,23,24 Again, the use of protamine was associated with a significantly lower rate of bleeding events (RR, 0.46; 95% CI, 0.38-0.73). As in the outcome of bleeding requiring reoperation, the test of heterogeneity was again borderline (P = .07; I2 = 42%). Sensitivity analysis was performed by study design and excluded 2 studies using a single-surgeon study design to create a statistically homogenous group, with similar effect (RR, 0.52; 95% CI, 0.38-0.73; P = .20; I2 = 27%).
Seven studies reported data on all-cause mortality among patients undergoing CEA.11-16 Pooling the findings, there was no significant difference between patients who received protamine (40/3458; 1.2%) and patients who did not (101/5933; 1.7%) (weighted pooled estimate: RR, 0.9; 95% CI, 0.62-1.29; I2 = 0%). Similarly, 3 studies assessed rates of MI following CEA and demonstrated no difference between study arms (RR, 0.89; 95% CI, 0.53-1.51; I2 = 0%)14-16 (Figure 2B and C).
We were able to perform a post hoc analysis of stroke risk in studies using 2 characteristics of surgical processes of care during CEA: shunt use and patch angioplasty. Three studies had data stratified by patch and shunt use.11,13,23 Protamine use was not associated with an increased risk for stroke in the patch angioplasty or shunt subgroups (Figure 4).
We performed sensitivity analyses, using only those studies with high-quality score/low risk for bias within each domain of the Newcastle-Ottawa Scale (selection, comparability, and exposure). This did not meaningfully alter the study findings for stroke. Within each restricted analysis, the findings between the protamine and no protamine groups did not differ significantly. Additional sensitivity analyses based on study age (published before 2000) or sample size (excluding the largest studies) did not meaningfully change the risk for stroke (full analysis shown in eAppendix 4 in the Supplement).
After repeating these restricted analyses for the major bleeding outcome, all 7 restricted estimates continued to favor protamine and 5 remained statistically significant. When findings were restricted to only those studies published after 2000, there was no change in stroke risk with protamine use. However, the benefit of protamine for decreasing bleeding was not significant when restricted only to studies published after 2000 or to larger studies, although these analyses had heterogeneity (I2 = 60% and I2 = 76%, respectively), which could not be eliminated through further sensitivity analyses.
We found no evidence of publication bias based on the funnel plot for stroke. However, we noted that all published small studies strongly favored protamine with regard to the major bleeding outcome (eFigure 1 and eFigure 2 in the Supplement).
We performed a meta-analysis of 12 studies, which included 9932 patients, 3907 (40%) of whom received protamine and 6025 (61%) of whom did not. We found that the use of protamine in CEA was not associated with a statistically significant increase in the risk for stroke. Additionally, protamine use was not related to a higher risk for any other thromboembolic complications, such as MI or death. However, protamine use was associated with a 43% decline in major bleeding risk. These findings were consistent across studies spanning several decades and remained unchanged when restricted to studies considered to be at low risk for bias.
A key aspect of any meta-analysis is the quality of the evidence. In our study, the quality of the included studies varied for several reasons. First, 11 of our 12 studies were observational in nature, and confounding may be a concern within these studies.12-18,21-24 We were able to perform a subgroup analysis of measured confounders, such as differences in operative technique, which did not change our primary finding. Second, because the studies spanned over a 20-year period, our findings were limited by temporal trends that have occurred. Overall stroke risk was higher in the older studies than in the newer studies (2.7% vs 1.7%; P = .002), as was bleeding risk (4.6% vs 1.7%; P < .001). We suspect this decline in adverse outcomes reflects improvements in patient selection, medical therapy, and preoperative management. Third, there were differences among studies in perioperative use of aspirin or β-blockers, both routinely recommended.29 Three studies continued aspirin through the day of surgery,11,13,21 while 2 studies explicitly stopped aspirin within 1 week of surgery for most patients.15,17 Only 1 study provided information regarding use of β-blockers or statins.14 Finally, most studies were performed in academic centers, so these findings may not be replicable across smaller private hospitals. However, 2 of the studies were single-surgeon trials11,15 and the largest database included all levels of hospitals,14 which allows the findings to be applicable to all surgeon practices.
The risk for bias was considered low overall, based on our assessment tool. The lack of heterogeneity in stroke and bleeding rates across studies, as well as the lack of between-study differences in other outcomes (transient ischemic attack, MI, and death) suggests that our findings reflect a real effect across a range of study designs and settings.
We minimized the possibility of missing trials using multiple methods to search both published and unpublished literature. Furthermore, contact with several study authors did not reveal any additional unpublished data on the topic. We were limited by missing data for the primary outcome, despite attempted contact with the 3 authors whose stroke outcome was not stratified by protamine use.17,18,21 That noted, we performed analysis with all strokes for each study occurring within the protamine arm, and our main finding remained unchanged.
Stroke remains a difficult outcome to standardize owing to differing definitions across studies. Our study was no exception: we were unable to discern thromboembolic from other stroke etiologies, as authors often included all perioperative stroke. However, hemorrhagic stroke during CEA is not thought to be associated with procedural factors, and so the rate should be equally distributed across both groups. Furthermore, hypotension due to protamine reaction has been cited as a possible etiology for stroke: no included study included patients who experienced protamine reaction.30,31
A second difficulty in studying stroke risk is that as it becomes increasingly rare following CEA, differences related to protamine use are more difficult to discern. This trend is unlikely to have biased our findings, as our findings were stable across sensitivity analyses of older and newer studies. Furthermore, our study of protamine use in CEA had similar findings to previously published meta-analyses of the use of protamine in coronary angioplasty, which found no increased risk of stroke.32 Finally, the trials included in our analysis varied widely with regard to operative technique and patient characteristics. Despite this, diversity did not affect heterogeneity on our main outcome measure: stroke. Given the wide variety of clinical practice in CEA, both in operative technique and preoperative medical management, we think this study is more representative of the use of protamine in a real-world environment.
Other potential sources of bias in our meta-analysis might include the broad inclusion criteria, limited randomized clinical trials, lack of studies that included other procedures, inability to explore subgroups that might have explained heterogeneity, and the varied definitions for bleeding complications across studies. We performed analyses with both narrow and broad definitions of bleeding, and the benefit of protamine use remained unchanged.
Our study has important implications. Surgeons should consider routinely using protamine during CEA owing to the decreased risk for bleeding with its use. While the net reduction in bleeding complications is small, reoperation for any reason after CEA has the potential for increased morbidity. Given that there were fewer studies examining protamine use in carotid artery stenting, further research on protamine and carotid stenting is needed to determine whether our findings are consistent across all types of carotid revascularization.
Corresponding Author: Karina A. Newhall, MD, Dartmouth Hitchcock Medical Center, Surgery, 1 Medical Center Dr, Lebanon, NH 03756 (email@example.com).
Accepted for Publication: July 8, 2015.
Published Online: October 21, 2015. doi:10.1001/jamasurg.2015.3592.
Author Contributions: Dr Newhall had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Newhall, Saunders, Larson, Goodney.
Acquisition, analysis, or interpretation of data: Newhall, Saunders, Larson, Stone.
Drafting of the manuscript: Newhall, Saunders, Goodney.
Critical revision of the manuscript for important intellectual content: Larson, Stone.
Statistical analysis: Newhall, Saunders, Goodney.
Obtained funding: Goodney.
Study supervision: Larson, Stone, Goodney.
Conflict of Interest Disclosures: None reported.
Funding/Support: Dr Goodney is supported by grants K08 HL05676 and R21 HS021581-01A funded through the Dartmouth Institute for Health Policy and Clinical Practice (Lebanon, New Hampshire).
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Additional Contributions: Christopher Caracciolo, BA, Suveera Dang, BA, and Achal Patel, BA, of the Dartmouth Institute for Health Policy and Clinical Practice (Lebanon, New Hampshire) provided data collection. They did not receive compensation for their contributions.
Disclaimer: The contents of this article do not represent the views of the US Department of Veterans Affairs or the US government.
Create a personal account or sign in to: