Assessing the Outcomes of Procedural Innovation | Surgery | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Views 986
Citations 0
Invited Commentary
February 22, 2021

Assessing the Outcomes of Procedural Innovation

Author Affiliations
  • 1Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
  • 2Cardiovascular Outcomes, Quality, and Evaluation Research Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia
  • 3Leonard Davis Institute of Health Economics, Philadelphia, Pennsylvania
  • 4Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
JAMA Netw Open. 2021;4(2):e210328. doi:10.1001/jamanetworkopen.2021.0328

Carotid arterial revascularization remains the evidence-based cornerstone of therapy for symptomatic or severe asymptomatic carotid atherosclerosis.1 Despite remarkable advances in surgical quality of care as well as the introduction of innovative endovascular device technologies during the past 15 years,2 carotid revascularization remains a high-risk procedure, with 4% to 5% of patients experiencing perioperative major adverse cardiovascular events (MACE).3 Transcarotid artery revascularization (TCAR) is an interventional technology designed to reduce the risk of thromboembolic periprocedural events. While TCAR adoption has been rapid in the past 5 years, there has been relatively little published evidence comparing TCAR with therapeutic alternatives. To address this evidence gap, Columbo et al4 used a robust analytic technique—a difference-in-differences regression—to analyze detailed clinical data from the Society of Vascular Surgery’s Vascular Quality Initiative (VQI). Their results show that TCAR adoption by a VQI hospital was associated with an impressive 10% reduction in MACE.4

A typical concern regarding the validity of observational studies is selection bias, but there is mitigation of selection bias in the current study: TCAR-adopting hospitals may have subsequently attracted increased numbers of technically difficult cases that were not suitable for carotid endarterectomy (CEA). Therefore, it seems unlikely that TCAR adoption would have reduced the average risk of a hospital’s carotid procedure case mix. Another strength is the analysis was confined to the approximately 600 hospitals that participated in the VQI, so there is less concern that TCAR-adopting hospitals had better outcomes simply by being more focused on and devoting more resources to vascular surgical quality than non–TCAR-adopting hospitals. By including only VQI hospitals, which provide only 10% of the CEAs performed in the United States, the authors in fact may have underestimated the benefits of TCAR adoption, given that non-VQI hospitals that only provided CEAs may have had even worse carotid procedural outcomes than VQI hospitals that only provided CEAs (ie, the current study’s control group).

The primary threat to this study’s validity is whether TCAR adoption by a hospital was associated with other concurrent quality-of-care initiatives that were the actual effectors of lower MACE rates. For example, it would not be surprising if TCAR-adopting hospitals were on a different trajectory for surgical quality than hospitals that did not adopt TCAR or did so slowly. Because unobserved confounding such as this cannot be excluded, a randomized clinical trial of patients who would be candidates for either CEA or TCAR is likely the only way to dispel any lingering doubts regarding TCAR’s comparative effectiveness. However, the preponderance of existing observational data and evidence from single-group trials certainly supports TCAR’s noninferiority to CEA, and in combination with a 2020 comparative effectiveness observational study using standard multivariable regression,5 the findings by Columbo et al4 are approaching the limit for nonclinical trial data to inform questions regarding TCAR’s safety and effectiveness.

There is more to learn about the potential causal mechanisms by which TCAR adoption may have reduced carotid procedural MACE rates. The simplest explanation is that TCAR may in fact be a safer procedure than CEA. It is also possible that the availability of TCAR resulted in better matching of patients to the more appropriate procedure, with high-risk patients who would have otherwise undergone CEA being treated with TCAR instead. In fact, it seems plausible that hospitals adopting TCAR changed their practice patterns along both the intensive margin (ie, TCAR drew patients at high surgical risk away from receiving CEA) and the extensive margin (ie, TCAR also drew high-risk patients to receiving interventional therapy who would otherwise have been deemed ineligible for any intervention in the pre-TCAR era).

The authors of the current study4 posit that the technological advantages of the innovative procedure, combined with better sorting of patients within hospitals that offered both therapies, are 2 essential ways that innovative procedures may improve health outcomes. Additional considerations are (1) whether the new procedure can be performed with sufficiently high technical quality at each hospital that newly adopts it; (2) whether the sorting of patients between the new and established procedures within hospitals remains firmly grounded in evidence-based medicine and does not become subject to turf battles;6 (3) whether the highest-risk patients who previously would not have been procedural candidates truly benefit from receiving a procedure; and (4) whether the new procedure’s appearance in the market changes the flow of patients between hospitals—ideally (but not necessarily) improving the likelihood that the highest-risk patients receive care at the hospitals that are the most likely to produce good clinical outcomes in such patients.

While the current study appropriately examines outcomes of all carotid procedural patients in the VQI hospitals rather than TCAR recipients only, the findings do not imply that all carotid procedural recipients at TCAR-adopting hospitals had comparable or better outcomes than all carotid procedural recipients at CEA-only hospitals. For example, it is possible that the benefits derived from improved sorting of patients to TCAR or CEA may have masked potential harms from inappropriate recruitment of high-risk cases that ought to have been managed nonsurgically.

An ongoing challenge for cardiovascular surgeons and interventionalists is to ascertain whether new therapeutic options for patients with a severe cardiovascular disease improve clinical outcomes across the full spectrum of patients with the disease. There are no guarantees that new therapies will produce net benefits across broad populations. For example, increased use of percutaneous coronary intervention as an alternative to coronary artery bypass grafting among patients with left main or triple-vessel coronary artery disease may have inappropriately shifted numerous patients away from the better treatment option (ie, bypass grafting).7 Conversely, transcatheter aortic valve replacement has increased the number of patients with severe aortic stenosis who can safely undergo an intervention by providing a viable option for patients with prohibitively high surgical risk for standard aortic valve replacement, thus potentially improving outcomes across the full spectrum of patients with aortic valve disease.

The pace of cardiovascular procedural innovation is constantly accelerating, and rapid innovation undoubtedly will require perpetual comparative effectiveness assessments of new vs established procedural therapies. Critical to these efforts will be the collection of comprehensive data on procedural recipients, the application of sophisticated statistical techniques to analyze these data, and judicious interpretation of the findings. Optimizing the clinical outcomes for cardiovascular procedure recipients will increasingly depend on data-driven clinical choices that leverage this multicenter collective learning process.

Back to top
Article Information

Published: February 22, 2021. doi:10.1001/jamanetworkopen.2021.0328

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Groeneveld PW. JAMA Network Open.

Corresponding Author: Peter W. Groeneveld, MD, MS, Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, 1204 Blockley Hall, 423 Guardian Dr, Philadelphia, PA 19104 (

Conflict of Interest Disclosures: None reported.

Kernan  WN, Ovbiagele  B, Black  HR,  et al; American Heart Association Stroke Council, Council on Cardiovascular and Stroke Nursing, Council on Clinical Cardiology, and Council on Peripheral Vascular Disease.  Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.   Stroke. 2014;45(7):2160-2236. doi:10.1161/STR.0000000000000024PubMedGoogle ScholarCrossref
US Food and Drug Administration. Premarket Approval (PMA) of Xact carotid stent system. Accessed January 4, 2021.
Brott  TG, Hobson  RW  II, Howard  G,  et al; CREST Investigators.  Stenting versus endarterectomy for treatment of carotid-artery stenosis.   N Engl J Med. 2010;363(1):11-23. doi:10.1056/NEJMoa0912321PubMedGoogle ScholarCrossref
Columbo  JA, Martinez-Camblor  P, O'Malley  AJ,  et al.  Association of adoption of transcarotid artery revascularization with center-level perioperative outcomes.   JAMA Netw Open. 2021;4(2):e2037885. doi:10.1001/jamanetworkopen.2020.37885Google Scholar
Dakour-Aridi  H, Ramakrishnan  G, Zarrintan  S, Malas  MB.  Outcomes of transcarotid revascularization with dynamic flow reversal versus carotid endarterectomy in the TCAR Surveillance Project.   Semin Vasc Surg. 2020;33(1-2):24-30. doi:10.1053/j.semvascsurg.2020.10.001PubMedGoogle ScholarCrossref
Veith  FJ.  Turf issues: how do we resolve them and optimize patient selection for intervention and ultimately patient care?   J Vasc Surg. 1998;28(2):370-372. doi:10.1016/S0741-5214(98)70178-8PubMedGoogle ScholarCrossref
Epstein  AJ, Polsky  D, Yang  F, Yang  L, Groeneveld  PW.  Coronary revascularization trends in the United States, 2001-2008.   JAMA. 2011;305(17):1769-1776. doi:10.1001/jama.2011.551PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words