The continued vigilance in safety monitoring in randomized clinical trials (RCTs) is critical as more data and experience are accumulated.1 Emerging safety profiles of therapeutic interventions during longer follow-up may cast doubt on earlier conclusions about benefit-risk assessment.1,2 Along this line, cumulative meta-analysis has been proposed as a tool to evaluate evidence aggregation. We retrospectively assessed how cumulative meta-analysis could serve as a safety monitoring tool to identify the time point when firm evidence for safety concerns of a rare outcome becomes available.
For this meta-analysis, we assessed the withdrawn polymeric everolimus-eluting coronary bioresorbable vascular scaffold (BVS) (Absorb; Abbott Vascular). The BVS received CE mark approval in January 2011 and US Food and Drug Administration approval in July 2016. In September 2017, the manufacturer voluntarily withdrew the device owing to safety concerns (increased risk of scaffold-related thrombosis) after it had been available for clinical use for more than 6 years in Europe and 1 year in the US. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.
We retrieved all available reports of RCTs comparing the BVS with metallic everolimus-eluting stents for percutaneous coronary interventions by searching PubMed, CENTRAL, and websites of major cardiology meetings occurring before May 31, 2019. Device-related (scaffold or stent) definite or probable thrombosis was the safety outcome of interest. We used Mantel-Haenszel (fixed-effects model) cumulative meta-analysis to summarize accumulated rare events over time and computed odds ratios (ORs) at each time point. All P values are 2-sided, and P < .05 was considered statistically significant. Analyses were performed using R, version 3.3.2 (The R Foundation for Statistical Computing).
A total of 22 reports describing 8 RCTs including 8180 patients randomized to BVS (4553 patients) or everolimus-eluting stents (3627 patients) were included, with 96 and 20 device-related thromboses for each intervention, respectively. Patient recruitment took place over 6 years, with considerable overlap of recruitment periods (Figure 1). The cumulative meta-analysis (Figure 2) revealed that the initial uncertainty regarding the treatment effect based on early trials with follow-up to 1 year gained precision through inclusion of additional trials and follow-up time. The analysis of accumulated evidence showed initial safety concerns after the publication of ABSORB III trial on October 12, 2015, for a clinically important but non–statistically significant increase in the risk of device-related thrombosis after use of BVS (OR, 2.22; 95% CI, 0.97-5.06, P = .06). The between-group difference became statistically significant on September 18, 2016 (OR, 2.52; 95% CI, 1.12-5.71; P = .03). Availability of longer follow-up and new trials resulted in an OR of 2.87 (95% CI, 1.34-6.16; P = .007) 11 months before the Absorb BVS was withdrawn in September 2017. Between-group differences reached on March 18, 2017, had an OR of 3.15 (95% CI, 1.48-6.72; P = .003) with a lower limit of the 95% CI above 1.00 (Figure 2). The final estimate was an OR of BVS-related thrombosis of 3.68 (95% CI, 2.25-6.00; P < .001), indicating that the experimental intervention was harmful.
Timely recognition of safety signals is important to patients, physicians, regulators, and the medical community at large to avoid unnecessary, clinically important adverse events and to prevent waste of research efforts, especially in studies of the comparative effectiveness of medical devices. In the absence of large clinical trials, some adverse events may not be known a priori when a new device is used and additional mechanisms, such as regulatory oversight for unexpected events, may be needed; continuously updated cumulative meta-analyses may contribute to this purpose. Of note, although cumulative statistical testing can bias this approach, it is not of particular concern in the present analysis because it was performed retrospectively and was not associated with a stopping rule for the meta-analysis. However, in a prospectively designed cumulative meta-analysis, correction for multiple testing should be considered because the examination of multiple outcomes and repeated analysis of the data over time may exacerbate the risks associated with multiplicity and further adjustments may be warranted.3,4 Under these scenarios, false-positive rates for significance tests at the conventional P < .05 are typically too high, and naive interpretations of statistical significance should be avoided.5,6
Accepted for Publication: June 1, 2020.
Published: September 4, 2020. doi:10.1001/jamanetworkopen.2020.15031
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Siontis GCM et al. JAMA Network Open.
Corresponding Author: George C. M. Siontis, MD, PhD, Department of Cardiology, Bern University Hospital, University of Bern, 3010 Bern, Switzerland (georgios.siontis@insel.ch).
Author Contributions: Dr Siontis had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Siontis, Nikolakopoulou, Räber, Jüni.
Acquisition, analysis, or interpretation of data: Siontis, Nikolakopoulou, Efthimiou, Windecker, Jüni.
Drafting of the manuscript: Siontis, Efthimiou, Jüni.
Critical revision of the manuscript for important intellectual content: Nikolakopoulou, Efthimiou, Räber, Windecker, Jüni.
Statistical analysis: Siontis, Nikolakopoulou, Efthimiou, Jüni.
Administrative, technical, or material support: Siontis.
Supervision: Räber, Windecker.
Conflict of Interest Disclosures: Dr Siontis reported receiving honoraria from Abbott outside the submitted work. Dr Efthimiou reported receiving grant support from the Swiss National Science Foundation. Dr Räber reported receiving research grants to the institution from Abbott, Biotronik, Boston Scientific, Heartflow, Sanofi, and Regeneron; receiving speaker honoraria from Abbott, AstraZeneca, Amgen, CSL Behring, and Sanofi outside the submitted work; receiving personal fees from Abbott during the conduct of the study; receiving personal fees from Amgen, AstraZeneca, Canon, Occlutech, Sanofi, and Vifor; and receiving grants from Boston Scientific, Biotronik, Heartflow, and Regeneron outside the submitted work. Dr Windecker reported receiving research and educational grants from Abbott, Amgen, Boston Scientific, Biotronik, Bayer, BMS, Cardinal Health, CSL Behring, Edwards Lifesciences, Medtronic, Polares, Sanofi, and Sinomed outside the submitted work. Dr Jüni reported receiving research grants to the institution from AstraZeneca, Biotronik, Biosensors International, Eli Lilly, and The Medicines Company; serving as an unpaid member of the steering group of trials funded by AstraZeneca, Biotronik, Biosensors, St. Jude Medical, and The Medicines Company outside the submitted work; and participating on advisory boards and/or consulting for Amgen, Ava, and Fresenius. No other disclosures were reported.