In the z curve (represented by grey dashed line), individual trials correspond to individual segments; trials are plotted in chronological order (from left to right). The x-axis indicates the cumulative number of patients; the starting point of the z curve is always at x = 0, ie, inclusion of no trials. At the cumulative number of 9343 included patients, the curve has already crossed the green boundaries and is in the futility area. The analysis estimates that at least 9775 patients would be the optimal information size to reach a conclusion in terms of superiority or inferiority or futility. The blue lines indicate the boundaries for superiority or inferiority; and green lines, the boundaries for futility. T indicates treatment group (receiving ω-3 fatty acid supplements); C, controls.
Messori A, Fadda V, Maratea D, Trippoli S. ω-3 Fatty Acid Supplements for Secondary Prevention of Cardiovascular Disease: From “No Proof of Effectiveness” to “Proof of No Effectiveness”. JAMA Intern Med. 2013;173(15):1466-1468. doi:10.1001/jamainternmed.2013.6638
In patients who have experienced cardiovascular events, ω-3 fatty acid supplements do not seem to be beneficial.1 However, there is not universal agreement on this conclusion.1- 3 On the one hand, after examining the data of 14 randomized placebo-controlled studies, the meta-analysis by Kwak et al1 found no reduction in cardiovascular events (risk ratio, 0.99; 95% CI, 0.89-1.09) as well as no improvement in other relevant end points. On the other hand, the aforementioned meta-analysis has been criticized because 2 positive randomized studies4,5 were excluded owing to their open-label design and no administration of placebo; furthermore, a query of clinicaltrials.gov (run on March 5, 2013) indicates that 8 trials, registered on this website, are presently under way, thus confirming that the effectiveness of ω-3 fatty acid supplements is still thought to be an open question.
The statistical question addressed by the meta-analysis by Kwak et al1 was whether the 14 randomized trials evaluating ω-3 fatty acid supplements showed superiority vs placebo based on standard clinical end points, and the conclusion was no proof of effectiveness (ie, no proof of superiority).
Trial-sequential analysis (TSA) is a novel statistical technique6- 8 that tries to better interpret negative results and, when appropriate, aims to provide a more informative conclusion in terms of proof of no effectiveness (otherwise denoted as futility or demonstration of both nonsuperiority and noninferiority). According to TSA, each meta-analysis can in fact be classified into only 1 of 4 categories (superiority, inferiority, futility, or inconclusive result). It should also be recalled that TSA adopts more conservative thresholds than standard meta-analysis for demonstrating superiority or inferiority.
We applied TSA to re-examine the 14 randomized studies evaluated by Kwak et al.1 Our analysis considered the end point of overall cardiovascular events. Our main assumptions included 2-sided testing, risk of type 1 error (5%), and power (80%). For cardiovascular events, the intervention effect was set at an event frequency of 12.2% for the control group (equal to the meta-analytical event frequency of the 14 control groups) and a relative risk reduction of 10%. As usual, the main result of TSA was expressed through the graph of cumulative z curve. With reference to this graph, the boundaries for concluding superiority or inferiority or futility were calculated according to the O’Brien-Fleming α-spending function. Our analysis used a specific statistical software (User Manual for TSA, Copenhagen Trial Unit 2011, software downloadable at http://www.ctu.dk/tsa). A secondary TSA was also carried out in which we included the 2 open-label randomized studies excluded by Kwak et al.1
The Figure shows the results of our analysis. The analysis of the 14 randomized placebo-controlled studies indicated futility, ie, proof of no effectiveness; interestingly enough, the final part of the curve was far beyond the boundary of futility. Our secondary analysis, which was based on 16 studies, gave essentially the same result (data not shown).
The results of our analysis can be seen as the proof of no effectiveness of ω-3 fatty acid supplements for secondary prevention of cardiovascular disease. Our evaluation is also a good example of the role that TSA can play to integrate the results of traditional meta-analyses.
Corresponding Author: Andrea Messori, PharmD, HTA Unit, Area Vasta Centro Toscana, Regional Health System, Via Guimaraes 9-11, 59100 Prato, Italy (email@example.com or firstname.lastname@example.org).
Published Online: June 17, 2013. doi:10.1001/jamainternmed.2013.6638.
Author Contributions: Dr Messori is independent of any commercial funder and had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Messori.
Acquisition of data: All authors.
Analysis and interpretation of data: All authors.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: Fadda, Maratea, Trippoli.
Statistical analysis: All authors.
Conflict of Interest Disclosures: None reported.