The Use of Not-Negative Conclusions to Describe Results of Formally Negative Trials Presented at Oncology Meetings

Presentations at medical meetings have great resonance within the scientific community, especially following the diffusion of social media. Oral presentations are not subject to peer review, and some authors’ conclusions may not be completely justified by the results. This is particularly critical when, despite the formally negative trial result, the authors’ conclusions are not-negative. The aim of this quality improvement analysis was to describe the frequency and type of not-negative conclusions used by presenters to discuss the results of formally negative trials at recent oncology meetings.

We reviewed oral presentations of phase 3 randomized clinical trials (excluding noninferiority trials) at American Society of Clinical Oncology and European Society for Medical Oncology congresses from 2017 through 2019. We classified trials as positive (not-negative) vs negative, in terms of formal results (rejection of null hypothesis for the primary end point) and in terms of conclusions (based on the sentences used in the Conclusions section of the oral presentation). We considered the conclusions to be not-negative when, more or less explicitly, authors consider the possibility of using the experimental treatment in that setting, without clear conclusions about the study negativity. This evaluation was not blinded to the positivity or negativity of formal results. The study did not require institutional review board or ethical committee review because it did not involve human subjects.

We classified formally negative trials in different categories according to the reasons of not-negative conclusions: (1) numerically better outcome in the experimental arm, despite a nonsignificant P value, (2) emphasis on positive subgroup(s), (3) emphasis on positive secondary end point(s), and (4) noninferiority interpretation of a negative superiority trial (eTable in the Supplement).

Overall, 208 randomized clinical trials were selected. Of the 91 formally negative studies, 26 (29%) had a not-negative conclusion. The proportion of negative studies with not-negative conclusions was 22%, 13%, and 47% in 2017, 2018, and 2019, respectively; 17 of 57 nonprofit studies (30%) and 9 of 34 for-profit studies (26%) have not-negative conclusions (Figure).

Within the 26 studies with a negative primary analysis, authors emphasized a numerically better outcome in the experimental arm in 13 cases (50%), the positive result in 1 or more subgroups in 12 cases (46%), and the positive result in 1 or more secondary end points in 10 cases (38%). In 7 cases (27%), authors interpreted post hoc the study as a noninferiority design.

Each of the reasons used to underline positive aspects of a formally negative result should be presented with caution, avoiding diffusion of methodologically equivocal statements. This is true for published papers,¹ and we show that it is relevant also for meeting presentations.

Many trials had a not-negative message despite a statistically nonsignificant primary analysis. We show that the risk of wrongly emphasizing borderline significance, already described in the oncology literature,² is present also in meeting presentations.

Positive subgroup analysis of negative trials can be misleading, should not support treatment adoption, and, at best, should be hypothesis generating.^3,4 In some cases, however, such as the STAMPEDE trial testing prostate radiotherapy in patients with advanced prostate cancer,⁵ methodological strengths and weaknesses of the subgroup analysis were correctly discussed by the study presenter.

Positive analyses of secondary end points in trials with a negative primary analysis carry the same risk of false-positive results inherent in multiple testing. Furthermore, the primary end point is the measure used for the study hypothesis and should condition study interpretation.

When a trial is designed to test the superiority of an experimental treatment, post hoc interpretation of noninferiority is methodologically debatable. The noninferiority hypothesis should be prospectively planned, with a clear definition of the margin acceptable to define noninferiority.

In conclusion, we believe that more attention should be paid to the statements included in the conclusions of oral presentations at meetings, and the discussants’ role is crucial. When the primary end point is not met, the word negative should be explicitly used.

Accepted for Publication: February 13, 2020.

Corresponding Author: Massimo Di Maio, MD, Division of Medical Oncology, Ordine Mauriziano Hospital, Via Magellano 1, Turin 10128, Italy (massimo.dimaio@unito.it).

Published Online: April 16, 2020. doi:10.1001/jamaoncol.2020.0475

Author Contributions: Dr Di Maio had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Di Maio, Perrone.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Di Maio.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Di Maio.

Administrative, technical, or material support: Audisio, Cardone, De Luca, Gargiulo.

Study supervision: Di Maio, Perrone.

Conflict of Interest Disclosures: Dr Di Maio reported receiving personal fees from AstraZeneca, Eli Lilly and Company, Bristol-Myers Squibb, Merck Sharp & Dohme, Eisai, Takeda, Pfizer, and Janssen. Dr Perrone reported receiving personal fees from Bayer, Ipsen, AstraZeneca, Bristol-Myers Squibb, Sandoz, Incyte, Celgene, Pierre Fabre, and Janssen-Cilag. No other disclosures were reported.