Context The superiority of innovative over standard treatments is not known.
To describe accurately the outcomes of innovations that are tested in randomized
controlled trials (RCTs) 3 factors have to be considered: publication rate,
quality of trials, and the choice of the adequate comparator intervention.
Objective To determine the success rate of innovative treatments by assessing
preferences between experimental and standard treatments according to original
investigators’ conclusions, determining the proportion of RCTs that
achieved primary outcomes’ statistical significance, and performing
meta-analysis to examine if the summary point estimate favored innovative
vs standard treatments.
Data Sources Randomized controlled trials conducted by the Radiation Therapy Oncology
Group (RTOG).
Study Selection All completed phase 3 trials conducted by the RTOG since its creation
in 1968 until 2002. For multiple publications of the same study, we used the
one with the most complete primary outcomes and with the longest follow-up
information.
Data Extraction We used the US National Cancer Institute definition of completed studies
to determine the publication rate. We extracted data related to publication
status, methodological quality, and treatment comparisons. One investigator
extracted the data from all studies and 2 independent investigators extracted
randomly about 50% of the data. Disagreements were resolved by consensus during
a meeting.
Data Synthesis Data on 12 734 patients from 57 trials were evaluated. The publication
rate was 95%. The quality of trials was high. We found no evidence of inappropriateness
of the choice of comparator. Although the investigators judged that standard
treatments were preferred in 71% of the comparisons, when data were meta-analyzed
innovations were as likely as standard treatments to be successful (odds ratio
for survival, 1.01; 99% confidence interval, 0.96-1.07; P = .5). In contrast, treatment-related mortality was worse
with innovations (odds ratio, 1.76; 99% confidence interval, 1.01-3.07; P = .008). We found no predictable pattern of
treatment successes in oncology: sometimes innovative treatments are better
than the standard ones and vice versa; in most cases there were no substantive
differences between experimental and conventional treatments.
Conclusion The finding that the results in individual trials cannot be predicted
in advance indicates that the system and rationale for RCTs is well preserved
and that successful interventions can only be identified after an RCT is completed.