Ridker PM, Torres J. Reported Outcomes in Major Cardiovascular Clinical Trials Funded by For-Profit and Not-for-Profit Organizations: 2000-2005. JAMA. 2006;295(19):2270-2274. doi:10.1001/jama.295.19.2270
Author Affiliations: Center for Cardiovascular Disease Prevention and the Divisions of Preventive Medicine and Cardiology, Brigham and Women's Hospital (Dr Ridker), and the Harvard Medical School (Dr Ridker and Mr Torres), Boston, Mass.
Context In surveys based on data available prior to 2000, clinical trials funded by for-profit organizations appeared more likely to report positive findings than those funded by not-for-profit organizations. Whether this situation has changed over the past 5 years or whether similar effects are present among jointly funded trials is unknown.
Objective To determine in contemporary randomized cardiovascular trials the association between funding source and the likelihood of reporting positive findings.
Design We reviewed 324 consecutive superiority trials of cardiovascular medicine published between January 1, 2000, and July 30, 2005, in JAMA, The Lancet, and the New England Journal of Medicine.
Main Outcome Measure The proportion of trials favoring newer treatments over the standard of care was evaluated by funding source.
Results Of the 324 superiority trials, 21 cited no funding source. Of the 104 trials funded solely by not-for-profit organizations, 51 (49%) reported evidence significantly favoring newer treatments over the standard of care, whereas 53 (51%) did not (P = .80). By contrast, 92 (67.2%) of 137 trials funded solely by for-profit organizations favored newer treatments over standard of care (P<.001). Among 62 jointly funded trials, 35 (56.5%), an intermediate proportion, favored newer treatments. For 205 randomized trials evaluating drugs, the proportions favoring newer treatments were 39.5%, not-for-profit; 54.4%, jointly funded; and 65.5%, for-profit trials (P for trend across groups = .002). For the 39 randomized trials evaluating cardiovascular devices, the proportions favoring newer treatments were 50.0%, not-for-profit; 69.2%, jointly funded; and 82.4%, for-profit trials (P for trend across groups = .07). Regardless of funding source, trials using surrogate end points, such as quantitative angiography, intravascular ultrasound, plasma biomarkers, and functional measures were more likely to report positive findings (67%) than trials using clinical end points (54.1%; P = .02).
Conclusions Recent cardiovascular trials funded by for-profit organizations are more likely to report positive findings than trials funded by not-for-profit organizations, as are trials using surrogate rather than clinical end points. Trials jointly funded by not-for-profit and for-profit organizations appear to report positive findings at a rate approximately midway between rates observed in trials supported solely by one or the other of these entities.
Surveys of randomized trials published between 1990 and 2000 raised awareness in the medical community that trials funded by for-profit organizations were more likely to report positive findings than those funded by not-for-profit organizations.1- 8 As a group, these surveys raised questions regarding the design and conduct of industry-funded clinical trials9- 11 as well as ethical concerns about potential violations of clinical equipoise.12- 15 Partly in response to these concerns, recommendations have been made to improve academic oversight in industry-sponsored research16,17 and to ensure registration and publication of all clinical trials.18,19
Whether this recognition has affected contemporary clinical trials is unknown. To address this question, we surveyed reported outcomes in high-impact cardiovascular clinical trials published between 2000 and 2005 in JAMA, TheLancet, and the New England Journal of Medicine, stratifying our analysis according to whether the trial was funded by for-profit or not-for-profit organizations. By focusing on manuscripts published in these 3 journals, we were able to limit our analysis to trials considered to be of high quality on the basis of prior rigorous peer and editorial review. We focused on trials of cardiovascular medicine because they are common, represent an area of considerable public health importance, and are almost equally funded by for-profit and not-for-profit entities. As a substantial number of cardiovascular clinical trials are funded jointly by for-profit and not-for-profit organizations, we were afforded the additional opportunity to address outcomes in the setting of these cost-sharing clinical trials. We also sought to evaluate reported outcomes among trials that used surrogate vs clinical end points.
We evaluated a consecutive series of 349 randomized trials related to cardiovascular medicine that were published between January 1, 2000, and July 30, 2005, in 3 prominent medical journals, JAMA, TheLancet, and the New England Journal of Medicine. Trials were initially identified using the search term cardiovascular diseases in PubMed/National Library of Medicine, limiting the search to human clinical trials. Trials were included in this analysis if they represented the primary report of the trial itself or were a major secondary report based on additional end points that could be critically evaluated on the basis of the randomization process. Reports were not included if they used the trial apparatus primarily as a prospective cohort or if the reported analyses were not fully dependent on the randomized interventions.
Following a methodology analogous to that used in prior investigations,5- 7 one of us (P.MR.) reviewed each clinical trial unaware of the funding source and scored the reported primary study end point on a 7-point ordinal scale designed to address both statistical significance and magnitude of effect: 1 indicated that the standard of care (SOC) was significantly better and highly preferred; 2, the SOC was significantly better and moderately preferred; 3, the SOC was nonsignificantly better; 4, the SOC and newer treatment were equivalent; 5, the newer treatment was nonsignificantly better; 6, the newer treatment was significantly better and moderately preferred; and 7, the newer treatment was significantly better and highly preferred. Trials were further evaluated according to study design (superiority trials vs noninferiority, equivalence, and safety trials), sample size, number of sites (single center vs multicenter), use of clinical or surrogate end points and whether the treatment of interest was a drug, a device, a patient-based behavioral intervention, or a novel medical or surgical procedure.
Simultaneous with this process, one of us (J.T.) reviewed each publication unaware of trial outcome and sought information regarding funding sources, categorizing trials into 1 of 4 groups: those financed exclusively by federal, state, or other not-for-profit foundations (not-for-profit trials); those financed exclusively by for-profit pharmaceutical or device manufacturers (for-profit trials); those financed jointly (not-for-profit and for-profit); and those for which no source of funding was provided. For the purpose of these analyses, trials financed in their entirety by federal, state, or foundation support for which drug or devices were donated without charge were considered not-for-profit trials.
To limit the potential for bias and subjectivity in the scoring of trial outcomes, we compared funding sources after regrouping all trials into 1 of 2 categories, those for which there was clear evidence of a statistical benefit in favor of the newer treatment (scores of 6 or 7) and those for which no statistical evidence of benefit was observed (scores of 1-5). We used the binomial distribution to evaluate whether the number of positive trials observed within a given funding group deviated significantly from a probability of 50%. Tests for trend were used to compare the proportion of trials significantly favoring newer treatments across funding groups (not-for-profit trials, jointly funded trials, and for-profit trials). On an a priori basis, we performed prespecified subgroup analyses among trials using clinical rather than surrogate end points and on the basis of whether the treatment was a drug or device. All analyses were performed using STATA release 8.0 (StataCorp, College Station, Tex). P<.05 was considered statistically significant; all P values were 2 sided.
Of the 349 trials evaluated, 109 (31%) were financed exclusively by not-for-profit organizations, 153 (44%) were financed exclusively by either for-profit pharmaceutical or device manufacturers, 66 (19%) were funded jointly by for-profit and not-for-profit organizations, and 21 (6%) noted no source of funding. Trials that did not disclose a funding source almost exclusively evaluated behavioral, procedural, dietary, or generic interventions. Three hundred twenty-four trials (93%) were designed to address superiority between the therapies under consideration and form the basis for our primary analyses, whereas 15 trials (4%) were designed to address equivalence or noninferiority and 10 trials (3%) were designed to address safety.
Characteristics of the 324 superiority trials evaluated in the primary analyses are shown in Table 1 according to funding source. Studies funded by for-profit organizations tended to have larger median sample size, were less likely to use surrogate end points, and more likely to be multicentered than were studies funded by not-for-profit organizations.
Overall, 190 (58.6%) of the 324 superiority trials reported evidence significantly favoring newer treatments, while 112 (34.6%) reported no significant difference between therapies, and 22 (6.8%) reported evidence significantly favoring SOC. These proportions, however, varied in analyses stratified by funding source.
Among not-for-profit trials, 51 (49%) of 104 reported evidence significantly favoring newer treatments, whereas 53 (51%) were either null or significantly favored SOC (P = .80). By contrast, among for-profit trials, 92 (67.2%) of 137 reported evidence significantly favoring newer treatments with 45 (32.8%) reporting null data or data favoring SOC (P<.001). The proportion of trials significantly favoring new treatments for studies jointly funded by for-profit and not-for-profit organizations was approximately midway between these 2 values (56.5%). In comparisons across groups, the difference in the proportions of positive findings among not-for-profit trials, jointly funded trials (not-for-profit and for-profit), and for-profit trials was statistically significant (Table 2; P = .005).
For randomized trials evaluating drugs, the proportions favoring newer agents were 39.5% for not-for-profit, 54.4% for jointly sponsored, and 65.5%for for-profit trials (P for trend across groups = .002). The greatest discrepancy between not-for-profit and for-profit trials in terms of the proportion significantly favoring newer therapies over SOC was observed among device trials. In this setting, 4 (50.0%) of 8 device trials funded by not-for-profit entities were positive compared with 9 (69.2%) of 13 funded jointly by not-for-profit and for-profit entities, and 14 (82.4%) of 17 trials funded by for-profit entities (P for trend across groups = .07).
Regardless of funding source, clinical trials using surrogate end points such as intravascular ultrasound, quantitative angiography, plasma biomarkers, and functional measures were more likely to report positive findings (67.0%) than were clinical trials using clinical end points (54.1%; P = .02). Of interest, and as shown in Table 1, the use of surrogate end points was more prevalent among not-for-profit trials than among for-profit trials. To avoid potential confounding on this basis, we also analyzed separately only those trials using hard clinical end points. As also shown in Table 2, among such hard end-point trials, the proportions favoring newer treatments were 34.6% for not-for-profit, 54.6% for jointly funded, and 66.7% for for-profit trials (P for trend across groups <.001).
Of the original 349 trials evaluated, 15 were specifically designed as noninferiority trials of which 13 (86.6%) resulted in an outcome consistent with noninferiority. All but 3 of these trials were funded in their entirety by for-profit organizations.
One of us (P.MR.) was the primary author or coauthor on 4 of the trials included in the current evaluation. Two of these favored newer treatments, 1 funded by a pharmaceutical company using a surrogate end point20 and 1 funded by federal sources using a clinical end point.21 Two others, funded by federal sources using clinical end points, were null.22,23
In this survey of 324 consecutive cardiovascular clinical trials published in JAMA, TheLancet, and the New England Journal of Medicine between 2000 and 2005, we found that those funded by for-profit organizations were significantly more likely to report positive findings than those funded by not-for-profit organizations. Furthermore, we observed that trials jointly sponsored by for-profit and not-for-profit organizations reported positive findings at a rate approximately midway between rates observed in trials supported solely by 1 or the other of these entities.
As suggested in surveys of randomized trials published prior to 2000,1- 8 these contemporary data appear to show that incentives surrounding for-profit organizations have the potential to influence clinical trial outcomes. Previous attempts to explain this phenomenon have focused largely on design bias, interpretation bias, data suppression, and differential data quality.5- 7,10,11,24
However, with regard to trial design bias, the direction of effect is not always predictable. Among studies reviewed herein, at least 2 major for-profit trials25,26 reported significant evidence of superiority favoring a competing manufacturer's innovation. For example, among 4809 patients randomized in a noninferiority trial comparing tirofiban to abciximab in the setting of percutaneous coronary intervention, abciximab was found to be statistically superior despite the trial being funded by the manufacturer of tirofiban.25 Similarly, among 4162 patients with acute coronary syndrome randomized in a noninferiority trial comparing pravastatin to atorvastatin, atorvastatin was found to be statistically superior despite the trial being funded by the manufacturer of pravastatin.26
With regard to author interpretation bias and limits on publication, it is possible that null or even adverse results from for-profit trials may be marginalized or suppressed in the clinical literature.10,11 Indeed, suspicion of adverse cardiovascular event suppression has very recently been raised in a major randomized trial comparing rofecoxib to naproxen among patients with rheumatoid arthritis.27 If pervasive, such effects would be highly problematic for the interpretation of published trials. Although our study design cannot address this issue directly, we believe recently adopted requirements for study registration is an important first step toward full publication of all clinical trial results, regardless of outcome.18
We think it unlikely that systematic differences between trials in terms of quality would have affected our results. By limiting our analysis to manuscripts published in JAMA, The Lancet, and the New England Journal of Medicine, all of the trials reviewed herein can be considered “high quality” based on journal acceptance after a rigorous peer review and editorial review process. It is also uncertain as to what direction any such bias might actually make in terms of reported study outcomes; in our survey of cardiovascular trials published between 2000 and 2005, those funded by for-profit organizations tended to have greater sample size, were more likely to be multicentered, and less likely to use surrogate end points, all characteristics typically associated with high quality.
Beyond these traditional concerns, we believe there are additional issues that help to explain, in part, the observed results. For example, when the first trial report of a truly novel therapy is null or negative, it becomes less likely that any funding source will support subsequent studies. On the other hand, when the first trial of a truly novel therapy is positive, the likelihood of further trials is increased. These subsequent trials understandably and perhaps appropriately are more likely to be funded by for-profit organizations.
Furthermore, although both for-profit and not-for-profit organizations are interested in the development of truly novel therapies, for-profit organizations are also commonly interested in evaluating proven therapeutic agents or devices in previously understudied patient populations (such as the elderly or in women) or in settings with different entry criterion than specified in previous trials (for example a lower baseline lipid value, ejection fraction, or arrhythmia hazard).
Although the probability of success for such trials is likely to be greater on an a priori basis compared with trials evaluating entirely novel therapies, such trials can be nevertheless useful for the broader application of medical innovations to clinical practice. By way of example, although the 4S trial of simvastatin in secondary prevention established the clinical efficacy of 3-hydroxy-3methylglutaryl coenzyme A reductase inhibitors as agents for the treatment of cardiovascular disease in high-risk secondary prevention,28 findings from the subsequent CARE,29 LIPID,30 WOSCOPS,31 AFCAPS/TexCAPS,32 HPS,33 ASCOT,34 PROVE-IT,26 and TNT35 trials extended data for statins into other patient groups.
Similarly, accrual of evidence from multiple trials involving antiplatelet and anticoagulant agents in acute ischemia, angiotensin-converting enzyme inhibitors and angiotensin-receptor blockers in congestive heart failure, and drug-eluting stents in percutaneous coronary revascularization was required for acceptance of each of these innovations. Moreover, replication is a typical requirement for US Food and Drug Administration approval and for the development of formal evidence-based treatment guidelines. Thus, part of the observed results may reflect the replication that is required in contemporary practice, at least with regard to phase 3 clinical trials. That being said, we also observed in our data instances in which a nonnovel therapeutic agent already in common use did not prove to be effective in rigorous evaluation. Most, but not all, of these latter trials were funded by not-for-profit organizations.
Prior work has suggested that for-profit trials directed and coordinated by recognized academic coordinating centers tend to report positive findings somewhat less often than those for which academic coordination is less evident.7- 10 Although not formally evaluated, we noted similar effects in our survey. However, we also observed that despite the relative independence, trials coordinated by recognized academic centers may nevertheless be under pressure to present results in the best possible light, in particular emphasizing favorable subgroup analyses when an overall neutral effect was observed for the primary end point. Two such examples of this effect have been reported in the very recent cardiovascular literature suggesting that this is an ongoing issue for investigators.36,37 We also noted that academic direction and oversight of for-profit trials does not always guarantee that the best possible study design available is used. As an example of this issue, a recent academically directed for-profit trial evaluating high-dose statin therapy as a method to achieve atherosclerotic regression was undertaken without an active control group.38
Limitations of our analysis should be considered. First, we limited our evaluation to contemporary cardiovascular trials and thus these data may not generalize to other fields of medical research. We note, however, that evaluations of clinical trials largely published prior to 2000 in several noncardiovascular fields1- 9 also suggest that industry-funded research is more likely than not to report favorable results.
Second, because we only evaluated trials published in 3 prominent clinical journals, it is possible that publication bias could account for part of our observations. We believe this unlikely for several reasons. First, in prior studies of industry influence that addressed all publications in a given field, almost identical findings have been observed.1- 4,7,8 Second, of phase 3 cardiovascular clinical trial reports reviewed as a pilot for this project, those that did not appear in 1 of the 3 journals reviewed herein were very often secondary reports that would not have qualified for study inclusion.
Third, contrary to the often-voiced concern that major journals do not report null studies, we found that a substantial proportion of the cardiovascular trials published in JAMA, TheLancet, and the New England Journal of Medicine between 2000 and 2005 reported either no significant difference between therapies (34.6%) or a significant difference favoring SOC over newer treatments (6.8%). Furthermore, among trials funded solely by not-for-profit organizations, the proportion of trials not favoring innovation was 51.0% suggesting that, at least for these trials, evidence of publication bias is minimal.
It must also be recognized that neutral trials, regardless of funding source, often provide important clinical information. For example, the not-for-profit Women's Health Initiative39 and the for-profit Heart and Estrogen/progestin Replacement Study (HERS)40 both found minimal evidence that hormone use after menopause reduces cardiovascular risk, data with implications for the care of millions of middle-aged women.
We additionally observed that trials jointly funded by not-for-profit and for-profit organizations reported findings favoring new treatments at a rate approximately midway between rates observed in trials supported solely by one or the other of these entities. We are aware of data from 1 study of noncardiovascular trials that also found a similar result.7 If confirmed in other areas of investigation, this observation could have implications for the design and conduct of jointly funded projects for which the primary intent is to share costs of pivotal trials with industry partners.9,16,17
Finally, we believe our observations of differential rates of positive trial reporting on the basis of end-point selection strongly reinforces the need for physician decision making and Food and Drug Administration approval to remain on the basis of clinical rather than surrogate end points.
Corresponding Author: Paul M Ridker, MD, Center for Cardiovascular Disease Prevention, Brigham and Women's Hospital, 900 Commonwealth Ave E, Boston, MA 02215 (firstname.lastname@example.org).
Author Contributions: Dr Ridker had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Ridker, Torres.
Acquisition of data: Ridker, Torres.
Analysis and interpretation of data: Ridker, Torres.
Drafting of the manuscript: Ridker.
Critical revision of the manuscript for important intellectual content: Ridker, Torres.
Statistical analysis: Ridker.
Study supervision: Ridker
Financial Disclosures: None reported.