Context Previous studies indicate that industry-sponsored trials tend to draw
proindustry conclusions.
Objective To explore whether the association between funding and conclusions in
randomized drug trials reflects treatment effects or adverse events.
Design Observational study of 370 randomized drug trials included in meta-analyses
from Cochrane reviews selected from the Cochrane Library, May 2001. From a
random sample of 167 Cochrane reviews, 25 contained eligible meta-analyses
(assessed a binary outcome; pooled at least 5 full-paper trials of which at
least 1 reported adequate and 1 reported inadequate allocation concealment).
The primary binary outcome from each meta-analysis was considered the primary
outcome for all trials included in each meta-analysis. The association between
funding and conclusions was analyzed by logistic regression with adjustment
for treatment effect, adverse events, and additional confounding factors (methodological
quality, control intervention, sample size, publication year, and place of
publication).
Main Outcome Measure Conclusions in trials, classified into whether the experimental drug
was recommended as the treatment of choice or not.
Results The experimental drug was recommended as treatment of choice in 16%
of trials funded by nonprofit organizations, 30% of trials not reporting funding,
35% of trials funded by both nonprofit and for-profit organizations, and 51%
of trials funded by for-profit organizations (P<.001; χ2 test). Logistic regression analyses indicated that funding, treatment
effect, and double blinding were the only significant predictors of conclusions.
Adjusted analyses showed that trials funded by for-profit organizations were
significantly more likely to recommend the experimental drug as treatment
of choice (odds ratio, 5.3; 95% confidence interval, 2.0-14.4) compared with
trials funded by nonprofit organizations. This association did not appear
to reflect treatment effect or adverse events.
Conclusions Conclusions in trials funded by for-profit organizations may be more
positive due to biased interpretation of trial results. Readers should carefully
evaluate whether conclusions in randomized trials are supported by data.
Empirical evidence shows that conclusions in randomized trials are more
positive toward experimental interventions if funded by for-profit organizations.1-7 Three
studies found this association in randomized trials published in high-impact
journals.1-3 Two
studies reached similar results in randomized trials on arthritis4 and myeloma.5 Two recent
systematic reviews6,7 highlight
the external validity of these findings. It is not known whether this association
reflects the quantitative trial results.8 More
positive conclusions in trials funded by for-profit organizations could reflect
either more beneficial treatment effects or less frequent occurrence of adverse
events. None of the previous studies1-7 assessed
these aspects. Furthermore, previous studies included relatively heterogeneous
trial cohorts. This case mix could confound the findings. It is possible that
the association simply reflects that trials funded by for-profit organizations
assess the most effective interventions.
The influence of methodological quality, type of control intervention,
sample size, and disease area also could be important. These variables were
assessed only in 1 study but did not appear to explain the association between
funding and conclusions.3 Moreover, financial
interests may influence the decision to submit trials with positive results
to high-impact journals.
We assessed whether an association between funding and conclusions in
randomized drug trials reflects the magnitude of the treatment effect or occurrence
of adverse events. Secondary objectives were to explore the impact of methodological
quality, type of control intervention, size of the trial, year of publication,
or publication in high-impact journals on this association.
We selected all randomized trials included in eligible meta-analyses
from a random sample of Cochrane reviews obtained in May 2001 and extracted
data on conclusions, funding, treatment effect, adverse events, and additional
confounding factors. The primary outcome for each trial was selected according
to the meta-analysis including the trial. We analyzed whether an association
between funding and conclusions reflected treatment effects or adverse events.
Selection of Meta-analyses
We selected meta-analyses of randomized trials because within each meta-analysis,
trials assessed comparable treatments for specific diseases. This allowed
us to adjust for disease areas and type of experimental and control intervention.
Meta-analyses of binary outcomes were selected to adjust our analyses for
the treatment effect estimated by an odds ratio (OR). To reduce the number
of empty cells in our analyses, the meta-analyses had to include at least
5 full-paper trials. To examine the impact of trial quality, the meta-analyses
had to include at least 1 trial with and 1 trial without adequate allocation
concealment.9-11 Our
inclusion criteria were adopted from previous methodological studies.9,11,12 We included all full-paper
trials that were contained in eligible meta-analyses.
Based on a pilot study of 100 systematic reviews, we estimated that
about 25% of all reviews published in the Cochrane Library would contain an
eligible meta-analysis. We also estimated that we had to include about 40
reviews to obtain a sample of about 500 trials. Previous studies1-3,5 have
indicated that this number would provide an acceptable risk of type II error.
The number of systematic reviews in the Cochrane Library 2001, Issue 2 was
1081.13 To obtain about 40 reviews, we used
a computer-generated list of random numbers in blocks of 26 and randomly selected
4 trials from each block. This resulted in 167 systematic reviews, which were
screened by one author (B.A.-N.) for eligible meta-analyses.
Selection of Primary Outcome
Two of the authors (B.A.-N., L.L.K.) independently selected the primary
binary outcome for each included meta-analysis. The primary binary outcome
from each meta-analysis was considered the primary outcome for all trials
included in each meta-analysis. If it was not explicitly reported or several
primary outcomes were listed, we chose the most clinically relevant outcome,
assessed by the largest number of trials.
Data Extraction and Definitions
All data were extracted from the original trial reports. We defined
conclusions as the interpretation of the extent to which the overall trial
results favored the experimental over the control intervention. We graded
conclusions according to the phrasing in the abstract and the summarized conclusion
on a scale of 1 to 6 points (Box).14 The higher the score, the more
positive the conclusion toward the experimental intervention. Because the
score did not conform to a normal distribution, we divided conclusions into
whether the experimental drug was recommended as the treatment of choice without
disclaimers (6 points) or not (1-5 points).
6 Points
Experimental intervention highly preferred and should now be considered
the standard intervention in all patients, or similar
5 Points
Experimental intervention preferred to control, but further trials still
indicated, experimental may be more costly, or similar
4 Points
Experimental and control intervention about equal, but the experimental
cheaper, easier to administer, or similar minor advantage
3 Points
Experimental and control intervention about equal, but the control may
be cheaper, easier to administer, or similar minor advantage
2 Points
Control intervention preferred to experimental intervention, but experimental
intervention might be promising under some circumstances, or similar
1 Point
Control intervention highly preferred and should now be considered the
standard intervention in all patients, or similar
We extracted the sources of funding from the text, statements of sources
of support, authors' affiliations, and acknowledgments. Funding sources were
classified as nonprofit organizations, not reported, both nonprofit and for-profit
organizations, or for-profit organizations. For-profit organizations were
defined as companies that might acquire financial gain or loss depending on
the outcome of the trial. Funding included provision of grants, study materials
(drug, placebo, assay kits, or similar materials), or manpower (authorship,
statistical analysis, or other assistance). For the primary binary outcome
measure, we extracted the number of outcomes and participants in the experimental
and control groups.
We classified the occurrence of adverse events as (1) no significant
difference between experimental and control group, (2) significantly more
frequent in the control group, (3) significantly more frequent in the experimental
group, or (4) not reported.
We assessed methodological quality from the original trial reports and
any additional information provided in the Cochrane review. We assessed the
following 3 components10,11: generation
of the allocation sequence (classified as adequate if based on a table of
random numbers, computer-generated, or similar), allocation concealment (classified
as adequate if based on central randomization, identical coded drug boxes,
sealed envelopes, or similar), and double blinding (classified as adequate
if the trial was described as double blind). We extracted the type of control
intervention (placebo/no intervention or active intervention), the number
of patients randomized, and whether a preset sample size was estimated and
reached. We registered the meta-analysis in which the trial was included,
the year of publication, and whether the trial was published in a high-impact
journal (impact factor ≥6).1-3
Two authors (B.A.-N., W.C.) independently extracted data from each trial
in an unblinded manner. Consensus was achieved before data entry. A third
author (C.G.) arbitrated disagreements. A fourth author (L.L.K.), who was
blinded with regard to funding, extracted conclusions in a random sample of
60 trials. The intraclass correlation coefficient between blinded and unblinded
assessment of conclusions was 0.93 (95% confidence interval [CI], 0.89-0.96).
For each trial, we estimated the OR of an unfavorable outcome (eg, mortality).
The SE of the logarithm of the OR was calculated as a measure of uncertainty.15 We calculated a z score
(log OR/SE to log OR) as a measure of treatment effect.15 The z score combines the magnitude of the point estimate (log
OR) with the level of uncertainty (SE).
We used the Kruskal-Wallis test for testing the overall null hypothesis
of no association between funding and conclusions assessed on the continuous
scale. We used a cutoff value between 5 and 6 points to divide trials into
whether the conclusions recommended the experimental drug as the treatment
of choice or not. We used logistic regression to assess the association between
funding and conclusions while adjusting for treatment effect, adverse events,
and other potentially confounding trial variables (methodological quality,
sample size, whether preset sample size was estimated and reached, meta-analysis,
year of publication, and journal impact factor). The logistic regression model
was fit using conclusions as the dependent variable and including trial variables
in a forward stepwise procedure. Meta-analysis was kept in the model irrespective
of statistical significance to adjust for the disease area and type of drug
and control intervention. All other variables were excluded if P>.05. The appropriateness of the logistic regression models was confirmed
by the Hosmer-Lemeshow test.16 All P values were 2-tailed and significance was defined as P<.05. Analyses were performed in SPSS version 11.0 for Windows
(SPSS Inc, Chicago, Ill).
Identification of Eligible Trials
From our random sample of 167 Cochrane reviews, we excluded 126 that
included fewer than 5 full-paper randomized trials in a meta-analysis (n =
105), included only trials with adequate (n = 6) or inadequate allocation
concealment (n = 13), or did not assess a binary outcome (n = 2). The remaining
41 reviews contained meta-analyses, which included 523 trials, that met our
inclusion criteria. Sixteen of these reviews, which included 153 trials, assessed
nonpharmacological interventions. Initial analyses revealed that only 4 (3%)
of these trials were funded by for-profit organizations. We therefore limited
our analyses to the 370 drug trials (references available on request from
the authors) from 25 reviews.17-41
Description of Included Trials
In most trials, conclusions favored the experimental drug (median score
[interquartile range], 5 [4-6]). In 36% of trials (n = 135), the experimental
drug was recommended as the treatment of choice (6 points).
Eighteen percent of trials (n = 67) were funded by nonprofit organizations
and in 29% (n = 106) funding was not reported. Fourteen percent of trials
(n = 51) were funded by both nonprofit and for-profit organizations and 39%
(n = 146) by for-profit organizations alone. The treatment effect assessed
by the mean (SD) z score was –1.39 (1.90) (range,
–10.99 to 5.45). In 50% of the trials (n = 185), the occurrence of adverse
events did not differ significantly between the intervention groups, and in
5% (n = 20) the occurrence was significantly higher in the control than that
in the experimental group. In 16% of trials (n = 60), the occurrence of adverse
events was significantly higher in the experimental group, and in 28% (n =
105) adverse events were not reported.
Adequate generation of the allocation sequence was reported in 28% of
trials (n = 105), adequate allocation concealment in 22% (n = 82), and 63%
(n = 234) were double blind. In 76% of trials (n = 283), the control intervention
was placebo or no intervention. The median number of patients randomized was
98 (range, 10-82 892). Preset sample size was estimated and reached in
21% of trials (n = 76).
The disease areas were intensive care (n = 85), smoking cessation (n
= 78), respiratory diseases (n = 54), gynecology/obstetrics (n = 48), gastroenterology
(n = 33), neurology (n = 26), psychiatry (n = 13), infectious diseases (n
= 12), rheumatology (n = 9), nephrology (n = 6), and dermatology (n = 6).
The primary outcome measures were smoking cessation (n = 78), mortality (n
= 64), blood transfusion (n = 61), withdrawals (n = 33), dysphagia (n = 23),
endometritis (n = 20), parasitemia (n = 17), depression (n = 13), admission
to hospital (n = 11), bronchiolitis (n = 8), neurologic deficit (n = 8), cesarean
delivery (n = 7), warts (n = 6), cytomegalovirus disease (n = 6), pregnancy
(n = 5), bacterial vaginosis (n = 5), and asthma (n = 5).
The year of publication ranged from 1971 to 2000 with 1990 (5.6 years)
as the mean (SD) publication year. Eighteen percent of trials (n = 65) were
published in high-impact journals.
Characteristics of Trials Stratified by Funding
The conclusions in trials stratified by funding are shown in Table 1. Conclusions were significantly
more favorable toward experimental drugs in trials funded by for-profit organizations
compared with those of trials funded by other sources (P<.001, Kruskal-Wallis test). The proportion of trials in which
conclusions recommended the experimental drug as the treatment of choice (6
points) was significantly higher among trials with for-profit funding compared
with that of other trials (P<.001, χ2 test).
The distributions of the potential confounding variables stratified
by funding are shown in Table 2.
Funding by for-profit organizations alone or by for-profit and nonprofit organizations
was associated with more complete reporting of adverse events, more adverse
events in the experimental group, more frequent report of adequate allocation
concealment and double blinding, and more frequent use of placebo or no treatment
as control intervention. Funding by both nonprofit and for-profit organizations
also was associated with a larger sample size and publication in high-impact
journals. No significant difference was observed between the groups regarding
the treatment effect, adequate generation of the allocation sequence, or whether
a preset sample size had been estimated and reached.
Funding and Conclusions Adjusted for Confounders
The logistic regression analyses showed that funding, treatment effect,
and double blinding were significantly associated with conclusions (Table 3). None of the remaining variables
were associated significantly with conclusions. After adjusting for the treatment
effect and double blinding, conclusions were significantly more likely to
recommend the experimental drug as treatment of choice in trials funded by
for-profit organizations alone compared with trials funded by nonprofit organizations
(OR, 5.3; 95% CI, 2.0-14.4). Compared with trials funded by nonprofit organizations,
conclusions were not significantly different in trials not reporting funding
or trials funded by both nonprofit and for-profit organizations. The likelihood
of recommending the experimental drug as the treatment of choice decreased
with decreasing treatment effect (OR, 0.6; 95% CI, 0.5-0.7) and increased
with adequate double blinding (OR, 2.9; 95% CI 1.4-6.0) (Table 3).
In randomized drug trials from a randomly selected sample of reviews
published in the Cochrane Library,13 we found
that conclusions of trials were significantly more likely to recommend the
experimental drug as the treatment of choice if trials were funded by for-profit
organizations. This result is in accordance with previous studies.1-7 The
present study adds to previous evidence by showing that this association does
not reflect the quantitative trial results; neither the magnitude of the treatment
effect nor the occurrence of adverse events could explain the association.
We selected meta-analyses from a random sample of Cochrane reviews because
the Cochrane Collaboration is a nonprofit organization that aims to minimize
the influence of financial and other competing interests. Furthermore, Cochrane
reviews appear to be of higher quality and are less prone to bias than reviews
published in traditional medical journals.42-44 We
excluded a high (75%) but expected number of the randomly selected reviews
because they did not contain a meta-analysis that fulfilled our inclusion
criteria. Furthermore, only 3% of the nonpharmacological trials in our original
sample were funded by for-profit organizations. Because of the low frequency
of for-profit funding, these trials would not contribute valuable information,
but rather introduce noise into the analyses.45 Therefore,
these trials also were excluded. Considering that more than 360 000 randomized
clinical trials have been published,46 our
results are based on a small sample. These factors may reduce the external
validity of our study. However, our sample was obtained randomly from the
Cochrane Library, which covers all medical areas.13 The
included trials covered a variety of medical areas, drugs, publication years,
and journals. This variability supports the external validity of our findings.
Within each meta-analysis, trials addressed the same clinical question,
compared the same treatments, and provided data on the same outcome measure.
We therefore were able to adjust our analyses for both disease areas and type
of experimental and control treatments. Therefore, the clinical importance
of the treatment corresponds to the magnitude of the treatment effect. Together
the point estimate (OR) and the level of uncertainty (SE) provide information
about the treatment effect. We used a z score to
capture these 2 aspects in 1 variable. We found similar results when including
the OR and the SE of the treatment effect separately in the logistic regression
model (data not shown). The magnitude of the treatment effect did not explain
the association between funding and conclusions.
To estimate the treatment effect, we selected the primary binary outcome
specified in the meta-analysis including the trial. We did this to obtain
clinically important and homogeneous data. Many randomized trial reports do
not specify the primary outcome measures,47 use
surrogate outcomes,48-51 or
use multiple primary outcomes.52 Cochrane reviews
are based on prespecified, peer-reviewed, and published protocols in which
the outcome measures most clinically relevant to patients are selected.53 Accordingly, the selection of the primary outcome
measure in Cochrane reviews aims to be unbiased and not data driven.52
The favoring of experimental drugs in trials funded by for-profit organizations
did not appear to reflect the occurrence of adverse events. Compared with
trials funded by nonprofit organizations, trials funded by for-profit organizations
reported a significantly higher number of adverse events in the experimental
arm. This might reflect differences in the quality of reporting. Possibly,
drug companies are more focused on reporting of adverse events because of
use of Good Clinical Practice guidelines.54
We assessed the effect of several potential confounders. These were
selected according to previous evidence and theoretical considerations. Several
of the confounders had some overlap (eg, the treatment effect and the use
of inactive control, double blinding and the use of placebo as control intervention).
However, this overlap only increased the SEs of the logistic regression model
but did not invalidate the model. Our results indicated that the methodological
quality, type of control intervention, sample size, whether a preset sample
size had been estimated and reached, year of publication, or publication in
high-impact journals did not explain the association between funding and conclusions.
Trials funded by for-profit organizations had better methodological
quality than trials funded by nonprofit organizations regarding allocation
concealment and double blinding. This finding is in accordance with previous
studies.5,7,55 We
are aware that discrepancies might occur between report and conduct of trials.56 The Cochrane Collaboration recommends that reviewers
correspond with the primary investigators of trials and companies to obtain
information about central methodological aspects.53 We
based our assessment of quality on both the trial reports and any additional
information provided in the Cochrane reviews.17-41
We assessed the reported conclusions using a scale developed by Gilbert
et al.14 The lack of validation of this scale
may be criticized.20 However, the scale has
been used in 3 studies,3,5,57 and
we found that it had high face and content validity as well as high reliability.
One study analyzed the reported conclusions on the continuous scale.3 Other studies divided the reported conclusions into
"positive" or "negative."1,2,4,5 In
the present study, only 20% of the trials scored 1 to 3 points. This skewed
distribution of data supported the use of a cutoff value between 5 and 6 points.
This allowed us to explore whether conclusions were more likely to recommend
the experimental drug as the treatment of choice without disclaimers. Such
conclusions must have a considerable impact on clinical decision making. Sensitivity
analyses revealed that selecting another cutoff (eg, between 4 and 5 points)
gave similar results. This increases the robustness of the evidence and supports
the existence of an association between funding and conclusions.1-7
A potential weakness of the present study is that the reported conclusions
were assessed unblinded with regard to the source of funding. However, we
found high interobserver agreement between blinded and unblinded assessment.
This concurs with previous findings3,58 and
suggests that blinding would not significantly change our conclusions.
Sponsor involvement and influence on the conduct and reporting of a
trial varies. The degree of influence is difficult to assess from trial reports.
We combined trials funded entirely by for-profit organizations with trials
in which only drugs and placebo were provided. Sensitivity analyses showed
that the significant association between funding and conclusions was present
both in the group of trials having only obtained for-profit funding in the
form of drug and placebo and in the group of trials having obtained more substantial
for-profit funding (data not shown).
A large proportion of trials did not report the sources of funding.
It is likely that funding by for-profit organizations is underreported in
trials. However, the degree of underreporting is not possible to assess. We
found that about 50% of the included trials were funded solely or in part
by for-profit organizations. This is in accordance with a recent review, in
which the median proportion of trials receiving for-profit funding was 39%
(interquartile range, 23%-64%).7
Conclusions reflect a trade-off between efficacy, safety, and cost-effectiveness.
We did not assess the impact of cost-effectiveness on conclusions. Economic
analyses are rarely included in randomized trials59,60 and
new interventions are generally more expensive than conventional ones. Friedberg
et al61 showed that studies in oncology funded
by for-profit organizations were nearly 8 times less likely to reach unfavorable
conclusions regarding economic assessments of experimental interventions than
studies funded by nonprofit organizations.
We based our analyses on all full-paper trials that were included in
the eligible meta-analyses. Trials that were only published as abstracts or
letters were excluded as they very rarely contained information on funding
and/or trial quality. We did not assess the studies that the authors of the
Cochrane reviews had excluded from meta-analyses. The reasons for exclusion
were described in all reviews. The majority of studies were excluded because
they were not randomized trials. It is possible that these studies estimated
a greater treatment effect because lack of randomization increases the risk
of selection bias. However, it is uncertain whether the design of a study
(randomized or nonrandomized) has a significant effect on the association
between funding and conclusions. Our study was not designed to address this
aspect.
The present study cannot show the causes or consequences of the association
between for-profit funding and conclusions favoring the experimental drug.
Our study was designed to assess if an association reflected the quantitative
trial results. We found a significant association between conclusions and
the estimated treatment effect. The likelihood of recommending the experimental
drug as the treatment of choice increased significantly with increasing treatment
effect (ie, decreasing z scores). However, this did
not appear to explain the association between funding and conclusions. We
found no significant difference in treatment effect between trials stratified
according to funding. This is in accordance with a recent pilot study.62 Our findings oppose the suggestion that conclusions
are more likely to be positive if funded by for-profit organizations because
these trials should be more likely to reach positive results.8
Likely explanations for the association could be violation of the uncertainty
principle, publication bias, emphasis on subgroup or secondary outcome analyses,
or bias in drawing conclusions. Violation of the uncertainty principle could
occur if for-profit organizations were more prone to sponsor trials that were
likely to favor the experimental drug.5 A main
objective of the pharmaceutical industry is to acquire financial gain. After
having conducted exploratory randomized trials in phase 2 drug development,
several confirmatory randomized phase 3 (proof of concept) trials usually
are launched. Such trials may have a higher likelihood of favoring the experimental
drug. Publication bias also has been suggested as a possible explanation.1-3,5,7,63 Concern
has been raised that for-profit organizations might discontinue ongoing studies
if accumulating results appear negative or if they avoid the publication of
negative studies.1,63,64 Because
of the design of the present study, we cannot refute or confirm these queries.
We found no significant difference in the estimated intervention effect of
the primary outcome measure between groups of trials stratified according
to funding. It is possible, that the favoring of experimental interventions
in conclusions of trials funded by for-profit organizations was due to emphasis
of results from surrogate outcomes,49 secondary
outcome measure analyses, or subgroup analyses.65,66 Finally,
the association between funding and conclusions might reflect a biased interpretation
of the overall trial results. This potential bias could be due to financial
conflicts of interest.67
In principle, about half of all trials should favor the control intervention.5 We found that trial results and conclusions rarely
favored the control. This finding concurs with previous studies.1-4,6,57,62,64,68 The
combined evidence underlines the need for an international register of all
initiated randomized clinical trials. Registration would enable the public
to follow the development of drugs from the beginning of phase 2 trials. Furthermore,
data from unpublished randomized trials irrespective of funding source or
results could be contained in an international register available to the public.69,70 The Consolidated Standards of Reporting
Trials (CONSORT) statement71 could consider
the importance of reporting of funding, as suggested by the International
Committee of Medical Journal Editors.72 Our
study suggests that editors, peer reviewers, and readers of trial reports
should evaluate carefully the trial data to determine if the reported conclusions
are supported by data.
1.Davidson RA. Source of funding and outcome of clinical trials.
J Gen Intern Med.1986;1:155-158.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=3772583
Google Scholar 2.Yaphe J, Edman R, Knishkowy B, Herman J. The association between funding by commercial interests and study outcome
in randomized controlled drug trials.
Fam Pract.2001;18:565-568.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11739337
Google Scholar 3.Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological
study of randomised clinical trials published in BMJ.
BMJ.2002;325:249-252.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12153921
Google Scholar 4.Rochon PA, Gurwitz JH, Simms RW.
et al. A study of manufacturer-supported trials of nonsteroidal anti-inflammatory
drugs in the treatment of arthritis.
Arch Intern Med.1994;154:157-163.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=8285810
Google Scholar 5.Djulbegovic B, Lacevic M, Cantor A.
et al. The uncertainty principle and industry-sponsored research.
Lancet.2000;356:635-638.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=10968436
Google Scholar 6.Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research:
a systematic review.
JAMA.2003;289:454-465.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12533125
Google Scholar 7.Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality:
systematic review.
BMJ.2003;326:1167-1170.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12775614
Google Scholar 8.Jacobs A. Association between competing interests and conclusions [letter].
BMJ.2002;325:1420.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12484375
Google Scholar 9.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated
with estimates of treatment effects in controlled trials.
JAMA.1995;273:408-412.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=7823387
Google Scholar 10.Kjaergard LL, Villumsen J, Gluud C. Reported methodological quality and discrepancies between large and
small randomized trials in meta-analyses.
Ann Intern Med.2001;135:982-989.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11730399
Google Scholar 11.Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment
of trial quality in systematic reviews? empirical study.
Health Technol Assess.2003;7:1-76.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12583822
Google Scholar 12.Balk EM, Boris PA, Moskowitz H.
et al. Correlation of quality measures with estimates of treatment effect
in meta-analyses of randomized controlled trials.
JAMA.2002;287:2973-2982.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12052127
Google Scholar 13. The Cochrane Database of Systematic Reviews. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
14.Gilbert JP, McPeek B, Mosteller F. Statistics and ethics in surgery and anesthesia.
Science.1977;198:684-689.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=333585
Google Scholar 15.Altman DG. Practical Statistics for Medical Research. London, England: Chapman & Hall; 1991.
16.Hosmer D, Lemeshow S. Applied Logistic Regression. New York, NY: Wiley & Sons; 1989.
17.Tinnion ON, Hanlon M. Acellular vaccines for preventing whooping cough in children [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
18.Olliaro P, Mussano P. Amodiaquine for treating malaria [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
19.Henry DA, Moxey AJ, Carless PA.
et al. Anti-fibrinolytic use for minimising perioperative allogeneic blood
transfusion [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
20.French LM, Smaill FM. Antibiotic regimens for endometritis after delivery [Cochrane Review
on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
21.Roos YBWEM, Rinkel GJE, Vermeulen M, Algra A, van Gijn J. Antifibrinolytic therapy for aneurysmal subarachnoid haemorrhage [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
22.Suarez-Almazor ME, Spooner CH, Belseck E, Shea B. Auranofin versus placebo in rheumatoid arthritis [Cochrane Review on
CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
23.Kellner JD, Ohlsson A, Gadomski AM, Wang EEL. Bronchodilators for bronchiolitis [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
24.Feigin VL, Rinkel GJE, Algra A, Vermeulen M, van Gijn J. Calcium antagonists for aneurysmal subarachnoid haemorrhage [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
25.Alderson P, Schierhout G, Roberts I, Bunn F. Colloids versus crystalloids for fluid resuscitation in critically
ill patients [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
26.Rowe BH, Spooner CH, Ducharme FM, Bretzlaff JA, Bota GW. Corticosteroids for preventing relapse following acute exacerbations
of asthma [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
27.Couchoud C. Cytomegalovirus prophylaxis with antiviral agents for solid organ transplantation
[Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
28.Lima MS, Moncrieff J. Drugs versus placebo for dysthymia [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
29.Rowe BH, Spooner C, Ducharme FM, Bretzlaff JA, Bota GW. Early emergency department treatment of acute asthma with systemic
corticosteroids [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
30.Candelise L, Ciccone A. Gangliosides for acute ischaemic stroke [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
31.Brocklehurst P, Hannah M, McDonald H. Interventions for treating bacterial vaginosis in pregnancy [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
32.Gibbs S, Harvey I, Sterling JC, Stark R. Local treatments for cutaneous warts [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
33.Davies H, Olson L, Gibson P. Methotrexate as a steroid sparing agent for asthma in adults [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
34.Silagy C, Mant D, Fowler G, Lancaster T. Nicotine replacement therapy for smoking cessation [Cochrane Review
on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
35.Wood-Baker R, Walters E.H, Gibson P. Oral corticosteroids for acute exacerbations of chronic obstructive
pulmonary disease [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
36.Soo S, Moayyedi P, Deeks J, Delaney B, Innes M, Forman D. Pharmacological interventions for non-ulcer dyspepsia [Cochrane Review
on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
37.Garner P, Gülmezoglu AM. Prevention versus treatment for malaria in pregnant women [Cochrane
Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
38.Martin-Hirsch PL, Jarvis G, Kitchener H, Lilford R. Progestagens for endometrial cancer [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
39.Tan BP, Hannah ME. Prostaglandins versus oxytocin for prelabour rupture of membranes at
term [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
40.Daya S, Gunby J. Recombinant versus urinary follicle stimulating hormone for ovarian
stimulation in assisted reproduction cycles [Cochrane Review on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
41.Ioannou G, Doust J, Rockey DC. Terlipressin for acute esophageal variceal hemorrhage [Cochrane Review
on CD-ROM]. Oxford, England: Cochrane Library, Update Software; 2001; issue 2.
42.Egger M, Davey SG, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test.
BMJ.1997;315:629-634.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=9310563
Google Scholar 43.Jadad AR, Cook DJ, Jones A.
et al. Methodology and reports of systematic reviews and meta-analyses: a
comparison of Cochrane reviews with articles published in paper-based journals.
JAMA.1998;280:278-280.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=9676681
Google Scholar 44.Jadad AR, Moher M, Browman GP.
et al. Systematic reviews and meta-analyses on treatment of asthma: critical
evaluation.
BMJ.2000;320:537-540.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=10688558
Google Scholar 45.Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their
Development and Use. 2nd ed. Oxford, England: Oxford Medical Publications; 1995.
46. The Cochrane Library. Oxford, England: Update Software; 2003; issue 2.
47.Zhang B, Schmidt B. Do we measure the right end points? a systematic review of primary
outcomes in recent neonatal randomized clinical trials.
J Pediatr.2001;138:76-80.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11148516
Google Scholar 48.Rossetti L, Marchetti I, Orzalesi N, Scorpiglione N, Liberati A. Is proper methodology associated with the use of a clinically relevant
outcome measure? the case of randomized clinical trials on medical treatment
of open-angle glaucoma.
Online J Curr Clin Trials [serial online].1993. Doc No. 100.Google Scholar 49.Gøtzsche PC, Liberati A, Torri V, Rossetti L. Beware of surrogate outcome measures.
Int J Technol Assess Health Care.1996;12:238-246.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=8707497
Google Scholar 50.Roberts L, Counsell C. Assessment of clinical outcomes in acute stroke trials.
Stroke.1998;29:986-991.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=9596247
Google Scholar 51.Duncan PW, Jorgensen HS, Wase DT. Outcome measures in acute stroke trials: a systematic review and some
recommendations to improve practice.
Stroke.2000;31:1429-1438.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=10835468
Google Scholar 52.Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials: a survey
of three medical journals.
N Engl J Med.1987;317:426-432.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=3614286
Google Scholar 53.Clarke M, Oxman A. Cochrane Reviewers' Handbook 4.1.6 [updated January 2003]. Oxford, England: Cochrane Library, Update Software; 2003; issue 2.
54.International Conference on Harmonisation Expert Working Group. Code of Federal Regulations & International Conference
on Harmonization Guidelines. Philadelphia, Pa: Parexel Barnett; 1997.
55.Kjaergard LL, Nikolova D, Gluud C. Randomized clinical trials in HEPATOLOGY: predictors of quality.
Hepatology.1999;30:1134-1138.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=10534332
Google Scholar 56.Huwiler-Müntener K, Jüni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic
quality.
JAMA.2002;287:2801-2804.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12038917
Google Scholar 57.Colditz GA, Miller JN, Mosteller F. How study design affects outcomes in comparisons of therapy, I: medical.
Stat Med.1989;8:441-454.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=2727468
Google Scholar 58.Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological
study of randomised clinical trials published in the BMJ [authors' electronic
response].
Available at: http://bmj.com/cgi/content/full/325/7358/249.
Accessibility verified July 7, 2003. 59.Adams ME, McCall NT, Gray DT, Orza MJ, Chalmers TC. Economic analysis in randomized control trials.
Med Care.1992;30:231-243.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=1538611
Google Scholar 60.Gluud C, Kjaergard LL. Quality of randomized clincial trials in portal hypertension and other
fields of hepatology. In: de Franchis R, ed. Portal Hypertension III:
Proceedings of the Third Baveno International Consensus Workshop on Definitions,
Methodology, and Therapeutic Strategies. Oxford, England: Blackwell
Science; 2001:204-218.
61.Friedberg M, Saffran B, Stinson TJ, Nelson W, Bennett CL. Evaluation of conflict of interest in economic analyses of new drugs
used in oncology.
JAMA.1999;282:1453-1457.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=10535436
Google Scholar 62.Clifford T, Barrowman N, Moher D. Funding source, trial outcome and reporting quality: are they related?
results of a pilot study.
BMC Health Serv Res.2002;2:18-23.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12213183
Google Scholar 63.Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine—selective reporting from studies sponsored
by pharmaceutical industry: review of studies in new drug applications.
BMJ.2003;326:1171-1173.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=12775615
Google Scholar 64.Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research.
Lancet.1991;337:867-872.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=1672966
Google Scholar 65.Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients
in randomized clinical trials.
JAMA.1991;266:93-98.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=2046134
Google Scholar 66.Freemantle N. Interpreting the results of secondary end points and subgroup analyses
in clinical trials: should we lock the crazy aunt in the attic?
BMJ.2001;322:989-991.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11312237
Google Scholar 67.Thompson DF. Understanding financial conflicts of interest.
N Engl J Med.1993;329:573-576.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=8336759
Google Scholar 68.Gøtzche PC. Methodology and overt and hidden bias in reports of 196 double-blind
trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis.
Control Clin Trials.1989;10:31-56.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=2702836
Google Scholar 69.Simes RJ. Publication bias: the case for an international registry of clinical
trials.
J Clin Oncol.1986;4:1529-1541.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=3760920
Google Scholar 70.Horton R. Medical editors trial amnesty.
Lancet.1997;350:756.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=9297993
Google Scholar 71.Moher D, Schulz KF, Altman DG.for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality
of reports of parallel-group randomized trials.
JAMA.2001;285:1987-1991.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11308435
Google Scholar 72.Davidoff F, DeAngelis CD, Drazen JM.
et al. Sponsorship, authorship, and accountability.
JAMA.2001;286:1232-1234.http://www.ncbi.nlm.nih.gov/htbin-post/Entrez/query?db=m&form=6&Dopt=r&uid=11559271
Google Scholar