From the University of Washington, Seattle (Dr Olson); the University of Chicago, Chicago, Ill (Dr Glass); and the Epidemiology Program Office, Centers for Disease Control and Prevention, Atlanta, Ga (Drs Thacker and Stroup). Dr Olson is Contributing Editor and Dr Glass is Deputy Editor, JAMA.
A protocol to prospectively study characteristics of meta-analyses submitted
to a weekly medical journal raised several ethical issues. In submitting a
manuscript for publication, authors do not implicitly consent to have their
work used for research. Authors must be free to refuse to consent, without
it affecting their chances for publication. Systematically analyzing data
on manuscript characteristics might influence the decision to publish. Having
investigators who are not on the editorial staff or peer reviewers extract
the manuscripts' characteristics breaks the confidentiality of the author-editor-reviewer
relationship. In response to these issues, we added a statement to our journal's
instructions for authors that submitted manuscripts may be systematically
analyzed to improve the quality of the editorial or peer review process. Authors
had to actively consent to participate, but editors and external reviewers
were unaware of which authors were participating. The manuscript characteristics
were not shared with authors, editors, or external reviewers. The investigators
were blinded to each manuscript's author and institution. After we addressed
ethical issues encountered in studying manuscripts submitted to a medical
journal, 99 of 105 authors submitting a meta-analysis during the study's first
24 months agreed to participate.
JAMA is a weekly general medical journal receiving more
than 4000 submissions a year with a 10.8% acceptance rate for unsolicited
manuscripts.1 Two meta-analysts (D.F.S. and
S.B.T.) approached the journal's staff proposing a study: editors and external
peer reviewers would systematically assess the methodological characteristics
of meta-analyses submitted to the journal; then, the meta-analysts would correlate
those characteristics with acceptance for publication. In response, the JAMA
editors raised several ethical concerns.
First, in submitting manuscripts for publication, authors do not implicitly
consent to have their work used for research. Yet, obtaining consent could
be logistically difficult and slow down the review process. In a written survey
of JAMA editors, with the inclusion of their names optional, 8 editors indicated
that authors should actively consent, 3 indicated that authors might be assumed
to consent unless they actively refused, and 1 supported either approach.
(We surveyed editors of JAMA only, since that was the study's site.)
Second, authors must feel free to refuse consent for their manuscript's
use in research, without fear that such refusal would affect their manuscript's
chances for publication. Third, systematically retrieving and analyzing data
on manuscript characteristics might delay review or influence the judgment
of external peer reviewers and editors. Also, manuscripts included in the
study should be assessed the same as those not included. Fourth, sending manuscripts
to outside experts who are not editors or reviewers would break the confidential
author-editor-reviewer relationship.2- 4
To address these concerns, we took the following steps. First, we added
a statement to our journal's instructions for authors2:
"Information from submitted manuscripts may be systematically collected and
analyzed as part of research to improve the quality of the editorial or peer
review process." Only those meta-analyses whose authors actively consented
to participate were included in the study. A senior editor (R.M.G.) identified
all meta-analyses as they were submitted. Corresponding authors were sent
a letter inviting their participation, which stated that we were studying
editorial decisions about publishing manuscripts, that the author's consent
or refusal to participate would not affect the reviewers' or editors' evaluation
or decision, and that all information would remain confidential with results
reported only in the aggregate.
Second, editors handling manuscripts and external peer reviewers were
blinded to whether an individual author was participating. Authors faxed their
response to an editor (C.M.O.) who works outside the main JAMA offices and
who did not edit or review any JAMA meta-analyses during this study. External
peer reviewers were not told that a study was being conducted, as reviewers
were not the subject of the study. The meta-analysts (D.F.S. and S.B.T.) did
not serve as external peer reviewers for meta-analyses submitted to JAMA
during this study.
Third, we changed the protocol to avoid affecting the review process.
Instead of editors and external peer reviewers, the meta-analysts (D.F.S.
and S.B.T.) extracted the manuscripts' characteristics. Their findings were
not shared with the authors, editors, or external peer reviewers.
Fourth, the meta-analysts were blinded to the manuscript's source by
removing the title page and any other information that might identify authors
or affiliations. After the meta-analysts assessed each manuscript independently
and resolved any differences by consensus, they returned the manuscript to
the JAMA office.
Finally, the Human Subjects Review Office of the Centers for Disease
Control and Prevention, Atlanta, Ga, initially reviewed the protocol and consent
form. After noting that no human interaction took place, data were not identifiable,
no sensitive questions were asked, and the risk to subjects was minimal, they
determined that the informed consent was adequate and the protocol was exempt
from full review. There is no equivalent to an institutional review board
at JAMA or the American Medical Association.
During the first 24 months, 105 meta-analyses were submitted. The authors
of 99 agreed to participate. Four actively refused and 2 did not respond to
initial or follow-up invitations.
Through obtaining authors' active consent to participate, keeping editors
handling meta-analyses unaware of authors' participation, maintaining the
usual review process, blinding investigators to authors and their institutions,
and submitting the protocol to an institutional review board, we responded
to ethical issues in conducting a study of unpublished manuscripts submitted
to a medical journal. Most authors were willing to participate in this prospective
observational research using their submitted manuscripts.
Olson CM, Glass RM, Thacker SB, Stroup DF. Ethical Issues in Studying Submissions to a Medical Journal. JAMA. 1998;280(3):290-291. doi:10.1001/jama.280.3.290