Author Affiliation:Obstetrics & Gynecology, Washington, DC.
Context Some journals routinely query potential referees before sending manuscripts
("askfirst"), whereas others just send manuscripts and allow referees to opt
out ("justsend"). It is not known which protocol results in more completed
reviews or shorter review time.
Methods Trial to assess proportion of referee turndowns and length of review
process, conducted at editorial office of Obstetrics &
Gynecology and involving 283 consecutive qualifying manuscripts. For
each, a referee was randomly assigned to askfirst (manuscript sent only after
affirmative response within 3 days) and another to justsend (manuscript sent
with request to review; could opt out).
Results Only 64% of askfirst referees assented initially (15% declined [vs 8%
for justsend, P = .008] and 21% failed to respond
within 3 working days, necessitating a replacement). But once manuscript was
mailed, mean time to file a review was significantly shorter for askfirst
(21.0 vs 25.0 days, P<.001); thus, overall time
to receipt of review did not differ significantly (24.7 vs 25.9 days, P = .19), nor did review quality (P
Conclusion Askfirst led to a higher rate of referee turndown than did justsend,
but assenting askfirst referees completed reviews faster. The overall time
for the review process did not differ between the 2 protocols.
In recruiting reviewers, some journals simply send the manuscript to
identified experts with a cover letter asking them to review the paper; if
unable or unwilling, they may decline ("justsend"). Other journals query potential
referees first, and only send manuscripts to those who specifically assent
("askfirst"). We are unaware of evidence as to which is better and authorities
provide little guidance. Bishop1 does not mention
the matter specifically, although his statement, "[s]ome journals send out
manuscripts for review with a very simple cover letter, ‘Could you please
review the enclosed paper as to its suitability for publication in this journal?'"
seems to imply justsend. The objective of this study was to compare these
Setting was the main editorial office of Obstetrics
& Gynecology, a monthly medical specialty journal. The editor chose
2 referees for each research article received between September 2, 1999 and
May 8, 2000; if fax numbers and mailing address were known for both, the manuscript
was enrolled in the study. Using a random-number generator, an editorial assistant
assigned one referee to justsend and the other to askfirst.
Referees assigned to justsend were mailed manuscripts and asked to return
their reviews by fax or e-mail within 3 weeks; if unable to comply, they were
to telephone the editorial office immediately, and a substitute was chosen.
Askfirst assignees were faxed information about the manuscript (title, authors,
length) and asked to indicate their willingness by return fax. If they accepted,
the manuscript and the same instructions and cover letter (asking for a return
within 3 weeks) were mailed. If a referee refused or failed to respond within
3 working days, a substitute chosen by the editor was contacted by fax, and
so on until a referee agreed to the review (up to 4 cycles) (Figure 1).
Main outcome variables were (1) proportion of initial referees who failed
to opt out (justsend) or who opted in (askfirst), (2) overall time for the
review process (days from enrolling manuscript in the study until review received),
and (3) days between mailing manuscript and receipt of review. Additionally,
quality ratings of reviews were compared for the 2 methods, based on a global
rating (5-point scale) assigned by the editor at manuscript disposition, without
knowledge of assignment for this study. One editor and an editorial assistant
Sample size, based on assumed 15% opt-out for justsend and assumed 30%
failure to opt-in for askfirst (power = 0.80, α = .05), was projected
at 300 manuscripts (final sample = 283).
Considering just the initial sample, of 283 justsend referees, 261 (92%)
did not decline and 247 (87%) ultimately produced reviews. Of 283 askfirst
referees, 181 (64%) agreed to review and 177 (63%) ultimately produced reviews;
43 (15%) declined and 59 (21%) failed to respond. The difference in frequency
of specific declines (8% vs 15%) was significant (χ2 = 7.66; P = .008). Viewed another way, whereas only 8% of justsend
referees opted out, 36% of askfirst candidates did not opt in. However, the
rate of producing reviews among initial justsend referees who did not opt
out (247/261 [95%]) and askfirst candidates who opted in (177/181 [98%]) did
not differ significantly (χ2 = 2.72; P
Considering all reviewers, the mean (SD) overall time required for the
review process was similar in askfirst and justsend (24.7 [9.7] vs 25.9[10.5]
days; P = .18). However, once the manuscript was
mailed to a referee (on day of study enrollment for justsend but not until
specific assent was given in askfirst), the time until a review was received
was significantly less with askfirst than with justsend (21.0 [9.2] vs 25.0
[10.1 days]; P<.005). Thus, askfirst reviewers
tended to complete their reviews sooner, even though all were given the same
instructions, and this difference made up for extra time spent in gaining
a potential reviewer's assent.
Quality of reviews was assessed in the subset of 155 manuscripts in
which both initial referees agreed and returned reviews, because initial assignment
was both random and masked, whereas selection of substitute referees could
well have been influenced by past quality ratings. There was no significant
difference in quality of reviews between justsend and askfirst groups (Wilcoxon
sign-rank test, P = .39).
We found, not surprisingly, that making potential referees opt in invited
approximately twice as many turndowns as an opt-out approach. Moreover, the
substantial segment of unanswered askfirst requests (21%) meant that only
64% were answered affirmatively. Although askfirst had a higher rate of turn-downs,
making it necessary to find a substitute referee, the time required for the
overall review process was similar to that with justsend. This resulted from
askfirst reviewers completing their reviews quicker than justsend reviewers,
even though both were allowed the same return time. Perhaps referees assent
in advance when they know their schedules will permit them to do the review,
whereas those who receive the manuscript without any warning add it to their
pile of work and do it when they can. We found no indication that soliciting
in advance affected review quality.
Pitkin RM, Burmeister LF. Identifying Manuscript ReviewersRandomized Comparison of Asking First or Just Sending. JAMA. 2002;287(21):2795-2796. doi:10.1001/jama.287.21.2795