Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
Peer Review
June 5, 2002

Identifying Manuscript ReviewersRandomized Comparison of Asking First or Just Sending

Author Affiliations

Author Affiliation:Obstetrics & Gynecology, Washington, DC.

JAMA. 2002;287(21):2795-2796. doi:10.1001/jama.287.21.2795

Context Some journals routinely query potential referees before sending manuscripts ("askfirst"), whereas others just send manuscripts and allow referees to opt out ("justsend"). It is not known which protocol results in more completed reviews or shorter review time.

Methods Trial to assess proportion of referee turndowns and length of review process, conducted at editorial office of Obstetrics & Gynecology and involving 283 consecutive qualifying manuscripts. For each, a referee was randomly assigned to askfirst (manuscript sent only after affirmative response within 3 days) and another to justsend (manuscript sent with request to review; could opt out).

Results Only 64% of askfirst referees assented initially (15% declined [vs 8% for justsend, P = .008] and 21% failed to respond within 3 working days, necessitating a replacement). But once manuscript was mailed, mean time to file a review was significantly shorter for askfirst (21.0 vs 25.0 days, P<.001); thus, overall time to receipt of review did not differ significantly (24.7 vs 25.9 days, P = .19), nor did review quality (P = .39).

Conclusion Askfirst led to a higher rate of referee turndown than did justsend, but assenting askfirst referees completed reviews faster. The overall time for the review process did not differ between the 2 protocols.

In recruiting reviewers, some journals simply send the manuscript to identified experts with a cover letter asking them to review the paper; if unable or unwilling, they may decline ("justsend"). Other journals query potential referees first, and only send manuscripts to those who specifically assent ("askfirst"). We are unaware of evidence as to which is better and authorities provide little guidance. Bishop1 does not mention the matter specifically, although his statement, "[s]ome journals send out manuscripts for review with a very simple cover letter, ‘Could you please review the enclosed paper as to its suitability for publication in this journal?'" seems to imply justsend. The objective of this study was to compare these 2 approaches.


Setting was the main editorial office of Obstetrics & Gynecology, a monthly medical specialty journal. The editor chose 2 referees for each research article received between September 2, 1999 and May 8, 2000; if fax numbers and mailing address were known for both, the manuscript was enrolled in the study. Using a random-number generator, an editorial assistant assigned one referee to justsend and the other to askfirst.

Referees assigned to justsend were mailed manuscripts and asked to return their reviews by fax or e-mail within 3 weeks; if unable to comply, they were to telephone the editorial office immediately, and a substitute was chosen. Askfirst assignees were faxed information about the manuscript (title, authors, length) and asked to indicate their willingness by return fax. If they accepted, the manuscript and the same instructions and cover letter (asking for a return within 3 weeks) were mailed. If a referee refused or failed to respond within 3 working days, a substitute chosen by the editor was contacted by fax, and so on until a referee agreed to the review (up to 4 cycles) (Figure 1).

Figure. Recruiting Reviewers: "Askfirst" vs "Justsend"
Image not available

Main outcome variables were (1) proportion of initial referees who failed to opt out (justsend) or who opted in (askfirst), (2) overall time for the review process (days from enrolling manuscript in the study until review received), and (3) days between mailing manuscript and receipt of review. Additionally, quality ratings of reviews were compared for the 2 methods, based on a global rating (5-point scale) assigned by the editor at manuscript disposition, without knowledge of assignment for this study. One editor and an editorial assistant were involved.

Sample size, based on assumed 15% opt-out for justsend and assumed 30% failure to opt-in for askfirst (power = 0.80, α = .05), was projected at 300 manuscripts (final sample = 283).


Considering just the initial sample, of 283 justsend referees, 261 (92%) did not decline and 247 (87%) ultimately produced reviews. Of 283 askfirst referees, 181 (64%) agreed to review and 177 (63%) ultimately produced reviews; 43 (15%) declined and 59 (21%) failed to respond. The difference in frequency of specific declines (8% vs 15%) was significant (χ2 = 7.66; P = .008). Viewed another way, whereas only 8% of justsend referees opted out, 36% of askfirst candidates did not opt in. However, the rate of producing reviews among initial justsend referees who did not opt out (247/261 [95%]) and askfirst candidates who opted in (177/181 [98%]) did not differ significantly (χ2 = 2.72; P = .14).

Considering all reviewers, the mean (SD) overall time required for the review process was similar in askfirst and justsend (24.7 [9.7] vs 25.9[10.5] days; P = .18). However, once the manuscript was mailed to a referee (on day of study enrollment for justsend but not until specific assent was given in askfirst), the time until a review was received was significantly less with askfirst than with justsend (21.0 [9.2] vs 25.0 [10.1 days]; P<.005). Thus, askfirst reviewers tended to complete their reviews sooner, even though all were given the same instructions, and this difference made up for extra time spent in gaining a potential reviewer's assent.

Quality of reviews was assessed in the subset of 155 manuscripts in which both initial referees agreed and returned reviews, because initial assignment was both random and masked, whereas selection of substitute referees could well have been influenced by past quality ratings. There was no significant difference in quality of reviews between justsend and askfirst groups (Wilcoxon sign-rank test, P = .39).


We found, not surprisingly, that making potential referees opt in invited approximately twice as many turndowns as an opt-out approach. Moreover, the substantial segment of unanswered askfirst requests (21%) meant that only 64% were answered affirmatively. Although askfirst had a higher rate of turn-downs, making it necessary to find a substitute referee, the time required for the overall review process was similar to that with justsend. This resulted from askfirst reviewers completing their reviews quicker than justsend reviewers, even though both were allowed the same return time. Perhaps referees assent in advance when they know their schedules will permit them to do the review, whereas those who receive the manuscript without any warning add it to their pile of work and do it when they can. We found no indication that soliciting in advance affected review quality.

Bishop CT. How to Edit a Scientific JournalPhiladelphia, Pa: ISI Press; 1984:53.