Prodding Tardy Reviewers: A Randomized Comparison of Telephone, Fax, and e-mail

Context To compare telephone, fax, and e-mail methods of prodding tardy reviewers.

Methods Randomized trial conducted January 1998 through June 1999 at the main editorial office of Obstetrics & Gynecology. Reviewers who had failed to file reviews by 28 days after being sent manuscripts (7 days after deadline) were sent identical messages in oral (telephone) or written (fax and e-mail) form inquiring as to the status of review, asking for its completion as soon as possible, and requesting it be sent by fax or e-mail.

Results Of 378 reviewers, proportions returning reviews within 7 days were essentially identical: telephone, 85 (68%) of 125; fax, 86 (67%) of 129; and e-mail, 84 (67%) of 124 (P = .59). In the two thirds who responded, the mean time to return reviews did not differ among the 3 groups.

Conclusion Contacting tardy reviewers resulted in a review being received within 7 days in about two thirds of cases, and it made no difference if the contact was made by telephone, fax, or e-mail.

When peer reviewers do not file reviews by the time requested, most journals contact them to urge completion of their review. How should such tardy reviewers be contacted? This study compared contact by telephone, fax, and e-mail with respect to effectiveness in prompting completion of the review.

The study was conducted in the main editorial office of Obstetrics & Gynecology, a monthly medical specialty journal, whose practice has been to send a manuscript to a potential reviewer with a request that the review be returned within 21 days of the date it was sent. When 28 days had elapsed since the original request, if telephone and fax numbers and e-mail address for the reviewer in question were on file and the reviewer lived no more than 4 time zones from Los Angeles, Calif, he or she was entered into the study. Using a computer-generated random number sequence, the tardy reviewer was assigned to be contacted by telephone, fax, or e-mail. Identical wording inquiring as to the status of the review and urging the report be sent by fax or e-mail was used in all cases. Telephone calls were timed to occur during the working day at the receiving end.

The main outcome assessed was receipt of review within 7 days of the contact. A secondary outcome was the number of days from contacting the tardy reviewer until a review was received among those who returned within 7 days.

Sample size was estimated a priori based on response rates of 50%, 50%, and 20%; to achieve 80% power at α = .05, 65 subjects were needed in each arm. We enrolled these numbers between January and July 1998, when analysis indicated underestimated response rate overall and overestimated difference among the 3 arms. Therefore, in November 1998, we reinitiated the study and continued it until June 1999 at which time the original sample size was essentially doubled. The χ² test was used to compare the proportions returning reviews and analysis of variance to compare the time for those returning reviews within 7 days.

The total number of enrollees was 383; 5 were excluded (2 reviewers claimed never to have received manuscripts and 3 claimed to have already sent reviews by fax or e-mail, but reviews were never received), for a final sample size of 378. The proportions who responded by producing a review within 7 days were virtually identical in the 3 groups (χ² = 1.102, P = .59): telephone, 85 (68%) of 125 (95% confidence interval [CI], 60%-76%); fax, 86 (67%) of 129 (95% CI, 59%-75%); and e-mail, 84 (67%) of 124 (95% CI, 59%-75%). Among respondents, the number of days from contact to receipt of a review also did not differ significantly (phone, 4.4 days; fax, 4.7 days; e-mail, 4.6 days; F = 1.49, P = .23).

Much has been said and written decrying the slowness of the traditional peer review system, but surprisingly little attention seems to have been given to the problem of the tardy reviewer. Bishop¹ states only that "journals should have a form letter to prompt those referees who have not sent in their reports on time."

Opinions differed among editors of our own journal and other editors we contacted with respect to how to prompt tardy reviewers. A telephone call is said to imply greater urgency and perhaps importance. A fax, it was argued, provides a tangible record and, if the recipient were unavailable, a fax might be seen by someone responsible for the recipient's messages. E-mail is a highly personal means of communication and has its enthusiastic devotees, and this, some contended, might increase the likelihood of a response. Our results indicate that, opinions notwithstanding, the 3 approaches did not differ in effectiveness at one specialty journal. Two thirds of tardy reviewers responded to the contact, irrespective of how it was made, by producing a review within 7 days.