Peer review is a critical element in the editorial process at Archives of Pediatrics & Adolescent Medicine. The goals are to provide expert advice to the authors regarding their work, a check on the scientific validity of the data and methods, and information to the editors for use in their decision about the suitability of the paper for publication in the ARCHIVES.
The quality of journals rests, to a large degree, on the quality of the peer review process. Like nearly all scientific journals, the ARCHIVES relies on a volunteer pool of talented individuals who take time out of their busy week to review manuscripts. The journal could not be published without their help, and we are extremely grateful for their efforts. However, reviewers are rarely given any advice, by their senior colleagues or anyone else, on the best way to conduct reviews. Our purpose in this brief article is to provide some suggestions for reviewers so that their time can be spent producing a review that will be helpful to authors and editors.
In approaching a review, keep in mind the Golden Rule: "Do to others as you would have them do to you." We prefer to approach a paper as if it were written by a colleague, and we try to give the authors suggestions that may improve their research design, their analysis, or their writing. Even if you think that the manuscript should be rejected, your suggestions may improve the next effort by these authors. Keep in mind how painful it is for authors when their work is rejected; avoid comments that are mean spirited. If you think that the manuscript is awful, by all means tell the editor, but try to be polite to the authors.
On the other hand, do not mislead authors. We prefer that you do not make statements about rejection or acceptance in your comments to authors because the final decision is not the prerogative of reviewers. However, you should not hesitate to point out problems with their study or their writing. If you think that something is wrong or poorly done, say so and try to tell them how they can fix the problem.
A little humility may help. Most of us have received rejections with harsh remarks, and more than once we have had a manuscript rejected and thought the reviewer was an ignoramus. Consider the possibility that what the authors have done or said is correct, even if it seems doubtful to you at first reading. If the manuscript is unclear, tell the authors that you could not grasp their meaning; only the most defensive author would not appreciate that if a thoughtful reviewer could not understand a sentence, it will not be clear to the average reader.
The more specific your advice, the more it may help the authors. For example, if you think that they have not adequately discussed the relevant literature, try to give them specific citations; just referring to "the paper by Smith" is not helpful. If you think that the authors should use a different statistical test, can you refer them to a textbook that might give them the details they need? We keep reference lists on bibliographic software for assistance in preparing our own articles. With this kind of software, it is easy to offer authors appropriate citations.
How long should a review be? Rarely is a single sentence or paragraph sufficient. Although authors may get a brief thrill from a 1-sentence review that says the manuscript should be accepted without any changes, we believe that nearly every manuscript can be improved. A very short review suggests to us that little review was actually done; similarly, a 1-sentence condemnation offers little help to the authors. A page or two may suffice for many reviews, whereas other manuscripts may require 10 pages.
It is not the job of the reviewers to rewrite the manuscript, but it is certainly permissible to comment on grammar and writing style. We have wonderful copy editors who work with the ARCHIVES and can substantially improve the quality of the writing. Many reviewers, however, have excellent editing skills. If you find that a sentence, phrase, or paragraph seems unclear or awkward to you, bear in mind that it probably will not seem any clearer or less awkward to other readers. The editors will appreciate any comments that may improve the writing. It is not necessary to point out to the authors of international submissions that English is not their first language; they (and the editors) know this. Offer them whatever help you can. If their article has scientific merit and is accepted, our copy editors will work to make it readable.
Treat the manuscript with the confidentiality that the authors have entrusted in the journal and the editors have entrusted in you. It should not be copied and shared with colleagues who might find it of interest. Asking a colleague to help with the review because of special expertise is certainly acceptable. This person should follow the same confidentiality requirements, and his or her work should be acknowledged in the information sent back to the editors.
When writing a review, start with general comments about the manuscript before moving on to specific parts of the paper. Every study has flaws. The question is how much the flaws bias the results and whether any of them can be fixed. Are there fatal and irreparable flaws in the paper that affect the validity of the work to such a degree that the results are useless? If so, let the editor know, and point out your concerns to the authors.
Is the paper concise, or could it be shortened to make it more readable? Do some parts need to be expanded to make it clearer?
Does the title accurately describe the study, or is it misleading or incomplete? Is the abstract informative enough that it can stand alone as a summary of the study? In particular, are the main numerical results of the study given in the results section of the abstract? Is all of the material in the abstract presented in the paper? Are the numbers in the abstract the same as those in the manuscript? Are the abstract's conclusions appropriate?
Do the authors succinctly present a rationale for the study? Do they clearly state the study question?
The reviewer should pay close attention to the methods section and understand what the authors did. Does the study follow a known study design? Did the authors follow the principles of this design? If you have specific references on methods or design that you think the authors should include, it is tremendously helpful to provide that information. The ARCHIVES asks authors of randomized controlled trials to follow the CONSORT guidelines1 in reporting the results and to complete a checklist.2 These guidelines have been designed to improve the completeness and quality of the reporting of such trials and to make future meta-analyses more feasible.
At the ARCHIVES, we are very conscious of the ethics of research as well as potential conflicts of interest. We wish to see evidence in all papers that an appropriate institutional review board evaluated the study. Do the authors clearly state the procedures for obtaining informed consent when the study involves patients? For industry-sponsored studies, is it clear what role was played by industry personnel, either in the methods section or in the acknowledgments?
We have a statistical consultant as part of our ARCHIVES editorial staff. That consultant or one of our peer reviewers with biostatistical training is asked to review manuscripts with complex methods or analyses. Peer reviewers should comment on the methods and analysis used and let us know if they feel that someone with statistical expertise should review the paper. Keep in mind that you, as a reviewer, are likely to have more expertise than the average reader. If you are left unclear regarding the study design or analysis, it will likely seem this way to others. A reasonable approach may be to tell the authors that you are not sure what they did and then ask them to try to explain their methods in a different way.
Are the results presented clearly? Do they too often repeat information that is included in the tables and figures? Did the authors account for all participants in the study? Are the tables and figures understandable and well labeled? Do they display the data in the clearest way possible? Are the numbers consistent in the abstract, results, tables, and figures? It is often useful to add up the columns in a few tables to make sure that they are correct.
Did the authors relate their study to prior work in the literature? Is it clear how this study supports or disagrees with previous ones and why? Did the authors acknowledge the limitations of their study and discuss how these might affect the results and their interpretation? Are the conclusions justified by their data?
Are the references appropriate for the statements they are meant to support? Are they up-to-date, or is more recent information available?
The editors want your advice about the fate of the manuscript. Should it be (1) accepted in its present form; (2) accepted after modest revisions; (3) reconsidered for acceptance after major changes; or (4) rejected at this stage? Several considerations should go into this decision. Do you think the data and methods are such that the results might be credible? Keep in mind that even if the results seem surprising or counterintuitive, we generally judge a study by the quality of its data and methods, not by the final result. Although results may confirm or dispute our prior beliefs, they cannot be used to measure the quality of research. Ask yourself the following question: Would your view regarding rejection (or acceptance) change if the results were the opposite of what was reported? If so, perhaps you should reconsider your thoughts about the paper.
How new and important is the information in the manuscript? How much does the paper add to the field? Even if a few articles have already reported the same finding, confirmation of that finding in a new location or population or by a new method may be a useful addition. You were chosen to review the paper because of your particular expertise. Editors rely on peer reviewers to tell them whether a study is an important contribution to the literature. If you think that the article is important, controversial, or for some other reason deserves an editorial, let the editor know.
All journals try to process manuscripts as rapidly as possible. Many authors know which journals have a relatively rapid turnaround time and which are black holes that will swallow a manuscript for many months. Most of the processing time for manuscripts is related to the peer review process. An editor's nightmare is a reviewer who keeps the manuscript for a month and, when called about the review, declines to do it. This may result in substantial delays in providing a decision to the authors. If you find that you cannot do a review because of time commitments or other reasons, please let the editorial office know right away (preferably by telephone or e-mail). This helps everyone involved. When you accept a manuscript for review, we suggest a 3-step process. First, read it soon after you receive it. If you are going to do the work, you might as well do it sooner. You would like your own work reviewed rapidly, so why not do so for others? You might then return to the paper in a day or so, read it carefully, and write your review. Finally, we suggest that you print out your review, read it, and edit it, perhaps after letting another day go by. Most of us take a lot of care with our own manuscripts; after all, when they are published, our name is there for all to see. Reviews, however, are anonymous and carry little reward, which may make us give in to sloppiness. Edit your work to be clear and to remove unnecessarily harsh language. Give a final consideration to what you say; remember, authors much like yourself are anxiously awaiting your thoughts. And use your spell checker to set a good example to the authors.
Reviewers may have a conflict of interest in reviewing a manuscript. In general, you should not review a manuscript from a close colleague at your institution. If you have a financial conflict of interest such as owning substantial equity interest in a company that makes a drug tested in the study, please declare that on the form for the editors. This does not preclude an individual from reviewing a paper, but it allows the editor to weigh this information when considering the review.
Being a good reviewer is, like writing and editing, a skill that improves with time. We believe that reviewing has many benefits:
Improving a manuscript is a service that many authors appreciate. Both of us can recall receiving thoughtful and helpful reviews regarding our own work. We are grateful to those who took the time to enhance our papers.
Improving published articles performs a service to readers.
Good reviews may attract investigators doing quality research back to the ARCHIVES, improving the quality of the journal.
Reviewing enhances our own skills as investigators and authors. It helps us improve the design of our own studies and the quality of our writing.
The next time you are struggling with a review, try to step back for a minute. The ultimate goal of biomedical publishing is to prevent illness, improve care, and promote health. Thoughtful peer review is a critical part of this process.
Corresponding author: Frederick P. Rivara, MD, MPH, Child Health Institute, Suite 300, 146 N Canal St, Seattle, WA 98103-8652 (e-mail: firstname.lastname@example.org).
Cummings P, Rivara FP. Reviewing Manuscripts for Archives of Pediatrics & Adolescent Medicine. Arch Pediatr Adolesc Med. 2002;156(1):11-13. doi:10.1001/archpedi.156.1.11