JSR Reviewer Guidelines

The Peer Reviews

The peer reviewers and Associate Editor play a critical role in helping determine which manuscripts get published and the amount of revision that is requested. The timeliness of the peer reviews and the amount of revision suggested also contribute to the amount of time that will pass between a manuscript’s initial submission and its eventual publication date. In spite of their importance, we’ve learned that peer reviews can be uneven in quality and in their apparent purpose. Therefore it is the editors who must make the final evaluation of a paper’s fate, and that decision will not always follow the recommendation of one or more of the reviewers.

In terms of quality, the best reviews are those that are thorough, constructive, polite, and honest in expressing the reviewer’s opinion of a manuscript's strengths and weaknesses. It is important for the reviewer to identify both the good and bad points of a paper, and to support and justify all their comments and recommendations. Neither authors nor editors are helped if changes are suggested without an explanation of how or why (this is especially true for the comment, “the paper can be shortened by x%”).

In terms of purpose, the reviewer must remember that he/she has a responsibility to both the editor and author. Both will benefit from a candid appraisal of the manuscript. The goal is two-fold —an analysis of the contents and presentation of the paper and a summary recommendation. In the analysis, the reviewer should assess the following:

• Does the paper articulate an issue of broad significance or is it cast only in parochial terms?
• Are the data separate from interpretations?
• Are the data sound and do they justify the interpretations?
• Is the reasoning sound and logical?
• Are the conclusions plausible?
• Is the paper focused?
• Do the presentation and organization make the paper easy to read and understand?
• Are the figures clear and are all necessary to make the author’s point?

With all reviews it is also very important that the process is as constructive as possible. A key purpose of the review is to try to identify how work can be refined and improved, so if reviewers identify problems with research described in a manuscript, they should also try to suggest practical ways the problems could be addressed. JSR and SEPM considers this to be particularly important when reviewing manuscripts from early-career researchers; in these cases, the review process should be seen as a mentoring process, and as much constructive and practical advice offered, so the authors can benefit as much as possible from the reviewer’s experience and knowledge.

In many reviews the most difficult task is settling on a summary recommendation. If the paper meets all the requirements listed in the bullet points, the decision to accept is straight forward. If it meets most of them, acceptance with instructions for minor, or perhaps more major revisions, to meet the rest of the criteria, is also often also a straight forward decision. And if the paper meets none of these criteria, the decision to reject it in its current form should be straight forward too.

If a review is identifies many problems with a manuscript it is important that the recommendation follows obviously from the content of the review. For example, when several pages of very critical comments suggest negative answers in the above list, it would be inconsistent to then suggest “accept after major revision”; reviewers should be prepared to reject manuscripts in their current form when they cannot see how revision could fix the issues identified, for example if there is a need for substantial new data, reinterpretations, and/or a complete restructuring of a study.

Finally, reviewers should not contact authors of manuscripts directly. We consider this to be somewhat unprofessional. All review comments should be channelled through the official review process, so that the AE and the editor can provide authors with a balanced summary of reviewers comments, not just the views of one person. If a reviewer thinks further contact with authors would be useful, s/he can highlight this in comments to the authors who can then make the choice themselves if they want to make contact to discuss the review further.