Don’t be cruel: how to write a fair peer review report

Publications in recognised academic journals undergo a rigorous peer review process. A key player in this process is the peer reviewer, who is a recognised expert in a research area or in using a specific research methodology. To inform journal editors’ decision-making, a select number of peer reviewers each write a report on the strengths and weaknesses of the submission. As well as evaluating the quality of the submission, in their report peer reviewers make suggestions for the author to consider.

What is a peer review report?

A peer review report is most often a stand-alone document that contains comments focusing on sections of the manuscript. Sometimes peer reviewers can use the “track changes” and “comments” functions of Microsoft Word to insert in-text comments (provided that their identities are hidden in double-blind peer review). They attach the marked-up document as their report. We have also seen cases of reviewers doing both.

How should I structure a peer review report?

There is no standard structure for a peer review report. Some journals include in-house peer review templates; we have seen journals ask reviewers to respond to questions about methodological rigour, relevance and originality. The way peer reviewers structure their report also depends on whether it is a first-time or revised submission. For a new manuscript, the peer reviewer is more inclined to comment more holistically on the submission. For a manuscript that has undergone revision (that is, manuscripts that have received a major-revision or minor-revision decision), the peer review report is likely to focus on author’s revisions.

• Understanding peer review: what it is, how it works and why it is important
• Peer review is frustrating and flawed – here’s how we can fix it
• Great citations: how to avoid referencing questionable evidence

From our analysis of our own peer review reports in a published study, a peer review report usually has a short introductory paragraph summarising the reviewer’s understanding of the manuscript. In this paragraph, the reviewer may clearly state their recommendation (for example, major revision) and general observations.

The introductory paragraph is usually followed by bullet points or short paragraphs focusing on various sections of the manuscript. Some reviewers divide their comments following the section headings of the manuscript, while others include specific observations and suggestions in no particular order. Regardless, it helps authors to easily act on the feedback if reviewers make clear the location of the changes requested (page, paragraph or even line number).

In some cases, a reviewer ends their report with an encouraging tone, especially when they recommend major revisions or a rejection, motivating the author to address the comments or submit their revised work elsewhere.

What should I focus on?

Most journal manuscripts are primary studies. So, peer reviewer comments will be expected to focus on the quality of research. Research quality can be exemplified in several ways: the author’s understanding of the topic or field and engagement with the extant literature, the impetus and originality of the study, rigour of how the research was conducted, insights from findings, and importance of the study. The focus of the peer review report may change slightly for other types of submission, such as a systematic literature review, a commentary, a conceptual paper or a theoretical paper.

A journal may specify additional areas that peer reviewers should focus on. For instance, if a journal focuses on a niche research area, a peer reviewer may be asked to comment on the compatibility between the manuscript topic and the remit of the journal.

When reviewing a manuscript, always remember to focus on what the submission is, not what the submission could have been. While a peer reviewer is tasked with assessing the quality of a submission, they should refrain from including comments that reflect their personal view or bias towards a topic or research method.

How long should a peer review report be?

Those new to peer review often ask how long their report should be. But this is not a question that can (or should) be answered definitively. For interest’s sake, reports analysed in a study conducted by Publons were on average 477 words. Reviews for articles in medical journal BMJ averaged about 530 words. Our reports included in a recent study average about 1,000 and 750 words respectively. The lack of consistency reveals considerable variation among reviewers, disciplines, research designs, and the quality, fullness and clarity of original submissions.

In our case, the recommended outcome largely influences the length of our individual reports. So, short reports of fewer than 150 words are those that recommend acceptance after previous rounds of review. The longest reports recommended major revisions or rejection. These reports not only detail limitations in the submission but also provide explicit advice for improving the manuscript and encourage authors.

So, don’t place too much emphasis on report length – rather, focus on the purpose of each part of the review.

How can I ensure the peer review report is constructive?

Many researchers have felt the sting of feedback that is unnecessarily cruel, personal or degrading. Such comments can be devastating on a personal level, particularly for those in their early career, but they also do nothing to progress scholarship. It is possible to highlight weaknesses and errors in studies and manuscripts while being constructive. Not all things that are written in a peer review report will be positive, that is the nature of critique, but it is important to remember that behind every manuscript are one or more authors who have invested considerable time and effort into the submission. The report they receive will have a professional as well as a psychological impact, and so empathy is advised. 

As you write a peer review report, ask yourself three questions.

1. What am I focusing on? The focus should never stray to value judgements about the research or the author/s.
2. What do I want the author to do? Even if your recommendation is to reject a paper, the feedback is provided not only to justify your decision but also to help the author. Consider what actions you would expect the author to take based on your comments.
3. Is there another way to word the comment? We suggest going back to a report after a day or two (before sending it to the journal) to consider this question. A slight change in wording can be the difference between cruel and constructive.

What other things should I consider?

Humility is important in peer review. Reviewers are chosen based on several factors, and some parts of a manuscript might be outside your area of expertise or knowledge. It is more than acceptable to make a note to the editor if you aren’t able to comment on, for example, a statistical test that you are not familiar with. The editor will then ensure this can be covered by other reviewers.

If you have been asked to review a manuscript, it is because you have expertise in the topic and/or the method employed. It is likely you also have publications that are relevant to the paper you are reviewing. In such cases, it may be appropriate to suggest your own work to the authors, with caveats. First, critically analyse whether your work is relevant and could add something new to the paper. If it is, it is appropriate to bring the work to the author’s attention, but it is unethical to make requests or requirements of citation of a particular paper in your report. 

The role of reviewer is different from that of the editor, who will make the final decision based on critical engagement with suggestions from all reviewers. The role is also pointedly not that of proofreader; this comes at a later stage. Use of non-standard English should not impact a reviewer’s evaluation of the rigour of the research. “Good” grammar, spelling or punctuation does not correlate to “good” research (nor the inverse). Asking the authors to proofread their work to aid comprehension is fine. Using it as a reason for recommending rejection is not. Asking for a “native speaker check” is also not only misguided but serves to uphold English linguistic hegemony in scholarship. The fact that researchers around the world engage in scholarly publishing in a language other than their mother tongue should be applauded but not penalised.

Sin Wang Chong is associate professor in language education at Moray House School of Education and Sport at the University of Edinburgh. He is associate editor of two SSCI-indexed journals, Innovation in Language Learning and Teaching and Higher Education Research & Development and a senior fellow of the Higher Education Academy.

Shannon Mason is associate professor in education in the Faculty of Education at Nagasaki University, Japan.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.