Introduction
Peer review is intended to support scientific integrity, correct errors, and democratize decisions about publication. Reference Aczel, Barwich and Diekman1 It continues to be the scientific control and gold standard of research validation in upholding quality, validity, and credibility of published work. 2
This standard relies almost entirely on the charitable contributions of scholars and experts who volunteer their time and expertise to maintain the quality of published papers through peer review. In providing this, peer review is best understood not as a process or transaction to achieve publication, but as a system of quality improvement with the intent of excellence in scientific rigor.
To maintain transparency and fairness, peer review should be embedded in a comprehensive journal Editorial review process. This can include components such as Editor review, statistical review, and language editing. This process can vary depending on the journal. A detailed overview of the Prehospital and Disaster Medicine (PDM) review process was recently published as an editorial. Reference Johnson and Franc3
The Role of the Peer Reviewer
Just as the review process varies between journals, so does the role of the peer reviewer. Many journals publish instructions for peer reviewers on their website and may include specific instructions about the peer reviewer’s role in the invitation email. For peer reviewers, spending a few minutes to read these instructions can save time and lead to a much higher quality review. For instance, the PDM instructions for reviewers listed on the journal’s website detail that all articles sent for peer review have already undergone initial screening by the Editor-in-Chief or another member of the Editorial Board to ensure that the manuscript aligns with the journal’s mission statement to publish innovative, high-impact, evidence-based research in Disaster and Prehospital Medicine. Prehospital and Disaster Medicine instructions also note that manuscripts accepted for publication will always undergo a thorough English language editing process prior to publication; thus, there is no need for peer reviewers to correct spelling, formatting, or grammar as part of their review.
Within the PDM review structure, the primary role of the peer reviewer is to assess the clarity, accuracy, and scientific validity of the manuscript.
Should I Be a Peer Reviewer, and if so, Why?
Given the time and effort related to providing a high-quality peer review as a service to the academic community, it begs the question of why scientists voluntarily choose to do so. While it is usually unpaid, it should be noted that peer review offers personal and professional advantages in the form of skill development, networking, and influence. Some of the benefits of engaging in peer review include:
-
1. Early insights into science and research related to your field. Reviewing allows you to see the latest in innovations well before they are officially published. This privilege places you at the cutting edge of emerging knowledge.
-
2. Personal improvement. The review and critique of work invariably lead to self-reflection of your own research practice, methodology, logic, and writing skills. Your personal blind spots are revealed and the opportunity to develop through peer review strengthens your academic endeavors.
-
3. Professional development. Peer review offers a practical approach to demonstrate your knowledge and skill, building your network and relationships. The act of peer review influences the quality of knowledge in your field and profession, ensuring that the highest standard of information is published, and gives back by supporting and developing authors contributing to this space.
A Practical Approach for Peer Reviewers
Reviewing a manuscript is a time-consuming process and there is no substitute for a careful and methodical approach. Although a single canonical method for manuscript review does not exist, there are critical questions pertaining to each section where peer reviewer comments have actionable value for the authors and editors.
Title
Does the title accurately reflect the research question and study type? Is the title comprehensible to the journal’s target audience?
Abstract
Does the abstract contain adequate detail to understand the key methods, results, and implications? Is the conclusion justified by the data presented? Are the findings reported in the abstract the same as those reported in the manuscript body?
Introduction
Is the research question clearly stated? Is a cohesive case made for why the research is important? Are the most relevant pieces of previous research properly cited? Is the primary objective clearly stated? Is the primary objective important?
Methods
Are the primary, secondary, and exploratory outcomes clearly defined? Have the methods been described clearly enough for others to repeat them? Are the methods valid? Are the methods described the best way to answer the research question? For animal and human subject research, is appropriate ethical approval cited? Is informed consent necessary and reported?
Statistics
Do the authors explicitly detail the test statistics used for each outcome reported in the methods? Are the test statistics chosen appropriate for each outcome?
Results
Do the authors report the results of all statistical tests reported in the methods? Are the results consistent with the methods? Are results presented clearly? Do tables and figures reflect the results?
Discussion
Have all research questions been addressed? Does the interpretation reasonably reflect the results? Is there reasonable comparison to previously published research? Has the potential impact been discussed?
Limitations
Does the paper contain a discrete limitations section that adequately and fairly addresses study limitations?
Conclusions
Are all conclusions supported by the data? Do the conclusions answer the research question? Do the conclusions avoid speculation? Do the conclusions avoid alluding to “future research?”
Ten Simple Rules for an Awesome Peer Review Report
The following are ten simple guidelines to consider when invited by a journal to perform a review:
-
1. Just Say No: If the manuscript is outside your area of expertise, or if you have a conflict of interest in reviewing the paper, it is always prudent to decline the invitation. Clicking the formal “decline” button and responding to the invitation is helpful for Editors as it signals to the Editorial Staff to search for other potential reviewers. Note that deadlines can often be flexible - contact the Editorial Staff if you likely need an extension.
-
2. Be Professional and Courteous: It costs nothing to be kind. Address the manuscript, not the authors or the research group. Try to be objective and provide specific areas for improvement.
-
3. Apply the Journal’s Guidelines: Always review manuscripts in the context of the journal. It can be confusing for authors if the peer reviewer’s comments contradict the journal’s guidelines. For instance, the PDM online instructions for authors give specific details about the use of P value reporting. When in doubt, it can help to review the online instructions for authors to ensure your review follows the journal guidelines.
-
4. Keep to Your Area of Expertise: Review the portions of the paper that are within your area of expertise. For instance, if you are not knowledgeable in the statistical methodology used in the paper, let the Editor know in the confidential comments to the Editor section and avoid making assumptions. Prehospital and Disaster Medicine Editors are required to send the unedited peer reviewer comments to the authors. In the event of a revision, authors will be required to directly address all comments. It can be very stressful and time consuming for authors to address comments that are structurally incorrect.
-
5. Avoid Use of Artificial Intelligence Tools: At PDM, Editors are interested in your professional opinion on the quality of the manuscript, not in what an AI tool thinks of the paper (which the authors could have easily done themselves if they wanted to). Furthermore, many submitted papers contain sensitive or proprietary information, and many authors may not be comfortable with their information uploaded to AI tools.
-
6. Avoid Requesting Further Research or Analysis: Generally, by the time papers reach peer review, the research has already been done. It is unrealistic to request changes to sample size, randomization procedures, or experimental methodology at this point. If the manuscript’s claims are not supported by the results of the study, it is those claims that require revision.
-
7. Skip the Grammar and Spellcheck: Most journals, including PDM, have a rigid final check of spelling and grammar prior to publication. Peer reviewers do not need to spend their valuable time correcting spelling, grammar, word count, reference format, or other style issues.
-
8. Avoid Recommending Your Own Research: Recommending your own research creates a conflict of interest that should be addressed by the Editorial Staff. At PDM, peer reviewers should contact the Editor-in-Chief directly in the confidential comments to the Editor section if they feel references to their own research are critical for the quality of the manuscript.
-
9. Give Private Feedback to the Editors as Needed: Comments such as “I’m not certain that this fits the journal’s scope,” “The authors use Fisher’s Exact Test and I’m not sure if that is correct,” “There have been several scoping reviews on the topic recently published,” or even “This article is really terrible” can be valuable private feedback to send to the Editors. Most journals, including PDM, have a dedicated section that allows private comments to the Editor that cannot be seen by the authors.
-
10. Make Sure Confidential Comments Reflect Comments to the Authors: If you do send private comments to the Editors, the comments should generally be reflective of those provided to the authors. Giving a series of positive comments to the authors and then privately telling the Editor that the paper should be rejected can make it very difficult for the Editor to provide accurate feedback to the authors regarding their manuscript.
The Stylish Review
Most journals do not enforce a specific style for peer reviewers. Some reviewers give numbered point-by-point instructions as a text file. Some use elegant prose. Others prefer to edit and annotate the manuscript file directly. Peer reviews are internal documents, and there is no need for peer reviewers to spend inordinate amounts of time obsessing over format, grammar, or style.
Peer reviewers should remember that authors will often be asked to provide a point-by-point response to the reviewers’ comments. Generally, clarity beats cleverness in these documents. High-quality peer reviews outline clearly what changes are needed. Providing each required change in its own sentence or list item makes it much easier for authors to address issues one-by-one. Use of section headings or line numbers for specificity can be helpful.
Along with clarity, reviews should be consistent. This is particularly important if the peer reviewer is asked to comment on the ultimate disposition of the article. Many journals ask peer reviewers to select a disposition such as Accept, Accept with Revisions, Revise and Resubmit, or Reject. It is very important that the comments to the authors give adequate justification as to why this decision was selected by the peer reviewer. It can be very confusing for authors when the reviewer selects the Reject disposition while comments to the authors are complementary and suggest only minor changes.
The Golden Rule of Peer Review
Creating a high-quality peer review report should be framed by an expectation that their own work will receive similar treatment. That is, to treat a manuscript as one would wish their own manuscript to be treated. With such an approach, reviewers carry a responsibility above that of just article review. Their feedback assists to refine arguments, filter error, and guide novel ideas and innovation in a clear manner to the public domain. A “best practice” peer review is more than a critique: it provides mentorship and support to early career researchers whilst maintaining rigor of scientific practice.
Acknowledgement
The authors would like to thank Dr. Manuela Verde for her editorial assistance.
Author Contributions
JMF, EJ, and JC contributed equally to the conception, writing, editing, and final review of all content.
Use of AI Technology
ChatGPT Version 5 (OpenAI; San Francisco, California USA) was used for assistance with writing; however, the authors remain responsible for all content.
Conflicts of interest
JMF is the CEO and founder of STAT59 and the Editor-in-Chief of Prehospital and Disaster Medicine. EJ is the Managing Editor for Prehospital and Disaster Medicine. Joe Cuthbertson is Associate Editor (Reviews) for Prehospital and Disaster Medicine.
