LEARNING OBJECTIVES

After reading this article you will be able to:

• • demonstrate a greater understanding of what the peer reviewing process entails

• • write a comprehensive review for a peer-reviewed medical journal

• • know where to go to for further information.

Peer review has been a formal part of scientific communication since the first scientific journals appeared more than 300 years ago. Philosophical Transactions, a journal launched in 1665, is thought to be the first journal to formalise the peer review process; it is also the first journal to be made freely available online (https://royalsocietypublishing.org/toc/rstl/1/1).

More recently, attempts have been made to establish peer reviewing guidelines. In 1978, a few editors met in Vancouver to establish guidelines for articles being submitted to their journals. This small group was named the Vancouver Group and continues to publish helpful advice and guidance in this area under the name of International Committee of Medical Journal Editors (http://www.icmje.org). This article uses some guidance from there as well as imparting knowledge and sharing learning from our own experience that should be useful for clinicians, academics, undergraduates and postgraduates in psychiatry.

Peer reviewing for medical journals is a richly rewarding experience benefiting the reviewer, author and journal. There appears to be a shortage of articles in psychiatry journals for those who have questions about the process and the details of peer review. This article aims to answer those questions and hopefully enthuse more people into becoming peer reviewers. It should be read in conjunction with our earlier article (Halder Reference Halder, Ramsay and Tyrer2011), which discussed ‘the nuts and bolts’ of how to do it. Since 2011, N.H. has adapted and changed the style of peer reviewing based on experience and feedback, and this is shared in the present article.

What is peer reviewing?

A peer-reviewed journal is one that publishes articles that have been checked by people who are ideally experts in the field and is available to the wider scientific community. Peer review is the critical assessment by unbiased and independent experts of manuscripts submitted to journals. It can be seen as an important extension of the scientific process and it aims to validate academic work.

Peer reviewers are people deemed suitably qualified to act as independent assessors of such articles. They will have a duty to the authors to try to improve the standard of the article by constructive feedback. It can be seen as a form of ‘quality control’ for the journal. Indeed, 82% in a survey agreed with the statement that ‘without peer review there is no control in scientific communication’ (Ware Reference Ware2016).

The reviewer will also have a duty to the journal to help the editor decide whether or not an article should be accepted (albeit with some amendments). Ultimately it is the editor who makes the decision to accept, revise or reject a paper.

The key stages of the peer review process are outlined in Box 1.

Why do peer reviews?

The new decade has brought with it people in growing numbers who appear to be sceptical or even dismissive of the notion of evidence-based information, whether that is reflected in growing numbers of so-called antivaxxers (Hussain Reference Hussain, Ali and Ahmed2018) or climate-change deniers (Gross Reference Gross2018) to name but two. It has therefore never felt more important a time to push back against this narrative and one part of doing that is either to engage in research or to act as a peer reviewer to ensure high quality publications.

There are a number of reasons to engage in the peer review process (Box 2). For one, it enhances critical appraisal skills. This helps to avoid losing the great skill that, for many, peaks at the time of their professional membership examinations. Some consultants mention a desire to engage in something different from their day-to-day routine. Not only does peer reviewing break up the clinical work, but it also adds a new skill set to consultants’ repertoire. It is something that can then be added to a CV and subsequently help in future job interviews. From a research perspective, reviewers benefit from reading about new research at the earliest stage and a chance to shape the final product. Simply having more knowledge about how and what reviewers look for increases the chances of having their own work published in future, as they will know what to look out for. As a general rule, it is desirable for peer reviewers to maintain an interest in a few specific areas of research where they have special knowledge.

Arguably most importantly, peer reviewing can be quite enjoyable. Part of the enjoyment could be the internal satisfaction of being part of an important process of quality control and making a contribution to the scientific process, ultimately benefiting their medical colleagues.

An added bonus, particularly for trainees and new consultants wishing to enhance their CV, is that many journals are now registered with Publons (publons.com), a free website that provides a service to track and verify the work of peer reviewers.

The most popular reasons for participating in peer review reported in one survey (Ware Reference Ware2016) related to social factors (playing a part as a member of the community was endorsed by 93%, and reciprocating others’ reviewing work by 75%) and intrinsic factors (enjoy helping improve the paper: 83%; enjoy seeing work ahead of publication: 72%). Instrumental or self-interested reasons were much less cited (e.g. to increase the chance of future acceptances: 16%; to increase the chance of a place on the editorial board: 24%; to enhance reputation or further career: 42%).

Who should do it?

The answer to this is simple. Anyone can become a peer reviewer. All consultants can make a contribution, given the experience they have already gained in reaching this level in their career. However, trainees and even undergraduates can also make meaningful contributions. If anyone has completed a masters or PhD, it is likely that they will know more about the subject matter than many others. They can put themselves forward to be a peer reviewer for that subject. Trainees can certainly comment on a wide variety of issues, including education- and training-related subjects, having been ‘experts by experience’. A growing number of medical student journals are peer reviewed, when appropriate, by other medical students. This will help home in on skills from a very early stage, which will continue to improve with experience and time.

How to become a reviewer

You can contact the editor or the administrator of a journal in the first instance. All journals have email addresses or contact details on their website or inside the journal itself. As a minimum, the details that you should include in this first contact are your qualifications and experience, followed by your areas of expertise and interests (with details why you consider this to be so). If you have any previous publications these should be listed. You should also provide your CV. If accepted, your name and details will be placed on a database so that the editor can pick you if a submission that matches your expertise or interest comes in.

There are many hundreds of peer-reviewed medical journals available and it may be difficult to choose which one or ones to be a peer reviewer for. There are different ways to choose. For example, you can simply choose journals that you enjoy reading. Others choose by looking at affiliated bodies (e.g. only choosing journals that belong to the Royal College of Psychiatrists or another body). Others choose journals that tend to take submissions from a particular group, such as medical students or trainees. Some search by looking at the impact factor, which is considered to be a proxy measure for the quality of a journal (although not all peer-reviewed journals have an impact factor). Or you can simply list your areas of expertise and search on the internet for the journals that most closely match those areas. Readers are sometimes surprised at how accurately their interests are matched with a particular journal. As an anecdote, N.H., currently working with people with brain injuries, searched for journals on neurology, neurosurgery and psychiatry, as his work involves liaising with these groups of people. The first search item located was the Journal of Neurology, Neurosurgery and Psychiatry.

What to do if invited to review?

In some cases, you may be asked by a journal to review a paper after they found your details on a database such as PubMed or via a recommendation. (If, however, you have not reviewed a paper for a long time, you may be removed from the database of reviewers at the discretion of the editor.)

The Committee on Publication Ethics (COPE) has published helpful guidelines on things to look for before agreeing to take on a review (Committee on Publication Ethics 2020). These include checking that you have the necessary expertise to assess the manuscript. Note that if you would like to do the review but do not quite feel experienced enough, some journals would accept a review co-signed by your supervisor. Other important issues include declaring any conflicts or competing interests and only accepting an invitation if you can meet the deadline or have agreed an extension with the journal. Unfortunately, and sadly increasingly, you also need to check that the journal is legitimate. There has been an increase in so-called ‘predatory journals’, where certain open access journals exist for revenue rather than scholarly activity and often send out spam emails to solicit contributions with little concern for quality (Kisely Reference Kisely2019).

It is important to note that you do not have to ‘accept’ all requests for peer reviewing. Clinicians go through phases of being busier at certain times and there is no pressure to accept all requests for reviews that you receive. When this arises, you may be given the option to nominate or suggest other suitable reviewers for the paper. This is much appreciated by editors, who often find it difficult to get reviewers for highly specialised subjects.

How long does it take?

Of course, this is subjective and dependent on a number of variables. With more practice the time taken can decrease significantly. When N.H. first started peer reviewing as trainee editor of the Psychiatric Bulletin, the process could take a weekend of dipping in and out. Now it can take an average of about 2 h. This is in keeping with findings from a study in the British Journal of Psychiatry (Walsh Reference Walsh, Rooney and Appleby2000), which reported that the mean time taken to complete an open review (where reviewers’ names were published) was 2.05 h and 1.65 h for masked (‘blind’) reviews (where reviewers’ names were not known to the authors).Footnote † However, another survey (Ware Reference Ware2016) found the median time per article spent reviewing was 5 h, with a reduction in the mean time needed per review with increasing age from the under-36s (9.6 h) to the over-65s (5.8 h).

You must resist the temptation of skim reading to save time. Authors have invested a significant amount of time from their busy schedule to complete the work that sits in front of you. It is crucial that you take enough time to give the authors the respect they deserve. You can, however, save time by only producing one review for both the authors and the editor, as these versions can be the same, with some very minor adjustments.

How do you approach a peer review?

Everyone may have their own way of approaching a review, and if it works for them then the advice would be to stick with this. Here is a suggested method. If you are not familiar with the journal, read the its aims and scope and instructions for authors to get an idea of what it is looking for. Once the full article is obtained, read it through once completely without breaks. Resist critiquing on the first read through, tempting though this may be. After short break, a second read through could be accompanied by written notes or questions that arise as you read the article. N.H. tends to print out the paper and write in the margins or between the lines. You may then write a first draft of the review (one way of setting this out is given in the next section). It is helpful then to leave it overnight or even longer. This allows you to organise your thoughts or formulate new ideas to help improve the article. Some say that if you go back to a review after a period it can sometimes appear that you are reading it with ‘fresh eyes’, generating fresh ideas. These can be incorporated into a revised draft. Finally, read through your review thoroughly to make sure it makes sense and does not itself have typos or grammatical errors. It is embarrassing to be critical of the authors’ poor grammar when your own writing is error-laden. For the review that the authors will see, you need to make sure that it addresses the main objective of improving the paper in a way that is encouraging and constructive. For the review for the editor, you have to ask whether this review helps the editor to make the decision about publishing the paper in their journal. Once you have completed and submitted your review you should destroy or delete any copies of the original article.

What do you write in a peer review?

Although a template was given in our earlier article (Reference Halder, Ramsay and TyrerHalder 2011), N.H. has changed his approach to laying out a review, in light of experience and feedback. The original article advocates for a summary, followed by major points and then minor points. It is now felt more helpful for all parties to lay out the review in the order of how the authors have written their paper, using the subheadings that they use. This makes it easier to follow and appears to be logical. A summary is still useful to both editors and authors, but any recommendations about whether to reject or accept a paper should only be sent to the editor and not authors. Box 3 outlines a revised template for writing a review, and we discuss each element in more details in the rest of this section.

How to write a review when the paper is seriously substandard

Because the peer review process includes confidential comments to the editor that will never be seen by the authors (unless they go so far as to request information via the Freedom of Information Act) the frank opinion of a very poor paper should be made primarily to the editor. The comments to the authors should be robust, accurate and relevant, but not rude. The first paper one of us (P.T.) sent many years ago to the British Journal of Psychiatry received an angry response, ‘Why on earth is this registrar carrying out research at his age when he should be carrying out his clinical duties?’ This now would not be tolerated.

A very poor paper will naturally tend to receive a shorter report than a better one.

Controversies in peer review

Journals that do not operate a peer-review process tend to be considered of not high quality because of the risk of editorial bias and, at worst, fraud. A survey revealed that, in addition to detecting the best manuscript for the journal, growing numbers of people believe that peer reviewers should also be looking to detect fraud (i.e. the results given are falsified) and plagiarism, although acknowledging difficulties in actually doing this (Ware Reference Ware2016). Indeed, one study found that, out of 400 consecutively submitted manuscripts to a major specialty medical journal, 17% of submissions contained unacceptable levels of plagiarised material, with 82% of plagiarised manuscripts submitted from countries where English was not an official language (Higgins Reference Higgins, Lin and Evans2016). However, to detect fraud, reviewers would potentially need to check and verify the raw data of a study, which could be impractical and discourage prospective reviewers from participating in the peer review process (Glonti Reference Glonti, Cauchi and Cobo2019). Some may argue that peer review should not involve ‘policing’ other people's work, especially when there is sophisticated software available, such as Turnitin (turinitin.com), that can be tasked to do this. The Royal College of Psychiatrists has recently formed a Research Integrity Group and this will be focused on, among other things, the role of reviewers in spotting fraudulent, plagiarised or poor research.

There is an ongoing debate whether peer reviewers should engage in copy-editing. Although the majority of articles examined stated that copy-editing does not fall within the duty of peer reviewers, several articles specifically mentioned that reviewers should offer grammatical and linguistic improvements (Glonti Reference Glonti, Cauchi and Cobo2019). Many journals have their own copy-editing team, who are specifically trained to identify and address such aspects of the manuscript. Reviewers may argue that their time should be focused on the improvement of scientific content rather than the linguistic fine-tuning.

Where else can I find help as a reviewer?

There are guidelines freely available online for almost all types of research (Box 4). These can be consulted to help inform your review, and they are especially useful if you are struggling to benchmark what has been written against a standard. The International Committee of Medical Journal Editors (ICMJE) provides guidelines for biomedical journals in general (http://www.icmje.org).

COPE produces guidelines on the peer reviewing process from an ethical standpoint, including what to do if you suspect fraudulent practices (https://publicationethics.org/peerreview).

The Royal College of Psychiatrists runs annual or biannual 1-day research methods courses. There are hands-on and practical, facilitated by experienced reviewers. These are advertised on the College website and in the newsletter.