Quality reviews are critical for peer review.

As a co-editor of Political Analysis, I ask a lot from our reviewers, in particular that they read and evaluate complex statistical, mathematical, and computational research.  In many of our submissions, authors provide computer code, with detailed supplementary materials, and reviewers need to examine those materials.  Political Analysis reviewers are excellent. Most produce detailed evaluations, usually within two or three weeks of our request.  These anonymous reviewers provide enormous value to the papers that go through our review process; in many cases reviewers are asked to evaluate revisions of a paper.

While we try to not ask for more than a review or two each year from anyone, the reality is that we sometimes ask some of our best reviewers to read a number of papers annually.  That’s a lot of work.  My guess is that these same reviewers are also seen as good reviewers by other top journals in the field, so I’ll bet that some of these excellent reviewers might be getting many dozens of review requests annually.

There are also people in the field who simply are not good reviewers.  They don’t respond to review requests, they are tardy in returning reviews, or they just don’t write helpful and informative reviews.  While we can keep track of poor reviewers internally, we don’t currently have any way to know whether or not people who have never reviewed for our journal will be excellent reviewers.

It’s important to note that most major journals are published by a small handful of presses, and that we all use similar manuscript management platforms.  Which leads to the inevitable question — should journals share reviewer information?

While there are logistical issues that need discussion, in principle, I think that journals should share information about reviewers.  Most importantly, it would be very helpful if we knew if a potential reviewer is currently reviewing a paper for another journal, when we are considering selecting that reviewer.  We don’t need to know which journal they have agreed to review for, nor which paper, but just knowing that a potential reviewer is currently committed to review for another journal would be helpful.

It would also be helpful to know how many review requests a potential reviewer had received in the past year, how many they completed, and how long it took them to complete those reviews.  That way we can try to avoid overloading some people with review requests.

We frequently hear complaints about the peer review process, in particular that colleagues feel they get too many review requests.  Reviewer overload likely slows down the peer review process, and probably means that we are not getting the highest quality reviews that our authors deserve.  As journal editors, we need consider better coordination with our fellow editors at other journals, to try to make sure that we are not overly burdening certain colleagues with review requests, which we can likely achieve by sharing some limited information about our reviewers with other journals.

Michael Alvarez

Professor of Political Science, Caltech

Co-editor, Political Analysis