Skip to main content
    • Aa
    • Aa

The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation

  • Domenic V. Cicchetti (a1)

The reliability of peer review of scientific documents and the evaluative criteria scientists use to judge the work of their peers are critically reexamined with special attention to the consistently low levels of reliability that have been reported. Referees of grant proposals agree much more about what is unworthy of support than about what does have scientific value. In the case of manuscript submissions this seems to depend on whether a discipline (or subfield) is general and diffuse (e.g., cross-disciplinary physics, general fields of medicine, cultural anthropology, social psychology) or specific and focused (e.g., nuclear physics, medical specialty areas, physical anthropology, and behavioral neuroscience). In the former there is also much more agreement on rejection than acceptance, but in the latter both the wide differential in manuscript rejection rates and the high correlation between referee recommendations and editorial decisions suggests that reviewers and editors agree more on acceptance than on rejection. Several suggestions are made for improving the reliability and quality of peer review. Further research is needed, especially in the physical sciences.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

R. N. Giere (1988). Explaining science: A cognitive approach. University of Chicago Press. [MEG]

H. O. Gulliksen (1950) Theory of mental tests. Wiley. [DL, LJS]

D. L. Hull (1988) Science as a process. University of Chicago Press. [LLH]

D. Kahneman , P. Slovic & A. Tversky , eds. (1982) Judgment under uncertainty: Heuristics and biases. Cambridge University Press. [HLR]

R. Rosenthal (1987) Judgment studies: Design, analysis, and meta-analysis. Cambridge University Press. [RR]

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Behavioral and Brain Sciences
  • ISSN: 0140-525X
  • EISSN: 1469-1825
  • URL: /core/journals/behavioral-and-brain-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 26 *
Loading metrics...

Abstract views

Total abstract views: 1030 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 27th June 2017. This data will be updated every 24 hours.