Examples of problems in peer review

Allen et al., raised concerns about inaccurate feedback provided in reviews, arguing that accuracy is not evaluated in developmental review. However, accurate feedback contributes to the perceived usefulness of feedback (Brett & Atwater, Reference Brett and Atwater2001) and skill acquisition (Brand et al., Reference Brand, Novak, DiGennaro Reed and Tortolero2020). In fact, effective developmental review requires reviewers to recognize that they are not all-knowing entities; thus, there is no reasonable justification for them to make unsubstantiated, inaccurate statements (Ragins, Reference Ragins2015). Although we may not reward reviewer accuracy, because accurate reviews contribute to author development, the argument that developmental review contributes to inaccurate reviews is tenuous at best.

Allen et al., also called for reviewers to “refrain from suggesting [questionable research practices (QRPs)]” (p. 18). We agree that peer review has played an unfortunate role in the prevalence of QRPs. For example, scholars have suggested that the exclusion of certain variables or results from reporting may be influenced by editorial direction (e.g., Crede & Harms, Reference Crede and Harms2019; Fiedler & Schwarz, Reference Fiedler and Schwarz2016). Researcher justifications for QRPs (e.g., reviewer expectations, fear of potential rejection) have been positively associated with engagement in QRPs (Latan et al., Reference Latan, Chiappetta Jabbour, Lopes de Sousa Jabbour and Ali2023). Even when reviewers don’t directly suggest QRPs, comments such as those critiquing null results have the unintended consequence of indirectly suggesting QRPs by reinforcing beliefs that unsupported hypothesis tests are unpublishable (Banks et al., Reference Banks, O’Boyle, Pollack, White, Batchelor, Whelpley, Abston, Bennett and Adkins2016). However, as Allen et al., also noted, in recent years, the field of industrial and organizational psychology has been focused on reducing engagement in QRPs. In fact, Aguinis’s (Reference Aguinis2025) recent research methodology textbook devotes considerable attention to the problem of QRPs. If developmental review is focused on authors’ growth, it should emphasize engaging in open science practices and eliminating QRPs. Although peer review may be influencing QRPs, the argument that developmental review is driving this relationship seems unlikely.

Allen et al. (p. 7) also claim that developmental review drives reviewers to inform researchers how they should have conducted research, “rather than [engage] with the research as conducted.” This argument contradicts guidance that in developmental review, reviewers are not ghostwriters (Ragins, Reference Ragins2015) and should be forward-looking in identifying solutions to problems (Carpenter, Reference Carpenter2009). Therefore, this concern seems to be the consequence of reviewers deviating from, rather than engaging in, developmental review. Collectively, these arguments question whether developmental review motivates the wrong reviewer behaviors or if contextual constraints beyond the trend of developmental review (as outlined below) prevent reviewers and editors from engaging in meaningful developmental review.

Constraint #1: Time pressures and performance metrics

Action editors (AE) often juggle multiple manuscripts at different stages of review, in addition to their own academic responsibilities (Aguinis et al., Reference Aguinis, De Bruin, Cunningham, Hall, Culpepper and Gottfredson2010; Aguinis & Vaschetto, Reference Aguinis and Vaschetto2011; Starbuck et al., Reference Starbuck, Aguinis, Konrad, Baruch, Baruch, Konrad, Aguinis and Starbuck2008). In fact, our look behind the scenes using data provided by a large academic publisher shows that AEs at top business journals handle an average of 44 papers per year, with annual workloads ranging from a manageable 13 to as many as 101 manuscripts. This volume limits capacity to verify reviewer claims or reconcile conflicting feedback. Thus, the AE’s role risks shifting from independent evaluator to aggregator and synthesizer of reviewer input. In practice, this compromises one of the primary safeguards the system is supposed to offer: a layer of editorial oversight that contextualizes, filters, and validates reviewer feedback. This dynamic is further reinforced by an emphasis on turnaround times over the accuracy or rigor of decision letters through embedded metrics such as days to first decision and overall production speed. Such metrics implicitly reward “moving things along” over deeply engaging with the content of reviews and send a signal to editors and authors alike that speed is the dominant measure of good editorial practice. Although efficiency has obvious practical value, prioritizing it can have unintended consequences for the integrity of editorial work that can result in less rigorous research becoming a part of the scientific record, whereas more robust contributions are not published (Adam, Reference Adam2025; Palayew et al., Reference Palayew, Norgaard, Safreed-Harmon, Andersen, Rasmussen and Lazarus2020). In a high-volume, efficiency-focused model, AEs are not positioned to fully engage with manuscripts and/or reviewer feedback.

Constraint #2: Reviewer incentives and availability

There is little tangible motivation for unpaid reviewers to devote the considerable time and care required for a high-quality review. This incentive gap likely narrows the reviewer pool and leads to overreliance on a small group of reviewers whose comments often shape outcomes more than they should. Furthermore, challenges in securing timely and qualified reviews can lead to compromises in reviewer selection that erode confidence in the rigor and fairness of the process (Adam, Reference Adam2025; Alberts et al., Reference Alberts, Hanson and Kelner2008). Reviewer scarcity also constrains editorial independence, as AEs may feel pressure to honor flawed or low-quality reviewer input to maintain goodwill. This tendency runs counter to explicit guidance from leading journals, which emphasize that editors are not merely “vote counters” (Campbell & Aguilera, Reference Campbell and Aguilera2022; Sarker et al., Reference Sarker, Whitley, Goh, Hong, Mähring, Sanyal, Su, Xu, Xu, Zhang and Zhao2023) and highlights the editor’s evaluative role. Yet in practice, AEs may hesitate to override reviewers as contradicting a reviewer can be perceived as undermining the process and disincentivize reviewers from reviewing manuscripts in the future.

Constraint #3: Power dynamics and deference

Status cues, such as author prominence or elite institutional affiliation, can measurably affect peer review outcomes, creating a “hierarchy of voices” in which established authors exert outsized influence (Teplitskiy et al., Reference Teplitskiy, Acuna, Elamrani-Raoult, Körding and Evans2018; Tomkins et al., Reference Tomkins, Zhang and Heavlin2017). If we apply this dynamic to reviewers and the review process, AEs may be disproportionately influenced by the opinions of reviewers with significant social capital. Furthermore, they may be more reluctant to override flawed reviews when authors lack social capital because the consequences of professional friction with high-status reviewers are greater than the consequences of friction with lower status authors. The reputational and relational costs of openly challenging reviewers (e.g., jeopardizing future collaborations, complicating reciprocal reviewing arrangements) can foster a “politics of deference” that allows problematic review practices to persist uncorrected.

Constraint #4: Cultural norms in the field

Deep-seated cultural norms within our field influence not only how reviewers evaluate manuscripts but also how AEs interpret and act upon those reviews and thus may perpetuate the very problems that Allen et al., and we highlight. A persistent cultural norm shaping editorial decisions is the bias toward confirmatory findings, whereby statistically significant results are implicitly valued over null or inconclusive outcomes (Franco et al., Reference Franco, Malhotra and Simonovits2014). Even when AEs recognize the problem, field-wide expectations can make it difficult to push back, and decision letters that highlight unsupported hypotheses can unintentionally reinforce QRPs. The absence of explicit guidance against QRPs often leaves reviewers’ problematic suggestions (e.g., unplanned analyses) unchallenged and, at times, endorsed by AEs. This is compounded by the lack of formal expectations for fact-checking reviewer claims that can result in inaccurate statements passing into decision letters uncorrected.

System-level reforms

Overcoming these constraints requires systemic reform to the peer review process. At the editorial level, journals could expand AE evaluation criteria to include qualitative indicators, such as review accuracy and the resolution of contradictory feedback, rather than focusing almost exclusively on speed. Clear policies and training should also reinforce that the AE’s role includes fact checking and reconciling reviewer claims. Several initiatives (e.g., COPE, 2017; Sage Journals, 2025) outline such responsibilities, but they have yet to be integrated into formal performance expectations. Explicit guidance that author–reviewer dynamics should not deter the AE from correcting inaccuracies or declining problematic suggestions would further help shift the perception of editorial independence from optional to essential.

Yet empowering AEs alone will not suffice if reviewer incentives remain weak and power dynamics remain strong. One way to begin addressing both of these issues is to recalibrate the reward structure for reviewing. Modest financial compensation can broaden the reviewer pool and improve reviewer timeliness (Chetty et al., Reference Chetty, Saez and Sandor2014), allowing AEs more time to critically review manuscript feedback. By widening the reviewer base, such models reduce overreliance on a small, high-prestige group, relieving AEs of the pressure to satisfy reviewers and empowering AEs to challenge inaccuracies and make decisions driven by evidence rather than reputation.

Finally, making editorial policies clear, visible, and operational is essential. Journals could discourage abstract conjectures by requiring reviewers to cite specific manuscript passages when critiquing methodology and mandating that any factual inconsistencies be flagged and resolved explicitly in editorial correspondence. Journals can affirm that sound methods are valued independent of outcome through formal editorial statements and policy innovations. In one such innovation, registered reports, manuscripts are provisionally accepted on the basis of theoretical significance and methodological rigor prior to data collection, with final publication guaranteed regardless of whether results are statistically significant (Briker & Gerpott, Reference Briker and Gerpott2024; Chambers & Tzavella, Reference Chambers and Tzavella2022; Miller & Bamberger, Reference Miller and Bamberger2016). Similarly, “results-blind” review holds potential for reducing editorial bias toward confirmatory findings (Findley et al., Reference Findley, Jensen, Malesky and Pepinsky2016).

Taken together, these reforms aim not merely to correct individual failings but to recalibrate the system itself, strengthening editorial independence by enabling AEs to resist default cultural pressures and to justify decisions that uphold rigor and transparency. The resulting alignment of editorial practices, reviewer incentives, and field-wide expectations should support a peer-review culture anchored in accuracy, transparency, and fairness, regardless of whether the focus of review is developmental or constructive.