In the interest of promoting open and reproducible science, the Journal of Experimental Political Science editorial team will pilot the pre-acceptance of preregistered reports. We note that the launch of this new submission option is a complement to, and does not replace, the option to submit other types of manuscripts. JEPS remains open to receiving and reviewing high quality manuscripts regardless of whether they are based on preregistered studies.
A preregistered report is like any other research paper in many respects. It offers a specific research question, summarizes the scholarly conversation in which the question is embedded, explicates the theoretically grounded hypotheses that offer a partial answer to the research question, and details the research design for testing the proposed hypotheses. It differs from most research papers in that a preregistered report stops here. The researchers do not take the next step of reporting results from the data they collected. Instead, they preregister the design in a third-party archive, such as the Open Science Framework (https://osf.io/), before collecting data.
At JEPS, we will send out preregistered reports for a review, just like other manuscripts, but we will ask reviewers to focus on whether the research question, theory, and design are sound. If the researchers carried out the proposed research a) would they make a contribution and b) would their proposed test do the job? If the answer is yes (potentially after a round of revisions), we will conditionally accept the paper and give the researchers a reasonable amount of time to conduct the study, write up the results, and resubmit the revised fully-fledged paper. At this point, we will seek the reviewers’ advice one more time and ask, “Did the researchers do what they said they were going to do?” If the answer is “yes,” we will publish the paper. It doesn't matter if the research produced unexpected results, null findings, or inconsistent findings. In fact, we will specifically instruct reviewers at the second stage to ignore statistical significance and whether they support the authors’ hypotheses when evaluating the paper.
We share the optimism of proponents in the open science movement that pre-acceptance of preregistered reports offers a way to burnish the credibility of science and increase our knowledge in the process. Why? As the crises surrounding the failure to replicate celebrated findings in social psychology makes plain, the current way in which we produce research makes it more likely that splashy, but potentially wrong, findings are published while null results are buried in a file drawer.
We do not believe that political science is immune from the challenges facing other disciplines. In fact, there is reason to believe that the same problems plague our research practices. Without preregistration, we cannot be sure if researchers are only reporting significant results. Without pre-acceptance of preregistered reports, we cannot prevent reviewers from judging the significance of a paper and soundness of the research design from how pleasing (or displeasing) they find the results.
The submission guidelines for pre-registered reports can be found on the Cambridge Core website (https://www.cambridge.org/core/journals/journal-of-experimental-political-science/information/instructions-contributors). We are indebted to the Open Science Framework's guidance and materials in developing our guidelines and process. Because we have no experience in this endeavor, we plan to pilot this initiative for one year and then reassess whether adjustments need to be made to the process.
