SPPQ Data Verification and Replication Policy
Overview
State Politics & Politics Quarterly is committed to publishing research of all methodological approaches so that it is open and available to the scientific community. As such, all accepted articles will undergo a data verification process to ensure replication materials enable researchers to reproduce all published analyses presented in the journal. SPPQ partners with the Odum Institute for Research in Social Science at the University of North Carolina, Chapel Hill, to verify all submitted materials are transparent, and interested researchers can replicate all published findings from the journal.
Upon an article acceptance, authors must follow the verification and replication procedures outlined below before an article enters the production stage of the publication process. Publication of an accepted article is contingent on successful completion of the verification and replication process.
The following guidelines will apply to most accepted articles at SPPQ and are provided to assist authors in this process. These general guidelines apply to both quantitative and qualitative research except where there are different expectations for those types of articles. We are currently developing more specific guidelines for qualitative articles. Until those are available, authors will receive specific instructions from the editors for the verification process for their individual manuscript. While these guidelines should cover most accepted articles, the editors may make minor exceptions to the general rules or process to accommodate individual articles.
Any exceptions or modifications to the verification policy must be approved by the SPPQ editors who retain final authority to grant any exceptions or modifications.
General Procedure
A manuscript moves into the verification stage once the author(s) have submitted the final draft of an accepted manuscript. After submission of the final draft, the corresponding author will be responsible for providing the materials necessary to verify the results in the manuscript are reproducible. Once the verification process is successfully completed, the manuscript moves into the production phase and the verified replication content is simultaneously made publicly available on the SPPQ Dataverse. Authors may also post their verification files elsewhere, but the material must remain on the SPPQ Dataverse.
The corresponding author starts the process by posting all verification and replication material on the SPPQ Dataverse. Please reference this checklist SPPQ Quantitative Checklist when working on the replication materials.
Replication files include the following materials:
README File:
Every manuscript will include a plain-text file titled “readme.txt.” This file provides the names of all other files contained in the study along with a brief description of each one. TheSPPQ README Guide provides more information and helpful tips.
Analysis Dataset(s):
This requirement may not apply to qualitative manuscripts.
All data necessary to reproduce the results must be included in the dataverse. Authors can choose their own data storage formats as long as the files are readily accessible to researchers in the social science community. The SPPQ Analysis Data Construction Guide provides more information and helpful tips.
In limited situations, the SPPQ editors may grant an exemption from the general requirement of publicly providing the source data. Most notably, if an author is using restricted access data or is in a situation where the data cannot be made public (e.g., the data contains confidential information), they can request a waiver from the editors. However, the editors may still ask that the data be made privately available (if that is possible) to complete the verification process, even if it not ultimately publicly posted. In these cases, the authors must provide the relevant information and procedures so that an interested researcher could apply for data access.
Additionally, the author must provide the software command files (see below) and the data construction information (see below) used to complete the analysis.
Codebook:
This requirement may not apply to qualitative manuscripts.
Codebooks must include information on all variables (and only those variables) provided in the analysis datasets as well as full data citations for all data sources. The SPPQ Codebook Guide provides more information and helpful tips.
Information to Reconstruct the Analysis:
Every manuscript must contain complete information for constructing the analysis dataset(s) from the original sources of the data. For qualitative manuscripts, until more specific instructions are posted, please consult with the editors on this procedure. For quantitative manuscripts, this information provides a roadmap that interested researchers will follow to produce the results in the manuscript. It traces the author(s)’process from collection of source data and data curation and management to the provided analysis dataset(s) in the dataverse.
Software Commands:
All manuscripts with any type of analysis dataset(s) must provide one or more files containing the software commands or code to reproduce all the analytic results reported in the manuscript. Typically, these will be plain text files. The exact format of the file contents will depend on the software used in the original analysis (i.e., Stata .do files, R command scripts, or text files).
Regardless of the format used, comment statements should be used extensively throughout the files to explain the steps of the analysis. Examples include comments like, “The following commands recode variables X and Y in preparation for . . .,” “The following commands produce the results in Table 1,” or “The following commands create Figure 1.”
Additionally, the authors need to list and provide information on any software packages that must be installed to complete the analysis (i.e., Stata .ado files or R packages).
Authors need to provide clear and specific information about the version of the software system used to conduct the analyses (e.g., Stata 16). This requirement is critical for successful replicability of the results.
We advise authors to pay particular attention to this step in the process as the verification process goes much smoother with well-written and detailed software command files.
Conclusion
These guidelines describe the minimum requirements for manuscript replication materials on the SPPQ Dataverse. These guidelines implement the principles of data access and research transparency (DART) to which SPPQ is a proud signatory. We encourage authors to provide as much information as possible to facilitate transparent, open, and diverse inquiries into the research published in the journal. Open, transparent research furthers efforts to establish powerful theories of political and social phenomena.