Introduction
As the professional journal of the European Consortium of Political Research (ECPR), European Political Science (EPS) aims to contribute to the discipline by providing a forum for the publication of work which shines a light on key debates in the profession. This includes the very process of undertaking political science research itself. Over recent decades, the profession has witnessed significant change with regard to the way that political scientists are trained, with a far greater focus on the use of data, being reflected of the shift towards big data. One of the ways that EPS has responded to these developments is by providing a forum for publishing datasets.
Discussions about data analysis in political science are not just a recent development (see Tufte Reference Tufte1969). They played an important role in the discussions that led to the founding of the ECPR, where the influence of political science scholars in North America was particularly instrumental in terms of the focus that was applied to the development and training of new generations of European political scientists (Budge Reference Budge2006; Daalder Reference Daalder2010). This included the ECPR Methods Schools. While much of these discussions focused on the use of data from a statistical point of view, the significance of transparency and reliability of qualitative research has more recently been highlighted in this journal (Closa Reference Closa2021).
Further significant efforts towards promoting a culture of open data in social sciences have been driven by funding institutions. The European Commission, for example, has contributed with initiatives like the Open Research Data Pilot, which “aims to improve and maximise access to and re-use of research data generated by Horizon 2020 projects”. Hence, H2020 beneficiaries have to make their research data findable, accessible, interoperable, and reusable (FAIR).
A crucial role, however, can be played by journal editors in raising awareness about open data by, for instance, adopting protocols and guidelines on research transparency for authors.
This editorial presents the new guidelines for research transparency and open data when publishing in EPS. Furthermore, it reflects about the importance of research transparency, the challenges that it faces, and offers a few proposals to foster a culture of open data. The editorial is structured as follows. First, we outline the role and responsibility of journal editors in promoting a culture and awareness of open data. Second, we provide a detailed outline of the EPS guidelines. Finally, we consider the future steps to promote and challenges to address, to enhance research transparency in the social sciences.
The role of journals in promoting a culture of research transparency and open data
Along with other ECPR journals (the European Journal of Political Research and the European Political Science Review), EPS was a driving force behind the ECPR’s statement on data access and research transparency (ECPR, n.d.). In line with other journals, it is important that we clearly state the importance of maintaining research quality and for that reason have agreed a set of guidelines for publishing in EPS that specifically relate to data access and transparency.
We view these guidelines as an important development in the discipline and place an emphasis on transparency and data sharing. Just as the origins of EPS can be traced to a newsletter format that focused on the sharing of knowledge and best practice, we consider that EPS equally holds these values integral to what it does today, albeit within the context of an academic journal that is equally measured by its impact factor. It is important for us as editors to acknowledge that just as EPS has published articles where there is a clear alignment to data sharing and transparency, there are also articles where data transparency has not always been clear. Going forward, we have therefore set out a code of practice that we expect authors to apply to. We recognise that an important aspect of this is that at the time of submission, authors need to make it clear to the journal where their contribution sits within these criteria.
As editors, however, we are aware that there are also caveats to consider when requiring authors to meet the requirements for research transparency. Firstly, there is not a “one size fits all” approach, as qualitative and quantitative data might have different needs. This is also mentioned in the ECPR statement on data access and research transparency (ECPR, n.d.). Secondly, there are personnel limits, meaning that replication standards cannot be checked by editors themselves. Thirdly, data is sometimes subject to restrictions like embargoes or privacy concerns. We consider that these caveats should allow some flexibility and thereby ensure that our policy is not too stringent.
By taking these limitations into account, the guidelines adopted by EPS adapt to the needs of qualitative and quantitative approaches, allow for some flexibility, and refer to trusted repositories as places where data can be stored. This will ensure that that replication standards can be checked by the scientific community itself, without relying on editors.
A related element to this discussion is the extent to which articles have ethical permission for the research undertaken. This reflects a broader trend in North American journals that request clarification of ethical approval at the point of submission. We recognise that this is not something that we have always emphasised at the point of submission. However, we hope that through these new guidelines, authors will now have a clearer understanding of the journal’s expectations in this regard, and in terms of the responsibility that rests with authors at the outset of the research process to ensure that their work meets the rigour that is required.
Introducing EPS guidelines
As a generalist journal with a methodological pluralist orientation to research, we acknowledge that each type of work requires a different approach to data access, replication, and transparency. For this reason, we encourage authors of accepted manuscripts to make public their data and codes necessary to replicate their results. However, when the authors cannot comply with this requirement, for instance because of embargoes, privacy or security concerns, these cases will be considered individually.
EPS guidelines are drawn from the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al. Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Buck2015), which set out standards that journals should apply to.Footnote 1 This also includes a TOP Factor which highlights the extent to which a journal complies with the TOP Guidelines.
TOP Guidelines identify standards of openness and transparency across eight key aspects (i.e., Citation standards, Data transparency, Analytic Methods (Code) Transparency, Research Materials Transparency, Design and Analysis Transparency, Study Preregistration, Analysis Plan Preregistration, Replication) of research design and reporting. Editors can decide which of these eight aspects is relevant for their journal and, for each category, they can choose between three different levels of stringency, according to the journal’s characteristics and scope.
Since EPS publishes research using different methodological approaches, both qualitative and quantitative, EPS opted for the Level 2 of stringency for Citation, Data Transparency, Analytic methods (Code), and Research Materials Transparency. It requires data sharing whilst allowing for some flexibility and provides guidance for what to do when open data is not possible. Authors are required to share data and materials when it is permitted, while provide appropriate justification when legal or ethical restrictions pose limitations to full sharing. For Replication and Study preregistration, EPS opted for Level 1 of stringency.
Furthermore, given the scope of EPS, its policy does not include requirements for the categories of Design and Analysis Transparency, and Analysis Plan.
Detailed information on the guidelines is published on the EPS journal’s webpage,Footnote 2 along with instructions for authors. Moreover, relevant emails with authors include a small statement informing them they can view our Open Data guidelines at the provided website’s link.
Steps forward towards a culture of open data in political science
Despite the importance of transparency, openness, and replicability for science, current practices in all disciplines, including political science, reveal that while much has been done, there is still a lot more to do.
Aside from the role of funding institutions and academic journals in this endeavour, university departments and faculties could further encourage this effort. Modules on FAIR data in PhD programmes, for instance, could help promote awareness on how to manage data soundly and instil these standards of practice at an early career stage.
As Nosek et al. (Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Buck2015) note, a further hurdle to address relates to the “academic reward system that does not sufficiently incentivise open practices”. Indeed, by relying on a publish or perish model, the academic system risks undermining the rigour and quality of research. By chasing mechanisms of competition based on the volume of scholars’ publications, scholars might be encouraged to publish quickly or to rely on easy experiments that rapidly produce data, which is decidedly the opposite of what good practice in research should be. At the same time, making data available through cleaning data, and preparing metadata for repositories, can be time consuming for researchers. However, such a time investment is still poorly rewarded in terms of recognition, for instance in competitions for tenured positions (Gernsbacher Reference Gernsbacher2018; Robson et al. Reference Robson, Baum, Beaudry, Beitner, Brohmer, Chin and Jasko2021). This trend could be reversed, however, with minor, yet albeit significant changes. For example, by emphasising evidence of open data practices in job applications for hiring into departments and faculties (Robson et al. Reference Robson, Baum, Beaudry, Beitner, Brohmer, Chin and Jasko2021). Furthermore, citations of datasets published in repositories should count as much as an original article in a peer-reviewed journal, when evaluating scholars’ research impact. Thus, the academic community can stimulate a change in the publish or perish culture, which would encourage greater attention to the time spent by researchers on making data open and transparent.
As journal editors, we hope that small steps such as the provision of guidelines for standards of openness and transparency will actively promote and raise awareness about the importance of open, accessible, and reproducible research for everyone. Together with funding institutions, research institutes, universities and academic departments, we hope our combined efforts can make a huge leap forward in this direction.
