Skip to main content Accessibility help
×
Home
Hostname: page-component-55597f9d44-5zjcf Total loading time: 0.198 Render date: 2022-08-17T02:08:02.952Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true } hasContentIssue true

Science Deserves Better: The Imperative to Share Complete Replication Files

Published online by Cambridge University Press:  29 December 2013

Allan Dafoe*
Affiliation:
Yale University

Extract

In April 2013, a controversy arose when a working paper (Herndon, Ash, and Pollin 2013) claimed to show serious errors in a highly cited and influential economics paper by Carmen Reinhart and Kenneth Rogoff (2010). The Reinhart and Rogoff paper had come to serve as authoritative evidence in elite conversations (Krugman 2013) that high levels of debt, especially above the “90 percent [debt/GDP] threshold” (Reinhart and Rogoff 2010, 577), posed a risk to economic growth. Much of the coverage of this controversy focused on an error that was a “perfect made-for-TV mistake” (Stevenson and Wolfers 2013) involving a simple error in the formula used in their Excel calculations. The real story here, however, is that it took three years for this error and other issues to be discovered because replication files were not publicly available, nor were they provided to scholars when asked. If professional norms or the American Economic Review had required that authors publish replication files, this debate would be advanced by three years and discussions about austerity policies would have been based on a more clear-sighted appraisal of the evidence.

Type
Symposium: Openness in Political Science
Copyright
Copyright © American Political Science Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

APSA. 2012. “A Guide to Professional Ethics in Political Science.” URL: http://www.apsanet.org/content86135.cfm.Google Scholar
Bhattacharjee, Yudhijit. 2013. “The Mind of a Con Man.” New York Times Magazine, April 26. URL: http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all&_r=1&.Google Scholar
Bowers, Jake. 2011. “Six Steps to a Better Relationship with Your Future Self.” Political Methodologist 18 (2): 9.Google Scholar
Bueno de Mesquita, Bruce, Gleditsch, Nils Petter, James, Patrick, King, Gary, Metelits, Claire, Ray, James Lee, Russett, Bruce, Strand, Håvard, and Valeriano, Brandon. 2003. “Symposium on Replication in International Studies Research.” International Studies Perspective 4 (1): 72107. URL: http://onlinelibrary.wiley.com/doi/10.1111/1528-3577.04105/full.CrossRefGoogle Scholar
Davenport, Christian, and Moore, Will H.. 2013. “Conflict Consortium Data Standards & Practices.” Manuscript. Google Scholar
Gandrud, C. 2013. Reproducible Research with R and Rstudio. Chapman & Hall/CRC. Google Scholar
Gerring, John. 2011. Social Science Methodology: A Unified Framework. New York: Cambridge University Press.CrossRefGoogle Scholar
Gherghina, Sergiu, and Katsanidou, Alexia. 2013. “Data Availability in Political Science Journals.” European Political Science 12: 333–49.CrossRefGoogle Scholar
Gleditsch, Nils Petter, Metelits, Claire, and Strand, Håvard. 2003. “Posting Your Data: Will You Be Scooped or Will You Be Famous?International Studies Perspectives 4 (1): 995. URL: http://gking.harvard.edu/files/replvdc.pdf.Google Scholar
Herndon, Thomas, Ash, Michael, and Pollin, Robert. 2013. “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff.” Political Economy Research Institute Working Paper Series (322). Google Scholar
Hook, Les A., Vanna, Suresh K. Santhana, Beaty, Tammy W., Cook, Robert B., and Wilson, Bruce E.. 2010. “Best Practices for Preparing Environmental Data Sets to Share and Archive.” URL: daac.ornl.gov/PI/BestPractices-2010.pdf. An earlier version of this was published in: 2001. Bulletin of the Ecological Society of America 82 (2): 138–41.Google Scholar
King, Gary. 1995. “Replication, Replication.” PS: Political Science and Politics 28 (3): 444–52.Google Scholar
Krugman, Paul. 2013. “The Excel Depression.” New York Times . URL: http://www.nytimes.com/2013/04/19/opinion/krugman-the-excel-depression.html.Google Scholar
Moravcsik, Andrew. 2010. “Active Citation: A Precondition for Replicable Qualitative Research.” PS: Political Science and Politics 43 (1): 2935.Google Scholar
Peng, Roger D. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–27.CrossRefGoogle Scholar
Piwowar, Heather A, Day, Roger S., and Fridsma, Douglas B. 2007. “Sharing Detailed Research Data Is Associated with Increased Citation Rate.” PLOSOne 2 (3): e308. CrossRefGoogle Scholar
Reinhart, Carmen M., and Rogoff, Kenneth S.. 2010. “Growth in a Time of Debt.” American Economic Review: Papers & Proceedings 100 (2): 573–78.CrossRefGoogle Scholar
Rich, Timothy S. 2013. “Publishing as a Graduate Student: A Quick and (Hopefully) Painless Guide to Establishing Yourself as a Scholar.” PS: Political Science & Politics 46 (2): 376–79.Google Scholar
Savage, C. J., and Vickers, A. J.. 2009. “Empirical study of data sharing by authors publishing in PLoS journals.” PLoS One 4 (9): e7078. CrossRefGoogle ScholarPubMed
Shalizi, Cosma Rohilla. 2013. “Advanced Data Analysis from an Elementary Point of View.” URL: http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/.Google Scholar
Shea, Christopher. 2012. “The Data Vigilante.” The Atlantic, December. URL: http://www.theatlantic.com/magazine/archive/2012/12/the-data-vigilante/309172/.Google Scholar
Simonsohn, Uri. 2012. “Just Post It: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone.” Psychological Science 24: 1875–88.CrossRefGoogle ScholarPubMed
Stevenson, Betsey, and Wolfers, Justin. 2013. “Six Ways to Separate Lies from Statistics.” Bloomberg. URL: http://www.bloomberg.com/news/2013-05-01/six-ways-to-separate-lies-from-statistics.html.Google Scholar
Wicherts, Jelte M., Borsboom, Denny, Katsand, Judith, and Molenaar, Dylan. 2006. “The Poor Availability of Psychological Research Data for Reanalysis.” American Psychologist 61 (7): 726–28.CrossRefGoogle ScholarPubMed
Wicherts, Jelte M., Bakker, Marjan, and Molenaar, Dylan. 2011. “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Statistical Results.” PLoS One 6 (11): e26828. CrossRefGoogle ScholarPubMed
Supplementary material: PDF

Dafoe supplementary material

Appendix A

Download Dafoe supplementary material(PDF)
PDF 365 KB
20
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Science Deserves Better: The Imperative to Share Complete Replication Files
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Science Deserves Better: The Imperative to Share Complete Replication Files
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Science Deserves Better: The Imperative to Share Complete Replication Files
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *