Skip to main content
×
×
Home

Verify original results through reanalysis before replicating

Abstract

In determining the need to directly replicate, it is crucial to first verify the original results through independent reanalysis of the data. Original results that appear erroneous and that cannot be reproduced by reanalysis offer little evidence to begin with, thereby diminishing the need to replicate. Sharing data and scripts is essential to ensure reproducibility.

Copyright
References
Hide All
Agnoli, F., Wicherts, J. M., Veldkamp, C. L. S., Albiero, P. & Cubelli, R. (2017) Questionable research practices among Italian research psychologists. PLoS One 12(3):e0172792. Available at: http://doi.org/10.1371/journal.pone.0172792.
Bakker, M., van Dijk, A. & Wicherts, J. M. (2012) The rules of the game called psychological science. Perspectives on Psychological Science 7(6):543–54. Available at: http://doi.org/10.1177/1745691612459060.
Bakker, M. & Wicherts, J. M. (2011) The (mis)reporting of statistical results in psychology journals. Behavior Research Methods 43(3):666–78. Available at: http://doi.org/10.3758/s13428-011-0089-5.
Bakker, M. & Wicherts, J. M. (2014) Outlier removal and the relation with reporting errors and quality of research. PLoS One 9(7):e103360. Available at: http://doi.org/10.1371/journal.pone.0103360.
Brown, N. J. L. & Heathers, J. A. J. (2017) The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science 8(4):363–69. Available at: http://doi.org/10.1177/1948550616673876.
Ebrahim, S., Sohani, Z. N., Montoya, L., Agarwal, A., Thorlund, K., Mills, E. J. & Ioannidis, J. P. (2014) Reanalysis of randomized clinical trial data. Journal of the American Medical Association 312(10):1024–32. Available at: http://doi.org/10.1001/jama.2014.9646.
Epskamp, S. & Nuijten, M. B. (2016) statcheck: Extract statistics from articles and recompute p-values. Available at: https://cran.r-project.org/web/packages/statcheck/ (R package version 1.2.2).
Giofrè, D., Cumming, G., Fresc, L., Boedker, I. & Tressoldi, P. (2017) The influence of journal submission guidelines on authors' reporting of statistics and use of open research practices. PLoS One 12(4):e0175583. Available at: http://doi.org/10.1371/journal.pone.0175583.
Ioannidis, J. P., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., Falchi, M., Furlanello, C., Game, L., Jurman, G., Mangion, J., Mehta, T., Nitzburg, M., Page, G. P., Petretto, E. & van Noort, V. (2009) Repeatability of published microarray gene expression analyses. Nature Genetics 41(2):149–55.
John, L. K., Loewenstein, G. & Prelec, D. (2012) Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science 23(5):524–32.
Kidwell, M. C., Lazarevic, L. B. Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., Errington, T. M., Fiedler, S. & Nosek, B. A. (2016) Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS Biology 14(5):e1002456. Available at: http://doi.org/10.1371/journal.pbio.1002456.
Nuijten, M. B., Borghuis, J., Veldkamp, C. L. S., Dominguez-Alvarez, L., Van Assen, M. A. L. M. & Wicherts, J. M. (2017) Journal data sharing policies and statistical reporting inconsistencies in psychology. Collabra: Psychology 3(1):122. Available at: http://doi.org/10.1525/collabra.102.
Nuijten, M. B., Hartgerink, C. H. J., Van Assen, M. A. L. M., Epskamp, S. & Wicherts, J. M. (2016) The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods 48(4):1205–26. Available at: http://doi.org/10.3758/s13428-015-0664-2.
Petrocelli, J., Clarkson, J., Whitmire, M. & Moon, P. (2012) When abc – c′: Published errors in the reports of single-mediator models. Behavior Research Methods 45(2):595601. Available at: http://doi.org/10.3758/s13428-012-0262-5
Royal Netherlands Academy of Arts and Sciences (2018) Replication studies. Improving reproducibility in the empirical sciences. KNAW.
Schönbrodt, F. D. (2018). p-checker: One-for-all p-value analyzer. Available at: http://shinyapps.org/apps/p-checker/.
Simmons, J. P., Nelson, L. D. & Simonsohn, U. (2011) False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22:1359–66. Available at: http://doi.org/10.1177/0956797611417632.
Vanpaemel, W., Vermorgen, M., Deriemaecker, L. & Storms, G. (2015) Are we wasting a good crisis? The availability of psychological research data after the storm. Collabra 1(1):15. Available at: http://doi.org/10.1525/collabra.13.
Veldkamp, C. L. S., Nuijten, M. B., Dominguez-Alvarez, L., van Assen, M. A. L. M. & Wicherts, J. M. (2014) Statistical reporting errors and collaboration on statistical analyses in psychological science. PLoS One 9(12):e114876. Available at: http://doi.org/10.1371/journal.pone.0114876.
Wicherts, J. M., Bakker, M. & Molenaar, D. (2011) Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One 6(11):e26828. Available at: http://doi.org/10.1371/journal.pone.0026828.
Wicherts, J. M., Borsboom, D., Kats, J. & Molenaar, D. (2006) The poor availability of psychological research data for reanalysis. American Psychologist 61:726–28. Available at: http://doi.org/10.1037/0003-066X.61.7.726.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Behavioral and Brain Sciences
  • ISSN: 0140-525X
  • EISSN: 1469-1825
  • URL: /core/journals/behavioral-and-brain-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed