Skip to main content

Publication Bias Reconsidered

  • Lee Sigelman

In political science and many other disciplines, statistically significant results—rejections of the null hypothesis—are achieved more frequently in published than in unpublished studies. Such “publication bias” is generally seen as the consequence of a widespread prejudice against statistically nonsignificant results. I argue that evidence of such a prejudice is in surprisingly short supply and that publication bias can occur even in the absence of such a prejudice and even if the review process is functioning perfectly. More importantly, publication bias may stem from dutiful application of standards of scientific inquiry rather than from irrational prejudice.

Hide All
Bartels, L. M. 1997. “Specification Uncertainty and Model Averaging.” American Journal of Political Science 41: 641674.
Begg, C. B. 1994. “Publication Bias.” In The Handbook of Research Synthesis, eds. Cooper, H. and Hedges, L. V. New York: Russell Sage Foundation, pp. 400408.
Begg, C. B., and Berlin, J. A. 1988. “Publication Bias: A Problem in Interpreting Medical Data.” Journal of the Royal Statistical Society, Series A 151: 419445.
Bero, L. A., Glantz, S. A., and Rennie, D. 1994. “Publication Bias and Public Health Policy on Environmental Tobacco Smoke.” Journal of the American Medical Association 272: 133136.
Card, D., and Krueger, A. B. 1995. “Time-Series Minimum-Wage Studies: A Meta-Analysis.” American Economic Review 85: 238243.
Coursol, A., and Wagner, E. E. 1986. “Effect of Positive Findings on Submission and Acceptance Rates: A Note on Meta-Analysis Bias.” Professional Psychology 17: 136137.
Cowles, M., and Davis, C. 1982. “On the Origins of the .05 Level of Statistical Significance.” American Psychologist 37: 553558.
De Long, J. B., and Lang, K. 1992. “Are All Economic Hypotheses False?Journal of Political Economy 100: 12571272.
Dickersin, K. 1990. “The Existence of Publication Bias and Risk Factors for Its Occurrence.” Journal of the American Medical Association 263: 13851389.
Dickersin, K., and Min, Y.-I. 1993. “Publication Bias: The Problem that Won't Go Away.” Annals of the New York Academy of Science 703: 135146.
Dickersin, K., Min, Y.-I., and Meinert, C. L. 1992. “Factors Influencing Publication of Research Results: Follow-up of Applications Submitted to Two Institutional Review Boards.” Journal of the American Medical Association 267: 374378.
Gill, J. 1999. “The Insignificance of Null Hypothesis Significance Testing.” Political Research Quarterly 52: 647674.
Greenwald, A. G. 1975. “Consequences of Prejudice Against the Null Hypothesis.” Psychological Bulletin 82: 120.
Kuhn, T. S. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
Lau, R. R., Sigelman, L., Heldman, C., and Babbitt, P. 1999. “The Effects of Negative Political Advertisements: A Meta-Analytic Assessment.” American Political Science Review (in press).
Mahoney, M. J. 1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy and Research 1: 161175.
Meehl, P. E. 1990. “Why Summaries of Research on Psychological Theories Are Often Uninterpretable.” Psychological Reports 66: 195244.
Peters, D. P., and Ceci, S. J. 1982. “Peer-Review Practices of Psychological Journals: The Fate of Published Articles Submitted Again.” Behavioral and Brain Sciences 5: 187255.
Rosenthal, R. 1979. “The File Drawer Problem and Tolerance for Null Results.” Psychological Bulletin 86: 638641.
Rosenthal, R. 1991. Meta-Analytic Procedures for Social Research. Newbury Park, CA: Sage.
Rosenthal, R., and Rubin, D. B. 1988. “Comment: Assumptions and Procedures in the File Drawer Problem.” Statistical Science 3: 120125.
Simon, R. 1988. “Comment on Begg and Berlin.” Journal of the Royal Statistical Society, Series A 151: 419459.
Sterling, T. D. 1959. “Publication Decision and the Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa.” Journal of the American Statistical Association 54: 3034.
Sterling, T. D., Rosenbaum, W. L., and Weinkam, J. J. 1995. “Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa.” American Statistician 49: 108112.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 3 *
Loading metrics...

Abstract views

Total abstract views: 108 *
Loading metrics...

* Views captured on Cambridge Core between 4th January 2017 - 21st March 2018. This data will be updated every 24 hours.