Skip to main content Accessibility help


  • Carole J. Lee


Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed.


Corresponding author


Hide All
Allison, D. B., Faith, M. S. and Gorman, B. S. 1996. ‘Publication Bias in Obesity Treatment Trials?International Journal of Obesity and Related Metabolic Disorders, 20: 931–7.
Altman, D. G.for the CONSORT Group. 2005. ‘Endorsement of the CONSORT Statement by High Impact Medical Journals: Survey of Instructions for Authors.’ British Medical Journal, 330: 1056–7.
Anderson, E. 1995. ‘Feminist Epistemology: An Interpretation and Defense.’ Hypatia, 10: 5084.
Angell, M. 2004. The Truth about the Drug Companies: How they Deceive us and What to Do about It. New York, NY: Random House.
Barbui, C., Hotopf, M. and Garattini, S. 2002. ‘Fluoxetine Dose and Outcome in Antidepressant Drug Trials.’ European Journal of Clinical Pharmacology, 58: 379–86.
Bardy, A. H. 1998. ‘Bias in Reporting Clinical Trials.’ British Journal of Clinical Pharmacology, 46: 147–50.
Begg, C., Cho, M., Eastwood, S. et al. 1996. ‘Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement.’ Journal of the American Medical Association, 276: 637–9.
Berlin, J. A., Begg, C. B. and Louis, T. A. 1989. ‘An Assessment of Publication Bias Using a Sample of Published Clinical Trials.’ Journal of the American Statistical Association, 84(406): 381–92.
Born, G. V. R. and Collins, R. 1997. ‘Aspirin versus Clopidogrel: The Wrong Question?Lancet, 349(9054): 806–7.
British Medical Journal. 2011. Welcome to Resources for Authors.
Brown, J. R. 2004. ‘Money, Method and Medical Research.’ Episteme, 1:4959.
Brown, J. R. 2008. ‘Politics, Method, and Medical Research.’ Philosophy of Science, 75: 756–66.
Bruijn, J. A. and van der Woude, F. 2002. ‘Tacrolimus and Ciclosporin Microemulsion in Renal Transplantation.’ Lancet, 360(9335): 800.
CAPRIE Steering Committee. 1996. ‘A Randomized, Blinded, Trial of Clopidogrel versus Aspirin in Patients at Risk of Aschaemic Events (CAPRIE).’ Lancet, 348(9038): 1329–40.
Casadevall, A. and Fang, F. C. 2012. ‘Reforming Science: Methodological and Cultural Reforms.’ Infection and Immunity, 80: 891–6.
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C. and Altman, D. G. 2004. ‘Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials: Comparison of Protocols to Published Articles.’ Journal of the American Medical Association, 291: 2457–65.
Dickersin, K. and Min, Y-.I. 1993. ‘NIH Clinical Trials and Publication Bias.’ Online Journal of Current Clinical Trials (Doc. No. 50).
Dickersin, K., Min, Y.-I. and Meinert, C. L. 1992. ‘Factors influencing Publication of Research Results: Follow-up of Applications Submitted to Two Institutional Review Boards.’ Journal of the American Medical Association, 267: 374–9.
Dorsey, E. R., de Roulet, J., Thompson, J. P. et al. 2010. ‘Funding of US Biomedical Research 2003–2008.’ Journal of the American Medical Association, 303: 137–43.
Douglas, H. 2000. ‘Inductive Risk and Values in Science.’ Philosophy of Science, 67: 559–79.
Easterbrook, P. J., Berlin, J. A., Gopalan, R. and Matthews, D. R. 1991. ‘Publication Bias in Clinical Research.’ Lancet, 337(8746): 867–72.
Egger, M. and Smith, G. D. 1995. ‘Misleading Meta-Analysis: Lessons from “an Effective, Safe, Simple” Intervention that Wasn't. (Study on Treatment of Heart Attacks).’ British Medical Journal, 310(6982): 752–4.
Elliott, C. 2010. ‘Making a Killing.’ Mother Jones, 35(5):5463.
Fanelli, D. 2009. ‘How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data.’ PLoS One, 4(5):111.
Fang, F. C. and Casadevall, A. 2011. ‘Retracted Science and the Retraction Index.’ Infection and Immunity, 79: 3855–9.
Fontanarosa, P. B. and DeAngelis, C. D. 2004. ‘Information for JAMA Authors – 2004.’ Journal of the American Medical Association, 291:107–8.
Garattini, S., Bertele, V. and Bassi, L. L. 2003. ‘How Can Research Ethics Committees Protect Patients Better?British Medical Journal, 326: 1199–201.
Grady, D., Herrington, D., Bittner, V. et al. 2002. ‘Cardiovascular Disease Outcomes During 6.8 Years of Hormone Therapy: Heart and Estrogen/Progestin Replacement Study Follow-up (HERSII).’ Journal of the American Medical Association, 288: 4957.
Grieneisen, M. L. and Zhang, M. 2012. ‘A Comprehensive Survey of Retracted Articles from the Scholarly Literature.’ PLoS One 7 (10). doi: 10.1371/journal.pone.0044118.
Horton, R. 2004. ‘Review: The Dawn of McScience.’ The New York Review of Books, March 11: 79.
Hulley, S., Grady, D., Bush, T. et al. 1998. ‘Randomized Trial of Estrogen Plus Progestin for Secondary Prevention of Coronary Heart Diease in Postmenopausal Women.’ Journal of the American Medical Association 280: 605–13.
ICMJE. ‘Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication [2004].’ Accessed March 1, 2013.
ICMJE. ‘Announcements, Updates & Editorials: Sponsorship, Authorship, and Accountability [2007].’ Accessed March 1, 2013.
Intemann, K. and de Melo-Martin, I. 2010. ‘Social Values and Scientific Evidence: The Case of the HPV Vaccines.’ Biology and Philosophy, 25: 203–13.
Ioannidis, J. P. A. 1998. ‘Effect of the Statistical Significance of Results on the Time to Completion and Publication of Randomized Efficacy Trials.’ Journal of the American Medical Association, 279: 281–6.
Ioannidis, J. P. A. 2005a. ‘Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.’ Journal of the American Medical Association, 294: 218–28.
Ioannidis, J. P. A. 2005b. ‘Why Most Published Research Findings are False.’ PLoS Medicine, 2: 696701.
Ioannidis, J. P. A. 2009. ‘Adverse Events in Randomized Trials: Neglected, Restricted, Distorted, and Silenced.’ Archives of Internal Medicine, 169: 1737–9.
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C. et al. 2004. ‘Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement.’ Annals of Internal Medicine, 141: 781–8.
Ioannidis, J. P. A. and Lau, J. 2001. ‘Completeness of Safety Reporting in Randomized Trials: An Evaluation of Medical Areas.’ Journal of the American Medical Association 285: 437–43.
Jennions, M. D. and Møller, A. P. 2002a. ‘Publication Bias in Ecology and Evolution: An Empirical Assessment Using the ‘Trim and Fill' Method.’ Biological Reviews, 77: 211–22.
Jennions, M. D. and Møller, A. P. 2002b. ‘Relationships Fade with Time: A Meta-Analysis of Temporal Trends in Publication in Ecology and Evolution.’ Proceedings: Biological Sciences, 269(1486): 43–8.
Johansen, H. K. and Gøtzsche, P. C. 1999. ‘Problems in the Design and Reporting of Trials of Antifungal Agents Encountered During Meta-Analysis.’ Journal of the American Medical Association, 282(18): 1752–9.
John, L. K., Loewenstein, G. and Prelec, D. 2012. ‘Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.’ Perspectives on Psychological Science 23: 524–32.
Kane, R., Wang, J. and Garrard, J. 2007. ‘Reporting in Randomized Clinical Trials Improved after Adoption of the CONSORT Statement.’ Journal of Clinical Epidemiology 60: 241–9.
Kirsch, I., Moore, T. J., Scoboria, A. and Nicholls, S. S. 2002. ‘The Emperor's New Drugs: Effect Size and Moderation Effects.’ Prevention and Treatment, 5(1): Art. 23.
Kitcher, P. 2001. Science, Truth, and Democracy. New York, NY: Oxford University Press.
Kravitz, R. L. and Feldman, M. D. 2011. ‘From the Editor's Desk: Self-Plagiarism and Other Editorial Crimes and Misdemeanors.’ Journal of General Internal Medicine, 26: 1.
Lawrence, B., Jones, C., Matthews, B., Pepler, S. and Callaghan, S. 2011. ‘Citation and Peer Review of Data: Moving towards Formal Data Publication.’ International Journal of Digital Curation, 6(2): 437.
Lee, C. J. 2012. ‘A Kuhnian Critique of Psychometric Research on Peer Review.’ Philosophy of Science, 79: 859–70.
Lee, C. J. and Schunn, C. D. 2010. ‘Philosophy Journal Practices and Opportunities for Bias.’ APA Newsletters: Newsletter on Feminism and Philosophy, 10(1): 510.
Lee, C. J. and Schunn, C. D. 2011. ‘Social Biases and Solutions for Procedural Objectivity.’ Hypatia: A Journal of Feminist Philosophy, 26: 352–73.
Lee, C. J., Sugimoto, C. R., Zhang, G. and Cronin, B. 2013. ‘Bias in Peer Review.’ Journal of the American Society for Information Science, 64: 217.
Lexchin, J., Bero, L. A., Djulbegovic, B. and Clark, O. 2003. ‘Pharmaceutical Industry Sponsorship and Research Outcome and Quality: A Systematic Review.’ British Medical Journal, 326: 1167–76.
Longino, H. E. 1990. Science as Social Knowledge. Princeton, NJ: Princeton University Press.
Lundh, A., Sismondo, S., Busuioc, O. A. and Bero, L. 2012. ‘Industry Sponsorship and Research Outcome (Review).’ Cochrane Database of Systematic Reviews (12). doi: 10.1002/14651858.MR000033.pub2.
Martin II, R. C. G., Brennan, M. F. and Jaques, D. P. 2002. ‘Quality of Complication Reporting in the Surgical Literature.’ Annals of Surgery, 235: 803–13.
Mathieu, S., Boutron, I., Moher, D., Altman, D. G. and Ravaud, P. 2009. ‘Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials.’ Journal of the American Medical Association, 302: 977–84.
Melander, H., Ahlqvist-Rastad, J., Meijer, G. and Beermann, B. 2003. ‘Evidence B(i)ased Medicine – Selective Reporting from Studies Sponsored by Pharmaceutical Industry: Review of Studies in New Drug Applications.’ British Medical Journal, 326: 1171–5.
Merton, R. K. 1973. ‘The Ambivalence of Scientists (1963).’ In Storer, N. W. (ed.), The Sociology of Science: Theoretical and Empirical Investigations, pp. 383412. Chicago, IL: The University of Chicago Press.
Merton, R. K. and Zuckerman, H. 1971. ‘Institutional Patterns of Evaluation in Science (1971).’ In Storer, N. W. (ed.), The Sociology of Science: Theoretical and Empirical Investigations, pp. 460–96. Chicago, IL: The University of Chicago Press.
Moher, D., Jones, A. and Lepage, L. 2001. ‘Use of the CONSORT Statement and Quality of Reports of Randomized Trials: A Comparative Before-and-After Evaluation.’ Journal of the American Medical Association, 285: 1992–5.
Neuliep, J. W. and Crandall, R. 1990. ‘Editorial Bias against Replication Research.’ Journal of Social Behavior and Personality, 5(4): 8590.
Neuliep, J. W. and Crandall, R. 1993. ‘Reviewer Bias against Replication Research.’ Journal of Social Behavior and Personality, 8(6): 21–9.
Oldenberg, H. 1966. Correspondence of Henry Oldenberg. Trans. by Rupert Hall, A. and Hall, Marie Boas. Vol. 2. Madison, WI: University of Wisconsin Press.
Olson, C. M., Rennie, D., Cook, D. et al. 2002. ‘Publication Bias in Editorial Decision Making.’ Journal of the American Medical Association, 287: 2825–8.
Palmer, A. R. 2000. ‘Quasireplication and the Contract of Error: Lessons from Sex Ratios, Heritabilities and Fluctuating Asymmetry.’ Annual Review of Ecology and Systematics, 31: 441–80.
Parsons, M. A., Duerr, R. and Minster, J.-B. 2010. ‘Data Citation and Peer Review.’ Eos, Transactions, American Geophysical Union, 91 (34): 297–8.
Pipkin, F. B., Crowther, C., de Swiet, M. et al. 1996. ‘Where Next for Prophylaxis against Pre-Eclampsia?British Journal of Obstetrics and Gynaecology, 103: 603–7.
Pitrou, I., Boutron, I., Ahmad, N. and Ravaud, P. 2009. ‘Reporting of Safety Results in Published Reports of Randomized Controlled Trials.’ Archives of Internal Medicine, 169: 1756–61.
Plint, A. C., Moher, D., Morrison, A. et al. 2006. ‘Does the CONSORT Checklist Improve the Quality of Reports of Randomised Controlled Trials? A Systematic Review.’ Medical Journal of Australia, 185: 263–7.
Prayle, A. P., Hurley, M. N. and Smyth, A. R. 2012. ‘Compliance with Mandatory Reporting of Clinical Trial Results on Cross sectional Study.’ British Medical Journal, 344: d7373.
Psaty, B. M. and Kornmal, R. A. 2008. ‘Reporting Mortality Findings in Trials of Rofecoxib for Alzheimer Disease or Cognitive Impairment: A Case Study Based on Documents from Rofecoxib Litigation.’ Journal of the American Medical Association, 299: 1813–7.
Reiss, J. 2010. ‘In favour of a Millian Proposal to Reform Biomedical Research.’ Synthese, 177: 427–47.
Reiss, J. and Kitcher, P. 2009. ‘Biomedical Research, Neglected Diseases, and Well-Ordered Science.’ Theoria, 66: 263–82.
Rochon, P. A., Gurwitz, J. H., Simms, R. W. et al. 1994. ‘A Study of Manufacturer-Supported Trials of Nonsteroidal Anti-Inflammatory Drugs in the Treatment of Arthritis.’ Archives of Internal Medicine, 154: 157–63.
Ross, J. S., Mulvey, G. K., Hines, E. M., Nissen, S. E. and Krumholz, H. M. 2009. ‘Trial Publication after Registration in ClinicalTrials.Gov: A cross-sectional analysis.’ PLoS Medicine, 6(9): 19.
Rothman, K. J. 1986. ‘Significance Questing.’ Annals of Internal Medicine, 105: 445–7.
Rudner, R. 1953. ‘The Scientist Qua Scientist Makes Value Judgments.’ Philosophy of Science, 20(1): 16.
Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W. and Haynes, R. B. 2000. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd edn.Oxford: Churchill Livingstone.
Safer, D. J. 2002. ‘Design and Reporting Modifications in Industry-Sponsored Comparative Psychopharmacology Trials.’ Journal of Nervous and Mental Disease, 190: 583–92.
Schulz, K. F., Altman, D. G., Moher, D. and Fergusson, D. 2010. ‘CONSORT 2010 Changes and Testing Blindness in RCTs.’ Lancet, 375: 1144–6.
Sekeres, M., Gold, J. L., Chan, A.-W. et al. 2008. ‘Poor Reporting of Scientific Leadership Information in Clinical Trial Registers.’ PLoS One, 3(2): 16.
Simes, R. J. 1986. ‘Publication Bias: The Case for an International Registry of Clinical Trials.’ Journal of Clinical Oncology, 4: 1529–41.
Simpson, G. M., and Lindenmayer, J.-P. 1997. ‘Extrapyramidal Symptoms in Patients Treated with Risperidone.’ Journal of Clinical Psychopharmacology, 17(3): 194201.
Smith, R. 2003. ‘Medical Journals and Pharmaceutical Companies: Uneasy Bedfellows.’ British Medical Journal, 326: 1202–5.
Snapinn, S. M. 2000. ‘Noninferiority Trials.’ Current Controlled Trials in Cardiovascular Medicine 1: 1921.
Solomon, M. and Richardson, A. 2005. ‘A Critical Context for Longino's Critical Contextual Empiricism.’ Studies in the History and Philosophy of Science, 36: 211–22.
Stern, J. M. 1997. ‘Publication Bias: Evidence of Delayed Publication in a Cohort Study of Clinical Research Projects.’ British Medical Journal, 315 (7109): 640–5.
The Lancet. 2011. Types of Article and Manuscript Requirements.
Thompson, J., Baird, P. and Downie, J. 2001. The Olivieri Report: The Complete Text of the Report of the Independent Inquiry Commissioned by the Canadian Association of University Teachers. Toronto: James Lorimer & Company.
Tramèr, M. R., Reynolds, D. J., Moore, R. A. and McQuay, H. J. 1997. ‘Impact of Covert Duplicate Publication on Meta-Analysis: A Case Study.’ British Medical Journal, 315(7109): 635–40.
Turner, E. H., Matthews, A. M., Linardators, E., Tell, R. A. and Rosenthal, R. 2008. ‘Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy.’ New England Journal of Medicine, 358: 252–60.
Viergever, R. F. and Ghersi, D. 2011. ‘The Quality of Registration of Clinical Trials.’ PLoS One, 6 (2): 18.
Weber, E. J., Callaham, M. L., Wears, R. L., Barton, C. and Young, G. 1998. ‘Unpublished Research From a Medical Specialty Meeting: Why Investigators Fail to Publish.’ Journal of the American Medical Association, 280: 257–9.
Weller, A. C. 2001. Editorial Peer Review: Its Strengths and Weaknesses. Medford, NJ: American Society for Information Science and Technology.
Zarin, D. A., Ide, N. C., Tse, T. et al. 2007. ‘Issues in the Registration of Clinical Trials.’ Journal of the American Medical Association, 297: 2112–20.
Ziman, J. 1969. ‘Information, Communication, Knowledge.’ Nature 224: 318–24.
Zuckerman, H. and Merton, R. K. 1971. ‘Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System.’ Minerva 9: 66100.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 1742-3600
  • EISSN: 1750-0117
  • URL: /core/journals/episteme
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed