Skip to main content Accessibility help
Hostname: page-component-78bd46657c-fstz4 Total loading time: 0.29 Render date: 2021-05-09T10:06:09.604Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false, "newCitedByModal": true }

Article contents


Published online by Cambridge University Press:  13 November 2013

E-mail address:


Under the traditional system of peer-reviewed publication, the degree of prestige conferred to authors by successful publication is tied to the degree of the intellectual rigor of its peer review process: ambitious scientists do well professionally by doing well epistemically. As a result, we should expect journal editors, in their dual role as epistemic evaluators and prestige-allocators, to have the power to motivate improved author behavior through the tightening of publication requirements. Contrary to this expectation, I will argue that the publication bias literature in academic medicine demonstrates that editor interventions have had limited effectiveness in improving the health of the publication and trial registration record, suggesting that much stronger interventions are needed.

Episteme , Volume 10 , Issue 4 , December 2013 , pp. 387 - 402
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below.


Allison, D. B., Faith, M. S. and Gorman, B. S. 1996. ‘Publication Bias in Obesity Treatment Trials?International Journal of Obesity and Related Metabolic Disorders, 20: 931–7.Google ScholarPubMed
Altman, D. G.for the CONSORT Group. 2005. ‘Endorsement of the CONSORT Statement by High Impact Medical Journals: Survey of Instructions for Authors.’ British Medical Journal, 330: 1056–7.CrossRefGoogle Scholar
Anderson, E. 1995. ‘Feminist Epistemology: An Interpretation and Defense.’ Hypatia, 10: 5084.CrossRefGoogle Scholar
Angell, M. 2004. The Truth about the Drug Companies: How they Deceive us and What to Do about It. New York, NY: Random House.Google Scholar
Barbui, C., Hotopf, M. and Garattini, S. 2002. ‘Fluoxetine Dose and Outcome in Antidepressant Drug Trials.’ European Journal of Clinical Pharmacology, 58: 379–86.CrossRefGoogle ScholarPubMed
Bardy, A. H. 1998. ‘Bias in Reporting Clinical Trials.’ British Journal of Clinical Pharmacology, 46: 147–50.CrossRefGoogle ScholarPubMed
Begg, C., Cho, M., Eastwood, S. et al. 1996. ‘Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement.’ Journal of the American Medical Association, 276: 637–9.CrossRefGoogle ScholarPubMed
Berlin, J. A., Begg, C. B. and Louis, T. A. 1989. ‘An Assessment of Publication Bias Using a Sample of Published Clinical Trials.’ Journal of the American Statistical Association, 84(406): 381–92.CrossRefGoogle Scholar
Born, G. V. R. and Collins, R. 1997. ‘Aspirin versus Clopidogrel: The Wrong Question?Lancet, 349(9054): 806–7.CrossRefGoogle ScholarPubMed
British Medical Journal. 2011. Welcome to Resources for Authors. Scholar
Brown, J. R. 2004. ‘Money, Method and Medical Research.’ Episteme, 1:4959.CrossRefGoogle Scholar
Brown, J. R. 2008. ‘Politics, Method, and Medical Research.’ Philosophy of Science, 75: 756–66.CrossRefGoogle Scholar
Bruijn, J. A. and van der Woude, F. 2002. ‘Tacrolimus and Ciclosporin Microemulsion in Renal Transplantation.’ Lancet, 360(9335): 800.CrossRefGoogle ScholarPubMed
CAPRIE Steering Committee. 1996. ‘A Randomized, Blinded, Trial of Clopidogrel versus Aspirin in Patients at Risk of Aschaemic Events (CAPRIE).’ Lancet, 348(9038): 1329–40.CrossRefGoogle Scholar
Casadevall, A. and Fang, F. C. 2012. ‘Reforming Science: Methodological and Cultural Reforms.’ Infection and Immunity, 80: 891–6.CrossRefGoogle ScholarPubMed
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C. and Altman, D. G. 2004. ‘Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials: Comparison of Protocols to Published Articles.’ Journal of the American Medical Association, 291: 2457–65.CrossRefGoogle ScholarPubMed
Dickersin, K. and Min, Y-.I. 1993. ‘NIH Clinical Trials and Publication Bias.’ Online Journal of Current Clinical Trials (Doc. No. 50).Google ScholarPubMed
Dickersin, K., Min, Y.-I. and Meinert, C. L. 1992. ‘Factors influencing Publication of Research Results: Follow-up of Applications Submitted to Two Institutional Review Boards.’ Journal of the American Medical Association, 267: 374–9.CrossRefGoogle ScholarPubMed
Dorsey, E. R., de Roulet, J., Thompson, J. P. et al. 2010. ‘Funding of US Biomedical Research 2003–2008.’ Journal of the American Medical Association, 303: 137–43.CrossRefGoogle ScholarPubMed
Douglas, H. 2000. ‘Inductive Risk and Values in Science.’ Philosophy of Science, 67: 559–79.CrossRefGoogle Scholar
Easterbrook, P. J., Berlin, J. A., Gopalan, R. and Matthews, D. R. 1991. ‘Publication Bias in Clinical Research.’ Lancet, 337(8746): 867–72.CrossRefGoogle ScholarPubMed
Egger, M. and Smith, G. D. 1995. ‘Misleading Meta-Analysis: Lessons from “an Effective, Safe, Simple” Intervention that Wasn't. (Study on Treatment of Heart Attacks).’ British Medical Journal, 310(6982): 752–4.CrossRefGoogle Scholar
Elliott, C. 2010. ‘Making a Killing.’ Mother Jones, 35(5):5463.Google Scholar
Fanelli, D. 2009. ‘How Many Scientists Fabricate and Falsify Research? A Systematic Review and Meta-Analysis of Survey Data.’ PLoS One, 4(5):111.CrossRefGoogle ScholarPubMed
Fang, F. C. and Casadevall, A. 2011. ‘Retracted Science and the Retraction Index.’ Infection and Immunity, 79: 3855–9.CrossRefGoogle ScholarPubMed
Fontanarosa, P. B. and DeAngelis, C. D. 2004. ‘Information for JAMA Authors – 2004.’ Journal of the American Medical Association, 291:107–8.CrossRefGoogle Scholar
Garattini, S., Bertele, V. and Bassi, L. L. 2003. ‘How Can Research Ethics Committees Protect Patients Better?British Medical Journal, 326: 1199–201.CrossRefGoogle ScholarPubMed
Grady, D., Herrington, D., Bittner, V. et al. 2002. ‘Cardiovascular Disease Outcomes During 6.8 Years of Hormone Therapy: Heart and Estrogen/Progestin Replacement Study Follow-up (HERSII).’ Journal of the American Medical Association, 288: 4957.CrossRefGoogle Scholar
Grieneisen, M. L. and Zhang, M. 2012. ‘A Comprehensive Survey of Retracted Articles from the Scholarly Literature.’ PLoS One 7 (10). doi: 10.1371/journal.pone.0044118.CrossRefGoogle ScholarPubMed
Horton, R. 2004. ‘Review: The Dawn of McScience.’ The New York Review of Books, March 11: 79.Google Scholar
Hulley, S., Grady, D., Bush, T. et al. 1998. ‘Randomized Trial of Estrogen Plus Progestin for Secondary Prevention of Coronary Heart Diease in Postmenopausal Women.’ Journal of the American Medical Association 280: 605–13.CrossRefGoogle Scholar
ICMJE. ‘Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication [2004].’ Accessed March 1, 2013. Scholar
ICMJE. ‘Announcements, Updates & Editorials: Sponsorship, Authorship, and Accountability [2007].’ Accessed March 1, 2013. Scholar
Intemann, K. and de Melo-Martin, I. 2010. ‘Social Values and Scientific Evidence: The Case of the HPV Vaccines.’ Biology and Philosophy, 25: 203–13.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. 1998. ‘Effect of the Statistical Significance of Results on the Time to Completion and Publication of Randomized Efficacy Trials.’ Journal of the American Medical Association, 279: 281–6.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. 2005a. ‘Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.’ Journal of the American Medical Association, 294: 218–28.CrossRefGoogle Scholar
Ioannidis, J. P. A. 2005b. ‘Why Most Published Research Findings are False.’ PLoS Medicine, 2: 696701.CrossRefGoogle Scholar
Ioannidis, J. P. A. 2009. ‘Adverse Events in Randomized Trials: Neglected, Restricted, Distorted, and Silenced.’ Archives of Internal Medicine, 169: 1737–9.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C. et al. 2004. ‘Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement.’ Annals of Internal Medicine, 141: 781–8.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. and Lau, J. 2001. ‘Completeness of Safety Reporting in Randomized Trials: An Evaluation of Medical Areas.’ Journal of the American Medical Association 285: 437–43.CrossRefGoogle ScholarPubMed
Jennions, M. D. and Møller, A. P. 2002a. ‘Publication Bias in Ecology and Evolution: An Empirical Assessment Using the ‘Trim and Fill' Method.’ Biological Reviews, 77: 211–22.CrossRefGoogle Scholar
Jennions, M. D. and Møller, A. P. 2002b. ‘Relationships Fade with Time: A Meta-Analysis of Temporal Trends in Publication in Ecology and Evolution.’ Proceedings: Biological Sciences, 269(1486): 43–8.Google Scholar
Johansen, H. K. and Gøtzsche, P. C. 1999. ‘Problems in the Design and Reporting of Trials of Antifungal Agents Encountered During Meta-Analysis.’ Journal of the American Medical Association, 282(18): 1752–9.CrossRefGoogle ScholarPubMed
John, L. K., Loewenstein, G. and Prelec, D. 2012. ‘Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.’ Perspectives on Psychological Science 23: 524–32.CrossRefGoogle ScholarPubMed
Kane, R., Wang, J. and Garrard, J. 2007. ‘Reporting in Randomized Clinical Trials Improved after Adoption of the CONSORT Statement.’ Journal of Clinical Epidemiology 60: 241–9.CrossRefGoogle ScholarPubMed
Kirsch, I., Moore, T. J., Scoboria, A. and Nicholls, S. S. 2002. ‘The Emperor's New Drugs: Effect Size and Moderation Effects.’ Prevention and Treatment, 5(1): Art. 23.Google Scholar
Kitcher, P. 2001. Science, Truth, and Democracy. New York, NY: Oxford University Press.CrossRefGoogle Scholar
Kravitz, R. L. and Feldman, M. D. 2011. ‘From the Editor's Desk: Self-Plagiarism and Other Editorial Crimes and Misdemeanors.’ Journal of General Internal Medicine, 26: 1.CrossRefGoogle Scholar
Lawrence, B., Jones, C., Matthews, B., Pepler, S. and Callaghan, S. 2011. ‘Citation and Peer Review of Data: Moving towards Formal Data Publication.’ International Journal of Digital Curation, 6(2): 437.CrossRefGoogle Scholar
Lee, C. J. 2012. ‘A Kuhnian Critique of Psychometric Research on Peer Review.’ Philosophy of Science, 79: 859–70.CrossRefGoogle Scholar
Lee, C. J. and Schunn, C. D. 2010. ‘Philosophy Journal Practices and Opportunities for Bias.’ APA Newsletters: Newsletter on Feminism and Philosophy, 10(1): 510.Google Scholar
Lee, C. J. and Schunn, C. D. 2011. ‘Social Biases and Solutions for Procedural Objectivity.’ Hypatia: A Journal of Feminist Philosophy, 26: 352–73.CrossRefGoogle Scholar
Lee, C. J., Sugimoto, C. R., Zhang, G. and Cronin, B. 2013. ‘Bias in Peer Review.’ Journal of the American Society for Information Science, 64: 217.CrossRefGoogle Scholar
Lexchin, J., Bero, L. A., Djulbegovic, B. and Clark, O. 2003. ‘Pharmaceutical Industry Sponsorship and Research Outcome and Quality: A Systematic Review.’ British Medical Journal, 326: 1167–76.CrossRefGoogle Scholar
Longino, H. E. 1990. Science as Social Knowledge. Princeton, NJ: Princeton University Press.Google Scholar
Lundh, A., Sismondo, S., Busuioc, O. A. and Bero, L. 2012. ‘Industry Sponsorship and Research Outcome (Review).’ Cochrane Database of Systematic Reviews (12). doi: 10.1002/14651858.MR000033.pub2.CrossRefGoogle Scholar
Martin II, R. C. G., Brennan, M. F. and Jaques, D. P. 2002. ‘Quality of Complication Reporting in the Surgical Literature.’ Annals of Surgery, 235: 803–13.CrossRefGoogle Scholar
Mathieu, S., Boutron, I., Moher, D., Altman, D. G. and Ravaud, P. 2009. ‘Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials.’ Journal of the American Medical Association, 302: 977–84.CrossRefGoogle ScholarPubMed
Melander, H., Ahlqvist-Rastad, J., Meijer, G. and Beermann, B. 2003. ‘Evidence B(i)ased Medicine – Selective Reporting from Studies Sponsored by Pharmaceutical Industry: Review of Studies in New Drug Applications.’ British Medical Journal, 326: 1171–5.CrossRefGoogle ScholarPubMed
Merton, R. K. 1973. ‘The Ambivalence of Scientists (1963).’ In Storer, N. W. (ed.), The Sociology of Science: Theoretical and Empirical Investigations, pp. 383412. Chicago, IL: The University of Chicago Press.Google Scholar
Merton, R. K. and Zuckerman, H. 1971. ‘Institutional Patterns of Evaluation in Science (1971).’ In Storer, N. W. (ed.), The Sociology of Science: Theoretical and Empirical Investigations, pp. 460–96. Chicago, IL: The University of Chicago Press.Google Scholar
Moher, D., Jones, A. and Lepage, L. 2001. ‘Use of the CONSORT Statement and Quality of Reports of Randomized Trials: A Comparative Before-and-After Evaluation.’ Journal of the American Medical Association, 285: 1992–5.CrossRefGoogle Scholar
Neuliep, J. W. and Crandall, R. 1990. ‘Editorial Bias against Replication Research.’ Journal of Social Behavior and Personality, 5(4): 8590.Google Scholar
Neuliep, J. W. and Crandall, R. 1993. ‘Reviewer Bias against Replication Research.’ Journal of Social Behavior and Personality, 8(6): 21–9.Google Scholar
Oldenberg, H. 1966. Correspondence of Henry Oldenberg. Trans. by Rupert Hall, A. and Hall, Marie Boas. Vol. 2. Madison, WI: University of Wisconsin Press.Google Scholar
Olson, C. M., Rennie, D., Cook, D. et al. 2002. ‘Publication Bias in Editorial Decision Making.’ Journal of the American Medical Association, 287: 2825–8.CrossRefGoogle ScholarPubMed
Palmer, A. R. 2000. ‘Quasireplication and the Contract of Error: Lessons from Sex Ratios, Heritabilities and Fluctuating Asymmetry.’ Annual Review of Ecology and Systematics, 31: 441–80.CrossRefGoogle Scholar
Parsons, M. A., Duerr, R. and Minster, J.-B. 2010. ‘Data Citation and Peer Review.’ Eos, Transactions, American Geophysical Union, 91 (34): 297–8.CrossRefGoogle Scholar
Pipkin, F. B., Crowther, C., de Swiet, M. et al. 1996. ‘Where Next for Prophylaxis against Pre-Eclampsia?British Journal of Obstetrics and Gynaecology, 103: 603–7.CrossRefGoogle Scholar
Pitrou, I., Boutron, I., Ahmad, N. and Ravaud, P. 2009. ‘Reporting of Safety Results in Published Reports of Randomized Controlled Trials.’ Archives of Internal Medicine, 169: 1756–61.Google ScholarPubMed
Plint, A. C., Moher, D., Morrison, A. et al. 2006. ‘Does the CONSORT Checklist Improve the Quality of Reports of Randomised Controlled Trials? A Systematic Review.’ Medical Journal of Australia, 185: 263–7.Google Scholar
Prayle, A. P., Hurley, M. N. and Smyth, A. R. 2012. ‘Compliance with Mandatory Reporting of Clinical Trial Results on Cross sectional Study.’ British Medical Journal, 344: d7373.CrossRefGoogle ScholarPubMed
Psaty, B. M. and Kornmal, R. A. 2008. ‘Reporting Mortality Findings in Trials of Rofecoxib for Alzheimer Disease or Cognitive Impairment: A Case Study Based on Documents from Rofecoxib Litigation.’ Journal of the American Medical Association, 299: 1813–7.CrossRefGoogle ScholarPubMed
Reiss, J. 2010. ‘In favour of a Millian Proposal to Reform Biomedical Research.’ Synthese, 177: 427–47.CrossRefGoogle Scholar
Reiss, J. and Kitcher, P. 2009. ‘Biomedical Research, Neglected Diseases, and Well-Ordered Science.’ Theoria, 66: 263–82.Google Scholar
Rochon, P. A., Gurwitz, J. H., Simms, R. W. et al. 1994. ‘A Study of Manufacturer-Supported Trials of Nonsteroidal Anti-Inflammatory Drugs in the Treatment of Arthritis.’ Archives of Internal Medicine, 154: 157–63.CrossRefGoogle ScholarPubMed
Ross, J. S., Mulvey, G. K., Hines, E. M., Nissen, S. E. and Krumholz, H. M. 2009. ‘Trial Publication after Registration in ClinicalTrials.Gov: A cross-sectional analysis.’ PLoS Medicine, 6(9): 19.CrossRefGoogle ScholarPubMed
Rothman, K. J. 1986. ‘Significance Questing.’ Annals of Internal Medicine, 105: 445–7.CrossRefGoogle ScholarPubMed
Rudner, R. 1953. ‘The Scientist Qua Scientist Makes Value Judgments.’ Philosophy of Science, 20(1): 16.CrossRefGoogle Scholar
Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W. and Haynes, R. B. 2000. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd edn.Oxford: Churchill Livingstone.Google Scholar
Safer, D. J. 2002. ‘Design and Reporting Modifications in Industry-Sponsored Comparative Psychopharmacology Trials.’ Journal of Nervous and Mental Disease, 190: 583–92.CrossRefGoogle ScholarPubMed
Schulz, K. F., Altman, D. G., Moher, D. and Fergusson, D. 2010. ‘CONSORT 2010 Changes and Testing Blindness in RCTs.’ Lancet, 375: 1144–6.CrossRefGoogle ScholarPubMed
Sekeres, M., Gold, J. L., Chan, A.-W. et al. 2008. ‘Poor Reporting of Scientific Leadership Information in Clinical Trial Registers.’ PLoS One, 3(2): 16.CrossRefGoogle ScholarPubMed
Simes, R. J. 1986. ‘Publication Bias: The Case for an International Registry of Clinical Trials.’ Journal of Clinical Oncology, 4: 1529–41.CrossRefGoogle ScholarPubMed
Simpson, G. M., and Lindenmayer, J.-P. 1997. ‘Extrapyramidal Symptoms in Patients Treated with Risperidone.’ Journal of Clinical Psychopharmacology, 17(3): 194201.CrossRefGoogle ScholarPubMed
Smith, R. 2003. ‘Medical Journals and Pharmaceutical Companies: Uneasy Bedfellows.’ British Medical Journal, 326: 1202–5.CrossRefGoogle ScholarPubMed
Snapinn, S. M. 2000. ‘Noninferiority Trials.’ Current Controlled Trials in Cardiovascular Medicine 1: 1921.CrossRefGoogle ScholarPubMed
Solomon, M. and Richardson, A. 2005. ‘A Critical Context for Longino's Critical Contextual Empiricism.’ Studies in the History and Philosophy of Science, 36: 211–22.CrossRefGoogle Scholar
Stern, J. M. 1997. ‘Publication Bias: Evidence of Delayed Publication in a Cohort Study of Clinical Research Projects.’ British Medical Journal, 315 (7109): 640–5.CrossRefGoogle Scholar
Thompson, J., Baird, P. and Downie, J. 2001. The Olivieri Report: The Complete Text of the Report of the Independent Inquiry Commissioned by the Canadian Association of University Teachers. Toronto: James Lorimer & Company.Google Scholar
Tramèr, M. R., Reynolds, D. J., Moore, R. A. and McQuay, H. J. 1997. ‘Impact of Covert Duplicate Publication on Meta-Analysis: A Case Study.’ British Medical Journal, 315(7109): 635–40.CrossRefGoogle ScholarPubMed
Turner, E. H., Matthews, A. M., Linardators, E., Tell, R. A. and Rosenthal, R. 2008. ‘Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy.’ New England Journal of Medicine, 358: 252–60.CrossRefGoogle ScholarPubMed
Viergever, R. F. and Ghersi, D. 2011. ‘The Quality of Registration of Clinical Trials.’ PLoS One, 6 (2): 18.CrossRefGoogle ScholarPubMed
Weber, E. J., Callaham, M. L., Wears, R. L., Barton, C. and Young, G. 1998. ‘Unpublished Research From a Medical Specialty Meeting: Why Investigators Fail to Publish.’ Journal of the American Medical Association, 280: 257–9.CrossRefGoogle Scholar
Weller, A. C. 2001. Editorial Peer Review: Its Strengths and Weaknesses. Medford, NJ: American Society for Information Science and Technology.Google Scholar
Zarin, D. A., Ide, N. C., Tse, T. et al. 2007. ‘Issues in the Registration of Clinical Trials.’ Journal of the American Medical Association, 297: 2112–20.CrossRefGoogle ScholarPubMed
Ziman, J. 1969. ‘Information, Communication, Knowledge.’ Nature 224: 318–24.CrossRefGoogle ScholarPubMed
Zuckerman, H. and Merton, R. K. 1971. ‘Patterns of Evaluation in Science: Institutionalisation, Structure and Functions of the Referee System.’ Minerva 9: 66100.CrossRefGoogle Scholar

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Available formats

Reply to: Submit a response

Your details

Conflicting interests

Do you have any conflicting interests? *