Skip to main content

Quality of reporting and risk of bias in therapeutic otolaryngology publications

  • N M Kaper (a1), K M A Swart (a1), W Grolman (a1) and G J M G Van Der Heijden (a1) (a2)
Abstract Background:

High-quality trials have the potential to influence clinical practice.


Ten otolaryngology journals with the highest 2011 impact factors were selected and publications from 2010 were extracted. From all medical journals, the 20 highest impact factor journals were selected, and publications related to otolaryngology for 2010 and 2011 were extracted. For all publications, the reporting quality and risk of bias were assessed.


The impact factor was 1.8–2.8 for otolaryngology journals and 6.0–101.8 for medical journals. Of 1500 otolaryngology journal articles, 262 were therapeutic studies; 94 had a high reporting quality and 5 a low risk of bias. Of 10 967 medical journal articles, 76 were therapeutic studies; 57 had a high reporting quality and 8 a low risk of bias.


Reporting quality was high for 45 per cent of otolaryngology-related publications and 9 per cent met quality standards. General journals had higher impact factors than otolaryngology journals. Reporting quality was higher and risk of bias lower in general journals than in otolaryngology journals. Nevertheless, 76 per cent of articles in high impact factor journals carried a high risk of bias. Better reported and designed studies are the goal, with less risk of bias, especially in otolaryngology journals.

Corresponding author
Address for correspondence: Dr Nina M Kaper, Department of Otorhinolaryngology and Head and Neck Surgery, Brain Center Rudolf Magnus, University Medical Center Utrecht, Heidelberglaan 100, PO Box 85500, 3508 GA Utrecht, Utrecht, The Netherlands Fax: +31 (0)30 254 1922 E-mail:
Hide All
1 Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K et al. Sicily statement on evidence-based practice. BMC Med Educ 2005;5:1
2 Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet 2003;362:1225–30
3 Burton MJ. Evidence-based medicine and otolaryngology–HNS: passing fashion or permanent solution. Otolaryngol Head Neck Surg 2007;137:4751S
4 Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, MacLeod MR, Moher D et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014;383:166–75
5 Shin JJ, Rauch SD, Wasserman J, Coblens O, Randolph GW. Evidence-based medicine in otolaryngology, part 2: the current state of affairs. Otolaryngol Head Neck Surg 2011;144:331–6
6 Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev 2012;1:60
7 The CONSORT statement. In: [11 August 2016]
8 STROBE statement. In: [11 August 2016]
9 Brody S. Impact factor is the best operational measure for medical journals. Lancet 1995;346:1300–1
10 The Clarivate Analytics Impact Factor. In: [11 August 2016]
11 Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314:498502
12 Science in Transition. In: [11 August 2016]
13 Journal Citation Report 2011. In: [6 November 2017]
14 McMaster University – HiRU's Approach to Search Filter Development. In: [11 August 2016]
15 Grobbee DE, Hoes AW. Clinical Epidemiology: Principles, Methods, and Applications for Clinical Research. London: Jones and Bartlett Publishers, 2009
16 Higgins JP, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 (updated March 2011). London: Cochrane Collaboration, 2011
17 Higgins JP, Altman DG, Sterne JA. Chapter 8: Assessing risk of bias in included studies. In: Higgins JP, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 (updated March 2011). London: Cochrane Collaboration, 2011
18 Glasziou P, Haynes B. The paths from research to improved health outcomes. ACP J Club 2005;142:A810
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The Journal of Laryngology & Otology
  • ISSN: 0022-2151
  • EISSN: 1748-5460
  • URL: /core/journals/journal-of-laryngology-and-otology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 3
Total number of PDF views: 19 *
Loading metrics...

Abstract views

Total abstract views: 138 *
Loading metrics...

* Views captured on Cambridge Core between 12th December 2017 - 17th January 2018. This data will be updated every 24 hours.