Skip to main content
×
×
Home

How reliable are scientific studies?

  • Marcus R. Munafò (a1) and Jonathan Flint (a2)
Abstract
Summary

There is growing concern that a substantial proportion of scientific research may in fact be false. A number of factors have been proposed as contributing to the presence of a large number of false-positive results in the literature, one of which is publication bias. We discuss empirical evidence for these factors.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      How reliable are scientific studies?
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      How reliable are scientific studies?
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      How reliable are scientific studies?
      Available formats
      ×
Copyright
Corresponding author
Marcus R. Munafò, Department of Experimental Psychology, University of Bristol, 12a Priory Road, Bristol BS8 1TU, UK. Email: marcus.munafo@bristol.ac.uk
Footnotes
Hide All

M.R.M. is supported by the Higher Education Funding Council for England (HEFCE). J.F. is supported by the Wellcome Trust.

Declaration of interest

Both authors hold opinions which are likely to influence their interpretation of evidence, including that presented here.

Footnotes
References
Hide All
1 Sullivan, PF. Spurious genetic associations. Biol Psychiatry 2007; 61: 1121–6.
2 Ioannidis, JP, Trikalinos, TA. An exploratory test for an excess of significant findings. Clin Trials 2007; 4: 245–53.
3 Munafò, MR, Durrant, C, Lewis, G, Flint, J. Gene x environment interactions at the serotonin transporter locus. Biol Psychiatry 2009; 65: 211–9.
4 Munafò, MR, Freimer, NB, Ng, W, Ophoff, R, Veijola, J, Miettunen, J, et al. 5-HTTLPR genotype and anxiety-related personality traits: a meta-analysis and new data. Am J Med Genet B Neuropsychiatr Genet 2009; 150B: 271–81.
5 Munafò, MR, Brown, SM, Hariri, AR. Serotonin transporter (5-HTTLPR) genotype and amygdala activation: a meta-analysis. Biol Psychiatry 2008; 63: 852–7.
6 Field, M, Munafò, MR, Franken, IH. A meta-analytic investigation of the relationship between attentional bias and subjective craving in substance abuse. Psychol Bull 2009; 135: 589607.
7 Ioannidis, JP. Why most published research findings are false. PLoS Med 2005; 2: e124.
8 Munafò, MR, Stothart, G, Flint, J. Bias in genetic association studies and impact factor. Mol Psychiatry 2009; 14: 119–20.
9 Munafò, MR, Attwood, AS, Flint, J. Bias in genetic association studies: effects of research location and resources. Psychol Med 2008; 38: 1213–4.
10 Martinson, BC, Anderson, MS, de Vries, R. Scientists behaving badly. Nature 2005; 435: 737–8.
11 Wellcome Trust Case Control Consortium. Genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls. Nature 2007; 447: 661–78.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The British Journal of Psychiatry
  • ISSN: 0007-1250
  • EISSN: 1472-1465
  • URL: /core/journals/the-british-journal-of-psychiatry
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 62 *
Loading metrics...

Abstract views

Total abstract views: 249 *
Loading metrics...

* Views captured on Cambridge Core between 2nd January 2018 - 23rd April 2018. This data will be updated every 24 hours.

How reliable are scientific studies?

  • Marcus R. Munafò (a1) and Jonathan Flint (a2)
Submit a response

eLetters

The dreaded P-value

saad f ghalib, consultant in old age psychiatry
24 November 2010

The article by Munafo and Flint is very timely.Certainly, attempts totackle the shortcomings of scientific methodology is not an easy task.Especially, if we agree that even poorly conducted science is hard todislodge as long it continues to be popular and well funded. However,I have picked on P-values as one area out of many where so much misunderstanding takes place and sometimes frank abuse. A p-value attempts to quantify our ignorance rather than establish any reality. It is about chance not material causation.Contrary to common perception, p-value is not the probability of the null hypothesis being true, it is the probability of the observed results really being the case if we assume the null hypothesis to be true. For some unclear reason the latter would have to be accepted a priori(face value). The null hypothesiscan not be statistically tested. Hence, the p-value is a measure of evidence against the NH, not for it. Sometime ago Ronald Fisher suggested that p-values should only be used for RCTs.Very often side effects are being interpreted based on p-values. Side effects are not hypotheses to betested,but simply observations to be reported in actual effect size. We also have the worrying situation where evidence from large meta-analyses is given a very high weight.This should not be the case if one considers that meta-analysis is an observational study of studies,therefore, the power of randomization is lost. I certainly agree with the authors, that more emphesis should be placed ona bayesian approach in designing and interpreting studies.It is the only way where causality can be assessed both ways,i.e. if an hypothesis (A) finds evidence (X), what is the likely chance that finding (X)is supportive that (A) is true? So far, research in psychiatry is mainly about the formar, not the latter. The human brain has an extra ordinary ability to accommodate noise.However, this can edit off important information unless one is wellequipped with good understanding of scientific methodology to tackle theseissues head on. Declaration of interest: none ... More

Conflict of interest: None Declared

Write a reply

Conformity with CONSORT

Vasudevan Krishnan, Specialty Registrar Year 2
10 November 2010

This article has come at a very interesting time when we at our localMRCPsych training were discussing about the reliability of research studies and biases involved.

One bias that I feel is pertinent to talk about is reporting bias.Theway an article is presented and the language used clearly has an impact onhow it is read and unknowingly leads many readers into making less robust appraisals of articles.

The CONSORT(Consolidated Standards of Reporting Trials) statement is one that was developed to help authors present research and trials in a manner that helps improve reporting.

It is amazing to see how acknowledged studies have failed to conform to the CONSORT guidelines.Whilst it may not be legally binding to concur with CONSORT,lack of so may indicate certain weaknesses in certain trials and therefore require more introspection.

I believe there should be more impetus put into making reporting of trials using guidelines such as those of CONSORT.

Declaration of interest: None

Reference

http://www.biomedcentral.com/content/pdf/1471-2288-1-2.pdf
... More

Conflict of interest: None Declared

Write a reply

×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *