Skip to main content
×
Home
    • Aa
    • Aa

Citation bias and selective focus on positive findings in the literature on the serotonin transporter gene (5-HTTLPR), life stress and depression

  • Y. A. de Vries (a1), A. M. Roest (a1), M. Franzen (a2), M. R. Munafò (a3) (a4) and J. A. Bastiaansen (a1)...
Abstract
Background

Caspi et al.'s 2003 report that 5-HTTLPR genotype moderates the influence of life stress on depression has been highly influential but remains contentious. We examined whether the evidence base for the 5-HTTLPR–stress interaction has been distorted by citation bias and a selective focus on positive findings.

Method

A total of 73 primary studies were coded for study outcomes and focus on positive findings in the abstract. Citation rates were compared between studies with positive and negative results, both within this network of primary studies and in Web of Science. In addition, the impact of focus on citation rates was examined.

Results

In all, 24 (33%) studies were coded as positive, but these received 48% of within-network and 68% of Web of Science citations. The 38 (52%) negative studies received 42 and 23% of citations, respectively, while the 11 (15%) unclear studies received 10 and 9%. Of the negative studies, the 16 studies without a positive focus (42%) received 47% of within-network citations and 32% of Web of Science citations, while the 13 (34%) studies with a positive focus received 39 and 51%, respectively, and the nine (24%) studies with a partially positive focus received 14 and 17%.

Conclusions

Negative studies received fewer citations than positive studies. Furthermore, over half of the negative studies had a (partially) positive focus, and Web of Science citation rates were higher for these studies. Thus, discussion of the 5-HTTLPR–stress interaction is more positive than warranted. This study exemplifies how evidence-base-distorting mechanisms undermine the authenticity of research findings.

Copyright
Corresponding author
*Address for correspondence: Y. A. de Vries, M.Sc., Department of Psychiatry, University Medical Center Groningen, Hanzeplein 1, 9713 GZ Groningen, The Netherlands. (Email: y.a.de.vries@umcg.nl)
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

JA Bastiaansen , YA de Vries , MR Munafò (2015). Citation distortions in the literature on the serotonin-transporter-linked polymorphic region and amygdala activation. Biological Psychiatry 78, E35E36.

JA Bastiaansen , MN Servaas , JBC Marsman , J Ormel , IM Nolte , H Riese , A Aleman (2014). Filling the gap: relationship between the serotonin-transporter-linked polymorphic region and amygdala activation. Psychological Science 25, 20582066.

I Boutron , S Dutton , P Ravaud , DG Altman (2010). Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303, 20582064.

A Caspi , AR Hariri , A Holmes , R Uher , TE Moffitt (2010). Genetic sensitivity to the environment: the case of the serotonin transporter gene and its implications for studying complex diseases and traits. American Journal of Psychiatry 167, 509527.

A Caspi , K Sugden , TE Moffitt , A Taylor , IW Craig , HL Harrington , J McClay , J Mill , J Martin , AW Braithwaite , R Poulton (2003). Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science 301, 386389.

RC Culverhouse , L Bowes , N Breslau , JI Nurnberger , M Burmeister , DM Fergusson , MR Munafò , NL Saccone , LJ Bierut (2013). Protocol for a collaborative meta-analysis of 5-HTTLPR, stress, and depression. BMC Psychiatry 13, 304.

P De Jonge , HJ Conradi , BD Thombs , JGM Rosmalen , H Burger , J Ormel (2011). Prevention of false positive findings in observational studies: registration will not work but replication might. Journal of Epidemiology and Community Health 65, 9596.

L Duncan , M Keller (2011). A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry. American Journal of Psychiatry 168, 10411049.

J-F Etter , J Stapleton (2009). Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. Journal of Clinical Epidemiology 62, 831837.

SA Greenberg (2009). How citation distortions create unfounded authority: analysis of a citation network. BMJ 339, b2680.

C Hammen (2005). Stress and depression. Annual Review of Clinical Psychology 1, 293319.

VE Heininga , AJ Oldehinkel , R Veenstra , E Nederhof (2015). I just ran a thousand analyses: benefits of multiple testing in understanding equivocal evidence on gene–environment interactions. PLOS ONE 10, e0125383.

X Hu , G Oroszi , J Chun , TL Smith , D Goldman , MA Schuckit (2005). An expanded evaluation of the relationship of four alleles to the level of response to alcohol and the alcoholism risk. Alcoholism: Clinical and Experimental Research 29, 816.

A-S Jannot , T Agoritsas , A Gayet-Ageron , TV Perneger (2013). Citation bias favoring statistically significant studies was present in medical research. Journal of Clinical Epidemiology 66, 296301.

S Kapur , AG Phillips , TR Insel (2012). Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Molecular Psychiatry 17, 11741179.

K Karg , M Burmeister , K Shedden , S Sen (2011). The serotonin transporter promoter variant (5-HTTLPR), stress, and depression meta-analysis revisited: evidence of genetic moderation. Archives of General Psychiatry 68, 444454.

LL Kjaergard , C Gluud (2002). Citation bias of hepato-biliary randomized clinical trials. Journal of Clinical Epidemiology 55, 407410.

J Martin , J Cleak , SAG Willis-Owen , J Flint , S Shifman (2007). Mapping regulatory variants for the serotonin transporter gene based on allelic expression imbalance. Molecular Psychiatry 12, 421422.

MR Munafò , C Durrant , G Lewis , J Flint (2009). Gene x environment interactions at the serotonin transporter locus. Biological Psychiatry 65, 211219.

SE Murphy , R Norbury , BR Godlewska , PJ Cowen , ZM Mannie , CJ Harmer , MR Munafò (2013). The effect of the serotonin transporter polymorphism (5-HTTLPR) on amygdala function: a meta-analysis. Molecular Psychiatry 18, 512520.

P Nieminen , G Rucker , J Miettunen , J Carpenter , M Schumacher (2007). Statistically significant papers in psychiatry were cited more often than others. Journal of Clinical Epidemiology 60, 939946.

BA Nosek , JR Spies , M Motyl (2012). Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7, 615631.

I-U Park , MW Peacey , MR Munafò (2014). Modelling the effects of subjective and objective decision making in scientific peer review. Nature 506, 9396.

N Risch , R Herrell , T Lehner , K Liang , L Eaves , J Hoh , A Griem , M Kovacs , J Ott , KR Merikangas (2009). Interaction between the serotonin transporter gene (5-HTTLPR), stressful life events, and risk of depression. JAMA 301, 24622471.

AM Roest , P de Jonge , CD Williams , YA de Vries , RA Schoevers , EH Turner (2015). Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry 72, 500510.

CF Sharpley , SKA Palanisamy , NS Glyde , PW Dillingham , LL Agnew (2014). An update on the interaction between the serotonin transporter promoter variant (5-HTTLPR), stress and depression, plus an exploration of non-confirming findings. Behavioural Brain Research 273, 89105.

JP Simmons , LD Nelson , U Simonsohn (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22, 13591366.

F Song , S Parekh-Bhurke , L Hooper , YK Loke , JJ Ryder , AJ Sutton , CB Hing , I Harvey (2009). Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Medical Research Methodology 9, 79.

PF Sullivan (2007). Spurious genetic associations. Biological Psychiatry 61, 11211126.

PF Sullivan , MC Neale , KS Kendler (2000). Genetic epidemiology of major depression: review and meta-analysis. American Journal of Psychiatry 157, 15521562.

JR Wendland , BJ Martin , MR Kruse , K-P Lesch , DL Murphy (2006). Simultaneous genotyping of four functional loci of human SLC6A4, with a reappraisal of 5-HTTLPR and rs25531. Molecular Psychiatry 11, 224226.

HA Whiteford , L Degenhardt , J Rehm , AJ Baxter , AJ Ferrari , HE Erskine , FJ Charlson , RE Norman , AD Flaxman , N Johns , R Burstein , CJL Murray , T Vos (2013). Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 382, 15751586.

S Zammit , MJ Owen , G Lewis (2010). Misconceptions about gene–environment interactions in psychiatry. Evidence-Based Mental Health 13, 6568.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Psychological Medicine
  • ISSN: 0033-2917
  • EISSN: 1469-8978
  • URL: /core/journals/psychological-medicine
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords:

Type Description Title
WORD
Supplementary Materials

de Vries supplementary material
Tables S1-S4

 Word (29 KB)
29 KB
UNKNOWN
Supplementary Materials

de Vries supplementary material
Figure image

 Unknown (2.4 MB)
2.4 MB
PDF
Supplementary Materials

de Vries supplementary material
Flow chart

 PDF (45 KB)
45 KB
UNKNOWN
Supplementary Materials

de Vries supplementary material
Figure image

 Unknown (2.6 MB)
2.6 MB
WORD
Supplementary Materials

de Vries supplementary material
Table

 Word (145 KB)
145 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 8
Total number of PDF views: 146 *
Loading metrics...

Abstract views

Total abstract views: 572 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 24th September 2017. This data will be updated every 24 hours.