Skip to main content
×
Home

Citation bias and selective focus on positive findings in the literature on the serotonin transporter gene (5-HTTLPR), life stress and depression

  • Y. A. de Vries (a1), A. M. Roest (a1), M. Franzen (a2), M. R. Munafò (a3) (a4) and J. A. Bastiaansen (a1)...
Abstract
Background

Caspi et al.'s 2003 report that 5-HTTLPR genotype moderates the influence of life stress on depression has been highly influential but remains contentious. We examined whether the evidence base for the 5-HTTLPR–stress interaction has been distorted by citation bias and a selective focus on positive findings.

Method

A total of 73 primary studies were coded for study outcomes and focus on positive findings in the abstract. Citation rates were compared between studies with positive and negative results, both within this network of primary studies and in Web of Science. In addition, the impact of focus on citation rates was examined.

Results

In all, 24 (33%) studies were coded as positive, but these received 48% of within-network and 68% of Web of Science citations. The 38 (52%) negative studies received 42 and 23% of citations, respectively, while the 11 (15%) unclear studies received 10 and 9%. Of the negative studies, the 16 studies without a positive focus (42%) received 47% of within-network citations and 32% of Web of Science citations, while the 13 (34%) studies with a positive focus received 39 and 51%, respectively, and the nine (24%) studies with a partially positive focus received 14 and 17%.

Conclusions

Negative studies received fewer citations than positive studies. Furthermore, over half of the negative studies had a (partially) positive focus, and Web of Science citation rates were higher for these studies. Thus, discussion of the 5-HTTLPR–stress interaction is more positive than warranted. This study exemplifies how evidence-base-distorting mechanisms undermine the authenticity of research findings.

Copyright
Corresponding author
*Address for correspondence: Y. A. de Vries, M.Sc., Department of Psychiatry, University Medical Center Groningen, Hanzeplein 1, 9713 GZ Groningen, The Netherlands. (Email: y.a.de.vries@umcg.nl)
References
Hide All
Bastiaansen JA, de Vries YA, Munafò MR (2015). Citation distortions in the literature on the serotonin-transporter-linked polymorphic region and amygdala activation. Biological Psychiatry 78, E35E36.
Bastiaansen JA, Servaas MN, Marsman JBC, Ormel J, Nolte IM, Riese H, Aleman A (2014). Filling the gap: relationship between the serotonin-transporter-linked polymorphic region and amygdala activation. Psychological Science 25, 20582066.
Boutron I, Dutton S, Ravaud P, Altman DG (2010). Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 303, 20582064.
Caspi A, Hariri AR, Holmes A, Uher R, Moffitt TE (2010). Genetic sensitivity to the environment: the case of the serotonin transporter gene and its implications for studying complex diseases and traits. American Journal of Psychiatry 167, 509527.
Caspi A, Sugden K, Moffitt TE, Taylor A, Craig IW, Harrington HL, McClay J, Mill J, Martin J, Braithwaite AW, Poulton R (2003). Influence of life stress on depression: moderation by a polymorphism in the 5-HTT gene. Science 301, 386389.
Culverhouse RC, Bowes L, Breslau N, Nurnberger JI, Burmeister M, Fergusson DM, Munafò MR, Saccone NL, Bierut LJ (2013). Protocol for a collaborative meta-analysis of 5-HTTLPR, stress, and depression. BMC Psychiatry 13, 304.
De Jonge P, Conradi HJ, Thombs BD, Rosmalen JGM, Burger H, Ormel J (2011). Prevention of false positive findings in observational studies: registration will not work but replication might. Journal of Epidemiology and Community Health 65, 9596.
Duncan L, Keller M (2011). A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry. American Journal of Psychiatry 168, 10411049.
Etter J-F, Stapleton J (2009). Citations to trials of nicotine replacement therapy were biased toward positive results and high-impact-factor journals. Journal of Clinical Epidemiology 62, 831837.
Greenberg SA (2009). How citation distortions create unfounded authority: analysis of a citation network. BMJ 339, b2680.
Hammen C (2005). Stress and depression. Annual Review of Clinical Psychology 1, 293319.
Heininga VE, Oldehinkel AJ, Veenstra R, Nederhof E (2015). I just ran a thousand analyses: benefits of multiple testing in understanding equivocal evidence on gene–environment interactions. PLOS ONE 10, e0125383.
Hu X, Oroszi G, Chun J, Smith TL, Goldman D, Schuckit MA (2005). An expanded evaluation of the relationship of four alleles to the level of response to alcohol and the alcoholism risk. Alcoholism: Clinical and Experimental Research 29, 816.
Jannot A-S, Agoritsas T, Gayet-Ageron A, Perneger TV (2013). Citation bias favoring statistically significant studies was present in medical research. Journal of Clinical Epidemiology 66, 296301.
Kapur S, Phillips AG, Insel TR (2012). Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Molecular Psychiatry 17, 11741179.
Karg K, Burmeister M, Shedden K, Sen S (2011). The serotonin transporter promoter variant (5-HTTLPR), stress, and depression meta-analysis revisited: evidence of genetic moderation. Archives of General Psychiatry 68, 444454.
Kjaergard LL, Gluud C (2002). Citation bias of hepato-biliary randomized clinical trials. Journal of Clinical Epidemiology 55, 407410.
Martin J, Cleak J, Willis-Owen SAG, Flint J, Shifman S (2007). Mapping regulatory variants for the serotonin transporter gene based on allelic expression imbalance. Molecular Psychiatry 12, 421422.
Munafò MR, Durrant C, Lewis G, Flint J (2009). Gene x environment interactions at the serotonin transporter locus. Biological Psychiatry 65, 211219.
Murphy SE, Norbury R, Godlewska BR, Cowen PJ, Mannie ZM, Harmer CJ, Munafò MR (2013). The effect of the serotonin transporter polymorphism (5-HTTLPR) on amygdala function: a meta-analysis. Molecular Psychiatry 18, 512520.
Nieminen P, Rucker G, Miettunen J, Carpenter J, Schumacher M (2007). Statistically significant papers in psychiatry were cited more often than others. Journal of Clinical Epidemiology 60, 939946.
Nosek BA, Spies JR, Motyl M (2012). Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science 7, 615631.
Park I-U, Peacey MW, Munafò MR (2014). Modelling the effects of subjective and objective decision making in scientific peer review. Nature 506, 9396.
Risch N, Herrell R, Lehner T, Liang K, Eaves L, Hoh J, Griem A, Kovacs M, Ott J, Merikangas KR (2009). Interaction between the serotonin transporter gene (5-HTTLPR), stressful life events, and risk of depression. JAMA 301, 24622471.
Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH (2015). Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry 72, 500510.
Sharpley CF, Palanisamy SKA, Glyde NS, Dillingham PW, Agnew LL (2014). An update on the interaction between the serotonin transporter promoter variant (5-HTTLPR), stress and depression, plus an exploration of non-confirming findings. Behavioural Brain Research 273, 89105.
Simmons JP, Nelson LD, Simonsohn U (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science 22, 13591366.
Song F, Parekh-Bhurke S, Hooper L, Loke YK, Ryder JJ, Sutton AJ, Hing CB, Harvey I (2009). Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Medical Research Methodology 9, 79.
Sullivan PF (2007). Spurious genetic associations. Biological Psychiatry 61, 11211126.
Sullivan PF, Neale MC, Kendler KS (2000). Genetic epidemiology of major depression: review and meta-analysis. American Journal of Psychiatry 157, 15521562.
Wendland JR, Martin BJ, Kruse MR, Lesch K-P, Murphy DL (2006). Simultaneous genotyping of four functional loci of human SLC6A4, with a reappraisal of 5-HTTLPR and rs25531. Molecular Psychiatry 11, 224226.
Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, Charlson FJ, Norman RE, Flaxman AD, Johns N, Burstein R, Murray CJL, Vos T (2013). Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 382, 15751586.
Zammit S, Owen MJ, Lewis G (2010). Misconceptions about gene–environment interactions in psychiatry. Evidence-Based Mental Health 13, 6568.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Psychological Medicine
  • ISSN: 0033-2917
  • EISSN: 1469-8978
  • URL: /core/journals/psychological-medicine
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords:

Type Description Title
WORD
Supplementary Materials

de Vries supplementary material
Tables S1-S4

 Word (29 KB)
29 KB
UNKNOWN
Supplementary Materials

de Vries supplementary material
Figure image

 Unknown (2.4 MB)
2.4 MB
PDF
Supplementary Materials

de Vries supplementary material
Flow chart

 PDF (45 KB)
45 KB
UNKNOWN
Supplementary Materials

de Vries supplementary material
Figure image

 Unknown (2.6 MB)
2.6 MB
WORD
Supplementary Materials

de Vries supplementary material
Table

 Word (145 KB)
145 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 10
Total number of PDF views: 165 *
Loading metrics...

Abstract views

Total abstract views: 636 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 23rd November 2017. This data will be updated every 24 hours.