Skip to main content Accessibility help

Bias in emerging biomarkers for bipolar disorder

  • A. F. Carvalho (a1), C. A. Köhler (a1), B. S. Fernandes (a2) (a3), J. Quevedo (a4) (a5), K. W. Miskowiak (a6), A. R. Brunoni (a7) (a8), R. Machado-Vieira (a9) (a10) (a11), M. Maes (a2), E. Vieta (a12) and M. Berk (a2) (a13)...



To date no comprehensive evaluation has appraised the likelihood of bias or the strength of the evidence of peripheral biomarkers for bipolar disorder (BD). Here we performed an umbrella review of meta-analyses of peripheral non-genetic biomarkers for BD.


The Pubmed/Medline, EMBASE and PsycInfo electronic databases were searched up to May 2015. Two independent authors conducted searches, examined references for eligibility, and extracted data. Meta-analyses in any language examining peripheral non-genetic biomarkers in participants with BD (across different mood states) compared to unaffected controls were included.


Six references, which examined 13 biomarkers across 20 meta-analyses (5474 BD cases and 4823 healthy controls) met inclusion criteria. Evidence for excess of significance bias (i.e. bias favoring publication of ‘positive’ nominally significant results) was observed in 11 meta-analyses. Heterogeneity was high for (I 2 ⩾ 50%) 16 meta-analyses. Only two biomarkers met criteria for suggestive evidence namely the soluble IL-2 receptor and morning cortisol. The median power of included studies, using the effect size of the largest dataset as the plausible true effect size of each meta-analysis, was 15.3%.


Our findings suggest that there is an excess of statistically significant results in the literature of peripheral biomarkers for BD. Selective publication of ‘positive’ results and selective reporting of outcomes are possible mechanisms.


Corresponding author

*Address for correspondence: A. F. Carvalho, MD, PhD, Department of Clinical Medicine, Faculty of Medicine, Federal University of Ceará, Rua Prof. Costa Mendes, 1608, 4 andar, 60430-040, Fortaleza, CE, Brazil. (Email:;


Hide All
APA (2013). Diagnostic and Statistical Manual of Mental Disorders (DSM-5). American Psychiatric Publishing: Arlington, VA.
Belbasis, L, Bellou, V, Evangelou, E, Ioannidis, JP, Tzoulaki, I (2015). Environmental risk factors and multiple sclerosis: an umbrella review of systematic reviews and meta-analyses. Lancet Neurology 14, 263273.
Berk, M, Kapczinski, F, Andreazza, AC, Dean, OM, Giorlando, F, Maes, M, Yucel, M, Gama, CS, Dodd, S, Dean, B, Magalhaes, PV, Amminger, P, McGorry, P, Malhi, GS (2011). Pathways underlying neuroprogression in bipolar disorder: focus on inflammation, oxidative stress and neurotrophic factors. Neuroscience and Biobehavioral Reviews 35, 804817.
Birmaher, B, Gill, MK, Axelson, DA, Goldstein, BI, Goldstein, TR, Yu, H, Liao, F, Iyengar, S, Diler, RS, Strober, M, Hower, H, Yen, S, Hunt, J, Merranko, JA, Ryan, ND, Keller, MB (2014). Longitudinal trajectories and associated baseline predictors in youths with bipolar spectrum disorders. American Journal of Psychiatry 171, 990999.
Brown, NC, Andreazza, AC, Young, LT (2014). An updated meta-analysis of oxidative stress markers in bipolar disorder. Psychiatry Research 218, 6168.
Button, KS, Ioannidis, JP, Mokrysz, C, Nosek, BA, Flint, J, Robinson, ES, Munafo, MR (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14, 365376.
Cuijpers, P, Smit, F, Bohlmeijer, E, Hollon, SD, Andersson, G (2010). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry 196, 173178.
Cuthbert, BN (2014). The RDoC framework: facilitating transition from ICD/DSM to dimensional approaches that integrate neuroscience and psychopathology. World Psychiatry 13, 2835.
Dargel, AA, Godin, O, Kapczinski, F, Kupfer, DJ, Leboyer, M (2015). C-reactive protein alterations in bipolar disorder: a meta-analysis. Journal of Clinical Psychiatry 76, 142150.
Davis, J, Maes, M, Andreazza, A, McGrath, JJ, Tye, SJ, Berk, M (2015). Towards a classification of biomarkers of neuropsychiatric disease: from encompass to compass. Molecular Psychiatry 20, 152153.
Diniz, BS, Sibille, E, Ding, Y, Tseng, G, Aizenstein, HJ, Lotrich, F, Becker, JT, Lopez, OL, Lotze, MT, Klunk, WE, Reynolds, CF, Butters, MA (2015). Plasma biosignature and brain pathology related to persistent cognitive impairment in late-life depression. Molecular Psychiatry 20, 594601.
Dupont, WD, Plummer, WD Jr. (1990). Power and sample size calculations. A review and computer program. Controlled Clinical Trials 11, 116128.
Egger, M, Davey Smith, G, Schneider, M, Minder, C (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal 315, 629634.
Fava, GA (2014). Road to nowhere. World Psychiatry 13, 4950.
Fava, GA, Guidi, J, Grandi, S, Hasler, G (2014). The missing link between clinical states and biomarkers in mental disorders. Psychotherapy and Psychosomatics 83, 136141.
Fernandes, BS, Gama, CS, Cereser, KM, Yatham, LN, Fries, GR, Colpo, G, de Lucena, D, Kunz, M, Gomes, FA, Kapczinski, F (2011). Brain-derived neurotrophic factor as a state-marker of mood episodes in bipolar disorders: a systematic review and meta-regression analysis. Journal of Psychiatric Research 45, 9951004.
Fernandes, BS, Steiner, J, Berk, M, Molendijk, ML, Gonzalez-Pinto, A, Turck, CW, Nardin, P, Goncalves, CA (2015). Peripheral brain-derived neurotrophic factor in schizophrenia and the role of antipsychotics: meta-analysis and implications. Molecular Psychiatry 20, 11081119.
First, MB (2014). Preserving the clinician-researcher interface in the age of RDoC: the continuing need for DSM-5/ICD-11 characterization of study populations. World Psychiatry 13, 5354.
Flint, J, Cuijpers, P, Horder, J, Koole, SL, Munafo, MR (2015). Is there an excess of significant findings in published studies of psychotherapy for depression? Psychological Medicine 45, 439446.
Frey, BN, Andreazza, AC, Houenou, J, Jamain, S, Goldstein, BI, Frye, MA, Leboyer, M, Berk, M, Malhi, GS, Lopez-Jaramillo, C, Taylor, VH, Dodd, S, Frangou, S, Hall, GB, Fernandes, BS, Kauer-Sant'Anna, M, Yatham, LN, Kapczinski, F, Young, LT (2013). Biomarkers in bipolar disorder: a positional paper from the International Society for Bipolar Disorders Biomarkers Task Force. Australian and New Zealand Journal of Psychiatry 47, 321332.
Fusar-Poli, P, Radua, J, Frascarelli, M, Mechelli, A, Borgwardt, S, Di Fabio, F, Biondi, M, Ioannidis, JP, David, SP (2014). Evidence of reporting biases in voxel-based morphometry (VBM) studies of psychiatric and neurological disorders. Human Brain Mapping 35, 30523065.
Girshkin, L, Matheson, SL, Shepherd, AM, Green, MJ (2014). Morning cortisol levels in schizophrenia and bipolar disorder: a meta-analysis. Psychoneuroendocrinology 49, 187206.
Goldstein, BI, Young, LT (2013). Toward clinically applicable biomarkers in bipolar disorder: focus on BDNF, inflammatory markers, and endothelial function. Current Psychiatry Reports 15, 425.
Higgins, J, Green, S (2013). Cochrane Handbook for systematic reviews of interventions, version 5.0. 2 (updated September 2009). The Cochrane Collaboration, 2009 ( Accessed 18 August 2014.
Higgins, JP (2008). Commentary: Heterogeneity in meta-analysis should be expected and appropriately quantified. International Journal of Epidemiology 37, 11581160.
Higgins, JP, Thompson, SG (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 21, 15391558.
Higgins, JP, Thompson, SG, Spiegelhalter, DJ (2009). A re-evaluation of random-effects meta-analysis. Journal of the Royal Statistical Society. Series A (Statistics in Society) 172, 137159.
Huedo-Medina, TB, Sánchez-Meca, J, Marin-Martinez, F, Botella, J (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods 11, 193.
Insel, TR, Landis, SC (2013). Twenty-five years of progress: the view from NIMH and NINDS. Neuron 80, 561567.
Ioannidis, JP (2005). Why most published research findings are false. PLoS Medicine 2, e124.
Ioannidis, JP (2013). Clarifications on the application and interpretation of the test for excess significance and its extensions. Journal of Mathematical Psychology 57, 184187.
Ioannidis, JP, Panagiotou, OA (2011). Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. Journal of the American Medical Association 305, 22002210.
Ioannidis, JP, Tarone, R, McLaughlin, JK (2011). The false-positive to false-negative ratio in epidemiologic studies. Epidemiology 22, 450456.
Ioannidis, JP, Trikalinos, TA (2007). An exploratory test for an excess of significant findings. Clinical Trials (London, England) 4, 245253.
Janelidze, S, Ventorp, F, Erhardt, S, Hansson, O, Minthon, L, Flax, J, Samuelsson, M, Traskman-Bendz, L, Brundin, L (2013). Altered chemokine levels in the cerebrospinal fluid and plasma of suicide attempters. Psychoneuroendocrinology 38, 853862.
Kapur, S, Phillips, AG, Insel, TR (2012). Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? Molecular Psychiatry 17, 11741179.
Lin, PY (2009). State-dependent decrease in levels of brain-derived neurotrophic factor in bipolar disorder: a meta-analytic study. Neuroscience Letters 466, 139143.
Looney, SW, el-Mallakh, RS (1997). Meta-analysis of erythrocyte Na,K-ATPase activity in bipolar illness. Depression and Anxiety 5, 5365.
McGorry, P, Keshavan, M, Goldstone, S, Amminger, P, Allott, K, Berk, M, Lavoie, S, Pantelis, C, Yung, A, Wood, S, Hickie, I (2014). Biomarkers and clinical staging in psychiatry. World Psychiatry 13, 211223.
Modabbernia, A, Taslimi, S, Brietzke, E, Ashrafi, M (2013). Cytokine alterations in bipolar disorder: a meta-analysis of 30 studies. Biological Psychiatry 74, 1525.
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Journal of Clinical Epidemiology 62, 10061012.
Munkholm, K, Brauner, JV, Kessing, LV, Vinberg, M (2013 a). Cytokines in bipolar disorder vs. healthy control subjects: a systematic review and meta-analysis. Journal of Psychiatric Research 47, 11191133.
Munkholm, K, Vinberg, M, Vedel Kessing, L (2013 b). Cytokines in bipolar disorder: a systematic review and meta-analysis. Journal of Affective Disorders 144, 1627.
Murck, H, Laughren, T, Lamers, F, Picard, R, Walther, S, Goff, D, Sainati, S (2015). Taking personalized medicine seriously: biomarker approaches in phase IIb/III studies in major depression and schizophrenia. Innovations in Clinical Neuroscience 12, 26s40s.
Parnas, J (2014). The RDoC program: psychiatry without psyche? World Psychiatry 13, 4647.
Phillips, ML, Chase, HW, Sheline, YI, Etkin, A, Almeida, JR, Deckersbach, T, Trivedi, MH (2015). Identifying predictors, moderators, and mediators of antidepressant response in major depressive disorder: neuroimaging approaches. American Journal of Psychiatry 172, 124138.
Polyakova, M, Stuke, K, Schuemberg, K, Mueller, K, Schoenknecht, P, Schroeter, ML (2015). BDNF as a biomarker for successful treatment of mood disorders: a systematic & quantitative meta-analysis. Journal of Affective Disorders 174, 432440.
Rosa, AR, Singh, N, Whitaker, E, de Brito, M, Lewis, AM, Vieta, E, Churchill, GC, Geddes, JR, Goodwin, GM (2014). Altered plasma glutathione levels in bipolar disorder indicates higher oxidative stress; a possible risk factor for illness onset despite normal brain-derived neurotrophic factor (BDNF) levels. Psychological Medicine 44, 24092418.
Scola, G, Andreazza, AC (2014). Current state of biomarkers in bipolar disorder. Current Psychiatry Reports 16, 514.
Sistrom, CL, Mergo, PJ (2000). A simple method for obtaining original data from published graphs and plots. American Journal of Roentgenology 174, 12411244.
Teixeira, AL, Barbosa, IG, Machado-Vieira, R, Rizzo, LB, Wieck, A, Bauer, ME (2013). Novel biomarkers for bipolar disorder. Expert Opinion on Medical Diagnostics 7, 147159.
Tsilidis, KK, Papatheodorou, SI, Evangelou, E, Ioannidis, JP (2012). Evaluation of excess statistical significance in meta-analyses of 98 biomarker associations with cancer risk. Journal of the National Cancer Institute 104, 18671878.
Tzoulaki, I, Siontis, KC, Evangelou, E, Ioannidis, JP (2013). Bias in associations of emerging biomarkers with cardiovascular disease. JAMA Internal Medicine 173, 664671.
Vieta, E (2014). The bipolar maze: a roadmap through translational psychopathology. Acta Psychiatrica Scandinavica 129, 323327.
WHO (1992). The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. World Health Organization: Geneva.
Young, SS, Bang, H (2004). The file-drawer problem, revisited. Science 306, 11331134; author reply 1133–1134.


Type Description Title
Supplementary materials

Carvalho supplementary material
Tables S1-S2

 Word (38 KB)
38 KB

Bias in emerging biomarkers for bipolar disorder

  • A. F. Carvalho (a1), C. A. Köhler (a1), B. S. Fernandes (a2) (a3), J. Quevedo (a4) (a5), K. W. Miskowiak (a6), A. R. Brunoni (a7) (a8), R. Machado-Vieira (a9) (a10) (a11), M. Maes (a2), E. Vieta (a12) and M. Berk (a2) (a13)...


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed