Skip to main content

How to prove that your therapy is effective, even when it is not: a guideline

  • P. Cuijpers (a1) (a2) and I. A. Cristea (a3) (a4)

Suppose you are the developer of a new therapy for a mental health problem or you have several years of experience working with such a therapy, and you would like to prove that it is effective. Randomised trials have become the gold standard to prove that interventions are effective, and they are used by treatment guidelines and policy makers to decide whether or not to adopt, implement or fund a therapy.


You would want to do such a randomised trial to get your therapy disseminated, but in reality your clinical experience already showed you that the therapy works. How could you do a trial in order to optimise the chance of finding a positive effect?


Methods that can help include a strong allegiance towards the therapy, anything that increases expectations and hope in participants, making use of the weak spots of randomised trials (risk of bias), small sample sizes and waiting list control groups (but not comparisons with existing interventions). And if all that fails one can always not publish the outcomes and wait for positive trials.


Several methods are available to help you show that your therapy is effective, even when it is not.

Corresponding author
* Address for correspondence: P. Cuijpers, Professor of Clinical Psychology, Department of Clinical Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands. (Email:
Hide All
Akobeng, AK (2005). Understanding randomised controlled trials. Archives of Disease in Childhood 90, 840844.
Barth, J, Munder, T, Gerger, H, Nüesch, E, Trelle, S, Znoj, H, Jüni, P, Cuijpers, P (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis. PLoS Medicine 10, e1001454.
Boyuan, Z, Yang, C, Ke, C, Xueyong, S, Sheng, L (2014). Efficacy of acupuncture for psychological symptoms associated with opioid addiction: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine: eCAM 2014, 313549.
Carey, B (2012). Feeling Anxious? Soon There Will Be an App for That. The New York Times 13 February.
Chalmers, I, Glasziou, P (2009). Avoidable waste in the production and reporting of research evidence. Lancet (London, England) 374, 8689.
Chen, P, Furukawa, TA, Shinohara, K, Honyashiki, M, Imai, H, Ichikawa, K, Caldwell, DM, Hunot, V, Churchill, R (2014). Quantity and quality of psychotherapy trials for depression in the past five decades. Journal of Affective Disorders 165, 190195.
Colagiuri, B, Schenk, LA, Kessler, MD, Dorsey, SG, Colloca, L (2015). The placebo effect: from concepts to genes. Neuroscience 307, 171190.
Coyne, J, Kok, RN (2014). Salvaging psychotherapy research: a manifesto. Journal of Evidence-Based Psychotherapies 14, 105124.
Crameri, A, von Wyl, A, Koemeda, M, Schulthess, P, Tschuschke, V (2015). Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy. Frontiers in Psychology 6, 1042.
Cuijpers, P, van Straten, A (2011). New psychotherapies for mood and anxiety disorders: necessary innovation or waste of resources? Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie 56, 251; author reply 251–252.
Cuijpers, P, Smit, F, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 a). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry: The Journal of Mental Science 196, 173178.
Cuijpers, P, van Straten, A, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 b). The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological Medicine 40, 211223.
Driessen, E, Hollon, SD, Bockting, CLH, Cuijpers, P, Turner, EH (2015). Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-analysis of US National Institutes of Health-Funded Trials. Plos One, in press.
Errington-Evans, N (2015). Randomised controlled trial on the use of acupuncture in adults with chronic, non-responding anxiety symptoms. Acupuncture in Medicine: Journal of the British Medical Acupuncture Society 33, 98102.
Fiksdal, BL, Houlihan, D, Barnes, AC (2012). Dolphin-assisted therapy: claims versus evidence. Autism Research and Treatment 2012, 839792.
Flint, J, Cuijpers, P, Horder, J, Koole, SL, Munafò, MR (2014). Is there an excess of significant findings in published studies of psychotherapy for depression? Psychological Medicine, 45, 439446.
Frank, JD, Frank, JB (1991). Antidepressants Versus Placebo in Major Depression: an Overview. John Hopkins University Press: Baltimore.
Furukawa, TA, Noma, H, Caldwell, DM, Honyashiki, M, Shinohara, K, Imai, H, Chen, P, Hunot, V, Churchill, R (2014). Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta psychiatrica Scandinavica 130, 181192.
Gibertini, M, Nations, KR, Whitaker, JA (2012). Obtained effect size as a function of sample size in approved antidepressants: a real-world illustration in support of better trial design. International Clinical Psychopharmacology 27, 100106.
Higgins, JPT, Green, S (eds) (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from
Higgins, JPT, Altman, DG, Gotzsche, PC, Juni, P, Moher, D, Oxman, AD, Savovic, J, Schulz, KF, Weeks, L, Sterne, JAC, Cochrane Bias Methods Group, Cochrane Statistical Methods Group (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928d5928.
Hollon, SD (1999). Allegiance effects in treatment research: a commentary. Clinical Psychology: Science and Practice 6, 107112.
Ioannidis, JPA (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279, 281286.
Ioannidis, JPA (2005). Why most published research findings are false. PLoS Medicine 2, e124.
Kamioka, H, Okada, S, Tsutani, K, Park, H, Okuizumi, H, Handa, S, Oshio, T, Park, S-J, Kitayuguchi, J, Abe, T, Honda, T, Mutoh, Y (2014 a). Effectiveness of animal-assisted therapy: a systematic review of randomized controlled trials. Complementary Therapies in Medicine 22, 371390.
Kamioka, H, Tsutani, K, Yamada, M, Park, H, Okuizumi, H, Honda, T, Okada, S, Park, S-J, Kitayuguchi, J, Abe, T, Handa, S, Mutoh, Y (2014 b). Effectiveness of horticultural therapy: a systematic review of randomized controlled trials. Complementary Therapies in Medicine 22, 930943.
Kaptchuk, TJ (2001). The double-blind, randomized, placebo-controlled trial: gold standard or golden calf? Journal of Clinical Epidemiology 54, 541549.
Khan, A, Brown, WA (2015). Antidepressants versus placebo in major depression: an overview. World Psychiatry: Official Journal of the World Psychiatric Association (WPA).
Leykin, Y, DeRubeis, RJ (2009). Allegiance in psychotherapy outcome research: separating association from bias. Clinical Psychology: Science and Practice 16, 5465.
Luborsky, L, Diguer, L, Seligman, DA, Rosenthal, R, Krause, ED, Johnson, S, Halperin, G, Bishop, M, Berman, JS, Schweizer, E (1999). The researcher's own therapy allegiances: a “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice 6, 95106.
Miller, S, Wampold, B, Varhely, K (2008). Direct comparisons of treatment modalities for youth disorders: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 18, 514.
Mohr, DC, Spring, B, Freedland, KE, Beckner, V, Arean, P, Hollon, SD, Ockene, J, Kaplan, R (2009). The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychotherapy and Psychosomatics 78, 275284.
Mohr, DC, Ho, J, Hart, TL, Baron, KG, Berendsen, M, Beckner, V, Cai, X, Cuijpers, P, Spring, B, Kinsinger, SW, Schroder, KE, Duffecy, J (2014). Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression. Translational Behavioral Medicine 4, 407423.
Moseley, JB, O'Malley, K, Petersen, NJ, Menke, TJ, Brody, BA, Kuykendall, DH, Hollingsworth, JC, Ashton, CM, Wray, NP (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. New England Journal of Medicine 347, 8188.
Munder, T, Gerger, H, Trelle, S, Barth, J (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 21, 670684.
Munder, T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis. Journal of Counselling Psychology 59, 631637.
Munder, T, Brütsch, O, Leonhart, R, Gerger, H, Barth, J (2013). Researcher allegiance in psychotherapy outcome research: an overview of reviews. Clinical Psychology Review 33, 501511.
Nezu, AM (1986). Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology 54, 196202.
Nezu, AM, Nezu (2008). Evidence-based Outcome Research; A Practical Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions. Oxford University Press: New York.
Ondo, WG, Sethi, KD, Kricorian, G (2007). Selegiline orally disintegrating tablets in patients with Parkinson disease and ‘wearing off’ symptoms. Clinical Neuropharmacology 30, 295300.
Pinniger, R, Brown, RF, Thorsteinsson, EB, McKinley, P (2012). Argentine tango dance compared to mindfulness meditation and a waiting-list control: a randomised trial for treating depression. Complementary Therapies in Medicine 20, 377384.
Preston, RA, Materson, BJ, Reda, DJ, Williams, DW (2000). Placebo-associated blood pressure response and adverse effects in the treatment of hypertension: observations from a Department of Veterans Affairs Cooperative Study. Archives of Internal Medicine 160, 14491454.
Psychiatry: Therapist-free therapy (2011). The Economist. March 5th 2011, downloaded from:
Rafiei, R, Ataie, M, Ramezani, MA, Etemadi, A, Ataei, B, Nikyar, H, Abdoli, S (2014). A new acupuncture method for management of irritable bowel syndrome: a randomized double blind clinical trial. Journal of Research in Medical Sciences: The Official Journal of Isfahan University of Medical Sciences 19, 913917.
Schmidt, NB, Richey, JA, Buckner, JD, Timpano, KR (2009). Attention training for generalized social anxiety disorder. Journal of Abnormal Psychology 118, 514.
Schulz, KF, Altman, DG, Moher, D, CONSORT Group (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research) 340, c332.
Schumi, J, Wittes, JT (2011). Through the looking glass: understanding non-inferiority. Trials 12, 106.
Siddiqui, O, Hung, HMJ, O'Neill, R (2009). MMRM vs. LOCF: a comprehensive comparison based on simulation study and 25 NDA datasets. Journal of Biopharmaceutical Statistics 19, 227246.
Turner, EH, Matthews, AM, Linardatos, E, Tell, RA, Rosenthal, R (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine 358, 252260.
Wampold, BE (2001). The Great Psychotherapy Debate: Models, Methods and Findings. Lawrence Erlbaum Associates: Mahwah, NJ.
Wu, J, Yeung, AS, Schnyer, R, Wang, Y, Mischoulon, D (2012). Acupuncture for depression: a review of clinical applications. Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie 57, 397405.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Epidemiology and Psychiatric Sciences
  • ISSN: 2045-7960
  • EISSN: 2045-7979
  • URL: /core/journals/epidemiology-and-psychiatric-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 21
Total number of PDF views: 285 *
Loading metrics...

Abstract views

Total abstract views: 3170 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 22nd March 2018. This data will be updated every 24 hours.