Hostname: page-component-5d84bcc8dc-95pw6 Total loading time: 0 Render date: 2026-09-09T09:25:46.093Z Has data issue: false hasContentIssue false
  • English
  • Français

How to prove that your therapy is effective, even when it is not: a guideline

Part of: Special Articles

Published online by Cambridge University Press:  28 September 2015

P. Cuijpers  and
I. A. Cristea
P. Cuijpers*
Affiliation:
Department of Clinical Psychology, VU University Amsterdam, The Netherlands EMGO Institute for Health and Care Research, The Netherlands
I. A. Cristea
Affiliation:
Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, Cluj-Napoca, Romania Clinical Psychology Branch, Department of Surgical, Medical, Molecular and Critical Pathology, University of Pisa, Pisa, Italy
*
* Address for correspondence: P. Cuijpers, Professor of Clinical Psychology, Department of Clinical Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands. (Email: p.cuijpers@vu.nl)
Get access Rights & Permissions [Opens in a new window]

Abstract

Aims.

Suppose you are the developer of a new therapy for a mental health problem or you have several years of experience working with such a therapy, and you would like to prove that it is effective. Randomised trials have become the gold standard to prove that interventions are effective, and they are used by treatment guidelines and policy makers to decide whether or not to adopt, implement or fund a therapy.

Methods.

You would want to do such a randomised trial to get your therapy disseminated, but in reality your clinical experience already showed you that the therapy works. How could you do a trial in order to optimise the chance of finding a positive effect?

Results.

Methods that can help include a strong allegiance towards the therapy, anything that increases expectations and hope in participants, making use of the weak spots of randomised trials (risk of bias), small sample sizes and waiting list control groups (but not comparisons with existing interventions). And if all that fails one can always not publish the outcomes and wait for positive trials.

Conclusions.

Several methods are available to help you show that your therapy is effective, even when it is not.

Keywords

Control groupsrandomised trialresearcher allegiancerisk of bias

Information

Type
Special Article
Information
Epidemiology and Psychiatric Sciences , Volume 25 , Issue 5 , October 2016 , pp. 428 - 435
Copyright
Copyright © Cambridge University Press 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akobeng, AK (2005). Understanding randomised controlled trials. Archives of Disease in Childhood 90, 840–844.CrossRefGoogle ScholarPubMed
Barth, J, Munder, T, Gerger, H, Nüesch, E, Trelle, S, Znoj, H, Jüni, P, Cuijpers, P (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis. PLoS Medicine 10, e1001454.Google Scholar
Boyuan, Z, Yang, C, Ke, C, Xueyong, S, Sheng, L (2014). Efficacy of acupuncture for psychological symptoms associated with opioid addiction: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine: eCAM 2014, 313549.Google Scholar
Carey, B (2012). Feeling Anxious? Soon There Will Be an App for That. The New York Times 13 February.Google Scholar
Chalmers, I, Glasziou, P (2009). Avoidable waste in the production and reporting of research evidence. Lancet (London, England) 374, 86–89.CrossRefGoogle ScholarPubMed
Chen, P, Furukawa, TA, Shinohara, K, Honyashiki, M, Imai, H, Ichikawa, K, Caldwell, DM, Hunot, V, Churchill, R (2014). Quantity and quality of psychotherapy trials for depression in the past five decades. Journal of Affective Disorders 165, 190–195.CrossRefGoogle ScholarPubMed
Colagiuri, B, Schenk, LA, Kessler, MD, Dorsey, SG, Colloca, L (2015). The placebo effect: from concepts to genes. Neuroscience 307, 171–190.CrossRefGoogle ScholarPubMed
Coyne, J, Kok, RN (2014). Salvaging psychotherapy research: a manifesto. Journal of Evidence-Based Psychotherapies 14, 105–124.Google Scholar
Crameri, A, von Wyl, A, Koemeda, M, Schulthess, P, Tschuschke, V (2015). Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy. Frontiers in Psychology 6, 1042.CrossRefGoogle ScholarPubMed
Cuijpers, P, van Straten, A (2011). New psychotherapies for mood and anxiety disorders: necessary innovation or waste of resources? Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie 56, 251; author reply 251–252.CrossRefGoogle ScholarPubMed
Cuijpers, P, Smit, F, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 a). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry: The Journal of Mental Science 196, 173–178.CrossRefGoogle ScholarPubMed
Cuijpers, P, van Straten, A, Bohlmeijer, E, Hollon, SD, Andersson, G (2010 b). The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological Medicine 40, 211–223.Google Scholar
Driessen, E, Hollon, SD, Bockting, CLH, Cuijpers, P, Turner, EH (2015). Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-analysis of US National Institutes of Health-Funded Trials. Plos One, in press.Google Scholar
Errington-Evans, N (2015). Randomised controlled trial on the use of acupuncture in adults with chronic, non-responding anxiety symptoms. Acupuncture in Medicine: Journal of the British Medical Acupuncture Society 33, 98–102.CrossRefGoogle ScholarPubMed
Fiksdal, BL, Houlihan, D, Barnes, AC (2012). Dolphin-assisted therapy: claims versus evidence. Autism Research and Treatment 2012, 839792.Google Scholar
Flint, J, Cuijpers, P, Horder, J, Koole, SL, Munafò, MR (2014). Is there an excess of significant findings in published studies of psychotherapy for depression? Psychological Medicine, 45, 439–446.Google Scholar
Frank, JD, Frank, JB (1991). Antidepressants Versus Placebo in Major Depression: an Overview. John Hopkins University Press: Baltimore.Google Scholar
Furukawa, TA, Noma, H, Caldwell, DM, Honyashiki, M, Shinohara, K, Imai, H, Chen, P, Hunot, V, Churchill, R (2014). Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta psychiatrica Scandinavica 130, 181–192.CrossRefGoogle ScholarPubMed
Gibertini, M, Nations, KR, Whitaker, JA (2012). Obtained effect size as a function of sample size in approved antidepressants: a real-world illustration in support of better trial design. International Clinical Psychopharmacology 27, 100–106.Google Scholar
Higgins, JPT, Green, S (eds) (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.cochrane-handbook.org Google Scholar
Higgins, JPT, Altman, DG, Gotzsche, PC, Juni, P, Moher, D, Oxman, AD, Savovic, J, Schulz, KF, Weeks, L, Sterne, JAC, Cochrane Bias Methods Group, Cochrane Statistical Methods Group (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928–d5928.Google Scholar
Hollon, SD (1999). Allegiance effects in treatment research: a commentary. Clinical Psychology: Science and Practice 6, 107–112.Google Scholar
Ioannidis, JPA (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279, 281–286.CrossRefGoogle ScholarPubMed
Ioannidis, JPA (2005). Why most published research findings are false. PLoS Medicine 2, e124.Google Scholar
Kamioka, H, Okada, S, Tsutani, K, Park, H, Okuizumi, H, Handa, S, Oshio, T, Park, S-J, Kitayuguchi, J, Abe, T, Honda, T, Mutoh, Y (2014 a). Effectiveness of animal-assisted therapy: a systematic review of randomized controlled trials. Complementary Therapies in Medicine 22, 371–390.CrossRefGoogle ScholarPubMed
Kamioka, H, Tsutani, K, Yamada, M, Park, H, Okuizumi, H, Honda, T, Okada, S, Park, S-J, Kitayuguchi, J, Abe, T, Handa, S, Mutoh, Y (2014 b). Effectiveness of horticultural therapy: a systematic review of randomized controlled trials. Complementary Therapies in Medicine 22, 930–943.CrossRefGoogle ScholarPubMed
Kaptchuk, TJ (2001). The double-blind, randomized, placebo-controlled trial: gold standard or golden calf? Journal of Clinical Epidemiology 54, 541–549.CrossRefGoogle ScholarPubMed
Khan, A, Brown, WA (2015). Antidepressants versus placebo in major depression: an overview. World Psychiatry: Official Journal of the World Psychiatric Association (WPA).Google Scholar
Leykin, Y, DeRubeis, RJ (2009). Allegiance in psychotherapy outcome research: separating association from bias. Clinical Psychology: Science and Practice 16, 54–65.Google Scholar
Luborsky, L, Diguer, L, Seligman, DA, Rosenthal, R, Krause, ED, Johnson, S, Halperin, G, Bishop, M, Berman, JS, Schweizer, E (1999). The researcher's own therapy allegiances: a “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice 6, 95–106.Google Scholar
Miller, S, Wampold, B, Varhely, K (2008). Direct comparisons of treatment modalities for youth disorders: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 18, 5–14.CrossRefGoogle ScholarPubMed
Mohr, DC, Spring, B, Freedland, KE, Beckner, V, Arean, P, Hollon, SD, Ockene, J, Kaplan, R (2009). The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychotherapy and Psychosomatics 78, 275–284.CrossRefGoogle ScholarPubMed
Mohr, DC, Ho, J, Hart, TL, Baron, KG, Berendsen, M, Beckner, V, Cai, X, Cuijpers, P, Spring, B, Kinsinger, SW, Schroder, KE, Duffecy, J (2014). Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression. Translational Behavioral Medicine 4, 407–423.CrossRefGoogle ScholarPubMed
Moseley, JB, O'Malley, K, Petersen, NJ, Menke, TJ, Brody, BA, Kuykendall, DH, Hollingsworth, JC, Ashton, CM, Wray, NP (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. New England Journal of Medicine 347, 81–88.CrossRefGoogle ScholarPubMed
Munder, T, Gerger, H, Trelle, S, Barth, J (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 21, 670–684.CrossRefGoogle ScholarPubMed
Munder, T, Flückiger, C, Gerger, H, Wampold, BE, Barth, J (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis. Journal of Counselling Psychology 59, 631–637.Google Scholar
Munder, T, Brütsch, O, Leonhart, R, Gerger, H, Barth, J (2013). Researcher allegiance in psychotherapy outcome research: an overview of reviews. Clinical Psychology Review 33, 501–511.CrossRefGoogle ScholarPubMed
Nezu, AM (1986). Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology 54, 196–202.CrossRefGoogle ScholarPubMed
Nezu, AM, Nezu (2008). Evidence-based Outcome Research; A Practical Guide to Conducting Randomized Controlled Trials for Psychosocial Interventions. Oxford University Press: New York.Google Scholar
Ondo, WG, Sethi, KD, Kricorian, G (2007). Selegiline orally disintegrating tablets in patients with Parkinson disease and ‘wearing off’ symptoms. Clinical Neuropharmacology 30, 295–300.CrossRefGoogle ScholarPubMed
Pinniger, R, Brown, RF, Thorsteinsson, EB, McKinley, P (2012). Argentine tango dance compared to mindfulness meditation and a waiting-list control: a randomised trial for treating depression. Complementary Therapies in Medicine 20, 377–384.CrossRefGoogle Scholar
Preston, RA, Materson, BJ, Reda, DJ, Williams, DW (2000). Placebo-associated blood pressure response and adverse effects in the treatment of hypertension: observations from a Department of Veterans Affairs Cooperative Study. Archives of Internal Medicine 160, 1449–1454.CrossRefGoogle ScholarPubMed
Psychiatry: Therapist-free therapy (2011). The Economist. March 5th 2011, downloaded from: http://www.economist.com/node/18276234.Google Scholar
Rafiei, R, Ataie, M, Ramezani, MA, Etemadi, A, Ataei, B, Nikyar, H, Abdoli, S (2014). A new acupuncture method for management of irritable bowel syndrome: a randomized double blind clinical trial. Journal of Research in Medical Sciences: The Official Journal of Isfahan University of Medical Sciences 19, 913–917.Google Scholar
Schmidt, NB, Richey, JA, Buckner, JD, Timpano, KR (2009). Attention training for generalized social anxiety disorder. Journal of Abnormal Psychology 118, 5–14.Google Scholar
Schulz, KF, Altman, DG, Moher, D, CONSORT Group (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research) 340, c332.Google Scholar
Schumi, J, Wittes, JT (2011). Through the looking glass: understanding non-inferiority. Trials 12, 106.Google Scholar
Siddiqui, O, Hung, HMJ, O'Neill, R (2009). MMRM vs. LOCF: a comprehensive comparison based on simulation study and 25 NDA datasets. Journal of Biopharmaceutical Statistics 19, 227–246.CrossRefGoogle ScholarPubMed
Turner, EH, Matthews, AM, Linardatos, E, Tell, RA, Rosenthal, R (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine 358, 252–260.CrossRefGoogle ScholarPubMed
Wampold, BE (2001). The Great Psychotherapy Debate: Models, Methods and Findings. Lawrence Erlbaum Associates: Mahwah, NJ.Google Scholar
Wu, J, Yeung, AS, Schnyer, R, Wang, Y, Mischoulon, D (2012). Acupuncture for depression: a review of clinical applications. Canadian Journal of Psychiatry. Revue Canadienne De Psychiatrie 57, 397–405.CrossRefGoogle ScholarPubMed