Skip to main content
    • Aa
    • Aa

How to prove that your therapy is effective, even when it is not: a guideline

  • P. Cuijpers (a1) (a2) and I. A. Cristea (a3) (a4)

Suppose you are the developer of a new therapy for a mental health problem or you have several years of experience working with such a therapy, and you would like to prove that it is effective. Randomised trials have become the gold standard to prove that interventions are effective, and they are used by treatment guidelines and policy makers to decide whether or not to adopt, implement or fund a therapy.


You would want to do such a randomised trial to get your therapy disseminated, but in reality your clinical experience already showed you that the therapy works. How could you do a trial in order to optimise the chance of finding a positive effect?


Methods that can help include a strong allegiance towards the therapy, anything that increases expectations and hope in participants, making use of the weak spots of randomised trials (risk of bias), small sample sizes and waiting list control groups (but not comparisons with existing interventions). And if all that fails one can always not publish the outcomes and wait for positive trials.


Several methods are available to help you show that your therapy is effective, even when it is not.

Corresponding author
* Address for correspondence: P. Cuijpers, Professor of Clinical Psychology, Department of Clinical Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands. (Email:
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

AK Akobeng (2005). Understanding randomised controlled trials. Archives of Disease in Childhood 90, 840844.

J Barth , T Munder , H Gerger , E Nüesch , S Trelle , H Znoj , P Jüni , P Cuijpers (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis. PLoS Medicine 10, e1001454.

Z Boyuan , C Yang , C Ke , S Xueyong , L Sheng (2014). Efficacy of acupuncture for psychological symptoms associated with opioid addiction: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine: eCAM 2014, 313549.

I Chalmers , P Glasziou (2009). Avoidable waste in the production and reporting of research evidence. Lancet (London, England) 374, 8689.

P Chen , TA Furukawa , K Shinohara , M Honyashiki , H Imai , K Ichikawa , DM Caldwell , V Hunot , R Churchill (2014). Quantity and quality of psychotherapy trials for depression in the past five decades. Journal of Affective Disorders 165, 190195.

B Colagiuri , LA Schenk , MD Kessler , SG Dorsey , L Colloca (2015). The placebo effect: from concepts to genes. Neuroscience 307, 171190.

A Crameri , A von Wyl , M Koemeda , P Schulthess , V Tschuschke (2015). Sensitivity analysis in multiple imputation in effectiveness studies of psychotherapy. Frontiers in Psychology 6, 1042.

P Cuijpers , F Smit , E Bohlmeijer , SD Hollon , G Andersson (2010 a). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry: The Journal of Mental Science 196, 173178.

E Driessen , SD Hollon , CLH Bockting , P Cuijpers , EH Turner (2015). Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-analysis of US National Institutes of Health-Funded Trials. Plos One, in press.

N Errington-Evans (2015). Randomised controlled trial on the use of acupuncture in adults with chronic, non-responding anxiety symptoms. Acupuncture in Medicine: Journal of the British Medical Acupuncture Society 33, 98102.

BL Fiksdal , D Houlihan , AC Barnes (2012). Dolphin-assisted therapy: claims versus evidence. Autism Research and Treatment 2012, 839792.

J Flint , P Cuijpers , J Horder , SL Koole , MR Munafò (2014). Is there an excess of significant findings in published studies of psychotherapy for depression? Psychological Medicine, 45, 439446.

TA Furukawa , H Noma , DM Caldwell , M Honyashiki , K Shinohara , H Imai , P Chen , V Hunot , R Churchill (2014). Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis. Acta psychiatrica Scandinavica 130, 181192.

M Gibertini , KR Nations , JA Whitaker (2012). Obtained effect size as a function of sample size in approved antidepressants: a real-world illustration in support of better trial design. International Clinical Psychopharmacology 27, 100106.

JPT Higgins , DG Altman , PC Gotzsche , P Juni , D Moher , AD Oxman , J Savovic , KF Schulz , L Weeks , JAC Sterne , Cochrane Bias Methods Group, Cochrane Statistical Methods Group (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928d5928.

JPA Ioannidis (1998). Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 279, 281286.

JPA Ioannidis (2005). Why most published research findings are false. PLoS Medicine 2, e124.

H Kamioka , S Okada , K Tsutani , H Park , H Okuizumi , S Handa , T Oshio , S-J Park , J Kitayuguchi , T Abe , T Honda , Y Mutoh (2014 a). Effectiveness of animal-assisted therapy: a systematic review of randomized controlled trials. Complementary Therapies in Medicine 22, 371390.

TJ Kaptchuk (2001). The double-blind, randomized, placebo-controlled trial: gold standard or golden calf? Journal of Clinical Epidemiology 54, 541549.

A Khan , WA Brown (2015). Antidepressants versus placebo in major depression: an overview. World Psychiatry: Official Journal of the World Psychiatric Association (WPA).

S Miller , B Wampold , K Varhely (2008). Direct comparisons of treatment modalities for youth disorders: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 18, 514.

DC Mohr , B Spring , KE Freedland , V Beckner , P Arean , SD Hollon , J Ockene , R Kaplan (2009). The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychotherapy and Psychosomatics 78, 275284.

DC Mohr , J Ho , TL Hart , KG Baron , M Berendsen , V Beckner , X Cai , P Cuijpers , B Spring , SW Kinsinger , KE Schroder , J Duffecy (2014). Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression. Translational Behavioral Medicine 4, 407423.

JB Moseley , K O'Malley , NJ Petersen , TJ Menke , BA Brody , DH Kuykendall , JC Hollingsworth , CM Ashton , NP Wray (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. New England Journal of Medicine 347, 8188.

T Munder , H Gerger , S Trelle , J Barth (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy Research: Journal of the Society for Psychotherapy Research 21, 670684.

T Munder , C Flückiger , H Gerger , BE Wampold , J Barth (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis. Journal of Counselling Psychology 59, 631637.

T Munder , O Brütsch , R Leonhart , H Gerger , J Barth (2013). Researcher allegiance in psychotherapy outcome research: an overview of reviews. Clinical Psychology Review 33, 501511.

AM Nezu (1986). Efficacy of a social problem-solving therapy approach for unipolar depression. Journal of Consulting and Clinical Psychology 54, 196202.

WG Ondo , KD Sethi , G Kricorian (2007). Selegiline orally disintegrating tablets in patients with Parkinson disease and ‘wearing off’ symptoms. Clinical Neuropharmacology 30, 295300.

R Pinniger , RF Brown , EB Thorsteinsson , P McKinley (2012). Argentine tango dance compared to mindfulness meditation and a waiting-list control: a randomised trial for treating depression. Complementary Therapies in Medicine 20, 377384.

RA Preston , BJ Materson , DJ Reda , DW Williams (2000). Placebo-associated blood pressure response and adverse effects in the treatment of hypertension: observations from a Department of Veterans Affairs Cooperative Study. Archives of Internal Medicine 160, 14491454.

NB Schmidt , JA Richey , JD Buckner , KR Timpano (2009). Attention training for generalized social anxiety disorder. Journal of Abnormal Psychology 118, 514.

KF Schulz , DG Altman , D Moher , CONSORT Group (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ (Clinical Research) 340, c332.

J Schumi , JT Wittes (2011). Through the looking glass: understanding non-inferiority. Trials 12, 106.

O Siddiqui , HMJ Hung , R O'Neill (2009). MMRM vs. LOCF: a comprehensive comparison based on simulation study and 25 NDA datasets. Journal of Biopharmaceutical Statistics 19, 227246.

EH Turner , AM Matthews , E Linardatos , RA Tell , R Rosenthal (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine 358, 252260.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Epidemiology and Psychiatric Sciences
  • ISSN: 2045-7960
  • EISSN: 2045-7979
  • URL: /core/journals/epidemiology-and-psychiatric-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 17
Total number of PDF views: 223 *
Loading metrics...

Abstract views

Total abstract views: 2438 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 19th September 2017. This data will be updated every 24 hours.