The past 15 years have witnessed an enthusiastic revival of research into the therapeutic potential of psychedelic drugs (e.g. psilocybin, MDMA) for people with serious psychiatric and addictive disorders (Bogenschutz & Ross, Reference Bogenschutz and Ross2018; Johnson, Hendricks, Barrett, & Griffiths, Reference Johnson, Hendricks, Barrett and Griffiths2019; Schimmel et al., Reference Schimmel, Breeksema, Smith-Apeldoorn, Veraart, van den Brink and Schoevers2021). If psychedelic drugs can be shown to ease the enormous human suffering caused by disorders like major depression, PTSD, and addiction, they should be approved for medical use like any other pharmacotherapy. But this approval requires rigorous scientific evidence on safety and efficacy.
Unfortunately, psychedelic drugs have come to recent prominence through the unwise lowering of research standards by some major medical journals and the inappropriate exaggeration of research results in the popular media by scientists. In this editorial, we describe these problems and argue that it is essential that the high scientific standards required of clinical research in other areas also be applied to the evaluation of psychedelic medicines.
Lowering of standards by major medical journals
The New England Journal of Medicine recently published a phase 2, randomized, controlled trial comparing the effects of psilocybin with those of a selective serotonin-reuptake inhibitor, escitalopram, in treating patients with long-standing major depressive disorder over a 6-week period (Carhart-Harris et al., Reference Carhart-Harris, Giribaldi, Watts, Baker-Jones, Murphy-Beiner, Murphy and Nutt2021). The trial was described as double-blinded but research participants in trials of psilocybin typically know that they have been given a psychedelic drug (Bender & Hellerstein, Reference Bender and Hellerstein2022; Colloca & Barsky, Reference Colloca and Barsky2020), and the study did not assess whether blinding was successful.
A post publication commentary (Burke & Blumberger, Reference Burke and Blumberger2021) on a Phase III trial of MDMA for PTSD published in Nature Medicine referred to the prior NEJM study as an example of why clinical trial guidelines should be strengthened. A review of methodological problems in the emerging field of psychedelic research cited the study in its discussion of ‘serious concerns that limit the generalizability of the results’ (Ona, Kohek, & Bouso, Reference Ona, Kohek and Bouso2022). A recent systematic review and meta-analysis (Kisely, Connor, Somogyi, & Siskind, Reference Kisely, Connor, Somogyi and Siskind2022) of the effect of psilocybin and methylenedioxymethamphetamine on mental, behavioral or developmental disorders noted limitations of this study.
The study assigned 30 patients to the psilocybin group and 29 to the escitalopram group. Psilocybin did not have a significant effect on the study's primary outcome, a change in depression score from baseline to six weeks follow up [−2.0, 95% (CI) −5.0 to 0.9]. Comparisons of the two treatments on 10 secondary outcomes (e.g. percentage achieving a clinical response) favored the psilocybin group but these analyses were not corrected for multiple comparisons. The report correctly noted that ‘no clinical conclusions can be drawn from these data’. As long-time readers and reviewers for the NEJM, familiar with their standard of upholding ‘a rigorous peer review and editing process’ that aims to prevent ‘overstated results from reaching physicians’ (NEJM Media Center, 2022), we wondered why the editors published an underpowered, short-term, phase 2 trial that could not support any clinical conclusions.
Similar questions are raised by Nature Medicine's decision to publish a small neuroimaging study based on participants from the NEJM trial and another small trial (Daws et al., Reference Daws, Timmermann, Giribaldi, Sexton, Wall, Erritzoe and Carhart-Harris2022). The investigators used a different measure of depression than the primary outcome in the clinical trial – the Beck Depression Inventory – because, they later conceded (Carhart-Harris, Daws, & Nutt, Reference Carhart-Harris, Daws and Nutt2022) in response to critics (Doss, Barrett, & Corlett, Reference Doss, Barrett and Corlett2022a, Reference Doss, Barrett and Corlett2022b), this measure was more likely to show anti-depressant effects of psilocybin. They reported that the participants who had been given psilocybin showed changes in the brain's default mode network (BMN), implicated in high-level cognitive functions including episodic memory, theory of mind, and self-referential processing. The authors interpreted this as evidence that psilocybin produced greater open-mindedness in patients than did escitalopram, despite the fact that tests of an interaction effect between drug received and changes in the BMN failed to achieve significance.
Critics of the study pointed out that other Nature journals had published articles explaining the flaws in this form of statistical inference and showing how it has contributed to failure to replicate the findings of many neuroimaging studies (Doss et al., Reference Doss, Barrett and Corlett2022a, Reference Doss, Barrett and Corlett2022b). Despite these major problems, this study appeared in a leading medical journal that has no history of accepting comparably limited work on other therapies.
Media hype by scientists
Carhart-Harris, the lead author on the trial in the NEJM, acknowledged in the published paper that ‘no clinical conclusions can be drawn from these data’ but provided a much more positive spin on his findings in an article in the Guardian newspaper (Carhart-Harris, Reference Carhart-Harris2021) where he proclaimed that:
Psilocybin worked more rapidly, decreasing depression scores as early as one day after the first dosing session. At the end of the trial, the average response rate to psilocybin therapy was more than 70%. … While we suspected that psilocybin might perform well compared to the SSRI, we had not expected it to perform as well as it did.
Many other media outlets echoed these unevidenced conclusions. The investigators of the Nature Medicine study also overstated their findings, leading to many breathless media stories claiming that their work had shown that psilocybin ‘rewires the brain’ (Love, Reference Love2022).
Trying to get the media to be responsible in reporting scientific work is often a fool's errand, but it is reasonable to ask why editors of prestigious journals and scientists who research psychedelics chose to ignite a media frenzy. The unwarranted excitement generated by media reports of these studies has prompted large investments by venture capital (Phelps, Shah, & Lieberman, Reference Phelps, Shah and Lieberman2022) and been used to justify popular initiatives in several US cities and states to legalize the medical use of psychedelic drugs (Smith & Appelbaum, Reference Smith and Appelbaum2022). We fear that this type of hype surrounding psychedelics will lead to their premature introduction to clinical practice in poorly regulated ways that risk patients' well-being and medicine's credibility, much as medical cannabis has been fraudulently marketed as a cure for COVID-19 and opioid addiction (Humphreys & Saitz, Reference Humphreys and Saitz2019; Shover & Humphreys, Reference Shover and Humphreys2020).
Let better science carry the day
It is challenging to conduct good quality clinical trials of psychedelic drugs, for substantive reasons (e.g. blinding is difficult), and for political ones (e.g. insufficient public funding, stringent regulations). Some might argue that these difficulties justify a preparedness to accept weak evidence from flawed scientific studies. But poor-quality science simply cannot provide a sound basis for administering medications to suffering patients. Nor should we accept the argument that psychedelic drugs should be given an easier ride through peer and regulatory review processes because they were treated unfairly in the past, a claim which has arguably been overstated (Hall, Reference Hall2021b).
Public funding is needed for longer-term, phase 3 controlled clinical trials of psychedelic drugs in rigorous designs conducted at equipoise (Hall, Reference Hall2021a). These trials need to recruit larger numbers of patients than trials to date and ensure that trial participants are representative of the patient populations in whom these drugs are likely to be used if they are approved for clinical use (Humphreys, Maisel, Blodgett, & Finney, Reference Humphreys, Maisel, Blodgett and Finney2013). Only the best of these studies should be published in leading journals, and all of them should be described to the public and media with meticulous accuracy.
Acknowledgement
We would like to thank Sarah Yeates for her assistance in proofreading and formatting the paper for publication.
Conflict of interest
The authors have no competing interests to declare and there was no funding received for the preparation of this article.
