Discussion

We characterised the quality of side-effects reporting in clinical trials of PAP for psychiatric disorders. We found that the body of evidence on PAP was variable in terms of quality of side-effects reporting, with 14 of 24 reports rated as being of either low or very low quality against the CONSORT harms guidelines, 4 as moderate and 6 as high quality. With side-effects as the outcome of interest, all RCTs had a high risk of bias. We also found several small inconsistencies between adverse events recorded in published reports and those in CTR, without evidence of systematic underreporting of adverse events in this body of literature.

These findings indicate somewhat superior side-effects reporting in the PAP literature compared with similar bodies of evidence. A recent evidence synthesis by our group found that none of 13 MDMA-AP trials met the criteria for adequate adherence to the CONSORT Harms recommendations, with a median adherence rate of 50%.^{Reference Colcott, Guerin, Carter, Meikle and Bedi25} Similar median rates were observed in the literature on esketamine (48%)^{Reference de Laportaliere, Jullien, Yrondi, Cestac and Montastruc26} and SSRIs (50%).^{Reference Meister, von Wolff, Mohr, Nestoriuc, Härter and Hölzel27} We report a similar median adherence (50%, range 19–100) in the PAP literature; however, side-effects reporting in 6 of 24 PAP studies (i.e. 25%) was of high quality, 42% of studies met criteria for adequate adherence (70%) and 1 met all 21 criteria.^{Reference Raison, Sanacora, Woolley, Heinzerling, Dunlop and Brown12} Notably, all six studies with high-quality side-effects reporting were published since 2020, with some explicitly seeking to address limitations in previous trials, including in side-effects reporting.^{Reference Goodwin, Aaronson, Alvarez, Arden, Baker and Bennett11,Reference Raison, Sanacora, Woolley, Heinzerling, Dunlop and Brown12} One possible explanation for the apparent increase in quality of side-effects reporting (Fig. 2) over time could be the publication of updated reporting standards by CONSORT in 2023.^{Reference Junqueira, Zorzela, Golder, Loke, Gagnier and Julious30} Additionally, early studies have been subject to critiques regarding methodological failings and associated issues with interpretability of findings,^{Reference Muthukumaraswamy, Forsyth and Lumley52} potentially leading to researchers designing more rigorous trials that include comprehensive reporting of adverse events. These are promising signs for strengthening of side-effects reporting in this field, and suggest that publications identified as high quality in relation to side-effects reporting should be viewed as a benchmark for future research. Evidence from these reports – particularly those RCTs that provide a higher level of evidence – should also be preferentially employed in policy decisions, and by clinicians seeking to inform patients about the potential risks of PAP.

Risk of bias was assessed using the RoB 2 tool for the 9 RCTs. Despite psilocybin-assisted psychotherapy being increasingly recognised as a potential treatment for several psychiatric indications, the limited number of RCTs published to date highlights the urgent need for more high-quality research to clarify its safety and efficacy. All nine RCTs were found to have a high risk of bias, which is similar to bias ratings in a recent MDMA-AP systematic review,^{Reference Colcott, Guerin, Carter, Meikle and Bedi25} where seven of eight trials were rated as high risk, with one having ‘some concerns’. Consistent with these findings, all esketamine trials included in a similar review^{Reference de Laportaliere, Jullien, Yrondi, Cestac and Montastruc26} were rated as having ‘some concerns’. Across these bodies of research, domain 4, ‘measurement of the outcome’, was the most likely to be violated, with functional unblinding presenting an ongoing issue in studies involving medications such as psychedelics that have strong psychoactive and subjective drug effects.^{Reference Wen, Singhal, Jones, Zeifman, Mehta and Shenasa53} This issue is compounded by the much-noted media hype surrounding these interventions.^{Reference Noorani, Bedi and Muthukumaraswamy54–Reference McNamee, Devenot and Buisson56} Attempts to ameliorate functional unblinding will require systematic methodological modifications in future clinical trials,^{Reference Muthukumaraswamy, Forsyth and Lumley52,Reference Wen, Singhal, Jones, Zeifman, Mehta and Shenasa53} with the aim of improving the certainty of evidence about PAP and related modalities.

Our final analysis examined adverse events reported in CTR compared with those reported in the corresponding published articles. Overall, our review revealed no evidence of systematic underreporting as suggested in a review of trials of esketamine.^{Reference de Laportaliere, Jullien, Yrondi, Cestac and Montastruc26} However, direct comparison between CTR and publications was challenging due to variability in the approaches to side-effects reporting across CTR and corresponding publications, with variations in the metrics used to quantify adverse events (i.e. total number of adverse events versus number of participants experiencing individual adverse events), timelines and thresholds for adverse events reporting. This lack of consistency in reporting itself highlights the need for more standardised approaches.

One issue that this review is unable to address is whether current approaches to identifying side-effects adequately capture some of the unique characteristics of psychedelic drugs such as psilocybin, and the potential impacts of their combination with psychotherapy. In particular, the potential for interpersonal harms occurring in PAP has been noted,^{Reference Schlag, Aday, Salam, Neill and Nutt57–Reference Yaden, Potash and Griffiths59} given the increased vulnerability of patients undergoing psychotherapy while affected by psychoactive drugs. How best to ensure that these rare but profoundly harmful events are prevented and, when they do occur, documented has yet to be determined. Moreover, the content of the psychotherapy delivered was often poorly defined in the studies reviewed. Improved transparency in the description of therapeutic models would enable future reviews to better assess how specific treatment components may influence the type, frequency and severity of harms. Similarly, there is little consensus as to how phenomena such as existential or spiritual crises related to psychedelic exposure^{Reference Meikle, Carter and Bedi60} are best understood within the biomedical framework, communicated to patients and managed in PAP research and clinical practice. Of additional concern, suicidality has been noted as an adverse event in some trials;^{Reference Anderson, Danforth, Daroff, Stauffer, Ekman and Agin-Liebes37,Reference Griffiths, Johnson, Carducci, Umbricht, Richards and Richards41} this may partly reflect the research focus on major depression. However, there is some indication of dose dependence in suicidality,^{Reference Goodwin, Aaronson, Alvarez, Arden, Baker and Bennett11} suggesting a causal effect of the treatment itself. Considering this, careful baseline screening, close monitoring throughout the trial and robust post-treatment support are crucial to ensure patient safety. Future studies should further investigate suicidality, and other similar safety issues, by examining each event systematically, tracking adverse events and how any changes in behaviour might be linked to the therapeutic process or drug effects. More broadly, efforts to stratify adverse events reporting by participant risk group or clinical indication are limited by the current evidence base. While such stratification could reveal important differences in reporting quality or adverse events profiles, particularly across populations with differing levels of baseline vulnerability, this remains difficult due to the small number of studies per indication and inconsistent reporting standards. A better understanding of these safety issues is critical for guiding future research, implementation decisions and clinical practice.

The findings of this review highlight variability and some limitations in side-effects reporting within PAP clinical trials. While most studies included adverse events information and some (25%) were rated as high quality regarding adherence to the CONSORT Harms guidelines, the overall quality of reporting was inadequate (median adherence 50%). Similar shortcomings have been observed in related research,^{Reference Colcott, Guerin, Carter, Meikle and Bedi25,Reference Meister, von Wolff, Mohr, Nestoriuc, Härter and Hölzel27} suggesting that side-effects reporting in psychiatry more broadly needs to be improved. Of note, however, is the apparent improvement in side-effects reporting in PAP trials published since 2020. To continue to enhance the quality of side-effects reporting, we echo our previous recommendation that a stronger emphasis be made on following the CONSORT Harms recommendations during peer review.^{Reference Colcott, Guerin, Carter, Meikle and Bedi25} Regarding PAP specifically, although recent trials show improvements in reporting, there remain some limitations in the overall body of evidence. From the clinical perspective, physicians seeking to provide information to patients considering PAP about its risk:benefit ratio should prioritise information from RCTs identified as having high-quality side-effects reporting. Moreover, patients should be informed of the relative uncertainty of existing evidence on potential side-effects given the risk of bias in existing studies. In future research, consistent and rigorous reporting practices, alongside methodological enhancements to reduce risk of bias, are essential to guide the safe translation of psilocybin-assisted psychotherapy into clinical practice.