Hostname: page-component-59f8fd8595-fsbp6 Total loading time: 0 Render date: 2023-03-22T06:40:04.245Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

Underestimating harm in mindfulness-based stress reduction

Published online by Cambridge University Press:  07 December 2020

Nicholas T. Van Dam*
School of Psychological Sciences, University of Melbourne, Melbourne, Australia
Julieta Galante
Department of Psychiatry, University of Cambridge, Cambridge, UK
Author for correspondence: Nicholas T. Van Dam, E-mail:
Rights & Permissions[Opens in a new window]


Copyright © The Author(s), 2020. Published by Cambridge University Press

In a recent study of harm following the completion of mindfulness-based stress reduction (MBSR) programs, the authors used a large observational dataset (n = 2155), alongside the active arm (n = 156) and wait-list arm (n = 118) of several amalgamated randomized controlled trials (Hirshberg, Goldberg, Rosenkranz, & Davidson, Reference Hirshberg, Goldberg, Rosenkranz and Davidson2020). The authors interpret their results as ‘strong evidence that MBSR is no more harmful than no treatment’ (p. 8). While seemingly conclusive, we would like to point out some potentially serious limitations to the methods used that undermine confidence in the authors' conclusions.

Considerable missing data with imputation of low prevalence events

It is well-known that those who experience adverse events are less likely to complete outcome measures (see e.g. Ioannidis et al., Reference Ioannidis, Evans, Gøtzsche, O'Neill, Altman and Schulz2004). It is also known that this phenomenon is exacerbated when researchers and instructors/clinicians are associated with the program (Allen, Chandler, Mandimika, Leisegang, & Barnes, Reference Allen, Chandler, Mandimika, Leisegang and Barnes2018); the authors themselves have recently published on this issue (Goldberg & Tucker, Reference Goldberg and Tucker2019). Thus, it is important to carefully consider what the data attrition actually reflects in the work by Hirshberg et al. (Reference Hirshberg, Goldberg, Rosenkranz and Davidson2020; as high as ~23% in their largest sample). Regardless of course completion numbers, the larger percentage of missing data is concerning because it could be masking harm (Schulz & Grimes, Reference Schulz and Grimes2002). The authors still considered that data were missing at random after adjusting imputation for just gender and participation year. Although these variables were predictive of missingness, they are insufficient to address potential causes for missingness when investigating harm (see e.g. Sterne et al., Reference Sterne, White, Carlin, Spratt, Royston, Kenward and Carpenter2009). Moreover, harm outcomes are expected to be low-prevalence events, making data imputation even more problematic.

Limited measurement of harm

The authors rightly assert that there is no consensus about ‘harm’ in psychosocial interventions. Acknowledgement of this lack of consensus is not, however, justification to ignore the many ways in which psychosocial interventions may potentially cause harm (Linden, Reference Linden2013). Hirshberg et al. (Reference Hirshberg, Goldberg, Rosenkranz and Davidson2020) focus their analyses on the deterioration of illness and the emergence of new symptoms as endorsed on a medical symptom checklist. It is worth noting that under-detection of adverse events is likely with traditional simple checklists (see Allen et al., Reference Allen, Chandler, Mandimika, Leisegang and Barnes2018). It is of additional note that MBSR guidelines not only encourage participants to practice outside of formal activity of the courses but also recommend that practice continue beyond the course (Santorelli, Meleo-Meyer, Koerbel, & Kabat-Zinn, Reference Santorelli, Meleo-Meyer, Koerbel and Kabat-Zinn2017). Continued practice outside the supportive period during which training is being provided, along with encouragement for participants to engage in meditation retreats (Santorelli et al., Reference Santorelli, Meleo-Meyer, Koerbel and Kabat-Zinn2017), makes the post-intervention period especially important to monitor. People cannot report harms if they are not asked about them and it is very possible that more serious harms only emerge when the practice is implemented without the support of MBSR instructors. It is essential that longer follow-up periods are examined prior to claiming MBSR does no harm.

Potential sample bias

The overall level of impairment (take Global Severity Index - GSI, for example) is significantly higher in the community sample than either of the two groups in the randomized controlled trials (RCTs; v. MBSR, Cohen's d = −0.99, t(2309) = −10.03, p < 0.001; v. WLC, Cohen's d = −1.17, t(2271) = −9.86, p < 0.001), and is above the T value (i.e. 60) commonly used to detect psychological distress (M = 61.78, s.d. = 42.67). Thus, the community sample represents a large selection of individuals who sought out and paid to receive MBSR, presumably, with the aim of helping them to manage their psychological distress. As this represents a non-blinded intervention, it likely suffers from inflated effect size estimates (pre-post change: Community d = −0.70; RCT d = −0.07) and attrition rates (~23% missing data in Community v. ~10% missing data in RCT; see Hróbjartsson, Emanuelsson, Thomsen, Hilden, and Brorson, Reference Hróbjartsson, Emanuelsson, Thomsen, Hilden and Brorson2014). Self-selection, desire to improve, along with financial and time investment in the treatment, makes it likely that participants were inclined to give favorable ratings at discharge. Those who did not receive benefits are more likely to have discontinued treatment earlier on in the process.

Collation of data across potentially different trials

The authors mention that they collated data from three trials and give two grant numbers but no trial protocols or publications. We were unable to find waitlist-controlled trials published under the grant U01AT002114-01A1. We found one waitlist and Health Enhancement Program (HEP; MacCoon et al., Reference MacCoon, Imel, Rosenkranz, Sheftel, Weng, Sullivan and Lutz2012) controlled trial under P01AT004952 that measured GSI; curiously enough they failed to report GSI results (Goldberg et al., Reference Goldberg, Wielgosz, Dahl, Schuyler, MacCoon, Rosenkranz and Davidson2016). We failed to find any information on two of the three trials. We also failed to identify a prospective protocol for the analyses presented by Hirshberg et al. While we recognize the time commitment of such efforts, it is widely recognized in the scientific community that trial reporting needs to be much more detailed and comprehensive to avoid major biases (Schulz, Altman, & Moher, Reference Schulz, Altman and Moher2010). Lack of transparency and pre-registration of detailed analysis plans increase chances of bias, compromising confidence in the results. Furthermore, data from trials were combined as if it was a single trial without any consideration for potential (unknown to us) variability among trials; such amalgamation can lead to bias and overprecise estimates of effect (Tierney et al., Reference Tierney, Vale, Riley, Smith, Stewart, Clarke and Rovers2015).

Inappropriate conclusion regarding preventative potential of MBSR

The authors use the deterioration of the wait-list group, compared to the improvement of the community group as a means of saying that community MBSR is preventative of harm. The comparison is inappropriate for three reasons. The first is that one cannot reasonably compare an active, open-label intervention to the control arm of an RCT. The second is that the estimation of effect in the community MBSR group is likely an overestimate, as discussed above. The third is the fact that participants allocated to wait-list groups in active randomized controlled trials are known to exhibit the nocebo effect (i.e. deterioration of symptoms generated by negative expectations pursuant to not getting the active intervention; Furukawa et al. Reference Furukawa, Noma, Caldwell, Honyashiki, Shinohara, Imai and Churchill2014; Gold et al. Reference Gold, Enck, Hasselmann, Friede, Hegerl, Mohr and Otte2017). In the study in question, participants in the wait-list condition deteriorated on the medical symptom checklist by Cohen's d = −0.64; well within the range of the observed nocebo effects (see e.g. Furukawa et al., Reference Furukawa, Noma, Caldwell, Honyashiki, Shinohara, Imai and Churchill2014). A recent meta-analysis showed that the difference in outcomes of active interventions can vary as much as d = 0.50 when compared against active controls v. waitlists (Furukawa et al., Reference Furukawa, Noma, Caldwell, Honyashiki, Shinohara, Imai and Churchill2014). A comparison with a control group such as the HEP, to control for non-specific effects, and used in at least some of the three trials analysed by the authors, would have provided the kind of evidence desired while mitigating the nocebo effect.

For all these reasons we do strongly disagree with the assertion that ‘these data provide strong evidence against claims that MBSR may increase harm’. The evidence presented in this article is insufficient to support the claim. Moreover, the authors appeal to a logical fallacy; absence of evidence rather than evidence of absence – not finding evidence of adverse effects in MBSR does not mean that there are none. We are also concerned that the abstract, often the only section people read, fails to acknowledge any of the limitations that undermine confidence in the results.

The claims made by the authors provide a much sought-after reassurance that is as yet unfounded in our view. A series of recent studies have suggested that adverse events can and do happen in meditation-based interventions including those with a focus on mindfulness (Cebolla, Demarzo, Martins, Soler, & Garcia-Campayo, Reference Cebolla, Demarzo, Martins, Soler and Garcia-Campayo2017; Farias, Marald, Wallenkampf, & Lucchetti, Reference Farias, Marald, Wallenkampf and Lucchetti2020; Schlosser, Sparby, Voros, Jones, & Marchant, Reference Schlosser, Sparby, Voros, Jones and Marchant2019). None of this is to say that people should not undertake MBSR or that MBSR is likely to be harmful. Much to the contrary, MBSR is likely to be helpful for most people for a wide array of issues (de Vibe et al., Reference de Vibe, Bjorndal, Fattah, Dyrdal, Halland and Tanner-Smith2017). However, methodological and reporting quality needs to improve before making strong evidential claims about adverse effects in interventions with high public health relevance such as MBSR.

Financial support

No financial support was received to undertake this work. Julieta Galante was funded by a National Institute for Health Research (NIHR) Post-doctoral Fellowship (PDF-2017-10-018). The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Conflict of interest

The authors declare no conflict of interest.


Allen, E. N., Chandler, C. I. R., Mandimika, N., Leisegang, C., & Barnes, K. (2018). Eliciting adverse effects data from participants in clinical trials. Cochrane Database of Systematic Reviews, 1(1), MR000039.Google ScholarPubMed
Cebolla, A., Demarzo, M., Martins, P., Soler, J., & Garcia-Campayo, J. (2017). Unwanted effects: Is there a negative side of meditation? A multicentre survey. PloS one, 12(9), e0183137. doi:10.1371/journal.pone.0183137.CrossRefGoogle Scholar
de Vibe, M., Bjorndal, A., Fattah, S., Dyrdal, G. M., Halland, E., & Tanner-Smith, E. E. (2017). Mindfulness-based stress reduction (MBSR) for improving health, quality of life and social functioning in adults: A systematic review and meta-analysis. Campbell Systematic Reviews, 13(1), 1-264. doi:10.4073/csr.2017.11.Google Scholar
Farias, M., Marald, E., Wallenkampf, K. C., & Lucchetti, G. (2020). Adverse events in meditation practices and meditation-based therapist: A systematic review. Acta Psychiatrica Scandinavica, 142, 374393.CrossRefGoogle Scholar
Furukawa, T. A., Noma, H., Caldwell, D. M., Honyashiki, M., Shinohara, K., Imai, H., … Churchill, R. (2014). Waiting list may be a nocebo condition in psychotherapy trails: A contribution from network meta-analysis. Acta Psychiatrica Scandinavica, 130, 181192. doi:10.1111/acps.12275.CrossRefGoogle Scholar
Gold, S. M., Enck, P., Hasselmann, H., Friede, T., Hegerl, U., Mohr, D. C., & Otte, C. (2017). Control conditions for randomised trials of behavioural interventions in psychiatry: A decision framework. The Lancet Psychiatry, 4, 725732. doi:10.1016/s2215-0366(17)30153-0CrossRefGoogle ScholarPubMed
Goldberg, S. B., & Tucker, R. P. (2019). Allegiance effects in mindfulness-based interventions for psychiatric disorders: A meta-re-analysis. Psychotherapy Research, 30, 753762. doi:10.1080/10503307.2019.1664783.CrossRefGoogle ScholarPubMed
Goldberg, S. B., Wielgosz, J., Dahl, C., Schuyler, B., MacCoon, D. S., Rosenkranz, M., … Davidson, R. J. (2016). Does the Five Facet Mindfulness Questionnaire measure what we think it does? Construct validity evidence from an active controlled randomized clinical trial. Psychological Assessment, 28(8), 10091014. doi:10.1037/pas0000233CrossRefGoogle ScholarPubMed
Hirshberg, M. J., Goldberg, S. B., Rosenkranz, M., & Davidson, R. J. (2020). Prevalence of harm in mindfulness-based stress reduction. Psychological Medicine, 19. Online ahead of print. doi:10.1017/s0033291720002834Google ScholarPubMed
Hróbjartsson, A., Emanuelsson, F., Thomsen, A. S., Hilden, J., & Brorson, S. (2014). Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. International Journal of Epidemiology, 43, 12721283. doi:10.1093/ije/dyu115.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., & Schulz, K., … CONSORT Group, (2004). Better reporting of harms in randomized trials: An extension of the CONSORT statement. Annals of Internal Medicine, 141, 781788.CrossRefGoogle ScholarPubMed
Linden, M. (2013). How to define, find and classify side effects in psychotherapy: From unwanted events to adverse treatment reactions. Clinical Psychology & Psychotherapy, 20(4), 286296. doi:10.1002/cpp.1765.CrossRefGoogle ScholarPubMed
MacCoon, D. G., Imel, Z. E., Rosenkranz, M. A., Sheftel, J. G., Weng, H. Y., Sullivan, J. C., … Lutz, A. (2012). The validation of an active control intervention for mindfulness based stress reduction (MBSR). Behaviour Research and Therapy, 50, 312. doi:10.1016/j.brat.2011.10.011.CrossRefGoogle ScholarPubMed
Santorelli, S. F., Meleo-Meyer, F., Koerbel, L., & Kabat-Zinn, J. (2017). Mindfulness-based stress reduction (MBSR) authorized curriculum guide. Worcester, MA: Center for Mindfulness in Medicine, Health Care, and Society. University of Massachusetts Medical School.Google Scholar
Schlosser, M., Sparby, T., Voros, S., Jones, R., & Marchant, N. L. (2019). Unpleasant meditation-related experiences in regular meditators: Prevalence, predictors, and conceptual considerations. Plos One, 14(5), e0216643. doi: 10.1371/journal.pone.0216643.CrossRefGoogle ScholarPubMed
Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMJ, 340, c332. doi:10.1136/bmj.c332CrossRefGoogle ScholarPubMed
Schulz, K. F., & Grimes, D. A. (2002). Sample size slippages in randomised trials: Exclusions and the lost and wayward. The Lancet, 359, 781785. doi:10.1016/S0140-6736(02)07882-0.CrossRefGoogle ScholarPubMed
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., ... Carpenter, J. R. (2009). Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ, 338, b2393.CrossRefGoogle ScholarPubMed
Tierney, J. F., Vale, C., Riley, R., Smith, C. T., Stewart, L., Clarke, M., & Rovers, M. (2015). Individual participant data (IPD) meta-analyses of randomised controlled trials: Guidance on their use. PLoS Medicine, 12(7), e1001855. doi:10.1371/journal.pmed.1001855.t001CrossRefGoogle ScholarPubMed