We noted with interest the recent publication of two articles on cognitive functioning in heavy users of ecstasy (an illicit preparation represented as containing MDMA; ±3,4-methylenedioxymethamphetamine) and other drugs (Bedi & Redman, Reference Bedi and Redman2008; Schilt et al. Reference Schilt, de Win, Jager, Koeter, Ramsey, Schmand and van den Brink2008). We are grateful to Dr Bedi and Dr Redman for discussing the numerous methodological limitations and inconsistencies with research on cognition in ecstasy users. Cautionary notes to the readers are still warranted. Both studies were non-randomized, retrospective, cross-sectional studies, with subjects recruited from different locations, increasing the risk for important baseline differences due to selection bias. The mean lifetime ecstasy use in the two studies was 170 tablets (range 13.5–2407) and 327 tablets (range 15–2000), respectively. In contrast, only 20–30% of ecstasy users have lifetime consumption over 25 tablets (De Win et al. Reference de Win, Jager, Vervaeke, Schilt, Reneman, Booij, Verhulst, Den Heeten, Ramsey, Korf and van den Brink2005).

Bedi & Redman found no group differences on verbal memory ability. Their groups of ecstasy-using and non-ecstasy-using polydrug users (total n=133) appeared relatively well-matched on a range of potential confounders, including education, depression, and anxiety in the subjects and substance abuse or psychiatric illness in first-degree relatives. After an automated variable reduction procedure that discarded most of the confounding variables, a weak negative correlation appeared between a ‘verbal memory’ factor and lifetime ecstasy use. Importantly, the authors fail to mention that such derived statistical models must always be verified on independent data (Good & Hardin, Reference Good and Hardin2006). It would be useful to see whether lifetime ecstasy use still predicted verbal memory in the Bedi & Redman sample when all measured confounders are included in the analysis or at least examined in more detail. Overall, Bedi & Redman conclude that the ‘hypothesis that ecstasy users would display lower cognition than non-users was not supported’.

Schilt and colleagues' report is part of a large multi-part project called the ‘Netherlands XTC Toxicity (NeXT)’ study (De Win et al. Reference de Win, Jager, Vervaeke, Schilt, Reneman, Booij, Verhulst, Den Heeten, Ramsey, Korf and van den Brink2005). Schilt and colleagues report a limited list of potential confounders, only gender, age, IQ, education level, and use of other substances. Although subjects with major psychopathology were excluded, no psychiatric family-history or lifestyle variables were reported. The limited demographic data was not reported separately for each group, and it is unclear if the groups were well-matched. Schilt and colleagues claim that in their sample ‘frequent ecstasy use is responsible for a drop of nearly two out of 15 words in a verbal delayed memory task’, an effect they call ‘quite substantial’; however, here they describe the raw difference between their non-randomized groups (total n=67), without adjusting for any confounders at all. After adjusting for age, gender, IQ, and other substance use, but not including education level or adjusting for multiple comparisons, weak associations appeared between ecstasy use and verbal delayed recall and verbal confabulations, but not verbal immediate recall or any of the other cognitive measures presented. In the same model, weak associations also appeared between alcohol use and verbal delayed recall and between gender and verbal immediate recall. Within the subjects who used ecstasy, a weak association appeared between lifetime ecstasy dose and verbal delayed recall (p=0.03, one-tailed), but not verbal confabulations. This dose–response analysis depends heavily on the unlikely assumption that the ecstasy-using subjects, with lifetime ecstasy doses ranging from 15 to 2000 tablets, are equivalent on all known and unknown potential confounders besides age, gender, IQ, and other substance use. The NeXT study design article lists three verbal memory subscores: immediate, delayed and recognition (De Win et al. Reference de Win, Jager, Vervaeke, Schilt, Reneman, Booij, Verhulst, Den Heeten, Ramsey, Korf and van den Brink2005). However, in the current cross-sectional study the NeXT team, without explanation, replaced verbal recognition with a verbal confabulation subscore. Overall, the NeXT team overstate their findings when they conclude that their non-randomized study ‘strongly suggest a specific negative effect of ecstasy use on verbal memory’.

Non-randomized and retrospective studies are notoriously misleading on causation (Smith & Ebrahim, Reference Smith and Ebrahim2002). Childhood neglect has been associated with decreased verbal memory in adulthood (Grassi-Oliveira et al. Reference Grassi-Oliveira, Stein, Lopes, Teixeira and Bauer2008). Ecstasy users are more likely to report childhood physical abuse and neglect (Singer et al. Reference Singer, Linares, Ntiri, Henry and Minnes2004; Montgomery et al. Reference Montgomery, Fisk and Craig2008). Thus, childhood neglect is an example of one of a multitude of pre-existing factors that might both decrease verbal memory ability and influence cumulative use of ecstasy – a drug well-known to increase compassion, and closeness to self and others. Lower verbal memory has also been reported in people with no psychiatric diagnosis but with limited symptoms of schizophrenia (Hurlemann et al. Reference Hurlemann, Jessen, Wagner, Frommann, Ruhrmann, Brockhaus, Picker, Scheef, Block, Schild, Moller-Hartmann, Krug, Falkai, Klosterkotter and Maier2008) or first-degree relatives diagnosed with depression (Mannie et al. Reference Mannie, Barnes, Bristow, Harmer and Cowen2008) or bipolar disorder (Arts et al. Reference Arts, Jabben, Krabbendam and van Os2008). A previous study by the NeXT team found that depression symptoms were correlated with lifetime ecstasy use (de Win et al. Reference de Win, Reneman, Reitsma, den Heeten, Booij and van den Brink2004), a large longitudinal study in The Netherlands found that anxiety and depression in childhood were risk factors for later ecstasy use (Huizink et al. Reference Huizink, Ferdinand, van der Ende and Verhulst2006), and the NeXT study design article describes psychiatric and lifestyle factors as serious potential confounders (De Win et al. Reference de Win, Jager, Vervaeke, Schilt, Reneman, Booij, Verhulst, Den Heeten, Ramsey, Korf and van den Brink2005). Schilt and colleagues do not adjust for, or even mention, psychiatric factors in their reports.

Another methodological issue that threatens the validity of all the published studies on the NeXT non-randomized cross-sectional sample (Jager et al. Reference Jager, de Win, van der Tweel, Schilt, Kahn, van den Brink, van Ree and Ramsey2007; de Win et al. Reference de Win, Jager, Booij, Reneman, Schilt, Lavini, Olabarriaga, Ramsey, Heeten and van den Brink2008; Schilt et al. Reference Schilt, de Win, Jager, Koeter, Ramsey, Schmand and van den Brink2008) is a possible sampling bias: subjects were recruited at different locations and settings and encouraged to recruit their friends. Confounding due to lifestyle differences, for instance regular attendance at dance parties (raves), cannot be dismissed. Moreover, heavy ecstasy users may have volunteered for a study entitled the ‘Netherlands XCT Toxicity’ study in order to confirm the existence of perceived ecstasy-related problems. Ecstasy users ‘primed’ to think that ecstasy is toxic performed worse than non-primed ecstasy users specifically on a verbal memory test (Cole et al. Reference Cole, Michailidou, Jerome and Sumnall2006). Since all the NeXT studies have recruited subjects from different locations with different methods, such as through a webpage of the project and snowballing, a serious selection bias cannot be excluded.

To study cognitive dysfunction in socially stigmatized groups is notoriously difficult (Gould, Reference Gould1996). Looking over 20 years of repeated studies looking for brain damage in ecstasy users, we see very few consistent findings and little consideration of pre-existing psychiatric factors that may influence young people to repeatedly risk criminal penalties in order to experience MDMA-mediated feelings of love and empathy. As Bedi & Redman acknowledge, cognitive functioning in ecstasy users is a highly debated topic and the data are inconclusive with no clear pattern of specific deficits.

In both articles under discussion, the authors speculate that any cognitive effects of ecstasy use could increase with age; however, there is no empirical basis for this often repeated warning. Most longitudinal studies of ecstasy users have found no change in cognitive function with continued ecstasy use, suggesting that any cognitive deficits may have been pre-existing (Gouzoulis-Mayfrank & Daumann, Reference Gouzoulis-Mayfrank and Daumann2006). Cross-sectional studies in moderate ecstasy users rarely find any effects (Gouzoulis-Mayfrank & Daumann, Reference Gouzoulis-Mayfrank and Daumann2006). Studies in non-randomized samples of heavy ecstasy users have little relevance for clinical studies involving infrequent doses of pharmacologically pure MDMA.

Given the accumulating evidence, it appears that ecstasy use is a comparatively minor overall problem for society compared to alcohol and many other drugs (Nutt et al. Reference Nutt, King, Saulsbury and Blakemore2007). Decades with studies of cognitive ability in ecstasy users continue to reveal small and inconsistent results and should therefore be interpreted with caution.