Skip to main content Accessibility help



  • Access
  • Cited by 8
  • Cited by
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Vrijsen, Janna N. Dainer-Best, Justin Witcraft, Sara M. Papini, Santiago Hertel, Paula Beevers, Christopher G. Becker, Eni S. and Smits, Jasper A. J. 2018. Effect of cognitive bias modification-memory on depressive symptoms and autobiographical memory bias: two independent studies in high-ruminating and dysphoric samples. Cognition and Emotion, p. 1.

    Wittekind, Charlotte E. Reibert, Evelyne Takano, Keisuke Ehring, Thomas Pogarell, Oliver and Rüther, Tobias 2018. Approach-avoidance modification as an add-on in smoking cessation: A randomized controlled study. Behaviour Research and Therapy,

    Field, Matt Jones, Andrew Kersbergen, Inge and Robinson, Eric 2018. Experimental Research Requires Valid and Sensitive Measures of Alcohol Intake, and This is a Step in the Right Direction: Commentary on Leeman and Colleagues (2018). Alcoholism: Clinical and Experimental Research, Vol. 42, Issue. 6, p. 1019.

    Kruijt, Anne-Will and Carlbring, Per 2018. Processing confusing procedures in the recent re-analysis of a cognitive bias modification meta-analysis. The British Journal of Psychiatry, Vol. 212, Issue. 4, p. 246.

    Vrijsen, Janna N. Fischer, Verena S. Müller, Bernhard W. Scherbaum, Norbert Becker, Eni S. Rinck, Mike and Tendolkar, Indira 2018. Cognitive bias modification as an add-on treatment in clinical depression: Results from a placebo-controlled, single-blinded randomized control trial. Journal of Affective Disorders, Vol. 238, Issue. , p. 342.

    Cristea, Ioana Alina 2018. Author’s reply. The British Journal of Psychiatry, Vol. 212, Issue. 4, p. 247.

    Mogg, Karin and Bradley, Brendan P. 2018. Anxiety and Threat-Related Attention: Cognitive-Motivational Framework and Treatment. Trends in Cognitive Sciences, Vol. 22, Issue. 3, p. 225.

    McNally, Richard J. 2018. Attentional bias for threat: Crisis or opportunity?. Clinical Psychology Review,




      • Send article to Kindle

        To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Invited commentary on … Confusing procedures with process in cognitive bias modification research
        Available formats

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Invited commentary on … Confusing procedures with process in cognitive bias modification research
        Available formats

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Invited commentary on … Confusing procedures with process in cognitive bias modification research
        Available formats
Export citation


The notion that cognitive bias modification should be appraised exclusively on the basis of trials where its postulated mechanisms were successfully changed starkly contradicts the standards of evidence-based psychotherapy. In the laboratory or as a treatment, cognitive bias modification cannot continue to eschew the rigorous scrutiny applied to other interventions.


See analysis, pp. 266–271, this issue.

Declaration of Interest


Selective exclusion of studies or outcomes from meta-analyses based on post-hoc criteria or applied without transparency or a solid theoretical justification is a pernicious practice that distorts results, 1 thus usually considered a grave error. Yet Grafton and colleagues seem to do exactly that with our meta-analysis. 2 They misconstrue it as investigating whether cognitive bias modification (CBM) alters ‘emotional vulnerability’, a vague concept of uncertain clinical relevance. Instead, as evident throughout, we examined post-intervention anxiety and depression outcomes. Using our data for anxiety outcomes at post-test, they employ three arbitrary filters to selectively exclude studies and subsequently perform a strictly qualitative and unclear classification of the remaining ones.

The most conspicuous filter is the exclusion of ten studies for measuring ‘resting mood state’ instead of ‘emotional vulnerability’. Eight of these used the State–Trait Anxiety Inventory (STAI)-State, one a specific phobia inventory, 3 another social anxiety measures. 4 Hence, we are at a loss as to what the authors mean, as all of these measure symptoms of ‘anxiety’. Moreover, all our effect size calculations were, as described, at post-test. If for instance authors of a trial would use the STAI-State at both post-test and after a so-called stressor task, we only considered the former. So, in this sense everything was ‘resting mood’. Furthermore, if Grafton et al deemed state anxiety measures as improper, they should have also excluded them from effect size calculations for the other included trials. Their analysis nonetheless retained five other studies that solely used the STAI-State. 5,6 Finally, Grafton et al do not substantiate their reanalysis with any actual data analysis, except for tallying findings as yes or no. We undertook this task for them. Presuming they intended to exclude state anxiety, we recalculated effect size for the 12 remaining studies measuring anxiety. The eight where bias change occurred resulted into a small Hedges g of 0.38, virtually identical to our original findings 2 (Duval-Tweedie publication bias adjusted g = 0.28). We conducted meta-regression analyses combining bias change with other significant moderators of outcome. 2 Bias change no longer predicted outcomes (Table 1).

Table 1 Meta-regression analysis for bias change, alone and in combination with other significant predictors of outcome 2

Bias change
Alone And participant

And delivery And impact

b = 0.42, P = 0.032 b = 0.37, P = 0.18 b = 0.25, P = 0.50 b = 0.24, P = 0.38

More generally, the claim that CBM should be assessed for effectiveness only in the presence of change in its postulated mechanisms conflicts with the current standards for evaluating psychotherapies. For instance, the effectiveness of cognitive–behavioural therapy 7 is not restricted to trials where dysfunctional thoughts were successfully changed. Process variables are commonly conjectural, unclear, multiple, confounded with outcome measures, assessed in miscellaneous ways, and produce contradictory results. Even when a hypothesised process changes in a trial, it does not follow this is indeed a mechanism of change. 8 For CBM, the nature and direction of bias change needed to engender symptom change have been targets of speculation and debate, 9,10 although posited as self-evident facts by Grafton et al.

Ultimately, CBM researchers should decide at which table they want to sit. If CBM is cast as a laboratory development, encouraging but as yet inconsequential for clinical practice, exploring procedures to modify assumed processes is an adequate goal. Conversely, if – as repeatedly claimed 10 – CBM is a promising psychotherapy for use on patients and in clinical trials, it should comply with the same standards as all psychotherapies. These standards involve evaluating effectiveness on clinically relevant outcomes, using all available evidence, as we did, 2 and cannot hinge on whether or not purported processes have changed. Grafton et al summarily gloss over other serious problems we evidenced, such as lack of effects for clinical samples, pervasive publication bias and low study quality. Vague and debatable distinctions qualitatively applied post-hoc to a subset of the available data cannot substitute for modest if extant symptom change.


1 Page, MJ, McKenzie, JE, Kirkham, J, Dwan, K, Kramer, S, Green, S, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev 2014; 10: MR000035.
2 Cristea, IA, Kok, RN, Cuijpers, P. Efficacy of cognitive bias modification interventions in anxiety and depression: meta-analysis. Br J Psychiatry 2015; 206: 716.
3 Harris, LM, Menzies, RG. Changing attentional bias: can it effect self-reported anxiety? Anxiety Stress Coping 1998; 11: 167–79.
4 Heeren, A, Reese, HE, McNally, RJ, Philippot, P. Attention training toward and away from threat in social phobia: effects on subjective, behavioral, and physiological measures of anxiety. Behav Res Ther 2012; 50: 30–9.
5 Hirsch, CR, Mathews, A, Clark, DM. Inducing an interpretation bias changes self-imagery: a preliminary investigation. Behav Res Ther 2007; 45: 2173–81.
6 Murphy, R, Hirsch, CR, Mathews, A, Smith, K, Clark, DM. Facilitating a benign interpretation bias in a high socially anxious population. Behav Res Ther 2007; 45: 1517–29.
7 Cuijpers, P, Berking, M, Andersson, G, Quigley, L, Kleiboer, A, Dobson, KS. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry 2013; 58: 376–85.
8 Kazdin, AE. Mediators and mechanisms of change in psychotherapy research. Annu Rev Clin Psychol 2007; 3: 127.
9 Koster, EHW, Bernstein, A. Introduction to the special issue on cognitive bias modification: taking a step back to move forward? J Behav Ther Exp Psychiatry 2015; 49: 14.
10 MacLeod, C, Mathews, A. Cognitive bias modification approaches to anxiety. Annu Rev Clin Psychol 2012; 8: 189217.