Kruijt & Carlbring misrepresent the position conveyed in our commentary,Reference Grafton, MacLeod, Rudaizky, Holmes, Salemink and Fox¹ wrongly attributing to us the suggestion ‘that we should only call CBM CBM if it is successful’. Our actual points are: (a) it cannot be claimed that cognitive bias has been modified when assessment data reveal that no modification of cognitive bias has taken place; and (b) the emotional impact of modifying cognitive bias cannot be determined from studies that fail to modify cognitive bias. Also, incorrectly, they describe our commentary as an ‘exposé on the correct question to meta-analyse’. We highlight the need to distinguish two quite different questions, without claiming that either is ‘correct’, and emphasise the resulting problems when meta-analyses fail to do so.

Our position adheres to the tenets of experimental medicine.Reference Sheeran, Klein and Rothman² The first step in experimental medicine is to identify a target mechanism that plausibly contributes to the dysfunction of interest. For example, high blood pressure represents a mechanism that may contribute to the dysfunction of elevated stroke risk, and attentional bias to threat represents a mechanism that may contribute to the dysfunction of anxious disposition. Step two involves developing a candidate intervention intended to manipulate this mechanism. This could involve a drug intended to reduce blood pressure, or a computer procedure intended to reduce attentional bias to threat. Step three involves delivering the intervention to determine: (a) whether the intervention impacts the mechanism, as intended; and if so (b) whether this impact on mechanism therapeutically attenuates dysfunction. Should the drug fail to reduce blood pressure, with no observed reduction in stroke risk, it cannot be concluded that reducing blood pressure has no impact on stroke risk. Likewise, should the computer procedure fail to modify attentional bias to threat, with no observed reduction in anxious disposition, it cannot be concluded that modifying attentional bias to threat has no impact on anxious disposition. If the drug sometimes reduces blood pressure, and whenever this occurs stroke risk also decreases, this suggests that blood pressure reduction attenuates stroke risk. Likewise, if the computer procedure sometimes reduces attentional bias to threat, and whenever this occurs anxious disposition also decreases, this suggests that attentional bias modification alters anxious disposition.

We re-analysed Cristea et al’sReference Cristea, Kok and Cuijpers³ effect sizes to demonstrate that, when procedures intended to modify cognitive bias elicit the process of cognitive bias modification, there is consistent impact on emotional disposition. Kruijt & Carlbring contend that Cristea et al’s method of computing effect sizes compromises sensitivity to emotional disposition, which would represent a further limitation of this meta-analysis. However, compelling evidence that when procedures intended to evoke the process of cognitive bias modification do so successfully then so too do they alter emotional disposition, is not restricted to our re-analysis, and has been reported elsewhere.Reference Price, Wallace, Kuckertz, Amir, Graur and Cummings⁴

We advocate adherence to the experimental medicine framework, by clearly distinguishing two questions: one asks whether successfully modifying cognitive bias yields therapeutic benefit, and the other asks whether procedures intended to modify cognitive bias successfully induce this process of cognitive change.