In a recent article in this Journal, Van der Zanden et al. tried to unravel the mechanisms that may explain the efficacy of an internet therapy for depression (van der Zanden et al. Reference van der Zanden, Galindo-Garre, Curie, Kramer and Cuijpers2014). Knowledge about mediating processes is scientifically interesting and may help to improve interventions. The authors suggested a circular therapeutic process including changes in depression, anxiety and mastery. However, one may question how clinically relevant these supposed mediating variables are. Would the therapy be adjusted to include more anxiety-reducing or mastery-promoting elements, if, for instance, a large mediating effect of anxiety or mastery on depression was found? The authors did not present their opinions. The assumption of a circular process suggests that it does not matter what you promote therapeutically; at least, in the way we interpreted the term ‘circular mediational process’. Van der Zanden and colleagues did not indicate what a circular mediational process would look like, but we assumed that it implies a causal chain of changes: For example, a decrease in depression leads to an increase in mastery, which in turn may lead to a further decrease in depression.

Unfortunately, the methods applied in this study were – in our view – not suited to demonstrate such a circular process. Neither were the applied methods very appropriate for demonstrating the activity of any mediating mechanism. We will present five arguments for this statement.

First, in order to determine whether a predictor causes a change in the supposed mediating variable and whether this change predicts a subsequent change in the outcome variable one needs at least three measurements. For the demonstration of the circular process one would need many more measurements. Van der Zanden and colleagues have obtained only three measurements and they used only two of these for their mediation analysis; a pre-treatment and a post-treatment measurement!

Second, to show how the variables affect each other, there should be a short interval between measurements, say of no more than one day. For instance, a person's feeling of mastery could have been increased by the accomplishment of a difficult task, which may lead to an improvement in one's depressed mood the next day, which in turn may lead to higher self-efficacy beliefs for performing the tasks of the day thereafter. The authors have applied a measurement interval of 12 weeks, however.

Third, the authors used the changes in depression, anxiety and mastery from pre- to post-treatment in three different path analyses. Each analysis included group membership (being in the intervention group or in the control group) as the predictor. In the first model, the change in depression was the outcome variable and the changes in anxiety and in mastery were the supposed mediating variables. In the second and third model the outcome and supposed mediating variables changed places. However, with this design one cannot discern whether a change in one variable caused a change in another variable, because the scales that were used to measure depression, anxiety and mastery have very similar content. For example, the item ‘Was bothered by things that usually don't bother me’ (Center for Epidemiologic Studies Depression scale; CES-D) is about the same as the item ‘Worrying thoughts go through my mind’ (Hospital Anxiety and Depression Scale; HADS-A). A person who endorses the item ‘I often feel helpless dealing with the problems of life’ (Mastery Scale) will probably also score high to the item ‘Felt depressed’ (CES-D). So, although depression, anxiety and mastery can be theoretically distinguished, they may not be different in the eyes of the participants; at least not with the scales used in the present study. The authors considered the high correlation between changes in depression and anxiety as support for their circularity hypothesis, but they can also be interpreted as evidence for conceptual overlap between the two constructs.

Fourth, the authors found different outcomes for their three mediation models, which they explained as evidence for different roles for the three variables; whether they were considered to be an outcome variable or a mediating variable. However, instead of their mediation explanation, the authors should have considered an alternative and simpler explanation, which considers all three variables as parallel outcome variables. This explanation suggests that a change in one variable partly implies a change in the other two variables. After all, the authors found that changes in anxiety and mastery were related to changes in depression, and that changes in depression were related to changes in mastery and anxiety; and these changes had all been assessed during the same time interval. The finding that the intervention was directly related to changes in depression as an outcome, but not to anxiety or mastery as outcomes can be arithmetically explained by a larger effect in one of the three variables; in this study the larger effect on depression.

Fifth, as a final step in their analyses, the authors determined the association between an early change in depression (between baseline and post-intervention) and a later change in anxiety (between post-intervention and follow-up), and vice versa. Their peculiar assumption was that the absence of such associations would be supportive for their circularity hypothesis. This is beyond our understanding. There may be several reasons for an absence of a relationship, such as the relatively large time interval between the measurements. Therefore, the absence of an association can not be used as prove for their specific hypothesis.

In fact, a negative relationship was found. The authors did not present an explanation for this finding, but the explanation may be simple: with a large early change in depression one would expect a relatively small later change in this variable, and with a small early change in depression one expects a relatively larger later change in the same variable. Because of the high correlation between depression and anxiety, one would thus expect that a greater early reduction in depression is associated with a smaller later change in anxiety (and a greater early reduction in anxiety is associated with a smaller later change in depression).

To summarize, the authors did not clearly explain the concept of circularity, which is central in their article, and did not thoroughly reflect on the consequences of their circularity assumption with respect to the aim of ‘further improvement of treatments’ and with respect to analytical methods. The design and analytical methods of their study were not suited for a demonstration of mediational processes.