Balanzá-Martinez et al made a detailed review of our article 1 and also provided several suggestions for future research.
Although we agree with the authors that the conclusions of our article are by no means definitive, we disagree with them on the following important issues.
First, Balanzá-Martinez et al suggest that our meta-analysis ‘more likely overestimates the potential for cognitive improvement’ as compared with other reviews (the authors cite as an example the review by Rund 2 ). On the contrary, we think that the comparison of results in individuals with schizophrenia with controls, which is an original feature of our review, has a sobering effect as it points out that ‘practice [is] more likely than cognitive remediation to account for most of the improvements observed’. On the other hand, reviews (such as the one cited earlier) in which performances in people with schizophrenia are not compared with those of controls could mistake improvement in results for improvement in cognitive abilities.
Second, commenting on the methods we used in our meta-analysis, Balanzá-Martinez et al criticise three of our options: (a) the inclusion of studies with a test–retest duration under 1 year; (b) the inclusion of open trials (especially clozapine trials); and (c) the fact that we did not differentiate between ‘patients with first-episode or chronic schizophrenia and geriatric patients’. They suggest therefore that it would be better to limit the analyses to a subset of the available data assuming, without formally testing, that some of the studies' characteristics significantly influence results.
At the time we made our analysis, there were only 11 studies with more than a year test–retest interval and only 4 reported data for a control group. It was not possible to limit our analysis only to this subset of studies, let alone further exclude studies or separately analyse subgroups of studies.
Instead, we chose to include all methodologically sound studies and test the role of potentially confounding variables (including test–retest interval). By using this method, we limited the loss of important available information. It is our belief that we could not gain more insight from fewer data.
As we report in our article, only 2 out of the 17 variables tested showed a larger improvement in studies with shorter test–retest intervals. Thus, for the vast majority of variables, excluding studies with shorter test–retest intervals would lead to an unjustified loss of information.
Third, Balanzá-Martinez et al consider that their suggestions may lead to results ‘more helpful for clinicians, patients and caregivers’. We think that limiting the analysis to studies with large test–retest intervals (which usually have high attrition rates) or excluding the more naturalistic, open trials would achieve just the contrary.
In conclusion, although we agree with Balanzá-Martinez et al that the conclusions of our meta-analysis are not definitive, we also consider that to improve our knowledge on the subject we need new data, not new analyses, in subsets of (the same) data.