Acceptability of cognitive remediation for schizophrenia: a systematic review and meta-analysis of randomized controlled trials

Background Acceptability is an important factor for predicting intervention use and potential treatment outcomes in psychosocial interventions. Cognitive remediation (CR) improves cognition and functioning in people with a diagnosis of schizophrenia, but its acceptability, and the impact of participants and treatment characteristics, remain to be investigated. Few studies provide a direct measure of acceptability, but treatment drop-out rates are often available and represent a valid surrogate. Method The systematic search conducted for the most comprehensive CR outcomes database for schizophrenia was updated in December 2020. Eligible studies were randomized clinical trials comparing CR with any other control condition in patients diagnosed with schizophrenia spectrum disorders and that also reported drop-out in treatment and control arms separately. Acceptability was measured as odd-ratios (OR) of drop-out. Results Of 2119 identified reports, 151 studies, reporting 169 comparisons between CR and control interventions with 10 477 participants were included in the analyses. The overall rate of drop-out was 16.58% for CR programs and 15.21% for control conditions. In the meta-analysis, no difference emerged between CR interventions and controls [OR 1.10, 95% confidence interval (CI) 0.96–1.25, p = 0.177]. Factors improving acceptability were: inpatient only recruitment, participants with fewer years of education and lower premorbid IQ, the presence of all CR core elements, and the presence of techniques to transfer cognitive gains into real-world functioning. Conclusions CR for people diagnosed with schizophrenia is effective and has a good acceptability profile, similar to that of other evidence-based psychosocial interventions.


Background
Cognitive remediation (CR) for people diagnosed with schizophrenia is a behavioral trainingbased intervention targeting cognitive processes (Wykes, Huddy, Cellard, McGurk, & Czobor, 2011). It produces significant gains not only in cognitive performance, but also in real-world psychosocial functioning, as demonstrated in systematic and recent meta-analyses (Kambeitz-Ilankovic et al., 2019;Lejeune, Northrop, & Kurtz, 2021;Vita et al., 2021), even for people who are more clinically compromised (Vita et al., 2021). So, there is little doubt that CR is an important asset in aiding people living with schizophrenia to achieve their recovery goals.
CR implementation is not widespread despite recent guidance (Keepers et al., 2020;National Institute for Health and Care Excellence, 2020), and there is still some reluctance to include it in everyday clinical practice (Vita & Barlati, 2019;Wykes, 2018). Acceptability is a key factor in real-world clinical settings as it is a predictor of intervention use and potential outcomes under real-world conditions. We know from the most recent meta-analysis that longer treatments produce better outcomes (Vita et al., 2021), so factors fostering participation as well as potential barriers to treatment delivery are important to discover. Acceptability is a complex construct, which can be assessed in different ways, depending on the theoretical framework in which it is evaluated (Carter, 2007). In healthcare interventions acceptability can be estimated through patient feedback, or by treatment drop-out rates (Sekhon, Cartwright, & Francis, 2017).
The core ingredients of CR (presence of an active and trained therapist, repeated practice of cognitive exercises, structured development of cognitive strategies, and use of techniques to improve the transfer of cognitive gains to the real world) identified in a recent expert consensus (Bowie et al., 2019) influence treatment effectiveness (Vita et al., 2021). We do not know if these same components have an impact on acceptability nor whether any patient-related characteristics could identify candidates who would complete treatment and therefore be more likely to receive a treatment benefit.

Aims of the study
To investigate CR acceptability (trial drop-out rate) between treatment and control arms in randomized clinical trials and assess if differences were moderated by patient characteristics or beneficial treatment components.

Methods
The review was conducted in accordance with the Preferred Reported Items for Systematic Review and Meta-analysis (PRISMA) guidelines (Moher, Liberati, Tetzlaff, Altman, & Group, 2009), and can be viewed as a complementary analysis to Vita et al. (2021). The literature search was completed on Dec 23, 2020, and comprised a systematic search of 3 electronic databases (PubMed, Scopus and PsycInfo) using the terms [('cognitive' or 'cognit*') AND ('training' or 'remediation' or 'rehabilitation' or 'enhancement') AND ('schizophrenia' or 'psychosis') AND ('random*' or 'randomized control trial' or 'clinical trial')], a manual search of Google Scholar with combinations of the same keywords, and manual inspection of reference lists of emerging reviews and included papers. Studies previously excluded due to the absence of outcomes of interest or not providing usable outcome data were re-assessed for eligibility.
Eligible studies were randomized trials recruiting participants with a diagnosis of schizophrenia spectrum disorders (at least 70% of the total sample), comparing CR interventions fulfilling the standard Experts Workshop definition (2010) to any control condition other than CR. For this meta-analysis we chose studies explicitly reporting drop-out in each treatment arm. CR could be either delivered as a stand-alone treatment or integrated with other adequately controlled psychosocial interventions. No restriction was applied either to the treatment setting or the duration and delivery mode of CR interventions. The screening was conducted by 2 independent reviewers with disagreements resolved by a third author. Only articles published in English in peer-reviewed journals were considered. The methodological rigor of included studies was evaluated using the Clinical Trials Assessment Measure (CTAM) (Tarrier & Wykes, 2004) by at least 2 reviewers, who also extracted data independently.

Outcomes of interest
The main outcome was the reported number of trial drop-outs due to any reason in each treatment arm. Drop-outs were individuals providing consent to participate in the study, randomized to one of the treatment arms and leaving the study before the end of the active treatment phase. Drop-out occurring after treatment was not considered. All drop-out definitions were considered valid, with no threshold for the minimum number of attended treatment sessions. If the reasons for drop-out were reported then this information was also extracted. A separate analysis was conducted including only drop-out due to lack of motivation, i.e., participants who withdrew consent to participate during the active treatment period or who explicitly considered the treatment tedious or too intensive.

Outcome measures
Acceptability was measured as odd-ratios (OR) of drop-out, with 95% confidence interval (CI), calculated according to the following formula: OR = (drop-out from CR interventions/completers of CR interventions)/(drop-out from control interventions/completers of control interventions). For studies with multiple treatment arms, each eligible comparison between CR interventions and control conditions was considered separately. As including multiple comparisons within a study could lead to a potential bias of dependent effects, sensitivity analyses were conducted introducing only one comparison per study (Rücker, Cates, & Schwarzer, 2017;Van den Noortgate, López-López, Marín-Martínez, & Sánchez-Meca, 2013).

Meta-analytic procedures
Given the expected high level of participant clinical heterogeneity of included studies, a random-effect approach was preferred. A sensitivity analysis was also performed using a fixed-effect approach. Statistical heterogeneity was investigated through visual inspection of forest plots and assessment of Q-test and I 2 statistic. All meta-analyses were performed using Comprehensive Meta-Analysis version 3.0 (Biostat, Englewood, NJ, USA, 2013). Descriptive statistics and analyses were performed using SPSS 22 (SPSS Inc., Chicago, IL, USA, 2005). p values <0.05 (2-tailed) were considered significant for all analyses.

Moderator effects
Subgroup, sensitivity, and meta-regression analyses were conducted to explore the effect of study methods, participants, and interventions characteristics on acceptability as identified or suggested by other investigators. We also carried out a sensitivity

3662
Antonio Vita et al.
analysis including only trials conducted in real-world settings, and subgroup analysis to compare pilot/feasibility studies to more rigorous trials. The list of moderator variables explored is available in Appendix 1.

Certainty of the evidence
The risk of publication bias was assessed by visual inspection of funnel plots and Egger test for asymmetry (Egger, Smith, Schneider, & Minder, 1997). Other determinants of the quality of evidence such as consistency, precision, and directness were explored using experts' recommendation (Schünemann, 2013).

Results
The study selection procedure is shown in Fig. 1. A total of 151 studies, reporting 169 comparisons between CR and control interventions with 10 477 participants were included in the analyses; 3 ongoing studies were also identified (see Appendix 2). Included participants were representative of people living with schizophrenia and using mental health services, from individuals experiencing first episodes to those with thirty-year illness duration, and at different stages of recovery. The mean CR treatment duration was 15.82 ± 15.45 (range 2-104) weeks; the four core ingredients of CR (Bowie et al., 2019) were well represented (single elements present in ⩾70% cases); however, only half (66, 48.88%) included all of them. CR was delivered in the context of a structured psychiatric rehabilitation program in 48 (28.9%) cases.

Included studies
Sixty-five studies were conducted in Europe, 41 in the US, 27 in Asia, 7 in Middle East countries, 5 in Canada, 4 in Australia and 2 in Brazil. Eighty-three studies recruited outpatients, 50 included inpatients from acute and residential care services, and 18 recruited from both. Most took place in real-world clinical practice sites (101, 66.9%), while some trials were conducted in specific settings (e.g. 20 research-oriented sites, 6 home-based or low-support programs, 4 forensic settings, 1 supported university program). Thirty-nine (25.8%) studies were explicitly defined as pilot/feasibility trials. Descriptive data are reported in detail in Table 1.
The overall rate of drop-out was 16.58% for CR programs and 15.21% for control conditions (18.62 and 17.28% respectively when removing trials with no drop-out). In the meta-analysis, no difference emerged between CR interventions and controls regarding attrition: OR 1.10 (95% CI 0.96-1.25), p = 0.177. Overall heterogeneity was statistically significant (Q = 163.68, df 134, p = 0.041), but low (I 2 = 18.13%). The sensitivity analysis conducted using a fixed-effects model yielded very similar results: OR 1.09 (95% CI 0.97-1.22), p = 0.132. No evidence of publication bias emerged (no funnel plot asymmetry at visual inspection, Egger test: p = 0.591). The results were also comparable when the analysis only included studies where drop-out was reported to be due to low motivation: [OR 1.064 (95% CI 0.821-1.378), p = 0.639], although this reduced the sample size (N = 36).
The type of control had a significant impact (χ 2 = 9.90, p = 0.019) with attrition being higher for CR compared to TAU [OR 1.43 (1.10-1.86)], but there was no consistent difference in comparison to an active control activity or an evidence-based psychosocial intervention. Surprisingly acceptability was higher when the comparison was an active TAU [OR 0.71 (0.50-1.03)]. Methodological quality, publication year, sample size, included diagnoses and payment of participants had no significant effect.
Acceptability of CR interventions was significantly higher in studies with participants showing fewer years of education [Coefficient = 0.150 (0.039-0.262), p = 0.008] and lower premorbid IQ [Coefficient = 0.026 (0.002-0.050), p = 0.033]. No significant effect was observed for age, gender, age of onset, duration of illness, or baseline symptom severity. Antipsychotic treatment dosage did not emerge as a significant moderator either.
Among the core CR ingredients in our subgroup analyses, the implementation of techniques to transfer cognitive gains to realworld functioning had a significant positive impact on acceptability [Including this element, OR 1.02 (0.89-1.18) not including this element OR 1.45 (1.06-1.99), χ 2 = 4.01, p = 0.045] (see Table 2). The presence of an active and trained therapist, the repetition of cognitive exercises or teaching novel cognitive strategies did not exert a significant effect when considered separately. However, treatments that included all the core ingredients had significantly better acceptability ) v. 1.25 (1.04-1.50), χ 2 = 3.99, p = 0.046], which further improved if psychiatric rehabilitation was considered as the optimal transfer technique [OR 0.82 (0.65-1.05) v. 1.20 (1.03-1.40), χ 2 = 6.83, p = 0.009]. No significant effect was observed for other treatment-related variables such as duration, intensity, format, and method of treatment delivery.

Discussion
This is the first systematic and comprehensive assessment of CR acceptability for people living with schizophrenia as measured by the study drop-out rate. The overall drop-out rate (16.58%) was equivalent to that observed in control interventions (15.21%), as confirmed by the quantitative meta-analytic synthesis that found no significant differences between the two groups. Of note, the analysis of drop-out due to low motivation, which can be viewed as the reason more specifically related to the intervention, yielded comparable results. The CR drop-out rate was also comparable to one found across different psychosocial interventions for individuals diagnosed with schizophrenia (Szymczynska et al., 2017;Villeneuve et al., 2010), and lower with respect to drop-out rates reported in pharmacological trials (Bighelli et al., 2020;Cramer & Rosenheck, 1998;Kemmler, Hummer, Widschwendter, & Fleischhacker, 2005;Lieberman & Hsiao, 2006), which often involve shorter observation periods (Bighelli et al., 2020). Acceptability of CR intervention was higher in studies recruiting inpatients. This result replicates previous literature reporting better feasibility of psychosocial interventions in inpatients setting, as confirmed by a systematic review (Villeneuve et al., 2010). Treatment programs that included all the core ingredients for CR effectiveness (Bowie et al., 2019), and particularly those including techniques to transfer cognitive gains into real-world functioning showed a lower drop-out rate. This is an important finding as it could be linked to a better treatment rationale, improved perceived efficacy by the participants themselves, and may also have positive effects on motivation. Other treatment elements despite having an impact on CR benefits (Vita et al., 2021), did not produce significant effects on acceptability. This result is of practical importance if we are to reduce barriers to successful recovery. Making the treatment rationale clear and linked to functional outcome is part of case formulation and a way of personalizing treatment [see (Wykes & Reeder, 2005) for a suggested method of CR case formulation]. To reduce drop-out (the surrogate for acceptability) and improve the cost-effectiveness of treatment, services choosing a CR intervention as part of their recovery programs should choose one that delivers this element.
The presence of an active and trained therapist did not emerge as a significant factor, despite its emphasis in previous literature

3664
Antonio Vita et al.  (Browne et al., 2021;Fiszdon et al., 2016), although it did have an effect on CR outcomes (Vita et al., 2021). This finding could be explained by therapists enhancing the treatment rationale, encouraging the transfer of CR skill improvement into everyday life and the chances of participants using these skills through the development of a positive therapeutic alliance. This more complex set of relationships requires further analysis. Participants with lower IQ and fewer years of education displayed better acceptability of CR programs; these characteristics have been already described as important moderators of CR efficacy on both cognition and functioning (Fiszdon, Choi, Bryson, & Bell, 2006;Tan et al., 2019;Twamley, Burton, & Vella, 2011;Vita et al., 2021); a previous study also found a correlation between lower cognitive performance, in particular in attention and working memory domains, and the level of participation to the intervention (Dillon et al., 2016). This finding suggests again that a link might exist between better effectiveness and better acceptability of the intervention. It may be that participants who are more inclined to commit to treatment are those who are more fragile or more compromised and more motivated to obtain an improvement in their clinical and personal condition (Saperstein et al., 2020). This hypothesis, however, remains to be more appropriately and thoroughly investigated.

Strengths and limitations
One of the main strengths of the present study is its novelty and uniqueness as the first meta-analysis to explore CR intervention acceptability for people living with schizophrenia. The conclusions were drawn from a comprehensive search of the literature and from the largest database reported of CR trials, so the robust results have implications for CR practical implementation.
The most obvious limitation is that we had to adopt drop-out rates as a measure of acceptability because very few studies asked participants what they thought of the treatment. Drop-out from treatment can be affected by many factors, including external life events, change in personal circumstances and symptom changes, although as we were investigating randomized trials these factors would equally occur in the comparison groups. This issue was partially addressed by performing a sub-analysis including only participants who explicitly left the trial due to low motivation. Some heterogeneity also occurred in the study drop-out definitions as they ranged from participants recruited who never received the intervention to those who did not attend the minimum number of treatment sessions.
Engagement in treatment programs might not always coincide with completion rate or sessions attendance (Mahncke et al., 2019): it is important to consider this and other indicators, such as satisfaction with treatment, as crucial complementary outcomes, when interpreting acceptability data derived from indirect proxies such as trial drop-out rates.
Although our findings are based on randomized clinical trials, the majority were conducted in real-world clinical settings and the results the same even after excluding trials in highly controlled or research-oriented sites, so generalization is probable.
The role of pharmacotherapy as a potential moderator of dropout has been marginally investigated in this study, only expressed as baseline therapy dose (chlorpromazine equivalents). In fact, concomitant antipsychotic treatment could be involved in either jeopardizing or enhancing the effects of CR (Biagianti, Castellaro, & Brambilla, 2021). Further exploring the role of pharmacotherapy in influencing the acceptability of CR, which did not emerge as significant in the current analyses, as well as the eventual role of medication adherence (Nuechterlein et al.,

3668
Antonio Vita et al. 2020), could represent a relevant target for future studies. In this light, it should be pointed out that only a few details on concomitant pharmacotherapy were reported in studies included in the present review, and this issue seems rather frequent in CR trials; besides, it could be better explored using a different methodology for quantitative synthesis of results, such as individual patient data meta-analyses. Analyzing the influence of pharmacological treatment on adherence to non-pharmacological interventions represents an interesting topic, also considering the growing literature interest for providing cognitive enhancement pharmacological strategies (Fleischhacker et al., 2021;Harvey & Sand, 2017;Michalopoulou et al., 2015;Otto et al., 2016).

Future studies
We urge two things of our scientist colleagues. To agree on the measure of drop-out as this will make it easier to compare different treatments and then make recommendations to national healthcare services. But just as vital is asking about acceptability directly, as the patient voice should be heard before making key decisions about treatment provision. A further recommendation is about the measurement of outcome in CR trials. We adopt a primary outcome measure and other assessments are then secondary. If a more nuanced approach was implemented then we could use the value placed by different groups (service providers, clinicians, and service users) and use a combined measure of the worth of treatment. These methods have rarely been used and are called Multi-Criterion Decision Modeling (MCDM) (Cinelli, Kadziński, Gonzalez, & Słowiński, 2020;Greco, Figueira, & Ehrgott, 2016), but would balance what are sometimes competing views of different parts of our health care system. Drop-out rates or satisfaction with treatment might be one element of an MCDM.

Conclusions
CR is an evidence-based intervention for people living with schizophrenia that is not only effective in producing cognitive and functional gains, but is also characterized by a good acceptability profile, similar to that of other evidence-based psychosocial interventions, and superior to that of pharmacological treatments. One core CR ingredient had an impact on acceptability, facilitation of transfer to everyday functioning. Other key elements also contribute to benefits, but this one is of great value in clinical practice as it values the subjective recovery preferences of people living with schizophrenia.