Reliable and valid criteria are now available to identify help-seeking individuals who are at an increased risk of imminently developing a first episode of schizophrenia or related psychoses (FEP). Yung and colleagues Reference Yung, McGorry, McFarlane, Jackson, Patton and Rakkar1 developed operational criteria to identify three subgroups experiencing an ‘at-risk mental state’ (ARMS) for psychosis. The following two subgroups specify state risk factors and are defined by the presence of transient psychotic symptoms: (a) Brief Limited Intermittent Psychotic Symptoms or (b) attenuated (subclinical) psychotic symptoms. The other subgroup comprises trait-plus-state risk factors, operationally defined by the presence of diminished functioning plus either a first-degree relative with a history of psychosis or a pre-existing schizotypal personality disorder. These criteria have now been widely adopted and have led to attempts to deliver interventions that will reduce the risk of development of a first episode of schizophrenia or related psychoses; effective interventions are attractive because of the significant personal, social and financial costs associated with the development of psychosis. In addition, it is clear that people meeting the ARMS criteria have significant mental health problems and impaired functioning, regardless of eventual transition. Several randomised controlled trials have evaluated the effect of psychosocial and pharmacological interventions Reference Addington, Epstein, Liu, French, Boydell and Zipursky2–Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9 on transition to psychosis and the severity of psychotic experiences within this population. These studies have found relatively encouraging results, suggesting that such interventions can reduce attenuated psychotic symptoms and may delay or prevent psychosis in some instances, and recent meta-analyses have concluded that cognitive therapy significantly reduces the risk of transition to psychosis at 12 months. A moderate effect was also reported at 18 months but was not sustained in a sensitivity analysis. Reference Stafford, Jackson, Mayo-Wilson, Morrison and Kendall10
Recent research suggests that the way in which cognitive therapy is delivered is an important factor in determining outcomes. For example, in a recent trial of cognitive therapy for people with psychosis, participants derived benefit if they received full therapy that involved active change strategies, such as evaluation of beliefs and the use of behavioural experiments, whereas cognitive therapy was potentially harmful if participants received partial therapy that only involved engagement and assessment. Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith11 Within the Early Detection and Intervention Evaluation (EDIE-2) trial for people at risk of psychosis, we found that those allocated to cognitive therapy received a mean of 9.11 sessions (s.d. = 6.69; range 0–26), and acceptability of cognitive therapy was reasonably good, with only 9/144 (6.25%) not attending any sessions, and 108/144 (75%) receiving at least four or more sessions; however, there was clearly considerable variance in the number of sessions received. Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9 To examine the contribution of the ‘dose’ of cognitive therapy, the effects of the number of sessions on 12-month outcomes were estimated through an instrumental variable regression using the Adjusted Treatment Received (ATR) algorithm Reference Hausman12,Reference Nagelkerke, Fidler, Bersen and Borgdorff13 (this method is described in detail in the context of dose–response effects in psychotherapy trials Reference Maracy and Dunn14 ); we found a significant effect of sessions on severity of psychotic experiences. However, quality of therapy received is likely to be as important, if not more so, than quantity alone. Cognitive therapy for people at risk of psychosis is a collaborative, problem-orientated approach, which requires the development of a problem list and shared goal. Reference French and Morrison15 Our treatment manual Reference French and Morrison15 also required that an idiosyncratic case formulation based on the cognitive model Reference Morrison16 be developed, and that the use of active change strategies such as provision of normalising information, evidential analysis and testing beliefs by modification of safety behaviours was present in as many sessions as possible. Similarly, our approach required extensive use of between-session tasks to facilitate change (i.e. homework tasks).
A recent Delphi study of expert opinion regarding the essential components of cognitive therapy for psychosis produced consensus regarding the importance of such factors. Reference Morrison and Barratt17 However, to date, the assumption that such components of cognitive therapy are associated with outcome is an untested hypothesis. Our study, which is a secondary analysis of the EDIE-2 trial, Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9 aims to test the specific hypotheses in which the development of a shared problem list, use of a case formulation based on the cognitive model, use of homework tasks between sessions and use of active intervention strategies outlined in the manual will act as process variables, increasing the effect of cognitive therapy, if received.
Method
Trial design
EDIE-2 is a multi-centre randomised controlled trial of psychotherapy for the prevention of psychosis in those at high risk. The trial compared cognitive behavioural therapy (CBT) with mental state monitoring versus mental state monitoring alone (TAU). A total of 288 help-seeking participants aged between 14 and 35 years who did not have a diagnosis of psychosis but satisfied the Comprehensive Assessment for At-Risk Mental State Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell'Olio18 (CAARMS) criteria for ARMS were randomised to CBT or TAU within each of the five sites: Manchester, Birmingham/Worcester, Glasgow, Cambridge and Norfolk. Exclusion criteria were previous or current antipsychotic medication for more than 2 days, moderate-to-severe intellectual disabilities and insufficient English.
Demographic and clinical characteristics of the participants were collected before randomisation. Clinical outcomes were collected at monthly intervals for the first 6 months and then every 3 months to a maximum of 24 months or until the end of the trial. The primary outcome is recorded 12 months after randomisation.
Assessment of putative treatment mechanisms
Therapist notes of cognitive therapy sessions carried out as part of the EDIE-2 trial were evaluated for evidence of particular aspects of therapy that should be present. They can broadly be grouped into four components in line with a recent Delphi study: Reference Morrison and Barratt17 having a shared problem list, development of a case formulation based on the cognitive model, use of homework tasks between sessions and use of active change strategies specified in the manual within sessions. The classifications are dependent upon detailed and accurate notes and so provide a conservative estimate of the occurrence of the practices, i.e. if there was no clear evidence in the therapy process notes that a particular aspect of therapy was conducted then it was not recorded as present.
Assessment of problems and goals
Problem assessment is considered present if in any session there was a formal list of the participant's problems and goals. This is a binary measure as it is not necessarily expected to occur more than once during the course of therapy.
Formulation
Formulation is a detailed account of the patient's difficulties, based on the specific cognitive model of psychosis that was utilised within the trial. It is recorded as present if there was clear evidence within the notes (i.e. a copy of the formulation was included). As with problem assessment, it is a binary measure since the requirement to demonstrate fidelity within therapy was to have at least one formulation evident in the notes. However, it is likely that many participants will have had multiple formulations.
Homework
Homework was recorded as present at each session if a review of the previous homework task was recorded in the notes. Additional detail on the type of homework given was also recorded under the groupings of behavioural experiment (changing reactive behaviours), monitoring and education. The proportion of sessions in which homework was involved was calculated as a continuous measure and also dichotomised at 50% of sessions to be analysed as a binary measure.
Active change strategies
The involvement of active change strategies was recorded for each session if any of the following intervention strategies were evident in the notes: provision of normalising information, generation of alternative explanations for problematic appraisals, manipulation of safety behaviours, evaluation of metacognitive beliefs or responses, evaluation of beliefs about self and others, efforts to reduce social isolation and attempts to promote relapse prevention. As with homework, the proportion of sessions involving any one of these other interventions as well as a dichotomised version (using 50%) of this variable was created.
Additionally, we created a binary variable indicating participants who received all of these components of therapy versus some or none of them, and a three-category variable indicating all, some and none. For all of these process variables, the processes were coded as being absent (i.e. equal to 0) for the participants in the control (TAU) group.
Outcome
We use the CAARMS to measure symptom severity; this a semi-structured interview to discuss recent experiences followed by a clinician-rated scale of symptoms, frequency and distress under the following subheadings: disorders of thought content, perceptual abnormalities, conceptual disorganisation, motor changes, concentration and attention, emotion and affect, subjectively impaired energy and impaired tolerance to normal stress. This measure accounts separately for the frequency and severity of symptoms and combines to form a product of the two so that it can be sensitive to small changes in experiences. Yung et al Reference Yung, Yuen, McGorry, Phillips, Kelly and Dell'Olio18 showed the measure to have good inter-rater reliability with an ICC of at least 0.62 on each subscale and an overall ICC of 0.85. In this study we use a shortened version consisting of only the unusual thought content, non-bizarre ideas, perceptual abnormalities and disorganised speech subscales. Inter-rater reliability of the CAARMS was assessed on eight occasions over the course of the EDIE-2 trial and reported in the baseline paper; the mean intra-class correlation was 0.90 (s.d. = 0.03). Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9
Covariates
Socio-demographic (age, sex, education, occupation) and health measures (anxiety: Social Interactions and Anxiety Scale (SIAS), Reference Mattick and Clarke19 depression: Beck's Depression Inventory (BDI), Reference Beck, Guth, Steer and Ball20 functioning: Global Assessment of Functioning (GAF) 21 and quality of life: EQ5D and Manchester Short Assessment of Quality of Life (MANSA) Reference Priebe, Huxley, Knight and Evans22 ) as well as the CAARMS severity and distress from symptoms were recorded before randomisation.
Statistical analysis
A descriptive account of the associations between baseline characteristics and content of therapy received in the intervention arm only are described using chi square and t-tests.
To determine the effect of receiving a specific aspect of therapy, it is assumed that there is an effect of being randomised to therapy even if this specific part of therapy is not given (through other mechanisms) and an additional effect of receiving the specific aspect of therapy in question. This can be considered similar to a mediation analysis. Since the content of therapy is not randomised, there will be confounding present: there will be factors that influence whether a patient has a problem assessment/formulation/homework/active change strategy that will also influence his/her treatment outcome, for example, age or education. To remove confounding from factors that have not or cannot be measured, we apply an instrumental variable analysis Reference Angrist and Krueger23 using a two-stage least squares estimation. Reference Dunn, Maracy and Tomenson24–Reference Dunn and Bentall26
The causal effect of each aspect of therapy on outcome at 12-month follow-up is investigated individually. Baseline covariates by randomisation group interactions are considered as instruments. To select the most effective instruments, the backward stepwise model selection method is used since no instrument has been set before the analysis. The selection is carried out to determine baseline variables associated with the process within the CBT arm only, and these are used as instruments when interacted with the randomisation group. The analyses allow for a direct effect of randomisation to CBT even if the particular aspect of therapy in question was not received. All analyses are adjusted for the baseline measure of symptom severity and bootstrapped with 1000 replications.
The data were initially analysed using only complete cases. Missing covariates are judged likely to be missing at random, meaning that the probability of missing data can be modelled by the observed data. To improve efficiency, missing covariate values are imputed using chained equations with the ‘ice’ command in Stata. Reference Royston27 This is carried out on the full dataset, including all variables used in this analysis as well as follow-up data from all time points and variables thought to be related to non-response. Five imputations were created using all variables. Variables were imputed using the appropriate model: linear, logistic or multinomial logistic. The matching method was used for variables with skewed distributions to maintain the shape and range of the distributions. The matching method predicts the missing value and then allocates the closest non-missing prediction as the imputed value. Missing values of the outcome variables are imputed together with the covariates; however, analyses only use imputed information for baseline covariates and do not include respondents without an outcome reported. Analyses are run on the imputed datasets separately and pooled using Rubin's rules in MS excel to give estimates and their variances.
Results
The EDIE-2 trial recruited 288 participants at high risk of psychosis who were randomised to mental state monitoring only (TAU = 144) or CBT plus mental state monitoring (CBT = 144). The average age of participants was 21 years (s.d. = 4.2), they were predominantly White (n = 252, 90%) and male (n = 180, 63%); full characteristics by treatment allocation are described in detail in the preliminary EDIE-2 trial paper. Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9
Heterogeneity was present in the treatment actually received by participants in the CBT arm. Our conservative analysis of the therapy notes indicated that the following numbers of participants clearly received these elements of therapy: 100 (69%) had an assessment of problems and goals, and 79 (55%) had received formulation at some point in their therapy. On average, 39% (s.d. = 32) of the sessions involved homework and 43% (s.d. = 34) involved other interventions. The notes clearly indicated that 36 (25%) participants received all four components of therapy, while the notes did not record any element for 28 (19%) participants. Individual associations between the characteristics of patients within the intervention arm and the treatment received are detailed in Tables 1 and 2. The tables show observed values with significant associations indicated for observed and imputed data. Greater anxiety (SIAS) was associated with receiving each aspect of therapy. Treatment centre was associated with receiving a problem assessment, formulation and other interventions but not homework. Lower functioning (GAF) was associated with having a problem assessment and other interventions. Receiving a problem assessment was also more likely in female participants. The components of intervention are correlated with each other and with the number of therapy sessions attended; the associations are detailed in Table 3.
Demographic associations with content of therapy received – CBT arm only
| Assessment of problems and goals |
Formulation | Homework | Active change strategies | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| No | Yes | P-value | No | Yes | P-value | No | Yes | P-value | No | Yes | P-value | |
| Gender | ||||||||||||
| Male: n (%) | 33 (37%) | 56 (63%) | 41 (46%) | 48 (54%) | 48 (54%) | 41 (46%) | 45 (51%) | 44 (49%) | ||||
| Female: n (%) | 11 (20%) | 44 (80%) | 0.03 | 24 (44%) | 31 (56%) | 0.78 | 33 (60%) | 22 (40%) | 0.48 | 27 (49%) | 28 (51%) | 0.86 |
| Age mean (s.d.) | 20 (3.9) | 21 (4.3) | 0.28 | 20 (4.1) | 21 (4.3) | 0.46 | 20 (3.7) | 21 (4.7) | 0.08 | 20 (3.9) | 21 (4.4) | 0.06 |
| Ethnicity | ||||||||||||
| Black: n (%) | 3 (21%) | 11 (79%) | 4 (29%) | 10 (71%) | 8 (57%) | 6 (43%) | 7 (50%) | 7 (50%) | ||||
| White: n (%) | 40 (31%) | 88 (69%) | 0.45 | 61 (48%) | 67 (52%) | 0.17 | 72 (56%) | 56 (44%) | 0.95 | 65 (51%) | 63 (49%) | 0.96 |
| Site | ||||||||||||
| Manchester: n (%) | 4 (10%) | 36 (90%) | 12 (30%) | 28 (70%) | 17 (43%) | 23 (58%) | 16 (40%) | 24 (60%) | ||||
| Birmingham: n (%) | 14 (36%) | 25 (64%) | 14 (36%) | 25 (64%) | 26 (67%) | 13 (33%) | 21 (54%) | 18 (46%) | ||||
| Cambridge: n (%) | 3 (21%) | 11 (79%) | 5 (36%) | 9 (64%) | 6 (43%) | 8 (57%) | 3 (21%) | 11 (79%) | ||||
| Norfolk: n (%) | 16 (76%) | 5 (24%) | 18 (86%) | 3 (14%) | 13 (62%) | 8 (38%) | 16 (76%) | 5 (24%) | ||||
| Glasgow: n (%) | 7 (23%) | 23 (77%) | <0.01 | 16 (53%) | 14 (47%) | <0.01 | 19 (63%) | 11 (37%) | 0.15 | 16 (53%) | 14 (47%) | 0.02 |
| Years of education mean (s.d.) |
12.8 (2.1) | 13.3 (2.3) | 0.27 | 12.9 (2.3) | 13.4 (2.2) | 0.16 | 12.9 (2.0) | 13.5 (2.5) | 0.08 | 12.9 (2.1) | 13.4 (2.3) | 0.12 |
| Continuing education | ||||||||||||
| No: n (%) | 16 (43%) | 21 (57%) | 21 (57%) | 16 (43%) | 22 (59%) | 15 (41%) | 22 (59%) | 15 (41%) | ||||
| Yes: n (%) | 22 (26%) | 62 (74%) | 0.06 | 35 (42%) | 49 (58%) | 0.13 | 42 (50%) | 42 (50%) | 0.34 | 35 (42%) | 49 (58%) | 0.07 |
| Degree | ||||||||||||
| No: n (%) | 33 (32%) | 71 (68%) | 49 (47%) | 55 (53%) | 55 (54%) | 49 (46%) | 49 (47%) | 55 (53%) | ||||
| Yes: n (%) | 5 (26%) | 14 (74%) | 0.64 | 8 (42%) | 11 (58%) | 0.69 | 11 (54%) | 8 (46%) | 0.69 | 9 (47%) | 10 (53%) | 0.98 |
| Occupation | ||||||||||||
| Seeking work/ other: n (%) |
12 (26%) | 34 (74%) | 21 (46%) | 25 (54%) | 25 (53%) | 21 (47%) | 21 (46%) | 25 (54%) | ||||
| Employed/student/ home: n (%) |
27 (34%) | 52 (66%) | 0.35 | 37 (47%) | 42 (53%) | 0.90 | 43 (58%) | 36 (42%) | 0.99 | 39 (49%) | 40 (51%) | 0.69 |
Associations between baseline health measures and each content of therapy received – CBT arm only.
| Assessment of problems and goals | Formulation | Homework | Active change strategies | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| P-value | P-value | P-value | P-value | ||||||||||||||
| No | Yes | Complete Case | Imputed | No | Yes | Complete Case | Imputed | No | Yes | Complete Case | Imputed | No | Yes | Complete Case | Imputed | ||
| SIAS: mean (s.d.) | 37.2 (15.3) | 45.6 (17.1) | <0.01 | 0.04 | 38.7 (16.4) | 46.7 (16.6) | <0.01 | 0.04 | 38.5 (17.0) | 48.1 (15.4) | <0.01 | <0.01 | 36.3 (16.1) | 49.4 (15.2) | <0.01 | <0.01 | |
| BDI: mean (s.d.) | 9.6 (3.7) | 10.8 (4.3) | 0.12 | 0.26 | 10.3 (3.8) | 10.5 (4.4) | 0.82 | 0.77 | 10.5 (4.3) | 10.4 (4.0) | 0.89 | 0.75 | 10.4 (4.3) | 10.4 (4.0) | 0.93 | 0.89 | |
| Severity: mean (s.d.) | 37.0 (16.2) | 39.5 (17.1) | 0.43 | 0.38 | 36.4 (16.8) | 40.7 (16.8) | 0.13 | 0.11 | 39.1 (17.4) | 38.8 (16.2) | 0.79 | 0.70 | 37.1 (18.3) | 40.4 (15.1) | 0.24 | 0.12 | |
| Distress: mean (s.d.) | 42.2 (20.9) | 43.0 (20.5) | 0.82 | 0.63 | 40.8 (20.2) | 44.3 (20.8) | 0.34 | 0.29 | 41.0 (20.5) | 44.9 (20.5) | 0.28 | 0.49 | 41.2 (22.3) | 44.2 (18.8) | 0.41 | 0.19 | |
| GAF: mean (s.d.) | 54.2 (10.9) | 49.6 (10.8) | 0.02 | 0.02 | 52.5 (11.5) | 49.7 (10.4) | 0.13 | 0.13 | 52.4 (11.0) | 49.2 (10.7) | 0.98 | 0.08 | 52.8 (12.2) | 49.1 (9.3) | 0.04 | 0.05 | |
| EQ5: mean (s.d.) | 0.65 (0.25) | 0.56 (0.29) | 0.08 | 0.07 | 0.62 (0.28) | 0.56 (0.29) | 0.28 | 0.23 | 0.58 (0.29) | 0.60 (0.28) | 0.71 | 0.45 | 0.61 (0.29) | 0.56 (0.28) | 0.35 | 0.52 | |
| MANSA: mean (s.d.) | 4.01 (0.75) | 3.77 (0.79) | 0.11 | 0.12 | 3.86 (0.80) | 3.83 (0.77) | 0.88 | 0.88 | 3.94 (0.82) | 3.73 (0.72) | 0.14 | 0.14 | 3.90 (0.79) | 3.79 (0.78) | 0.46 | 0.46 | |
SIAS, social interactions and anxiety; BDI, Beck's depression inventory; GAF, global assessment of functioning; EQ5D, European quality of life; MANSA, Manchester Short Assessment of Quality of Life.
Associations between components of therapy
| Sessions attended Mean (s.d.) |
Problem agreement | Formulation | > 50% homework | ||||
|---|---|---|---|---|---|---|---|
| No | Yes | No | Yes | No | Yes | ||
| Problem agreement | |||||||
| No | 4.5 (5.2) | ||||||
| Yes | 10.9 (6.3) | ||||||
| Formulation | |||||||
| No | 4.8 (4.9) | 37 (84%) | 28 (28%) | ||||
| Yes | 12.4 (6.0) | 7 (16%) | 72 (72%) | ||||
| > 50% sessions with homework | |||||||
| No | 7.1 (6.4) | 34 (77%) | 47 (47%) | 47 (72%) | 34 (43%) | ||
| Yes | 11.3 (6.7) | 10 (23%) | 53 (53%) | 18 (28%) | 45 (57%) | ||
| >50% sessions with active change strategy | |||||||
| No | 5.6 (5.0) | 35 (80%) | 37 (37%) | 51 (78%) | 21 (27%) | 58 (72%) | 14 (22%) |
| Yes | 12.3 (6.5) | 9 (20%) | 63 (63%) | 24 (22%) | 58 (73%) | 23 (28%) | 49 (78%) |
All associations are significant at the 5% level.
The primary analysis of the EDIE data indicated a significant improvement in severity of symptoms for those randomised to receive CBT (coefficient = −5.12, 95% CI −8.60 to −1.64, P = 0.004). Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley9 When each aspect of therapy is considered separately as a mechanism of CBT, there is no longer a significant direct effect of randomisation on the severity of symptoms (Table 4). The results are very similar between the complete case analysis and the imputed datasets. The instrumental variables analysis indicates that there is no significant additional treatment effect through problem assessment (imputed coefficient = −10.9, 95% CI −29 to 7.4, P = 0.243). There is a significant additional decrease in the symptom score estimated for case formulation (estimated 23 point decrease, 95% CI −44 to −1.7, P = 0.036) and the proportion of sessions involving homework (estimated 0.26 point decrease, 95% CI −0.51 to −0.001, P = 0.049). The results suggest that including active change strategies in therapy improves outcomes although the estimate is of borderline significance when assessed at the 5% level (estimated 0.23 point decrease, 95% CI −0.47 to 0.005, P = 0.056).
Instrumental variables analysis of each potential mechanism of CBT on severity of symptoms at 12 months follow-up
| Complete case | Imputed | ||||||
|---|---|---|---|---|---|---|---|
| Effect | Coefficient | s.e. | Normal | 95% CI | Coefficient | SE estimate | P |
| Agreement of problems and goals | |||||||
| Randomisation | −0.24 | 7.64 | −12.48 | 17.47 | −0.38 | 6.73 | 0.955 |
| Agreement of problems and goals | −6.15 | 9.83 | −28.18 | 10.33 | −10.94 | 9.37 | 0.243 |
| Case formulation | |||||||
| Randomisation | −3.97 | 6.28 | −14.95 | 9.68 | 4.49 | 6.49 | 0.490 |
| Formulation | −1.73 | 9.16 | −21.20 | 14.70 | −22.99 | 10.85 | 0.036 |
| % of sessions involving homework | |||||||
| Randomisation | 11.064 | 10.109 | −5.908 | 20.417 | 1.952 | 5.701 | 0.732 |
| % sessions with homework* | −0.361 | 0.209 | −0.542 | −0.032 | −0.258 | 0.131 | 0.049 |
| > 50% of sessions involving homework | |||||||
| Randomisation | −1.21 | 5.04 | −10.44 | 9.32 | −0.58 | 5.44 | 0.915 |
| >50% sessions with homework | −8.46 | 7.75 | −25.27 | 5.13 | −17.44 | 10.27 | 0.093 |
| % of sessions involving active change strategies | |||||||
| Randomisation | 1.779 | 7.475 | −7.493 | 21.807 | 2.174 | 5.815 | 0.709 |
| % sessions with active change strategies* | −0.135 | 0.139 | −0.504 | 0.043 | −0.233 | 0.121 | 0.056 |
| >50% of sessions involving active change strategies | |||||||
| Randomisation | 0.48 | 5.98 | −6.01 | 17.43 | 1.73 | 5.64 | 0.760 |
| >50% sessions with active change strategies | −9.44 | 8.93 | −34.62 | 0.36 | −18.52 | 10.21 | 0.076 |
| 2-category fidelity measure | |||||||
| Randomisation | 10.69 | 11.95 | −3.58 | 43.27 | −3.18 | 3.58 | 0.374 |
| All components | −18.21 | 13.85 | −55.33 | −1.02 | −20.13 | 10.38 | 0.053 |
| 3-categority fidelity measure | |||||||
| Some components | −3.75 | 4.00 | −10.81 | 4.88 | −4.31 | 5.29 | 0.416 |
| All components | −6.60 | 8.40 | −26.47 | 6.47 | −22.81 | 8.55 | 0.008 |
* Change in treatment effect for each 1% increase in the process measure.
Receiving all aspects of therapy is expected to reduce symptom severity by 20 points (95% CI −40 to 0.22, P = 0.053) compared to those who receive only some or none of the components; this is again of borderline significance. When it is assumed that participants who do not receive any of the four components of therapy have no treatment effect, there is no significant improvement in symptoms from receiving some aspects of therapy but a significant improvement is seen when all four components are applied.
Discussion
Summary of findings
This analysis of components of therapy as possible mechanisms of change within cognitive therapy for people at risk of developing first episode of schizophrenia or related psychoses accounts for potential unmeasured confounding between the mechanism and outcome, which is an important advance on previous research. We found that there was a greater effect of treatment if formulation and homework are involved in therapy and a suggestion that active change strategies also improve symptoms, which confirms the consensus of therapist opinion and the approach recommended in our treatment manual. When examining the effects on outcomes, there was no direct effect of randomisation, suggesting that allocation to therapy per se is not sufficient to produce change, rather it is the quality of the therapy provided that results in change in clinical outcomes.
The importance of a case formulation in cognitive therapy for psychosis has been discussed, but previous studies have failed to show a relationship between formulation and treatment outcomes in people with psychosis; Reference Chadwick, Williams and Mackenzie28 this is the first study to empirically demonstrate an effect of formulation on outcome in cognitive therapy for people at risk of psychosis. Similarly, attempts to demonstrate the effects of homework tasks on outcome have previously been unsuccessful; Reference Dunn, Morrison and Bentall29 it is likely that this is due to the limited statistical power resulting from small sample sizes in the studies to date. The effects of active change strategies have been shown to be related to clinical response to cognitive therapy in people with psychosis; Reference Dunn, Fowler, Rollinson, Freeman, Kuipers and Smith11 we have replicated this finding in people at risk of psychosis. The lack of an effect of having a problem list is interesting; it may be that this is a necessary, but not sufficient, condition for achievement of clinical improvement. Given that cognitive therapy is a collaborative and problem-orientated approach, it is likely to be very difficult to develop a formulation, collaboratively set homework tasks and utilise active change strategies in the absence of a shared list of problems and goals. The high correlation between the components of therapy and with the number of therapy sessions attended means that we cannot attribute the treatment effect seen for one component apart from another. This is reflected in the similar magnitudes of estimated treatment effects and it is likely that they are all indicating the general effect of fidelity to treatment. The strong effect of having any versus none of these is consistent with such a suggestion.
Methodological limitations and recommendations for future research
The presence of treatment components was determined by inspection of the therapy process notes, and assumed to be not present if not explicitly evident; this may have lead to components of therapy being under-reported (which at least provides a conservative test of the hypotheses), and may be confounded by therapists administrative abilities. Also, we did not specify at the start of the trial that the process notes would be utilised for this purpose; this may have resulted in greater variability in the recording of such information than if it had been explicitly stated from the outset but would additionally not be influenced by the knowledge that the data are being monitored. Females were found to be more likely to receive an assessment of problems and goals. Given that no gender difference is found for the other components, it may be a chance finding due to multiple testing when looking at individual associations, but it is also possible that the ability to engage in a therapeutic relationship and develop collaborative goals may be greater in females, which could help to explain the widely accepted finding that women with psychosis have better outcomes than men. Reference Grossman, Harrow, Rosen, Faull and Strauss30 Trial site, however, was individually associated with each component of therapy apart from homework. Although regular homework use is not statistically associated with trial site, the pattern of use among the sites is similar to the other components. Differences may be due to variations in application of the therapy, characteristics of patients or completion of notes but highlights the importance of consistency in application of the therapeutic model.
To test the hypotheses, we would ideally have randomised participants to receive specific aspects of therapy, but this approach is not feasible within a definitive trial of a complex intervention such as cognitive therapy; however, the instrumental variable approach tries to account for the problems of confounding associated with not being randomised. While this approach to mediation analysis represents a significant advance over traditional methods that do not account for unmeasured confounding, there are several issues that need to be considered. Two assumptions are made about the instruments, the first is that they are associated with the component of therapy in question and the second is that they only influence severity of symptoms at follow-up through their effects on receipt of the therapy component. The instruments used are interactions of baseline covariates with randomisation arm. The first assumption is therefore valid since the covariates selected for the interactions are specifically chosen as the variables most associated with receipt of the component of therapy in the CBT arm of the trial. The second assumption cannot be tested; we assume that the benefits of randomisation hold and there is no reason to expect a difference in the association between covariates and outcome in the two treatment arms other than through the baseline characteristics affecting the components of therapy received. However, since the variables associated with one component may be associated with others, the instruments could affect the outcome through these other pathways invalidating the assumption. If it is the case that the components are in fact measuring a more general fidelity to treatment, this may not be a concern.
The confidence intervals for the estimates are very wide, indicating a degree of uncertainty in the magnitude of the effects. Instruments must be selected that explain a large proportion of the heterogeneity in the treatment received. To ensure that the most effective instruments are selected, we have used the data reduction technique, backward stepwise selection. There may be some impact on the results over the choice of instrument; however, a sensitivity analysis, using all covariate by randomisation arm interactions and selection using a shrinkage approach, the Least Absolute Shrinkage and Selection Operator produces similar results giving confidence in our analysis.
Implications
There are several clinical implications of this study. It is important to emphasise the development of a case formulation, the use of homework tasks between sessions and the incorporation of active change strategies within sessions in both training of therapists and delivery of therapy. Early development of a shared formulation would be desirable, for example, an analysis of a recent incident in terms of events, appraisal of events, associated physical and emotional feelings and the cognitive and behavioural responses. It is likely that a more comprehensive formulation that incorporates information regarding life experience and beliefs will occur later in the therapy process. The use of homework tasks is clearly important in determining outcome; thus, ensuring that previous tasks are reviewed and new tasks are set will be important. It is likely that collaborative development of such tasks, allocation of sufficient time to developing such tasks and planning for obstacles to implementation are likely to promote successful completion; however, such hypotheses could be tested in future research. The use of active change strategies is likely to be facilitated by use of therapist reminders or prompts and routine monitoring of the implementation of such intervention strategies. Sharing the findings of this research with patients is also likely to be beneficial; thus, ensuring that patients are aware that a shared formulation, conducting tasks between sessions and an active stance within sessions are all associated with good clinical outcomes is likely to facilitate the implementation of such components.
In conclusion, we have found that there is a greater treatment effect if formulation and homework are involved in therapy; however, given the high correlations between such components, it is possible that these may be indicators of overall treatment fidelity and it may be that the other aspects of adherence to the therapy protocol may also be important.
Funding
This work was supported by funding from the Medical Research Council (G0500264) and the Department of Health. C.F. is supported by an MRC-ESRC Interdisciplinary Research Studentship. MB is part-funded by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care West Midlands.
Acknowledgements
We thank the Mental Health Research Network, the Scottish Mental Health Research Network and the OpenCDMS team for their support and assistance. We also thank the other EDIE-2 investigators and our trial therapists for their involvement in the trial and the EDIE-2 research assistants for their data collection work. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
eLetters
No eLetters have been published for this article.