Primary Measures

Hamilton Program for Schizophrenia Voices Questionnaire (HPSVQ; Van Lieshout and Goldberg, Reference Van Lieshout and Goldberg 2007 ). The HPSVQ is a 9-item self-report measure of voice hearing. Items are scored on a 5-point rating scale, with higher scores indicating increased severity. The HPSVQ has excellent test–retest reliability, sensitivity to change, good internal consistency (Cronbach’s alpha=0.74–0.85), and convergent validity with the Psychotic Symptom Rating Scales-Auditory Hallucination Subscale (PSYRATS-AH) (Berry et al., Reference Berry, Newcombe, Strauss, Rammou, Schlier, Lincoln and Hayward2021; Kim et al., Reference Kim, Jung, Hwang, Chang, Kim, Ahn and Kim2010). It consists of two subscales – the Physical Subscale (HPSVQ-PS) examines the physical voice characteristics, and the Emotional Subscale (HPSVQ-ES) examines voice-related distress.

Choice of Outcome in Cbt for psychoses – Short form (CHOICE-SF; Webb et al., Reference Webb, Bartl, James, Skan, Peters, Jones, Garety, Kuipers, Hayward and Greenwood 2021 ). The CHOICE-SF is a shortened version of the service-user defined self-report measure of recovery developed by Greenwood et al. (Reference Greenwood, Sweeney, Williams, Garety, Kuipers, Scott and Peters2010). The measure was co-created with experts-by-experience to comprise a measure of patient-reported ‘psychological recovery’ and is aligned with treatment targets of CBTp. It consists of 11 items which are scored on an 11-point Likert scale (0=worst, 10=best), and one additional personal goal item. The measure has good test–retest reliability (Cronbach’s alpha=0.93), high sensitivity to change, and good construct validity (Webb et al., Reference Webb, Bartl, James, Skan, Peters, Jones, Garety, Kuipers, Hayward and Greenwood2021).

Secondary measures

General Anxiety Disorder Scale (GAD-7; Spitzer et al., Reference Spitzer, Kroenke, Williams and Löwe 2006 ). The GAD-7 is a 7-item self-report measure of anxiety symptoms and associated severity over the previous 2 weeks. The measure has been indicated to have good reliability, excellent internal consistency (Cronbach’s alpha=0.92), as well as good construct, criterion and factorial validity (Spitzer et al., Reference Spitzer, Kroenke, Williams and Löwe2006). It has been validated in both clinical and general populations (Löwe et al., Reference Löwe, Decker, Müller, Brähler, Schellberg, Herzog and Herzberg2008).

Patient Health Questionnaire (PHQ-9; Kroenke et al., Reference Kroenke, Spitzer and Williams 2001 ). The PHQ-9 is a 9-item self-report questionnaire measuring depression symptom severity over the previous 2 weeks. The measure displays good psychometric properties, excellent internal reliability (Cronbach’s alpha=0.89), test–retest reliability (intraclass correlation=0.84), good construct and criterion validity and sensitivity to change (Kroenke et al., Reference Kroenke, Spitzer and Williams2001). The measure has displayed validity in both clinical (Beard et al., Reference Beard, Hsu, Rifkin, Busch and Björgvinsson2016) and non-clinical samples (Martin et al., Reference Martin, Rief, Klaiberg and Braehler2006).

Guided self-help cognitive behaviour intervention for VoicEs (GiVE)

GiVE is a brief, targeted form of CBT for distressing voices. The intervention focuses on five modules implemented over eight 1-hour sessions, held weekly where possible. It was developed from the ‘Overcoming Distressing Voices’ CBT self-help book (Hayward et al., Reference Hayward, Strauss and Kingdon2018). Intervention delivery within session was supported through a published workbook (Hazell et al., Reference Hazell, Hayward, Strauss and Kingdon2018b), and a mobile app. The five modules comprising the intervention are listed below:

1. (1) Coping – an exploration of triggers and responses to voices (1 session);

2. (2) Me – identifying and evaluating negative beliefs about the self (2 sessions);

3. (3) My Voices – identifying and addressing unhelpful beliefs about voices (2 sessions);

4. (4) My Relationships – promoting assertiveness in difficult relationships with voices and others (2 sessions);

5. (5) Looking to the Future – supporting the continued application of the skills learned in previous modules (1 session).

Service users were invited to express a preference for the method of intervention delivery – face-to-face, videocall or telephone. Intervention completion was defined as attending at least six of the eight sessions. Service users were encouraged to engage with the self-help materials between sessions as the intervention proceeded.

Intervention delivery

Therapists who delivered the intervention had a variety of roles (e.g. support workers, graduate mental health workers, nurses and clinical psychologists) and had varying levels of therapy training/experience (39% had formal training in CBT; 58% had no formal therapy training; data were missing for 3% of therapists). The intervention was delivered to service users either face-to-face (56%), via telephone call (28%), videocall (12%), or through a mixture of modalities (4%).

Clinical and demographic characteristics

Descriptive demographic and clinical characteristic statistics were calculated, including age, gender, employment status, relationship status, ethnic group, level of education, length of voice hearing, and diagnosis, with count (n), and percentage of total provided for categorical variables, and mean and standard deviation (SD) provided for continuous variables. Diagnoses were clinician-reported and confirmed by a review of clinical records, where necessary.

Pre–post outcome measures

Mean pre- and post-intervention scores on primary and secondary measures were calculated, including standard deviations, minimum and maximum values, and 95% confidence intervals (CIs) for each. A comparison of pre- and post-intervention scores on primary and secondary measures was carried out using paired sample t-tests to determine whether statistically significant changes (i.e. p-values <0.05) occurred following the intervention (William, Reference William1908).

Potential predictors and classification of need

All clinical assessment measures and demographic characteristics were utilised as potential predictors due to evidence indicating potential impact on treatment outcomes and engagement (Ajnakina et al., Reference Ajnakina, Stubbs, Francis, Gaughran, David, Murray and Lally2021; Freeman and Garety, Reference Freeman and Garety2003; Hayward et al., Reference Hayward, Slater, Berry and Perona-Garcelán2016; O’Keeffe et al., Reference O’Keeffe, Conway and McGuire2017; Waters and Fernyhough, Reference Waters and Fernyhough2017). Consistent with the approach employed in Paulik et al. (Reference Paulik, Jones and Hayward2018), participants who scored highly for voice-related distress (13.14 or above on HPSVQ-ES), and also depression (19.33 or above on PHQ-9), or anxiety (16.01 or above on GAD-7) were classed as high need, and all other participants were categorised as low need. This distinction was defined a posteriori based on mean scores on each measure within the sample, where scores above the mean indicated high scores and those below indicated lower scores. For all analyses, to increase the predictive power of the test, categorical variables were re-categorised into binary variables (e.g. employment status: employed/not-employed).

Prediction of engagement

Service users were classified as non-commencers (offered GiVE but did not attend any sessions), non-completers (attended 1–5 sessions), and completers (attended 6–8 sessions). Consistent with the approach applied in Paulik et al. (Reference Paulik, Jones and Hayward2018) and Jones et al. (Reference Jones, Strauss and Hayward2021), models were built to compare completers, non-completers, and non-commencers via ANOVA for continuous variables, as application of Levene’s (Reference Levene, Olkin, Ghurye, Hoeffding, Madow and Mann1960) test revealed that the assumption of homogeneity of variances was not met for some measures. Chi-square tests were applied to compare levels of engagement across categorical variables.

Where results were statistically significant, post hoc analyses were applied to help interpret the findings, exploring the pattern of association between each level of engagement and the predictor variable. This was carried out to identify whether engagement could be predicted for various levels of the predictor variable, and thus inform clinical decision making. Multinomial regression models were applied here. Continuous variables were converted to categorical variables based on an initially agreed grouping. We also checked the total number of participants in each category was calculated to ensure categories were relatively balanced in size. This was implemented to help ensure results from measures where the sample was skewed would lead to more meaningful and clinically informative comparisons across levels, increased reliability of the model, and reduced bias. In this model, engagement served as the dependent variable, while the statistically significantly predictor was the independent variable. Odds ratios were calculated to illustrate results.

Prediction of outcome

Outcome was predicted using data from completers only, as follow-up data were not available for those who disengaged from the service. Separate linear regression models for each primary outcome measure (with post-intervention scores for voice-related distress or recovery as dependent variables) were built for each potential predictor. Independent variables were the corresponding primary outcome baseline score, the potential predictor (either a baseline clinical measure or demographic characteristic) and covariates. All regression coefficients are based on raw (unstandardised) scores to preserve interpretability. As such, regression coefficients represent the expected unit change for the outcome variable per one-unit increase in the predictor. For example, in models predicting HPSVQ-ES post-intervention (maximum score 16), a coefficient of 1 indicates that a 1-point increase in the predictor is associated with an expected 1-point increase on the HPSVQ-ES, reflecting an increase in voice-related distress.

Effect size, treatment response rate, and minimal clinically important change

Consistent with the approach employed in Jones et al. (Reference Jones, Strauss and Hayward2021), effect sizes were reported as Cohen’s d (Cohen, Reference Cohen1988), and treatment response rate (TRR) was calculated for each completer for each primary and secondary measure.

TRR was calculated through categorising service users as unchanged, deteriorated, minimally improved, or much improved, where minimal improvement was at least 20% change in a favourable direction over baseline, and much improvement corresponded to a 50% or greater change in a favourable direction from baseline scores (Bighelli et al., Reference Bighelli, Huhn, Schneider-Thoma, Krause, Reitmeir, Wallis, Schwermann, Pitschel-Walz, Barbiu, Furukawa and Leucht2018). Those who improved or deteriorated less than 20% from their original score were counted as unchanged. Favourable direction involved decreases on all measures with the exception of the CHOICE, where increased scores indicate favourable change.

To serve as a further indicator of clinical change, minimal clinically important change (MCID) was calculated for the two primary outcome measures (HPSVQ-ES and CHOICE-SF). MCID was defined a priori, for voice-related distress as 2 points below baseline assessment scores on the HPSVQ-ES (Hazell et al., Reference Hazell, Hayward, Cavanagh, Jones and Strauss2018a), and for recovery as 1.45 increase from baseline on the CHOICE-SF (Jolley et al., Reference Jolley, Garety, Peters, Fornells-Ambrojo, Onwumere, Harris, Brabban and Johns2015).

Missing data

Missing data were quantified, analysed and those assumed to be missing at random were treated using multiple imputation for chained equations (MICE; Van Buuren and Groothuis-Oudshoorn, Reference Van Buuren and Groothuis-Oudshoorn2011; Van Buuren, Reference Van Buuren2018). If missing data amounted to 5–40% of the total dataset, MICE was deemed appropriate (Newman, Reference Newman2014; Schafer, Reference Schafer1999). For the imputation stage, consistent with recommendations (Collins et al., Reference Collins, Schafer and Kam2001; Jakobsen et al., Reference Jakobsen, Gluud, Wetterslev and Winkel2017; White et al., Reference White, Royston and Wood2011), all variables which reasonably predicted missingness (auxiliary variables) or the outcome, and all covariates used in the analysis model which had a minimum correlation of r=0.1 with missingness were used for imputation (Van Buuren and Goothuis-Oudshoorn, Reference Van Buuren and Groothuis-Oudshoorn2011). Auxiliary variables were identified using logistic regression models with missingness as the outcome variable and potential continuous auxiliary variables set as predictors. Potential categorical auxiliary variables were identified using a chi-square test. The number of imputed datasets m, was selected through the total percentage of missingness in the dataset. Once the imputed datasets were obtained, these were used to fit the same models as with the complete case dataset. Model imputations were then pooled into a single entity, i.e. m parameter estimates were pooled into one estimate for each variable and the variance estimated (Rubin, Reference Rubin1987; Van Buuren, Reference Van Buuren2018).