Voice-hearing is experienced by around 75% of people diagnosed with schizophrenia.Reference Waters and Fernyhough1 The impact of antipsychotic medication is variable, and many people continue to experience voice-hearing.Reference Ascher-Svanum, Nyhuis, Stauffer, Kinon, Faries and Phillips2,Reference Sinkeviciute, Hugdahl, Bartz-Johannessen, Kroken, Løberg and Kjelby3 It is increasingly recognised that a non-medical, recovery-oriented approach is needed to support voice-hearers. This is in line with increasing expectations and requirements from governments and mental health policies for mental health services to adopt a recovery-oriented approach.4,Reference Slade, Amering and Oades5 In this context, ‘recovery’ does not refer to erasing symptoms but to living a meaningful, socially connected and personally valued life.Reference Anthony6 Adopting a recovery-oriented approach to voice-hearing means supporting voice-hearers to flourish and do the things that they want to do, even in the presence of ongoing voices.
The hearing voices approachReference Romme and Escher7 challenges common biologically oriented conceptualisations of voices as meaningless symptoms of illness to be suppressed or ignored. Rather, voice-hearing is seen as a meaningful part of human experience and a response to people’s histories and circumstances, including trauma.Reference Corstens, Longden, McCarthy-Jones, Waddingham and Thomas8 Furthermore, the impact of voices on a person’s distress in daily life is largely determined by their interpretation of those voices.Reference Mawson, Cohen and Berry9 By exploring, understanding and changing their relationships with their voices, voice-hearers can learn to accept, cope with and successfully manage their voices.Reference Corstens, Escher, Romme, Longden, Moskowitz, Dorahy and Schafer10 A systematic review and synthesis of qualitative research about recovery processes in voice-hearing found that making sense of voice-hearing experiences, accepting voices and changing one’s beliefs about and relationships with voices facilitated recovery.Reference Hall, Terry and Hayward11 The reviewers identified that searching for meaning in voices tended to result in less continued distress than blocking out or fighting against voices, which was often associated with seeing them as meaningless symptoms.
The hearing voices approach is being used increasingly across the world,12 predominantly in voice-hearing groups, which have been found to have benefits including improved coping skills, developing a better understanding of and a different relationship with voices, reducing stress and improving self-esteem.Reference Corentin, Fitzgerald and Goodwin13 Participants also reported that these groups provided a kind of support unavailable to them elsewhere.Reference Corentin, Fitzgerald and Goodwin13
Nevertheless, not all voice-hearers have access to, or want to attend, hearing voices groups. Individual interventions that use the hearing voices approach have been reported as useful, but participants highlighted that these specialised services are difficult to access for many people and poorly understood within mainstream services.Reference Middleton, Cooke and May14 Many voice-hearers have identified a desire to openly discuss the content and meaning of their voices.Reference Coffey and Hewitt15,Reference Holt and Tickle16 Given that hearing voices groups and specialised services are unlikely to be available to many voice-hearers, it is important for mental health workers to provide them with the opportunity to hear about and access this evidence-based approach. Nonetheless, mental health workers are often unaware of hearing voices approaches or lack confidence to engage with voice-hearing.Reference Coffey and Hewitt15,Reference Jones and Jacobsen17–Reference White, Stein-Parbury, Orr and Dawson19 Voice-hearers have reported experiencing mental health services as disempowering and unhelpful when they provide only ineffective medical options or invalidate voice-hearers’ experiences.Reference Hall, Terry and Hayward11,Reference Holt and Tickle16,Reference Clements, Coniglio and Mackenzie20 Researchers have identified a need for materials and resources to support mental health workers in engaging in meaningful conversations with voice-hearers about their voices.Reference McCluskey and Help21,Reference Rammou, Berry, Fowler and Hayward22
The current study evaluates such a resource: Let’s Talk About Voices (LTAV). In this paper we address the research question: What is the impact of engaging with LTAV on workers’ self-reported capacity to work with voice-hearers using a hearing voices approach?
Method
Design
A randomised, controlled crossover study design was used. Participants were randomly allocated to one of two groups after completing the first assessment. The immediate group were given access to LTAV immediately, whereas the delayed group received no intervention prior to the second assessment. Both groups were assessed again 4 weeks later, when the delayed group were given access to LTAV. In a further 4 weeks, both groups completed the third and final assessment.
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation, and with the Helsinki Declaration of 1975 as revised in 2013. All procedures involving human subjects were approved by The University of Sydney Human Research Ethics Committee (protocol no. 2023-819). The study is reported according to the CONSORT checklist, noting that it is not a clinical trial (as it does not measure a health outcome). Because this was not a clinical trial, prospective registration of the protocol on a publicly accessible platform was not possible. However, all analyses were completed according to the originally approved human research ethics submission.
Instrument
Because no measure of mental health worker capacity to work with voice-hearers was available, the Supporting Voice-Hearers Measure (SVHM) was developed (Appendix 1). This consists of 23 questions and takes less than 5 min on average to complete. It was co-designed by the researchers, the clinicians who developed LTAV, peer workers and voice-hearers who were familiar with the hearing voices approach. Development was an iterative process that occurred across four workshops. At the first workshop, co-developers brainstormed ideas related to concepts that should be included in the measure. These concepts were then transformed into draft items, which were presented to the co-developers at the second workshop alongside discussion of the Theory of Planned Behaviour (described below). The co-developers revised the item list (removing, adding and changing wording of items). At the third workshop, following presentation of the revised list, co-designers provided further feedback and this was incorporated. Prior to the final workshop, co-designers were provided with a copy of the penultimate version of the measure and piloted completion of the measure on the survey platform. These experiences were discussed at the final workshop, with minor updates suggested and incorporated into the final version.
Questions were developed based on the aims of LTAV and framed around the Theory of Planned Behaviour. The Theory of Planned BehaviourReference Ajzen and Boslaugh23–Reference Ajzen25 asserts that, assuming a person is able to perform a behaviour, their doing so is predicted by their intention. This intention is determined by the interaction between three factors and the beliefs that shape them. Behavioural beliefs are those about the outcome of a behaviour and how positive and negative that would be, and these determine the person’s attitude toward the behaviour, whether positive or negative. Normative beliefs are those about what other people think about the behaviour and the person’s wish to meet their expectations, resulting in subjective norms or social pressure to engage or not engage in the behaviour. Control beliefs are beliefs about the degree to which internal or external factors will assist or prevent the person from performing the behaviour, leading to perceived behavioural control. While not without its detractorsReference Sniehotta, Presseau and Araújo-Soares26 this theory has been found helpful in explaining, predicting and promoting behaviour change. For example, by 2020 it had been used and examined in more than 4200 papers across a variety of disciplines, including health.Reference Bosnjak, Ajzen and Schmidt27 The Theory of Planned Behaviour not only scaffolded instrument development but is also a useful framework through which to interpret the study findings (see Discussion).
Items are rated on a five-point Likert scale, from strongly disagree to strongly agree. Examples of items include: ‘There are many helpful strategies to support voice-hearers’ and ‘The only effective approach to deal with hearing voices is medication’. Negatively worded items are reverse scored so that higher total scores represent more positive responses. Total scores can range from 23 to 115. The measurement properties of the SVHM have been evaluated in a separate study (currently in production) using Rasch analysis. In summary, the SVHM demonstrated suitable measurement properties, including unidimensionality, rating scale validity and overall construct validity. Additionally, the SVHM demonstrated good internal consistency (Cronbach α = 0.88), concurrent validity (correlation of 0.49 with overall confidence in working with voice-hearers at baseline) and responsiveness/sensitivity to change.
LTAV
LTAV was developed in collaboration with voice-hearers and clinical experts, uses a hearing voices approach and is freely available at https://www.nslhd.health.nsw.gov.au/hearingvoices. Mental health workers within the organisation in which it was developed have described it as easy to use and extremely valuable for supporting voice-hearers. It consists of seven short, animated videos and accompanying worksheets covering a range of recovery-focused, strength-based topics. These are voice-hearer facing but designed for flexible use by mental health workers. For example, a worker could refer a voice-hearer to the online resources, use these in a group setting or work through them together with the voice-hearer. They can choose the most relevant resources to use, without needing to utilise them all. Table 1 provides a summary of the topics covered in the videos and worksheets.
Contents of Let’s Talk About Voices modules28
Procedure
Upon being given access to LTAV, mental health workers were asked to watch each of the videos and read through all the worksheets within a 7-day period. After 7 days they were sent a reminder that, if they had not done so, to please finish going through LTAV in the next week. They were not required to use the resources with voice-hearers, although they were welcome to do so if they found them helpful.
Sampling and recruitment
Participants were eligible if they were currently employed as mental health workers (including peer workers), worked with voice-hearers and had no previous experience with LTAV. To avoid contamination, the name of the resource was not revealed until each participant was given access to the resource, at which point they were asked whether they had seen it previously. The few participants who responded in the affirmative were diverted to a related study.
Mental health workers across Australia and other English-speaking countries were recruited through the newsletters and social media of a wide range of professional and mental health organisations. Researchers also sent project flyers to their professional networks with a request to circulate them. Posts were made on social media, such as Facebook and Instagram. People who were interested in participating were invited to follow a link to the participant information sheet and consent form. They indicated their consent using REDCap’s e-consent capability. Participants were randomised to groups via a list generated by an electronic random sample generator. The randomised list was input into the electronic survey tool, which assigned each participant to a group according to the list. Occasionally a participant commenced the first questionnaire twice (usually due to closing prior to completion or losing the link to LTAV following allocation to the immediate group). In these cases, the most complete record was retained and the other deleted; if both records were equally complete, the second was deleted. There was no evidence that this caused bias in group allocation because, in 12 of the 15 cases, both records were allocated to the same group. The intended sample size was 128 full completions, calculated using an estimated effect size of 0.25.
Data collection
Data were collected via online surveys using REDCap electronic data capture tools hosted at The University of Sydney,Reference Harris, Taylor, Minor, Elliott, Fernandez and O’Neal29,Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde30 between December 2023 and August 2024. The first survey was completed immediately after consent was received, following which participants were informed of their group allocation. Participants received email invitations with links to complete the second and third surveys 4 weeks after completion of each previous survey. They were emailed reminders if they did not complete each survey within 1 week, and were asked to view the resources within 1 week of being given access to LTAV. Demographic and practice data were collected in the first questionnaire.
Data analysis
The primary analyses were completed using data from participants who had completed measures at the time points included in the relevant analysis. All analyses were completed using IPM SPSS Statistics (version 28.0). Intention-to-treat analyses were also completed to determine whether the preliminary results held true for the full sample. For the intention-to-treat analyses, the most recent observation was carried forward for participants who did not complete all measures (i.e. if a participant had measures only for time 1, these were then carried forward to time 2 and time 3; if a participant had time 1 and time 2 measures, the latter were carried forward to time 3).
Between-group comparisons
To test for the changes associated with engaging with LTAV resources, an analysis of covariance was conducted comparing time 1 and time 2 data for the immediate and delayed groups. Time 2 scores on the SVHM were included as the dependent variable, with time 1 scores entered as a covariate. The grouping variable was group allocation (i.e. immediate or delayed group). Effect size estimates (partial η 2) were also calculated. Values of 0.01, 0.06 and 0.14 were used as indicators of small, medium and large effect size, respectively.
Pairwise repeated-measures comparisons
To further explore the influence of LTAV, a series of pairwise comparisons were completed for each group. The immediate and delayed groups were analysed separately, and repeated-measures t-tests were completed for time 1 and time 2, time 2 and time 3, and time 1 and time 3. Effect size estimates (Cohen’s d) were also calculated. Values of 0.2, 0.5 and 0.8 were used as indicators of small, medium and large effect size, respectively.
Tests of individual items
To provide greater detail on the changes associated with engaging with the LTAV resources, averages scores for each item before and after access to LTAV (i.e. time 1 and time 2 for the immediate group and time 2 and time 3 for the delayed group) were compared using repeated-measures t-tests. Bonferroni correction was used to adjust for the inflated risk of type I errors associated with multiple comparisons. This adjustment (0.05/23) resulted in the significance value being set at α = 0.00217 for these analyses. Effect size estimates (Cohen’s d) were also calculated.
Differences in change score according to demographic characteristics
To examine whether access to LTAV had a variable impact on the different participant groups, a series of analyses of variance were also completed. For these analyses, a ‘change score’ was calculated for each participant based on scores before and after receiving access to LTAV (i.e. change from time 1 to time 2 for the immediate group and from time 2 to time 3 for the delayed group). In analysis of variance, change score was the dependent variable and grouping variables were: profession; years working in mental health; work setting; frequency of working with voice-hearers; and confidence in working with voice-hearers at time 1.
Results
Participants
A total of 296 individuals commenced the initial survey. Randomisation and progress through the study are outlined in the CONSORT flow diagram (Fig. 1). Of these, 40 were removed due to either (a) not providing consent to participate, (b) completing the initial survey twice or (c) providing only name and contact email but no other details. This left a total commencing sample of 256 individuals. Of these 256 individuals, 120 provided measures at all 3 time points and an additional 54 provided data at time 1 and time 2 but not at time 3 (of these, 14 were in the immediate group, for whom time 2 represented the post-intervention assessment). As can be seen from the CONSORT diagram, there was significantly higher attrition from the immediate group from time 1 to time 2 compared with the delayed group. This is likely to be related to the former already having received access to the resources, and therefore perhaps having less incentive to complete the time 2 measures. Two participants who completed all measures but did not look at the resources (i.e. were not exposed to LTAV), and eight participants who identified on presentation of the resources that they had previously engaged with the resources were excluded from the primary analyses but were included in the intention-to-treat analyses. Most participants did not provide reasons for withdrawing from the study – they simply failed to respond to the survey invitations. Only six participants, all from the immediate group, cited reasons including insufficient time (n = 3), information technology problems, health problems and disagreement with content.
CONSORT flow diagram.
Demographic characteristics of the sample are presented in Table 2. There were no significant differences in the demographic characteristics of completers compared with non-completers. Comparisons of demographics for participants with different patterns of survey completion are presented in Appendix 2.
Demographic characteristics, all participants (N = 256)
a. Other professions include advocate, art therapist, case coordinator, counsellor, recovery coach, dietician, diversional therapist, psychotherapist and student.
b. Other work settings include education, custody, housing, administration and unclear.
c. Australian participants came from New South Wales (n = 84), Western Australia (n = 29), Victoria (n = 28), Queensland (n = 21) and South Australia (n = 21).
Figure 2 depicts participants’ reported engagement with the LTAV resources for those included in the primary analysis. While not everyone engaged with all the resources, 67% of participants watched all videos and 60% read all worksheets.
Participant engagement with videos and worksheets.
Between-group comparisons
The between-group comparison, testing the changes associated with engaging with the LTAV resources compared with not having access to them, was highly significant. For the primary analyses, the result was F 1,171 = 40.2, P < 0.001, with an effect size (partial η 2) of 0.19 indicating a large effect size. In the intention-to-treat analysis, the result remained significant (F 1,252 = 22.9, P < 0.001), with an effect size of 0.08 indicating a medium effect size.
Pairwise repeated-measures comparisons
Change over time for the immediate and delayed groups is presented in Table 3 and visually summarised in Fig. 3.
Supporting Voice Hearers Measure scores, across time points for immediate and delayed groups
a. For these figures, n = 132 because one participant completed the time 2 measure but not time 1 or time 3; this participant was included in intention-to-treat analysis with time 2 data carried over to time 3.
Visual representation of change over time for the Supporting Voice-Hearers Measure, immediate and delayed groups.
Results from the pair-wise comparisons for each group are summarised in Table 4. For the immediate group, who received access to LTAV between time 1 and time 2, there was significant change from time 1 to time 2 and that change was sustained at time 3. For the delayed group, who received the access to LTAV between time 2 and time 3, there was no change between time 1 and time 2 but a significant change from time 2 to time 3. These results were consistent between the per-protocol and intention-to-treat analyses, with effect sizes (Cohen’s d) of approximately 0.9 in the primary analyses (indicating a large effect size) and 0.5 in the intention-to-treat analyses (indicating a medium effect size).
Results from pre- to post-pairwise comparisons
T1, time 1; T2, time 2; T3, time 3; ITT, intention-to-treat analysis.
a. Analyses completed including participants who completed measures at both relevant time points.
b. Analyses completed on an intent-to-treat basis (missing time points were replaced with most recent observation).
c. For this analysis, n = 132 because one participant completed the T2 measure but not T1 or T3; this participant was included in ITT analysis, with T2 data carried over to T3.
Tests of individual items
Of the 23 test items, 14 showed significant change between pre- and post-intervention, with 11 having effect size of 0.4 or larger. These are presented in Table 5, grouped by their Theory of Planned Behaviour domain and ordered by effect size.
Individual items pre- to post-test
a. Negatively worded item, ratings were reversed prior to analysis. *P < 0.00217 (Bonferroni correction applied for 23 comparisons).
Individual items with the lowest significance and effect sizes were those, with one exception, where the initial mean score was relatively high in this sample, indicating limited room for improvement (items 1, 3, 5, 6, 7, 16, 18 and 23).
Differences in change score according to demographic characteristics
Exploration of change scores across different demographic categories revealed no significant differences according to profession (F 8,125 = 1.69, P = 0.108), number of years working in mental health (F 3,130 = 0.89, P = 0.450), work setting (F 3,130 = 0.36, P = 0.786) or frequency of working with voice-hearers (F 4,129 = 0.67, P = 0.616). Comparison of change scores based on ratings of confidence in working with voice-hearers revealed highly significant differences (F 4,129 = 8.0, P < 0.001). Average change scores were significantly higher for those who rated themselves as ‘Not at all confident’ (average change score 21.5), ‘Not very confident’ (average change score 13.4) or ’Somewhat confident’ (average change score 6.4) when compared with individuals rating themselves as either ‘Confident’ (average change score 3.8) or ‘Very confident’ (average change score 4.5).
Discussion
This study is the first, to the authors’ knowledge, to evaluate an online resource that uses a hearing voices approach. It investigated the self-reported confidence and capabilities of mental health workers to work with voice-hearers using this approach. The findings provide an early indication of the potential utility of LTAV, with participants showing significant improvements after accessing LTAV, which were sustained at follow-up, and differences between intervention and control conditions with large effect sizes. Significance remained following intention-to-treat analysis with moderate effect sizes, indicating that the results cannot be attributed to non-random attrition of participants. The apparent impact of LTAV cut across profession, work setting, level of experience and frequency of work with voice-hearers.
While our findings do not guarantee that practitioners will either take a hearing voices approach or use LTAV in their future practice, the individual item analysis provides a positive indication. First, significant changes were observed in the 4 items (items 4, 9, 14 and 21) that reported actual behaviours. Second, even after correcting for family-wise error, significant changes after engaging with LTAV were observed for the majority of items measuring each of the three factors that, according to the Theory of Planned Behaviour, influence behavioural intentions and therefore behaviour.Reference Ajzen and Boslaugh23–Reference Ajzen25 The significantly improved behavioural beliefs and attitudes items may address a particular area of need. For example, mental health workers have expressed fears that talking about voices might make these worseReference Coffey and Hewitt15,Reference McMullan, Gupta and Collins31 and, in a study of mental health workers’ knowledge and attitudes about hearing voices groups, only 25% were aware of the evidence base for these groups.Reference Jones and Jacobsen17 A qualitative study of nurses in an acute setting found that distraction was often the tool of choice in dealing with voice-hearing, the impact of which was modest and short term.Reference McCluskey and Help21 Due to volunteer bias, these qualitative studies probably report attitudes of a group of people who are relatively open to hearing voices approaches; beliefs in the benefits of voice-hearing approaches may well be far less widespread among non-volunteers.
Previous studies also indicate the potential impact of improvement in subjective norms. This factor, defined as whether a person thinks that other people who matter to them think that they should use such an approach,Reference Ajzen25 has been found to significantly predict intention to assess distressing voice-hearing.Reference Rammou, Berry, Fowler and Hayward22
Lastly, an increase in control beliefs and perceived behavioural control is important given that many mental health workers express lack of confidence in supporting voice-hearers.Reference Jones and Jacobsen17,Reference White, Stein-Parbury, Orr and Dawson19,Reference McCluskey and Help21,Reference McMullan, Gupta and Collins31–Reference Kramarz, Lyles, Fisher and Riches33 It is encouraging that people who expressed less confidence with working with voice-hearers on intake showed particularly strong improvement in our study, compared with more confident participants. Previous research has shown that providing useful voice-hearing information to voice-hearers is an area in which clinicians across services have indicated relatively low self-efficacy.Reference Rammou, Berry, Fowler and Hayward22
That items measuring attitudes, subjective norms and perceived behavioural control (or self-efficacy) all showed significant improvement following engagement with LTAV bodes well for the future practice of participating mental health workers. Even if they do not choose to use LTAV in the future, the change in their attitudes towards, and confidence in working with, voice-hearers may well be beneficial for the latter.
It is clear from the implementation science literature, however, that more than individual change is required if hearing voices approaches are to be adopted into routine practice: organisational and policy commitment is also needed. A systematic review of studies of new implementation of recovery-based practice into adult mental health services found that implementation was affected by setting factors such as the culture, structural characteristics and leadership commitment within the organisation.Reference Piat, Wainwright, Sofouli, Vachon, Deslauriers and Préfontaine34 Similarly, mental health workers have talked about how the expectations and practices of the organisation and more senior staff influence how they work with voice-hearers.Reference White, Stein-Parbury, Orr and Dawson19 Perhaps a review of these resources by senior colleagues would support greater organisational commitment. Future research examining workplace cultures and the perspectives of mental health workers about barriers to such an approach in the workplace would provide helpful information in assisting with implementation.
Limitations
As with any research relying on volunteers, people who agreed to participate are likely to at least have an interest in non-pharmacological approaches to voice-hearing. However, given that those who did not are unlikely to engage with LTAV in a real-world context, the results are likely to reflect the impact on the target market for the resources. Additionally, the studies use of a self-report measure of confidence and capabilities, rather than an objective measure of skills, is another limitation, because self-ratings may not always translate into improved practice. Finally, increasing the self-reported confidence and capabilities of mental health workers in using a hearing voices approach, and providing them with the resources to do so, does not automatically translate into positive outcomes for voice-hearers. Our research team is engaged in ongoing research to assess the impact on voice-hearers of engaging with LTAV.
Notwithstanding the limitations, this study provides preliminary evidence of the usefulness of LTAV, a freely accessible set of videos and accompanying worksheets that mental health workers can use with voice-hearers. The findings suggest that accessing and reviewing these resources results in positive changes, not only to mental health workers’ self-reported behaviour with regard to supporting voice-hearers, but also to their attitudes towards and confidence in doing so.
Supplementary material
The supplementary material can be found online at https://doi.org/10.1192/bjo.2025.10071
Data availability
Anonymised data to support the findings are available upon reasonable request from the corresponding author (A.H.).
Acknowledgements
The authors thank the stakeholder group of clinicians, peer workers and voice-hearers for their contributions to this study. We also thank Xinyi (Rachel) Yu for assistance with recruitment.
Author contributions
A.H. conceptualised the study, with input from all authors; and led the development of the SVHM, with input from J.N.S, N.H., L.S., H.Z. and K.W. A.H. and G.J. were responsible for participant recruitment, and A.H. led data collection. J.N.S. completed the analyses and, with A.H., drafted sections of the original manuscript. A.H., J.N.S., N.H., L.S., H.Z., K.W. and G.J. reviewed the analyses and revised the manuscript.
Funding
This research was funded through the Moyira Elizabeth Vine Fund for Research into Schizophrenia, administered by The University of Sydney, Australia.
Declaration of interest
L.S. and H.Z. are the developers of LTAV. However, there is no financial interest involved because the resources are available free of charge. The authors declare no other conflicts of interest.
Transparency declaration
The authors affirm that the manuscript is an honest, accurate and transparent account of the study being reported.
Open access
eLetters
No eLetters have been published for this article.