Frontline emergency service workers are often disproportionately exposed to demanding workloads, long working hours, shift work and exposure to traumatic and stressful events. These established psychosocial risks 1–3 place frontline workers at disparate risk of mental ill health. Reference Harvey, Milligan-Saville, Paterson, Harkness, Marsh and Dobson4–Reference Petrie, Milligan-Saville, Gayed, Deady, Phelps and Dell7 Mental illness is currently the leading cause of long-term sickness absence and work incapacity in most developed countries, Reference Duijts, Kant, Swaen, van den Brandt and Zeegers8,Reference Whiteford, Degenhardt, Rehm, Baxter, Ferrari and Erskine9 with work-related factors acknowledged as a determinant of mental health. Reference Rugulies, Aust, Greiner, Arensman, Kawakami and LaMontagne10,Reference Frank, Mustard, Smith, Siddiqi, Cheng and Burdorf11 For many frontline personnel, mental ill-health can cause disruption to career, impact on family and relationships and increase rates of suicide. Reference Harvey, Milligan-Saville, Paterson, Harkness, Marsh and Dobson4,Reference Haugen, McCrillis, Smid and Nijdam12
Challenges in seeking or accessing treatment for such mental health issues are especially prevalent in first responders Reference Jones, Agud and McSweeney13 and other high-risk occupational groups. Reference Hoge, Castro, Messer, McGurk, Cotting and Koffman14 One recent meta-analysis highlighted that one in ten first responders experience barriers to mental healthcare Reference Haugen, McCrillis, Smid and Nijdam12 such as limited knowledge of appropriate sources of help, lack of support from the workplace to access treatment and concerns surrounding public and self-stigma, confidentiality and the potential for negative impact on career progression. Reference Hoge, Castro, Messer, McGurk, Cotting and Koffman14–Reference Rona, Burdett, Khondoker, Chesnokov, Green and Pernet16 These barriers can impact screening participation and responses, Reference Greenberg and Forbes17 or accessing treatment, and potentially delay recovery. Nevertheless, early detection and treatment of mental ill-health is associated with the need for less intensive treatments and improved recovery, possibly because early treatment reduces the risk of secondary issues such as relationship problems or work impairment. Reference McNally, Bryant and Ehlers18,Reference Muñoz, Cuijpers, Smit, Barrera and Leykin19 In attempts to detect individuals at risk of, or currently experiencing, a mental health condition and to reduce its impact by promoting appropriate help-seeking, many high-risk industries such as first responder agencies are increasingly implementing mental health screening within their workplaces. Reference Greenberg and Forbes17
Workplace mental health screening has received criticism for the cost and resources required, safety of information and potential consequences of disclosure biasing reporting. Reference Marshall, Milligan-Saville, Petrie, Bryant, Mitchell and Harvey20,Reference Warner, Appenzeller, Grieger, Belenkiy, Breitbach and Parker21 These concerns are especially pertinent when considering first responder populations, given the potential negative repercussions Reference Murphy, Noble, Chakraborty, Michlig, Michalak and Greenshaw22 of stigma from colleagues or impact on role due to mandatory reporting and notifiable conduct. Nevertheless, workplace mental health screening continues to be implemented in first responder workplaces, despite the limited evidence supporting its intended benefits. Reference Strudwick, Gayed, Deady, Haffar, Mobbs and Malik23
A recent systematic review Reference Strudwick, Gayed, Deady, Haffar, Mobbs and Malik23 revealed only eight controlled trials investigating the effectiveness of workplace mental health screening to improve employee mental health. A positive effect of screening for help-seeking was found in only one trial at 3-month follow-up, which was attenuated after 6 months. Reference Gärtner, Nieuwenhuijsen, Ketelaar, van Dijk and Sluiter24 Limited benefits regarding improved mental health outcomes following screening were observed only when screening was paired with facilitated access to treatment. Reference Gärtner, Nieuwenhuijsen, Ketelaar, van Dijk and Sluiter24–Reference Farzanfar, Locke, Heeren, Stevens, Vachon and Nguyen26 One randomised controlled trial (RCT) of over 10 000 military personnel did not find an effect of post-deployment mental health screening on help-seeking behaviour for mental health symptoms 10–24 months later. Reference Rona, Burdett, Khondoker, Chesnokov, Green and Pernet16 Such mixed results are probably due, in part, to a range of methodological factors including differing interventions, varying worker populations and sample size.
Considering its use within first responder agencies, despite limited evidence supporting its implementation, we sought to investigate the impact of mental health screening among currently employed firefighters. Previous findings suggest that the provision of a generic list of recommendations following mental health screening is not sufficient to improve mental health symptoms Reference Strudwick, Gayed, Deady, Haffar, Mobbs and Malik23 and, although there is some evidence supporting screening followed by facilitated access to treatment, confidentiality concerns among first responders may hinder employees from participating in a workplace initiative that facilitates access to treatment. Considering these findings, this RCT aimed to test whether the provision of tailored online feedback, following a brief and strictly confidential online mental health screening assessment, increased appropriate help-seeking behaviour among first responders.
Method
Study design and participants
We conducted a clustered RCT with firefighters currently employed within Fire and Rescue New South Wales (FRNSW), Australia. FRNSW, the largest fire and rescue organisation in the southern hemisphere, responds to fire, rescue and hazardous material emergencies across the state of New South Wales (NSW). Clustering occurred at the level of the station at which the firefighters were based. Supplementary File 1 details the stages of participation. Prior to study commencement the trial protocol, including selection of the primary outcome and analysis plan, was registered on ANZCTR (registration no. ACTRN12621001457831). The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation, and with the Helsinki Declaration of 1975 as revised in 2013. All procedures involving human subjects were approved by the Human Research Ethics Committee of the University of New South Wales (no. HC219527).
Inclusion criteria
Participants were required to be 18 years or older and currently employed either full- or part-time as a paid firefighter or rescue personnel within FRNSW. While Australia has a large number of volunteer firefighters, particularly in remote areas, these volunteer fire services were not included in this study. Work email addresses of all eligible staff were provided to the research team and uploaded to the system prior to study commencement; these acted as ‘tokens’ or ‘keys’ for potential participants to access the study.
Randomisation and masking
To increase the chance that participants remained blinded to their condition, clusters were defined by the fire station at which firefighters were based. Randomisation of station occurred prior to individual participant recruitment commencement using the cluster ra() function from the ‘randomizeR’ package in R (v4.0.3, for Windows). Complete randomisation is equivalent to tossing a fair coin for the allocation of each station to either condition.
Measures of assessment
The mental health screening tool comprised 13 questions across 3 scales and took approximately 2 min to complete. Self-reported help-seeking for mental health reasons was assessed using an adapted version of the Actual Help-Seeking Questionnaire. Reference Rickwood, Deane, Wilson and Ciarrochi27 This scale asked participants whether and where they had sought help for mental health reasons. At follow-up, this question was repeated for the period following baseline. Levels of psychological distress were assessed using Kessler-6 (K6). Total scores ranged from 6 to 30 for the 6 items. A score less than 14 indicated none to mild psychological distress, 14 to 18 suggested a moderate level of psychological distress and 19 or more a severe level of psychological distress. Reference Kessler, Green, Gruber, Sampson, Bromet and Cuitan28 Exposure to critical incidents was measured using an adapted version of a questionnaire previously used in trials with firefighters Reference Milligan-Saville, Choi, Deady, Scott, Tan and Calvo29 assessing the number and type of trauma exposure experienced while on duty.
The researchers co-developed the screening tool with an advisory group of representatives from FRNSW. Its usability, feasibility and appropriateness were piloted with a group of 36 firefighters identified by FRNSW. This pilot informed some minor changes to processes that were implemented for the RCT.
Outcomes
The primary outcome for this trial was a change over time in help-seeking behaviour among participants who, at baseline, scored 14 or over on the K6, indicating at least moderate psychological distress. Appropriate help-seeking included seeking help or advice from external clinical services such as GPs, counsellors, psychologists or psychiatrists, as well as employer-provided supports such as an employee assistance programme (EAP).
The secondary outcomes for this trial were a change over time in self-reported new help-seeking behaviour and psychological distress for participants who scored 14 or above on K6 at baseline.
Trial procedures
Prior to recruitment commencement, demographic and employment details of all eligible participants were provided by FRNSW to the research team. Data included age, gender, years of service, whether they were employed as a retained (on-call staff who respond to emergency incidents from their home or other workplaces) or permanent firefighter, type and location of station at which they were based and work email address. Collection of this information allowed the research team to reliably perform cluster randomisation prior to recruitment, and was retained and analysed only for those who provided consent to participate in the trial. No additional demographic questions were asked during screening.
Recruitment, written consent and data collection were conducted online. An invitation to participate in a study delivering screening to first responders was sent to all eligible participants via email on 15 November 2021. The study was also advertised at fire stations through printed flyers, organisation newsletters and other internal communication channels, each linking to the study website. Although recruitment had already commenced, once approvals were received the employee union representative body also promoted the study through their communication channels.
Firefighters who wished to participate were required to input their work email address into the trial registration. This ensured that only currently employed firefighters were able to register and, following registration, were allocated to the correct study condition as per the cluster randomisation to which their workstation was assigned. The Participant Information and Consent Form (PISCF) detailed the study aims. Participants in both conditions were provided the same information and advised that their responses would help determine the effectiveness of mental health screening for first responders.
Assessments
Once written informed consent was obtained online, the online baseline screening assessment commenced. Irrespective of intervention or control study condition, all participants completed the brief, 2-min, 13-item mental health screening questionnaire. At completion, participants were given the option to be contacted by mobile phone, email or both for the next stage of the trial. Participants were free to withdraw from the study at any point by discontinuing the screening.
Three months following baseline, participants received notification to complete their follow-up assessment. In the event of a non-response, individuals were contacted a maximum of two additional times. The follow-up assessment asked the same 13 questions as at baseline.
Interventions
At the completion of the baseline assessment, participants in the intervention group were immediately provided with tailored feedback according to their reported level of distress, as categorised by their scores on K6. Those reporting minimal psychological distress (K6 < 14) received feedback stating that it was unlikely they were experiencing a common mental health disorder at this time but, if they ever felt they would benefit from more support or a mental healthcare plan, to contact their GP or workplace support team to discuss available options. Those reporting moderate psychological distress (K6 = 14–18) were informed that they were currently experiencing symptoms of psychological distress that may be due to (or progress to) a common mental health disorder, and that these symptoms could be addressed by seeking either internal supports from their workplace (with details of their well-being team and EAP provided) or external supports (mental health tools, websites and contact details of services relevant for moderate levels of distress provided). Those reporting severe psychological distress (K6 ≥ 19) were informed that they were currently experiencing significant symptoms of psychological distress, which may be due to a mental health disorder, and that help could be sought from either their workplace (with details of their well-being team and EAP provided) or external supports (relevant mental health tools, websites and contact details of services provided). Importantly, among the external supports provided for the intervention group, participants with case-level symptoms could access free, external, independent and confidential treatment via the UNSW Traumatic Stress Clinic.
The control group did not receive any information regarding levels of their psychological distress, as assessed during baseline screening, nor specific recommendations of available services or resources. Instead, control participants all received a copy of the same generic feedback advising them to contact their GP or internal peer support team if they experienced any symptoms or felt they would benefit from more support for their mental health. Once follow-up was completed, all participants irrespective of group allocation received tailored feedback according to their currently reported level of psychological distress.
Sample size
Sample size was calculated using the CRTSize package in R version 4.0.3 for Windows (R Core Team; https://CRAN.R-project.org/package=CRTSize). Power was set at 0.80, α = 0.05 (two-tailed); the anticipated proportion of individuals with the outcome of interest (i.e. actual help-seeking behaviour) at follow-up was 35% in the control group and 50% in the experimental group; intraclass correlation coefficient (ICC) was 0.05, with an anticipated average cluster size of 5. Together, this estimated that a sample of 400 participants was required (200 per arm). An earlier power calculation, using the same parameters, was incorrectly completed and reported on ANZCTR. This error was detected early in the recruitment phase, and the power calculation and sample size requirements were updated on the Registry before any analysis began.
Statistical analyses
Analyses for the primary and secondary outcomes were undertaken with an intent-to-treat analysis. For the primary outcome, a mixed-effects logistic regression model in Stata (v18, for Mac) was utilised to estimate differential change in help-seeking behaviour over time between groups. This model included random intercepts for participants and for sites to accommodate within-individual and cluster dependencies, respectively. For the secondary outcome a mixed-model, repeated-measures (MMRM) analysis of variance in SPSS (v30, for Windows) was utilised to analyse the change in mean K6 score between the intervention and control groups across study time points. Clustering was accounted for by a random site intercept. An unstructured variance–covariance matrix accommodated dependency of observations at different occasions of measurement. The group-by-time interaction, and differences between the intervention and control groups between baseline and follow-up, were examined as planned contrasts. As per the preregistered, a priori analysis plan, primary analysis compared change over time on overall help-seeking behaviour for participants in the intervention and control groups with moderate or severe psychological distress, as indicated by a score of ≥14 on the K6 scale at baseline.
Role of funding source
The trial was conducted with funding from the iCare foundation. R.B. and S.B.H. were supported by NHMRC Investigator Grants. The funders had no role in the study design, data collection, data analysis, data interpretation or writing of the report.
Results
As outlined in Fig. 1, from 7291 invited firefighters, 495 (6.9%) consented to participate in the trial between November 2021 and April 2022. Of these, 459 completed baseline (211 intervention and 248 control participants). Compared with data published on all firefighters employed in NSW, the recruited sample appeared to be representative in terms of gender (89% male in recruited sample compared with 88% for all FRNSW firefighters, χ 2 = 0.60, P = 0.44), although the recruited sample had a greater proportion of permanent firefighters (64% compared with 53%, χ 2 = 24.2, P < 0.01). Of those whose K6 baseline score was ≥14 (n = 141), follow-up data were available for 46/60 (76.7%) intervention and 64/81 (79%) control group participants (total N = 110, 78.0%) forming the sample for our analysis. Demographic details of this sample at baseline are presented in Table 1. There were no baseline differences between those who completed both assessments and those lost to follow-up in terms of age (P = 0.20), gender (P = 0.23), mental health symptoms (P = 0.66) and previous help-seeking (P = 0.23). There was also no evidence of variation in follow-up rates between the intervention and control groups (P = 0.74).
CONSORT trial profile. K6, Kessler-6 psychological distress score.
Baseline characteristics of participants who scored moderate or severe levels of psychological distress (14 or above) on the Kessler-6 psychological distress score (K6) at baseline (N = 141)
a. Where both full- and part-time firefighters receive additional training to allow them to respond to more complex rescue situations.
b. This question was asked if participant answered ‘no’ to seeking help in the previous 90 days (n = 68).
Primary outcome: help-seeking
As shown in Fig. 2, at baseline 58.0% (n = 47) of those with at least moderate K6 symptoms in the control group had sought help from professional services for mental health, compared with 43.3% (n = 26) in the intervention group. This type of baseline imbalance is not unusual in a cluster RCT. Significance testing was not carried out Reference Moher, Hopewell, Schulz, Montori, Gøtzsche and Devereaux30 because these differences were, by definition, random and were accounted for in the mixed logistic regression analysis that considered change in levels of help-seeking over time. At follow-up there was no increase in overall rates of help-seeking within either condition, and no statistical evidence of a difference over time between the two groups (P = 0.31).
Help-seeking behaviour for participants who scored in the moderate or severe Kessler-6 psychological distress score (K6) categories at baseline: percentage (with 95% CI) who sought help in the previous 90 days.
Secondary outcomes
New help-seeking
To examine for an effect on new help-seeking behaviour, additional analysis focused on participants (n = 68) whose K6 baseline score was ≥14 and had responded ‘no’ to receiving help at baseline. There was a trend towards more new help-seeking in the intervention group (odds ratio 2.04 (95% CI [0.5, 8.6]), but this fell well short of statistical significance (P = 0.33).
Psychological distress
The mean change in psychological distress between the intervention and control groups for firefighters who at baseline received a K6 score of ≥14 (n = 141) was examined (see Fig. 3). While psychological distress reduced over time in both groups, there was a greater persistence of symptoms among the intervention group. Across the follow-up period, there were significant differences in the level of change in psychological distress between the intervention and control groups (t[111] = 2.29, 95% CI: 0.24, 3.23, P = 0.024). By the end of the follow-up period, this difference equated a Cohen d effect size of −0.42 (95% CI: −0.04, −0.79), with intervention participants having an average K6 score of 1.8 points (95% CI: 0.17, 3.43) higher than the control group.
Change in mean Kessler-6 psychological distress score (K6) (with 95% CI) for participants who scored in the moderate or severe categories at baseline (n = 141).
A post hoc analysis examined this difference further by stratifying participants based on whether their baseline distress was moderate or severe. Among participants (n = 53) who received a K6 score in the severe category, psychological distress was higher by an average of 1.12 points (95% CI: −1.85, 4.23) for the intervention compared with the control group, but this difference was not statistically significant (t[39] = 0.58, P = 0.57). By contrast, among participants (n = 88) who received a K6 score in the moderate category, psychological distress was higher by an average of 2.31 points (95% CI: 0.69, 3.90) for the intervention compared with the control group, and this difference was statistically significant (t[69.6] = 2.59, P = 0.01).
Exposure to critical incidents
Analysis of the number of critical incidents attended from baseline to follow-up showed that just over half of the sample (n = 73, 51.8%) had attended at least one event. Of the remaining 48.2%, 37 (26.2%) had not attended any and 31 (22.0%) did not provide a response. There was no difference in frequency of exposure between intervention and control groups (χ 2 = 0.047, P = 0.84).
Discussion
This cluster RCT is the largest controlled trial conducted to date evaluating the efficacy of workplace mental health screening in promoting help-seeking behaviour and reducing self-reported levels of psychological distress for firefighters. Our findings suggest that a brief, validated and confidentially administrated opt-in screening tool, paired with tailored online feedback providing resources for support and links to free, confidential treatment, does not significantly increase help-seeking behaviour or reduce levels of psychological distress at 3-month follow-up. In fact, with levels of psychological distress at follow-up significantly higher for the intervention than the control group, the results suggest that mental health screening conducted in this way has the potential to be harmful for individual workers. This difference was particularly marked at follow-up for participants who scored in the moderate level of psychological distress at baseline. This finding raises the question as to whether this sustained level of psychological distress observed in the intervention group may be due to labelling or pathologising what may have otherwise been transient symptoms. It is likely that the risks associated with labelling are likely to be highest amongst those with only moderate levels of symptoms. This is important, because this group are often considered key targets for programmes aimed at increasing early intervention. This one-off process of providing information and highlighting potential symptoms may promote a similar adverse response to that found in psychological debriefing following a critical incident. Reference Mayou, Ehlers and Hobbs31,Reference Bisson, Jenkins, Alexander and Bannister32
It has been noted previously that health screening has the potential to do harm, and it is essential to understand whether the potential benefits outweigh these risks. Reference Gilbody, Sheldon and Wessely33 Our findings are valuable because they demonstrate the potential for such mental health screening, paired with tailored online feedback, to be harmful for firefighters. We found that the provision of tailored feedback following mental health screening within the environment studied does not lead to a greater likelihood that someone experiencing psychological distress will seek help for that distress. Furthermore, it is possible that this process of receiving tailored online feedback about levels of distress may orient the individual’s attention towards catastrophising their distress, leading to persistently elevated levels and potentially prevent against the natural remission/reduction of distress with time. Given these findings, the wide-scale use of mental health screening, paired with online feedback, among first responders cannot be recommended at present and should not form part of an organisation’s response to psychosocial risk management.
The study had a number of methodological strengths guided by the literature. Previous research conducted with police officers Reference Marshall, Milligan-Saville, Petrie, Bryant, Mitchell and Harvey20 highlighted the potential for underreporting of symptoms in employer-administered mental health screening programmes, compared with screening led by independent third parties. Therefore, the use of a trusted independent organisation to administer the screening was included in the design to control risks and alleviate concerns associated with confidentiality. In addition, support from the associated representative union body was obtained, with the intention of engendering credibility in the trial and promoting participation numbers. The delayed introduction of this recruitment strategy saw an increase in participation numbers. This approach may also have contributed to the high adherence rates at follow-up. Furthermore, our study deliberately attempted to move beyond screening that provided generic information that has been found to be ineffective Reference Strudwick, Gayed, Deady, Haffar, Mobbs and Malik23 and, rather, to replicate in a more confidential way facilitated access to appropriate treatment. Rather than participants receiving a call to arrange an appointment, which one recent review found to have a small positive effect, Reference Strudwick, Gayed, Deady, Haffar, Mobbs and Malik23 online information with direct access to appropriate services, without the need for a referral and with reduced waiting time, was provided.
Alongside these strengths, there are several methodological limitations to be considered. Despite co-design with the partnering organisation and the above-mentioned union endorsements, the participation rate was low, representing just over 7% of personnel. Difficulties with recruitment may have led to some sample bias towards those motivated by a pre-existing interest in mental health. This potentially increased the likelihood of a null finding due to mitigation of the benefits of screening in this group, and it remains possible that different findings might have been seen if mental health screening was mandatory. The small sample in the cohort who experienced moderate and severe levels of psychological distress also limits the power for these analyses, and potentially the study’s ability to detect changes in new help-seeking amongst this group, although it is important to note that the study had adequate power to detect a clear harm associated with the intervention. Given this detection of harm, it would be difficult to justify a larger sample size. The necessity of having the screening tool as short as possible, to minimise burden on participants, also limited the measures available for interpretation. The use of a two-point assessment rather than an ongoing programme of screening delivered on a regular basis to track individual trajectory means that our results cannot be generalised to screening programmes that might utilise repeated or face-to-face clinical assessments. Similarly, it is uncertain whether the results obtained would have been different if alternative feedback mechanisms, including face-to-face discussions, had been used. At baseline, just over half of participants with at least moderate symptom levels had sought help in the previous 90 days; previous large surveys of frontline workers in Australia suggest that this level of help-seeking is typical for this population. 34 However, this highlights further limitations around generalisability and whether similar results would have been obtained in a setting with lower levels of baseline help-seeking. It is also important to note the baseline difference in levels of previous help-seeking between the two study groups. A final limitation to consider is the follow-up period of 3 months, which was selected due to funding requirements. Although 3 months would ordinarily be sufficient time for an individual to action a referral, it may not have been long enough to detect any treatment-related improvements in symptoms, particularly if a delay was experienced in accessing the recommended services. Although an important limitation, the lack of any significant difference in help-seeking suggests that a longer follow-up may have been unlikely to uncover any additional symptomatic benefits.
Notwithstanding these limitations, our study demonstrated that mental health screening paired with tailored online feedback should not be implemented at scale for firefighters. Such screening does not appear to increase help-seeking, and is associated with harm in the form of poorer mental health outcomes compared with those who did not receive tailored feedback. The reasons for this harmful effect are not clear but, given that it was observed most prominently among those with moderate distress, supports a hypothesis that providing individuals with labelled information about their moderate levels of distress through online communication, in the absence of directly facilitated follow-up with a health professional, may not be beneficial and may interfere with the ordinary management of such symptoms. Considering these results, mental health screening should cease to be implemented at scale in organisations and any further research on mental health screening should be conducted in a way that ensures that no harm is occurring. Researchers might investigate alternative means to identify and direct at-risk workers into treatment without producing net harm, or whether organisations can achieve better outcomes for their employees by improving internal communication and culture, promoting support and help-seeking in general within the workplace and managing work environments to reduce psychosocial risk. Reference Rugulies, Aust, Greiner, Arensman, Kawakami and LaMontagne10,Reference Greenberg and Forbes17
Supplementary material
The supplementary material is available online at https://doi.org/10.1192/bjp.2025.106
Data availability
Deidentified participant data that underlie the results reported in this article will be shared with investigators up to 3 years post publication date, and the proposed use of the data has been approved by an independent review committee identified for this purpose and will be available from the corresponding author (A.G.).
Acknowledgements
We thank the firefighters who participated in the trial and the staff at FRNSW and the Fire Brigade Employees Union (FBEU) who supported the implementation of this study.
Author contributions
A.G., R.B. and S.B.H. conceptualised the study. A.G., J.S. and S.B.H. designed the intervention and trial. K.K., T.A.B., A.D., K.E., B.M., D.A.J.C., M.D. and R.B. supported the development of the intervention and implementation of the trial. K.K. collected the data. A.G., K.K., A.M. and S.B.H. analysed and interpreted the data. A.G. and S.B.H. drafted the manuscript, which was revised for additional interpretation by all authors. All authors approved the final manuscript.
Funding
This study was funded by a research grant from iCare NSW and was supported by NHMRC Investigator grants awarded to R.B. and S.B.H.
Declaration of interest
None.
Open access
eLetters
No eLetters have been published for this article.