The COVID-19 pandemic continues to have a devastating impact on the UK job economy. In the third quarter of 2020, the number of redundancies rose to a record of 314 000 (Office for National Statistics 2020a). Unemployment rates have risen more rapidly than in the Great Depression (Institute for Employment Studies 2020) and were predicted to peak in the second quarter of 2020 at 7.75% (Bank of England 2020) – a rate similar to the aftermath of the 2008 financial crisis (Office for National Statistics 2020b). In response to unprecedented economic impact, the UK has embarked on unprecedented spending – through a second ‘furlough’ job retention scheme, business subsidies, employment coaching (BBC 2020) and £300 billion of quantitative easing (Bank of England 2020).
There is a complex and reciprocal relationship between health and unemployment (Stauder Reference Stauder2019). Becoming unemployed is associated with developing poorer health, and poorer health is associated with becoming unemployed (van Rijn Reference van Rijn, Robroek and Brouwer2014). Those unemployed with poorer health face a reduced likelihood of re-employment (Schuring Reference Schuring, Robroek and Otten2013).
This is pertinent for psychiatry. Mental illness is estimated to cost the world economy one trillion dollars per year (World Health Organization 2019), mediated partly through high rates of unemployment among those with mental illness (Ramsay Reference Ramsay, Stewart and Compton2012; Trades Union Congress 2017). As a result, the National Institute for Health and Care Excellence (2014) recommends employment interventions for all patients with severe mental illness.
The health drawbacks of unemployment are mediated largely through poorer mental health. Unemployment causes and worsens anxiety, depressive and adjustment disorders, lowers well-being and self-esteem (Norström Reference Norström, Waenerlund and Lindholm2019) and increases risk of suicide (Mattei Reference Mattei and Pistoresi2019). Those aged 25 and below are 2.5 times more likely to have lost their jobs as a result of the COVID-19 pandemic (Powell Reference Powell2020), overlapping with the modal age at onset of depression. Factors that traditionally increase unemployment, such as low socioeconomic status and low level of education, are also more common in those with mental illness, and have been even more strongly associated with unemployment during the pandemic (Organisation for Economic Cooperation and Development 2020).
Systematic reviews of health-improving interventions in the unemployed have been performed, as have reviews of employment interventions. Reviewing whether improving health can increase employment had not been previously attempted. At a time of great economic and healthcare uncertainty, this could provide valuable information for clinicians.
Summary of this month's Cochrane Review
Hult and colleagues (Reference Hult, Lappalainen and Saaranen2020) sought to question how health-improving interventions alone and combined with job-search training can affect re-employment. They included 15 randomised controlled trials (RCTs) involving 6397 unemployed job seekers. Their findings suggest that health-improving interventions alone have a small but very uncertain effect on increasing re-employment, whereas health-improving interventions combined with job-search training are associated with modestly increased rates of re-employment at long-term follow-up. The authors found no evidence that these interventions improved the health of job seekers.
What did the authors do?
Although initially opting to include non-randomised studies, the authors found sufficient randomised studies and subsequently only included RCTs in the meta-analysis. Studies conducted with unemployed (but able to work) participants over 16 years of age were considered. The authors excluded studies focused on specific cohorts, such as those with schizophrenia, to improve the generalisability of their results.
Studies were included if the intervention contained a ‘health-improving element at the individual level’. Interventions were grouped as combined interventions or therapeutic interventions (Table 1), depending on whether job-search training was offered alongside the health-improving element. The control group intervention could vary, from no intervention at all to multi-appointment social support programmes.
Interventions in the analysed studies
JOBS: Michigan Prevention Research Center Job Search Program; AmigA-M, Arbeitsförderung mit Gesundheitsbezogener Ausrichtung in München.
The authors’ primary outcome was re-employment rates. Studies were divided into short term (where re-employment was reported less than 3 months after the intervention ended) and long term (where re-employment was reported at least 3 months after the intervention ended). The authors were also interested in a number of secondary outcomes, although studies did not need to report these to be included in the analysis. These secondary outcomes included various measures of health, subjective work ability, adverse events and cost-effectiveness.
To find eligible studies, the authors searched a number of electronic databases (CENTRAL, MEDLINE, Scopus, PsycINFO, CINAHL, SocINDEX, NIOSHTIC, NIOSHTIC-2, HSELINE and CISDOC) for articles in English and non-English languages. They also searched ClinicalTrials.gov and the World Health Organization (WHO) trials portal for unpublished trials, checked reference lists of all studies and spoke with experts in the field to identify additional eligible materials.
The initial search yielded 3931 original articles. Two authors screened titles and abstracts and identified 68 potential articles. The same two authors then independently read the full text, marking studies as either ‘include’ or ‘exclude’. Disagreements were resolved by the decision of a third author. At the end of this process, 15 studies were included in the meta-analysis.
Risk of bias across a number of domains was assessed independently for each study by the two authors using the criteria in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins Reference Higgins and Green2011); they resolved disagreements by discussion among themselves. The GRADE approach was used to assess the certainty of the evidence (Schünemann Reference Schünemann, Broze˙k and Guyatt2013: chapters 4 and 5).
Intervention effects on employment rates were calculated using risk ratios, whereas other outcomes were continuous and were analysed using mean difference, or standardised mean difference if studies employed different methods to measure the same outcome.
Heterogeneity across studies within an analysis was calculated using the I 2 statistic (Box 1). If I 2 exceeded 50%, the authors intended to use subgroup analysis to explore causes. If I 2 exceeded 75%, the results of these studies were deemed too heterogeneous to be synthesised and the analysis was not performed. If data from studies were sufficiently homogeneous, the authors performed a meta-analysis using a random-effects model. Potential reporting biases were explored using a funnel plot (Box 2).
The I 2 statistic
As methods vary across studies in a meta-analysis, a degree of variation in results is expected. Authors must decide whether there are genuine differences (heterogeneity) between the studies or whether the variation observed is due to chance (the studies are homogeneous). I 2 can range from 0%, where any variation is entirely due to chance, to 100%, where all variation is due to heterogeneity across studies and cannot be explained by chance. The inventors of I 2 assign values of 25% to low, 50% to moderate and 75% to high study heterogeneity (Higgins Reference Higgins and Green2011).
Where I 2 is high, it can be difficult to reach clear conclusions. Authors can then use subgrouping to examine causes of heterogeneity. For example, the authors could divide studies into male and female, then calculate I 2 across the genders and within the genders. If gender were causing heterogeneity in the overall results, we might expect I 2 across the genders to rise and I 2 within the genders to fall. This implies that the variation in results observed could be caused by males and females responding differently to the intervention.
Funnel plots
A funnel plot is a scatter plot in which each point is a different study. The x-axis is the estimate of effect and the y-axis is the study's precision (usually the s.e. of the estimate of effect). The y-axis is inverted, so that the larger studies, with smaller s.e. and higher precision, group at the top, whereas the smaller studies, with larger s.e. and lower precision, scatter at the bottom. If no bias or heterogeneity is present, a symmetrical funnel appears, with 95% of studies falling within an isosceles triangle, its vertex being the summary effect estimate and its base 1.96 s.e. either side of this (Fig. 1(a)).
The plot should be symmetrical. If not, causes of heterogeneity or bias should be considered. If studies are missing from areas of non-significance, then this might represent a publication bias (Fig. 1(b)). Asymmetry from heterogeneity could be examined by dividing studies into subgroups and repeating the funnel plots (Fig. 1(c)) to see whether symmetry is achieved (Fig. 1(d)).
Funnel plots.
What did the authors find?
Results were separately analysed for therapeutic interventions and combined interventions. Therapeutic interventions encompassed 585 participants in intervention groups and 557 in control groups. Participants had a mean age of 38, the majority (54.6%) were female and had been unemployed between 1.8 months and 6.8 years. After interventions, participants were followed up for between 2 months and 3 years.
There was very low-quality evidence that therapeutic interventions alone increased re-employment (RR = 1.41, 95% CI 1.07–1.87, n = 1142), moderate-quality evidence of no effect on mental health (s.m.d. = 0.12, 95% CI −0.06 to 0.29, n = 530) and low-quality evidence of no effect on general health (s.m.d. = 0.19, 95% CI −0.04 to 0.41, n = 318).
Across combined interventions, there were 4101 participants, with 2343 in intervention groups and 1758 in control groups. At short-term follow-up, there was very low-quality evidence of no effect on re-employment (RR = 1.18, 95% CI 0.78–1.79, n = 66). At long-term follow-up, there was moderate-quality evidence of increased re-employment (RR = 1.12, 95% CI 1.06–1.20, n = 209).
Additionally, the authors divided study populations into the short-term unemployed (less than 12 months) and those with long-term unemployment (more than 12 months). In this analysis, therapeutic interventions alone had a modest effect, increasing employment in both groups (short-term: RR = 2.02, 95% CI 1.3–3.15; long-term: RR = 1.67, 95% CI 1.19–2.35), but combined interventions had no effect. All these subgroup analyses suffer from low numbers of studies, broad confidence intervals and consequent very low quality of evidence.
Are the findings useful?
This review showed unclear evidence for the effect of health-improving interventions in those who are unemployed. There is some moderate-quality evidence that combined interventions can increase employment at long-term follow-up but otherwise the evidence presented is unclear and of poor quality. Because of this and several other limitations, the generalisability and clinical relevance of these findings are doubtful.
A small number of relevant RCTs were identified, despite a broad study question and comprehensive search strategy. Opting to include only RCTs because of risk of bias is sensible, but including relevant, non-randomised studies might have improved the generalisability. Meta-analysis is predicated on identified studies having a degree of homogeneity that allows their findings to be combined. All studies employed a common binary outcome: ‘was a participant employed or not at follow-up?’ but heterogeneity across study participants, interventions and controls limits the review's usefulness. Given the methodological variation, using a random-effects model (Box 3) was appropriate.
Random-effects models
Meta-analyses are generally analysed using either a ‘fixed-effect’ or a ‘random-effects’ model. A random-effects model is used when authors assume that the ‘true’ effect may differ from study to study. The fixed-effect model is used when they have assumed that there is a single, ‘true’ effect that underlies all studies in the analysis. In the fixed-effect model, any variation across studies results is due to a sampling error. In the random-effects model, variation in results will additionally be explained by changes in methodology across studies.
Not surprisingly, most meta-analyses utilise random-effects models, which produce a summary effect which is an estimate of the distribution's mean. Fixed-effect models produce a summary effect which is an estimate of the true value. Consequentially, only a random-effects model allows authors to explore how the effect size changes across subgroups.
Participants
The majority of recent UK job losses have been among 16- to 24-year-olds, working in the hospitality and entertainment industry (Office for National Statistics 2020b). Although the average age of participants in one study was 19, another study recruited only those aged over 50. Length of unemployment varied from less than 13 weeks to more than 12 months. In two studies, participants were unemployed professionals. Age, length of unemployment and professional background are all likely to affect health status, employability and response to a health-promotion intervention.
In the review, participants were grouped only according to whether they were short-term unemployed (<12 months) or long-term unemployed (>12 months). The authors originally intended to address heterogeneity with further subgroup analysis, but the samples were too small. Their analysis had insufficient power to detect a true difference between the subgroups. They correctly conclude that more trials are needed.
The authors decided to exclude studies focused on improving employability for specific patient groups, for example those with severe mental illness, but four identified studies excluded participants with any evidence of mental illness. Two studies excluded participants if they did not engage with the intervention. Given that the employment rate of those with severe mental illness has been estimated to be far below rates of employment in those without mental illness, this approach possibly selects for an unrepresentative group of patients who are more likely to be re-employed anyway and ignores a group who may maximally benefit from health-improving interventions.
Interventions and controls
Interventions were classed as similar and grouped for analysis if they were purely health improving, or if they combined this with job-search training (Table 1). Often this division seemed arbitrary. It could be argued that some interventions classed as solely therapeutic seemed also to utilise job-search training, such as a cognitive–behavioural therapy (CBT) programme that aimed to teach skills for coping with unemployment, or a career transition programme that involved participants using imagery to construct a successful self and rehearse a job interview. Other interventions stretched the definition of therapeutic, such as an expressive writing course, where participants wrote down their thoughts, feelings and the consequences of unemployment. Other interventions included internet and traditional CBT programmes, physical exercise and yearly health screening. These are diverse interventions, which are hard to justify as similar for meta-analysis.
Of combined interventions, four studies employed JOBS, a job-search seminar teaching strategies for re-employment and methods of dealing with anxiety, depression and low self-esteem. Overall, the combined analysis similarly suffers from the heterogeneity of interventions.
Control groups also varied considerably. Most studies employed nothing at all, being on a waiting list, or literature on effective job-searching. Participants in these studies would still be able to access regular employment and health services, but this would be an insufficient control for the effects of engaging with an intervention independent of any therapeutic and job-search training components. A minority of studies had better controls, such as comparing CBT with a social support programme of equal length, but these tended to have lower numbers and therefore had less weight in the analysis.
Bias
The authors assessed bias across a number of domains. Random allocation was adequate in 8 of the 15 studies, but only 6 studies reported concealing allocation. This might introduce selection bias. Adequate masking was reported in only 3 studies, which is not necessarily a problem for assessing whether someone is employed or not, but might introduce biased assessment of continuous outcomes such as health.
A level of publication bias for studies employing combined interventions was identified using a funnel chart. This was ultimately deemed to be non-significant as the two studies driving this effect had little weighting in the results.
The wider context
The 15 studies were carried out over a period of 26 years, between 1989 and 2015 and across 11 different countries. Over this time and space, the economic context varied dramatically; re-employment rates would increase in times of economic growth as more jobs are created and conversely decrease in times of recession. Interventions might disproportionately benefit when job supply is higher and vice versa, making the true effect unclear. All studies took place in high-income countries, with developed social benefit and unemployment programmes. This heterogeneous landscape makes the evidence very difficult to generalise and apply to the current peri-pandemic health landscape.
Conclusions
The question put forth by the reviewers is important, but somewhat vague (Box 4). Consequently, the evidence is uncertain and unlikely to affect clinical practice. There was some evidence that, when combined with job-search training, therapeutic interventions may increase re-employment, but whether job-search training alone would have the same effect is unclear. It was disappointing that health-promoting interventions were not found to improve participants’ health. Despite this, clinicians should be aware of, and address, the deleterious effects unemployment has on the health of their patients.
Choosing a research question for meta-analysis
The research question is important, as it guides the planning, analysis and reporting of a study. It is useful to think in terms of the population, intervention, comparator and outcomes of interest (the PICO). For each category, a balance must be struck. A more specific PICO reduces the number of confounding variables and makes researchers more confident that their findings are free from bias. If too specific, researchers might not identify enough studies for analysis, or their conclusions may have limited real-world relevance.
Refining a research question is iterative and adapts as the study proceeds. If the literature is limited, researchers may broaden their PICO. This introduces confounders which have an impact on the outcome independently of the intervention. The more inclusive a PICO, the more bias is introduced and the less confidence we can have in the findings.
Funding
This article received no specific grant from any funding agency, commercial or not-for-profit sectors.
Acknowledgement
I thank Dr Riccardo De Giorgi for his feedback on an earlier draft of the manuscript.
Author contributions
The manuscript was written by J.B.
Declaration of interest
None.
An ICMJE form is in the supplementary material, available online at https://doi.org/10.1192/bja.2020.100.
eLetters
No eLetters have been published for this article.