Study design and sample characteristics

The SWAN (Supported Work and Needs) study is a pragmatic RCT of the IPS model. Participants were recruited from community mental health teams in two boroughs of South London (ISRCTN96677673). Borough A has an average index of deprivation score of 21.31 (a rank average score of 125 out of 354 for the UK), a 6.3% unemployment rate (compared with 5.3% nationally) and incapacity benefit (a benefit received by people unable to work because of ill health) is received by 5.6% of working-age residents (compared with 7.2% nationally). Borough B has an average index of deprivation score of 34.95 (a rank average score of 19 out of 354, i.e. a high level of deprivation), a 9.5% unemployment rate and incapacity benefit received by 7%.

Inclusion criteria were that participants should be receiving out-patient or community psychiatric care from local mental health services, have severe mental illness (duration of illness over 2 years, global asssessment of functioning (GAF) ^{Reference Endicott, Spitzer, Fleiss and Cohen8} score of 60 or less, and a diagnosis of a psychotic or chronic affective disorder), aged 18–65, able to read and speak English to a high enough standard to give informed written consent, to have been unemployed for at least 3 months and want to obtain competitive employment. Participants were excluded if they had been referred for IPS in the previous 6 months. Mental health professionals were asked to identify potential participants and ask them if they agreed to being approached by a researcher. After participants were given a complete description of the study by the researcher, written informed consent was obtained. Ethical approval was obtained from the Joint Institute of Psychiatry/South London and Maudsley Research Ethics Committee (reference no 319/03).

Participants were randomly allocated to the intervention group or the control group. Treatment allocation was stratified by gender and age (10-year bands). Randomisation with minimisation was used, performed by the Institute of Psychiatry Mental Health and Neuroscience Clinical Trials Unit, a unit independent of the study to maintain concealment. The intervention condition was an IPS programme integrated within community mental health teams provided by a well-established not-for-profit non-governmental supported employment agency. The intervention involved linking four experienced employment specialists (two for each borough) with community mental health teams. In addition to seeing clients, the employment specialists attended team meetings and multidisciplinary care planning meetings, and met with care coordinators whenever appropriate. They focused on rapid placement with continued follow-up support and sought to find employment opportunities that were consistent with participants' preferences, skills and abilities. However, the integration of the IPS programme was not structural or managerial, reflecting supported employment as it is provided in the vast majority of settings in the UK, which is the current provision referred to by the UK government in its implementation of IPS. ⁹

The control condition (treatment as usual, TAU) consisted of existing psychosocial rehabilitation and day care programmes available in the local area. There were 33 such services in the catchment areas of the RCT; these were visited or telephoned and a senior manager was asked about their operational characteristics. Each service employed, on average, 7 vocational staff, had a median of 58 places available to clients (range 6–3000) and 79% of services received referrals from clinical teams. A range of courses were offered, most commonly pre-employment preparation (e.g. interview skills, curriculum vitae coaching and application form practise), computers/information technology and confidence building/motivation.

The participating community mental health teams were given presentations on the study with opportunities for questions and answers and written information on the study, and employment consultants working with the community mental health teams would regularly explain the study when they attended team meetings and when addressing individual questions and concerns. Teams were also regularly sent newsletters with updates on study progress and examples of success in obtaining employment. Written information on the study was revised when the study team were aware of specific issues that needed to be addressed (e.g. concerns about equipoise or the referral process).

Statistical and health economic analysis

All data were analysed using SPSS for Windows (version 15.0). The primary and secondary hypotheses were tested on the whole group. For comparing groups, t-tests and χ²-tests were used to compare means and proportions respectively, unless the data were highly skewed, in which case non-parametric tests were used. Logistic and linear regression models were also fitted including potential confounding variables: (grouped) age, gender, ethnic group, educational level, symptomatology and diagnosis. All data were analysed in groups as randomised, whether or not receiving an intervention (i.e. intention-to-treat). Data were compared for those followed up with those not followed up, overall and by each treatment arm.

The amount of time spent by employment workers with clients was recorded and this was multiplied by their hourly cost. This was based on annual costs per employment worker of £40 000, a working week of 35 h and 42 weeks per year, and a ratio of direct to indirect client time of 0.67 (this ratio was not directly available from the employment services and therefore was based on the ratio of occupational therapist direct to indirect client time ^{Reference Curtis19} and used to calculate the cost of the IPS intervention). The resulting cost was £45 per hour. The costs of other services were calculated by combining service data with appropriate unit cost information. ^{Reference Curtis19} Total costs at follow-up were compared between the two groups using a regression model with the group indicator as the single independent variable. Confidence intervals around the cost difference were estimated using bootstrapping with 10 000 resamples due to the likely skewed cost distribution.

Process data

Individual placement and support fidelity was found to be high; the IPS programme in the two boroughs received a good IPS rating of 67 (Borough A) and 69 (Borough B); the IPS programme scored less well on the organisation dimension (specifically, integration of rehabilitation with mental health treatment) and, for Borough A, the services dimension (specifically, for the rapid search for a competitive job) compared with other dimensions. Of the 109 patients in the intervention arm, 73 (67%) engaged with staff (i.e. had at least one direct contact with an employment consultant); of these individuals, the mean number of contacts with or on behalf of clients was 14 (s.d. = 10).

The researchers who conducted the 1-year follow-up interviews were masked to allocation status, but guessed 119 correctly out of the 197 (60%) clients assessed compared with a hypothesised 50% (with random guesses); this is significant at P = 0.005 (single sample, double-sided test of a proportion).

One-year follow-up

One protocol deviation was noted in that one participant randomised to TAU received the intervention. Out of 219 individuals recruited, 20 (9%) reported having worked between baseline and the 1-year follow-up, in jobs that met the competitive employment criteria. Of those followed up, 7 out of 99 (7%) were from the control group and 13 out of 98 (13%) were from the intervention group (risk ratio 1.35, 95% CI 0.95–1.93, χ²(1) = 2.07, P = 0.15). Controlling for variables associated with loss to follow-up (BPRS and GAF) made no difference to the conclusions (P = 0.127). Sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, but this made little difference (P = 0.288 and P = 0.153 respectively). If the primary outcome had been alternatively defined (as in some other studies ^{Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7} – working at least 1 day) there would still have been no significant difference in obtaining competitive employment: 22/197 (11%) of participants worked in a job for at least 1 day, 8/99 (8%) from the control group and 14/98 (14%) from the intervention group (χ²(1) = 1.911, P = 0.167). When sensitivity analyses were conducted assuming the 22 participants who were lost to follow-up had either all worked or all not worked, this made little difference (P = 0.296 and P = 0.170 respectively).

The median number of hours worked per week was not significantly different in the control group (32.5 h, interquartile range (IQR) 4–45, range 3–63 for 7 jobs obtained) and the intervention group (8 h, IQR 4–16, range 3–50 for 15 jobs obtained) (Mann–Whitney Z = –1.45, P = 0.15). Similarly, the length of job tenure (median 13 weeks, IQR 8–15, range 4–26 for control group; 18 weeks, IQR 7–30, range 4–52 for intervention group; Mann–Whitney Z = –1.20, P = 0.23) and the median salary (£7.73, IQR 6.15–14.18, range 6–16 for control group; £6.00, IQR 5.35–7.20, range 4.85–30 for intervention group; Mann–Whitney Z = 1.48, P = 0.14) were not significantly different in the two groups. Out of the 20 participants who had worked in competitive employment in the 1-year follow-up, 2 had worked in two different competitive employment roles. The hypothesis that the intervention would increase net contributions to the national economy is closely linked to the hypotheses regarding net earnings and hours worked. Higher earnings would potentially lead to increased tax revenues. However, because there was no significant difference in earnings there is also no significant net gain to the economy.

Within the intervention arm, there was a significant difference in employment outcome by borough: within Borough A, 1 participant (3%) obtained competitive employment, whereas within Borough B, 12 participants (17%) obtained competitive employment (Fisher's exact test P = 0.03). At 1-year follow-up there were no adverse events for any clients.

Forty-five per cent (10 of 22) of the jobs gained at 1-year follow-up were elementary occupations (e.g. bar work, kitchen porter); 23% (5 of 22) of jobs were sales and customer service occupations; 14% (3 of 22) were administrative and secretarial occupations, 9% (2 of 22) were professional occupations. Other jobs obtained were skilled trade's occupation (n = 1), and process, plant and machine operative (n = 1).

Table 3 shows there were no significant differences in secondary outcomes when comparing outcomes by randomisation group. We also found no significant differences in outcomes for participants who had obtained work and those who had not.

Table 3

Clinical outcome for participants by randomisation status

	Intervention		Control		Statistical test	P
		IQR		IQR
Camberwell Assessment of Need unmet needs (n = 195), median (range)	1 (0.9)	0-3	1 (0-9)	0-3.5	Z = -0.409	0.68
Rosenberg Self Esteem questionnaire (n = 196), mean (s.d.)	2.26 (0.53)		2.25 (0.55)		t(194) = -0.124	0.90
Overall Manchester Short Assessment (MANSA) (n = 195), mean (s.d.)	4.03 (0.82)		3.95 (0.87)		t(194) = -0.705	0.48
Work	3.05 (1.50)		2.85 (1.39)		t(193) = -0.988	0.32
Finance	3.29 (1.64)		3.39 (1.58)		t(194) = 0.443	0.66
Leisure	3.44 (1.56)		3.47 (1.56)		t(194) = 0.137	0.89
Social life	4.39 (1.39)		4.30 (1.40)		t(189) = -0.424	0.67
Indiana Job Satisfaction Scale Reference Resnick and Bond16 (n = 16),a mean (s.d.)	2.08 (0.53)		1.78 (0.45)		t(14) = -1.10	0.29
Brief Psychiatric Rating Scale (n = 195), median (range)	36 (24-80)	29-44	37 (24-58)	30-44	Z = -0.484	0.63
Global Assessment of Functioning (n = 196), mean (s.d.)	53.34 (14.75)		52.46 (13.16)		t(194) = -0.440	0.66

Regression analyses were applied to control for ethnicity, age, gender, diagnosis, symptom level and level of functioning in order to test if there were any factors that were masking a difference between the two treatment arms in any of the outcomes. However, none of the possible masking or confounding factors were found to be significant, and there were no significant interactions between these variables and treatment group.

Service use and costs

During the 1-year follow-up, two-thirds of the intervention group received input from employment workers at mean cost of £296 (Table 4). There were not substantial differences in the number of people using other services. However, control group participants who were admitted spent substantially more days in hospital than in-patients in the intervention group. This resulted in a difference in in-patient costs of £1522. Total costs were £2176 significantly higher in the control group (bootstrapped 95% CI £445–4168).

Table 4

Service use and costs at 1-year follow-up

	Control						Intervention
	Using service		Contactsa		Costb		Using service		Contacts Reference Priebe, Huxley, Knight and Evans11		Cost Reference Thornicroft, Tansella, Becker, Knapp, Leese and Schene2
	n	%	Mean	s.d.	Mean	s.d.	n	%	Mean	s.d.	Mean	s.d.
Received individual placement and support	1	1	12.0	-	5	47	73	67	9.3	7.8	296	416
Psychiatric in-patient	13	13	80.4	55.2	2241	7111	10	10	33.9	22.1	719	2533
Psychiatrist	45	46	1.7	2.0	190	390	33	34	2.0	2.4	117	229
Other doctor	13	13	2.8	2.7	31	114	20	20	1.8	2.0	30	98
Day care/education	35	35	18.8	20.0	320	858	35	36	10.9	12.9	149	424
Psychologist/therapy	4	4	5.8	5.9	21	139	11	11	5.5	5.6	67	320
Social care	7	7	4.6	4.0	17	90	8	8	5.9	4.9	25	111
General in-patient	5	5	27.2	26.7	323	1893	4	4	6.3	3.8	60	331
General practitioner	52	53	2.2	1.8	55	91	48	49	3.0	4.4	58	99
District nurse	1	1	3.0	-	< 1	1	5	5	4.2	1.8	2	11
Community mental health nurse	59	60	4.6	2.9	65	99	55	56	4.1	2.7	49	77
Occupational therapist	2	2	2	2.8	1	6	3	3	4.3	2.9	4	27
Total				5701	8677					3525	3854

Main findings

In this largest RCT of supported employment in the UK to date, there was no evidence of significant benefit of the IPS model in helping people with severe mental illness obtain competitive employment. The study context – a socioeconomically deprived area with a somewhat rigid welfare benefits system, and an intervention provided by an external agency – may have made successful implementation of the IPS model more challenging. Although in this sample the IPS model achieved higher rates of competitive employment compared with standard services, this was not significant, and it is striking that in both arms the rates of competitive employment were very low (considerably lower than the rates reported for control groups in other studies). This suggests that even with a larger sample size that may have allowed us to detect a significant difference between the intervention arm and TAU, a very large number of people would need to receive the intervention in order to achieve relatively low rates of competitive employment.

Although we did not find evidence that IPS was effective in this study, we did not find the intervention arm to be significantly more expensive than the control arm; in fact, the control group spent more days in hospital and therefore was significantly more expensive (although this is as a result of a small number of outliers and has therefore skewed the results). This may be a chance finding or may reflect the supportive nature of the intervention, which may prevent in-patient admissions.

Successful effective implementation of IPS may be context-dependent, and it may be difficult to achieve even moderate rates of competitive employment in some settings. Our results contrast markedly with previous results reported in RCTs of the IPS model and there are a number of possible explanations for our findings. First, the IPS model may be less helpful in non-USA settings – a recent review of RCTs of supported employment found that two of the three studies with the lowest employment rates were non-US studies, ^{Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7,Reference Latimer, Lecomte, Becker, Drake, Duclos and Piat20,Reference Drake and Bond21} although some non-US studies have found IPS to be effective (e.g. Latimer et al). ^{Reference Latimer, Lecomte, Becker, Drake, Duclos and Piat20} These findings may be because of the different nature of the labour market; the lack of employer incentives in employing individuals with severe mental illness outside the USA; payment for healthcare in the USA, which may increase the pressure to find work in US settings; and the differences in welfare benefit systems found in different countries – Burns et al's EQOLISE study ^{Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7} found that the degree of economic disincentives for working across several countries predicted the success of IPS programmes. They also reported considerable recruitment difficulties, despite carrying out the study in a relatively more affluent area of London (Richmond) that is likely to have more entry-level jobs; this suggests that some of these disincentives existed at the London site in their study. However, whereas local unemployment rates do predict vocational outcomes for people with severe mental illness, ^{Reference Cook, Mulkernb, Greya, Millera, Blylerc and Razzanoa22} there is evidence from the USA that even in areas with high unemployment, people who receive IPS have superior outcomes to those in the control condition, ^{Reference Cook, Mulkernb, Greya, Millera, Blylerc and Razzanoa22} so local unemployment rates are unlikely to be the main reason for our results.

Second, the same review ^{Reference Drake and Bond21} found lower employment rates in a trial that did not require clients to have a goal of competitive employment. ^{Reference Resnick and Bond16} Whereas our eligibility criteria included individuals only if they wanted to obtain competitive employment we did not test participants' motivation or restrict access to the trial to people with high levels of motivation. However, the lack of engagement with employment consultants that occurred in a quarter of the participants suggests that participants may have been less motivated compared with those in other trials. Recent studies have suggested that the most robust predictors of work outcomes include recent employment history, motivation and self-efficacy. ^{Reference Rinaldi, Perkins, Glynn, Montibeller, Clenaghan and Rutherford23} Ethnographic research suggests that people who say they would like to work can be categorised as ‘active job-seekers and passive job-seekers’, and the latter make little progress in obtaining a job. ^{Reference Alverson, Carpenter and Drake24} We may therefore have demonstrated greater effectiveness of the intervention if we had selected only participants who were ‘active job-seekers’; 25% of participants obtained employment in the UK Sesami study, where participants were self-selected and had already engaged with supported employment programmes. ^{Reference Schneider, Slade, Secker, Rinaldi, Boyce and Johnson25} Indeed, some of the American trials have held two participant consent meetings, to screen out those who are less highly motivated to participate, whereas we held one consent meeting only. These findings suggest that IPS programmes should routinely include tests of motivation before entry into the programme.

Third, the IPS programme was not structurally or managerially integrated with community mental health services and these may be important components that determine the effectiveness of IPS. Previous research has found that the degree to which integration of IPS with mental health services occurs predicts success. ^{Reference Drake and Bond21} It may therefore be relevant that the borough with the higher fidelity rating (i.e. where IPS was more successfully implemented, although still without structural or managerial integration) had better employment rates than the other borough. We have not been able to find many studies that report data on the number of contacts between IPS staff and clients to compare with our other process data, but the two studies (of 11 RCTs included in a recent review) ^{Reference Bond, Drake and Becker4} with such data report higher numbers of contacts, which also suggests IPS implementation may not have been optimal. Other aspects of IPS implementation, which are not captured by the IPS fidelity scale, such as the nature of the relationship between the IPS worker and the client (which was an independent predictor of outcome in the EQOLISE study) ^{Reference Catty, Lissouba, White, Becker, Drake and Fioritti26} may also explain the poorer outcomes found in this study. A recently developed modified IPS fidelity scale (IPS+) also includes rating essential processes such as consumer choice, ^{Reference Paulson, Post and Herinckx27} and the IPS fidelity rating carried out in this study may have been lower if the IPS+ had been used.

Fourth, the relatively high proportion of participants from ethnic groups other than White (62%) could also limit the success of IPS in the UK, as people from other ethnic groups are more likely to be unemployed in the UK. ²⁸ We also used an inclusion criteria of a maximum level of functioning of 60 (as measured by the GAF scale) in order to examine the effectiveness of IPS in people with severe mental illness. However, this means that our study population may have been more impaired than those recruited in other studies. In addition, qualitative research reported elsewhere ^{Reference Howard, de Salis, Tomlin, Thornicroft and Donovan29} found that the clinicians involved in this RCT had idiosyncratic interpretations of the eligibility criteria used for the trial (e.g. some staff would not refer individuals with positive symptoms to the study team even though this was not an exclusion criterion). This may have led to motivated potential participants not being referred and entering the trial, as staff wished to avoid the ‘stress’ of job-seeking in case this led to relapse. Finally, attitudes towards employing people with severe mental illness may differ in different contexts; there is evidence that experienced and anticipated discrimination do vary across countries, ^{Reference Thornicroft, Brohan, Leese and Sartorius30} but there are no current data on international differences in discrimination for employment.

Strengths and limitations

Strengths of this study include the largest sample size for an RCT of IPS in the UK, the comparatively diverse group of participants and high levels of retention at follow-up. The limitations include the pragmatic nature of this trial in a ‘real world’ setting in a socially deprived inner-city catchment area where most people with severe mental illness do not obtain employment, and as is typical in the UK, the IPS programme was provided by an external supported employment agency. Such well-established but financially insecure non-governmental agencies may have less IPS-focused staff than new staff employed to set up IPS programmes (as in previous RCTs) where the enthusiasm and commitment of staff is likely to be particularly high. Despite the high IPS fidelity rating, the separation of the supported employment agency from mental health services may also have led to differences in the intervention received by participants in this study compared with that delivered in more fully integrated models of care.

Other studies have not usually reported on rates of engagement with IPS so it is difficult to compare data but the failure to engage all participants may reflect an individual's lack of motivation, severity of illness or staff attitudes. Mental health professionals involved had low expectations of their clients' ability to obtain competitive employment, as reported elsewhere. ^{Reference Howard, de Salis, Tomlin, Thornicroft and Donovan29,Reference Boardman, Grove, Perkins and Shepherd31,Reference Thornicroft32} The UK site of the only European RCT of IPS also found recruitment difficult and slow, ^{Reference Burns, Catty, Becker, Drake, Fioritti and Knapp7} suggesting these factors are not unique to our study. There is evidence from other UK studies that mental health professionals do not focus on vocational needs ^{Reference Bertram and Howard33} and may actually advise people not to work. ^{Reference Rinaldi and Hill34}

There was evidence for researchers guessing allocation arm but as the primary outcome was derived from administrative methods in addition to participant and healthcare professional recall this is unlikely to affect the main findings. The methods to calculate patient costs relied on recall of service utilisation. There may be some concerns about recall accuracy but this is an accepted method (especially in the absence of databases designed to measure service use for insurance systems) and previous studies have shown the method to be acceptable. ^{Reference Calsyn, Allen, Morse, Smith and Templehoff35,Reference Goldberg, Seybolt and Lehman36}

Implications

This study therefore suggests that implementing IPS in the context studied here can be challenging. Even in the studies carried out in North America, IPS is successful in obtaining employment for only around 50% of people with serious mental illness in the total population of people with disablility. Thus it is not clear that IPS alone can overcome an individual's reluctance to join the workforce, deficits in social skills and neurocognitive impairments that militate against enduring employment, ^{Reference Liberman37} and research into additional interventions such as motivational interviewing, cognitive interventions and benefits counselling ^{Reference Drake and Bond21} may help identify further ways of helping people with severe mental illness into work. We conclude from our study that IPS will not be the only intervention necessary to improve social inclusion for people with severe mental illness. Alternative or additional interventions such as motivational interviewing and/or cognitive remediation may be needed to help obtain work; for those who do not obtain work, other measures will be necessary to promote social inclusion.