Surveys indicate that most people with severe mental illness would like to work (Reference Hatfield, Huxley and MohamadHatfield et al, 1992; Reference LehmanLehman, 1995), but only about 10-20% actually do so, at least in Europe (Marwaha et al, 2004). Individual placement and support, a form of supported employment for people with severe mental illness (Reference Becker and DrakeBecker & Drake, 2003), has been shown in several experimental studies to be more effective than traditional approaches at enabling people with severe mental illness to work in competitive settings (Reference Drake, McHugo and BeboutDrake et al, 1999; Reference Crowther, Marshall and BondCrowther et al, 2001; Reference Lehman, Goldberg and DixonLehman et al, 2002; Reference Twamley, Jeste and LehmanTwamley et al, 2003; Reference Mueser, Clark and HainesMueser et al, 2004). Randomised trials published to date, however, have all been carried out in the USA. Cross-country differences in rules governing income and benefits for people with disabilities, employment opportunities and other factors could influence the relative effectiveness of supported employment.
This article reports on a randomised controlled trial of supported employment conducted in Montreal, Canada. Our main hypothesis was that supported employment would remain more effective than standard vocational rehabilitation services in terms of competitive employment outcomes. We also hypothesised that competitive employment might improve symptoms, quality of life, self-esteem, social support, client functioning and substance misuse outcomes.
METHOD
Study setting
The study was carried out at the Douglas Hospital, a teaching psychiatric hospital in Montreal, Canada. The hospital provides psychiatric treatment and rehabilitation services to people who live within a geographical sector in the south-west of Montreal. In addition to one central facility, which provides in-patient as well as various out-patient services, the hospital operates seven satellite out-patient clinics and a vocational rehabilitation centre located about 1 mile from the main facility.
Planned interventions
Supported employment
Participants allotted to the supported employment condition were assigned an employment specialist attached to their clinical service. The employment specialist helped the client to:
-
(a) define a competitive job corresponding to his or her interests and capabilities;
-
(b) obtain such a job;
-
(c) continue in employment, once a job was obtained;
-
(d) recover from job loss, identifying what went wrong and looking for a new employment opportunity.
Employment specialists were integrated into their client's clinical team and thus could easily communicate with the client's case manager or psychiatrist to coordinate services. Each of the four employment specialists was assigned to two out-patient services, among a set of eight: four of the seven satellite out-patient clinics, an assertive community treatment team, a programme to support clients living in foster homes, the hospital's general out-patient department and a case management programme for people who had been discharged from a residential programme. Each had a case-load of about 20. Employment specialists met weekly as a team with their supervisor to discuss challenges with particular people on their case-loads, help each other through exchanging job leads, etc.
Employment specialists received initial training in the model, in some cases including on-site visits to successful USA programmes in Hartford, Connecticut, and Lebanon, New Hampshire. For approximately the first 1.5 years of operation, employment specialists also received ongoing telephone and in-person consultation with one of the authors (D.B.).
The decision to implement a supported employment programme was made by the hospital administration. The research team was assembled subsequently, and had no influence over engagement and retention of supported employment staff, decisions which were in any event constrained by union rules.
Usual vocational services
Clients assigned to the control group were invited to an interview at the hospital's vocational rehabilitation centre. There they were given an opportunity to sign up for one of the many vocational services normally available. These included sheltered workshops, creative workshops, a client-run boutique and horticultural programmes. Job-finding-skills training, as well as psychosocial interventions administered through two day-treatment centres, were also available. None of these programmes had competitive employment as their immediate goal.
In addition, clients could be offered a social integration measure, that is a Québec government programme that offers clients part-time work in competitive settings, in exchange for a Can$120 top-up to their monthly welfare cheque and a free public transport pass. Finally, clients could also be referred to a non-profit community agency that sought to place clients either in competitive jobs or in government-subsidised adapted businesses, in which wages equal or exceed the legal minimum wage but where the majority of jobs are reserved for people who have disabilities. This agency was not integrated with clinical services, nor did it provide ongoing support to clients, two hallmarks of supported employment.
After their initial interview at the hospital's vocational centre, clients assigned to the control condition were left to avail themselves of this array of services as they chose, reflecting usual practice at the hospital.
Study participants
Study recruitment methods were based on those suggested by Drake et al (Reference Drake, Becker and Anthony1994). To participate in the study, clients had to attend two introductory meetings, offered at various times and locations each week between January 2001 and February 2003. The meetings were advertised through clients' case managers, flyers posted on bulletin boards and pizza parties targeting all eligible clients at specific sites. Thus, any individuals interested in working could enter the study, regardless of their case managers' beliefs concerning their suitability for work. At the conclusion of the first introductory meeting, clients who were potentially interested in participating in the study were asked to sign a consent form allowing the research team to check their eligibility. Individuals who met study inclusion and exclusion criteria (see below), and who were able to answer a set of questions concerning the study, were invited to sign the consent form indicating their willingness to participate at the conclusion of their second introductory meeting. Clients who agreed that their eligibility could be checked, but who did not proceed to sign up for the study, did not differ from study participants on any of the variables used as inclusion or exclusion criteria.
Inclusion and exclusion criteria
To be included in the study, individuals had to:
-
(a) be between 18 and 64 years of age;
-
(b) have a diagnosis of schizophrenia-spectrum disorder (schizophrenia, schizoaffective disorder, schizophreniform disorder, psychosis not otherwise specified), bipolar disorder, or major depression;
-
(c) if their principal diagnosis was one of major depression, be classified as disabled due to mental illness by the provincial welfare system;
-
(d) express interest in competitive employment; and
-
(e) be unemployed at the time of signing the consent form.
Individuals were excluded who:
-
(a) had a learning disability (IQ<70);
-
(b) had a physical or organic handicap that seriously impeded work; or
-
(c) did not have a case manager willing to see them at least once per month.
Randomisation procedure
Stratified randomisation was used, with two factors that were expected to influence vocational outcomes: previous work history (⩾1 year of continuous work experience at some point in the past, or less) and clinical site. Earlier studies suggested that previous work history could influence success in finding work (Reference Drake, McHugo and BeckerDrake et al, 1996a ; Reference Bond, Becker and DrakeBond et al, 2001). The varying intensities of different clinical teams (from multiple contacts per week at the assertive community treatment team to as few as one per month at the satellite out-patient teams), and the fact that different employment specialists were assigned to different clinical teams, were thought likely to influence outcomes. The biostatistician associated with the study (H.X.) generated 16 random assignment sequences (one for each study site × work history combination) and emailed them to the project secretary. The secretary who kept the 16 sequences indicated to the project coordinator (I.D.) the group to which a new study participant was to be assigned, according to that person's clinical site and work history. The project coordinator then prepared an opaque envelope containing the assignment and gave it to the interviewer before the baseline interview. Assignment was revealed (to both interviewer and participant) at the conclusion of the baseline interview.
Masking
Because of the nature of the intervention and of some of the questions asked, we did not attempt to mask interviewers to group assignment. Interviewers were not, however, reminded or told (in the case of a change in interviewer subsequent to the baseline interview) of a client's group assignment. Interviewers were aware of the main hypotheses the trial was intended to test, but they had no stake in the outcome of the study.
Contamination
No preliminary results were released, including to supported employment staff, until data collection had been completed, in order to minimise the risk of contamination. Supported employment staff did, however, keep track of their own employment outcomes.
Measures
Comprehensive measures were obtained, from clients and staff, concerning vocational and non-vocational domains. Face-to-face interviews were conducted with clients and staff at baseline, 6 months and 12 months. In addition, telephone interviews to assess vocational outcomes were conducted at 2-month intervals.
Diagnostic and background information
All study clients had one or more psychiatric diagnosis in their hospital records. Trained research assistants reviewed patient charts using the Structured Clinical Interview for DSM-IV Disorders checklist (SCID; Reference First, Spitzer and GibbonFirst et al, 1997) to validate the principal diagnosis. Other background data (demographics, work experience during the 5 years before study entry) were obtained during the baseline interview.
Employment outcomes
Clients were interviewed at 2-month intervals to ascertain job start and end dates, hours per week, salary conditions and type of work. Corroborating information that could have been obtained from supported employment staff was not used, so as to render data collection procedures the same for the two groups - there was no equivalent corroborating source of information for control participants. Jobs were categorised as competitive if:
-
(a) they paid the minimum wage or better, or on a commission basis (e.g. sales);
-
(b) they were not reserved for people with disabilities; and
-
(c) fewer than 50% of the person's coworkers had disabilities (information ascertained by contacting the employer directly).
Self-employment (one client in the usual-services group attempted to start a bicycle repair shop) was also classified as competitive. Other jobs were classified as non-competitive. Job type was coded according to the Dictionary of Occupational Titles (US Department of Labor and Employment and Training Administration Affairs, 1991).
Non-vocational outcomes
Quality of life, social network, self-esteem and psychiatric symptoms were assessed by means of interviews with clients conducted at baseline, 6 months and 12 months. The Canadian version of the Wisconsin Quality-of-Life scale contains 58 items grouped into 8 subscales: general life satisfaction, activities and occupations, psychological well-being, symptoms, physical health, social support, finances and activities of daily living (Reference Becker, Diamond and SainfortBecker et al, 1993; Reference Diaz and MercierDiaz & Mercier, 1996; Reference Caron, Corbière and MercierCaron et al, 2003). The total score is scaled between -3 and 3, a higher number indicating better self-perceived quality of life. Social network information was assessed using a standard 24-item scale, the Social Provision Scale (Reference Cutrona and RussellCutrona & Russell, 1987), the items of which fall into 6 subscales: attachment, social integration, reassurance of worth, material support, advice and information, and need to feel useful. The Self-Esteem Rating Scale (Reference Nugent and ThomasNugent & Thomas, 1993; Reference Lecomte, Corbière and LaisnèLecomte et al, 2006) contains 40 items each on a 7-point scale. The total score can be positive or negative, with a higher score indicating greater self-esteem. Interviewers (eight in total over the course of the study) were trained to carry out the 24-item Brief Psychiatric Rating Scale (BPRS; Reference Ventura, Green and ShanerVentura et al, 1993; Reference Miller, Faustman, Sederer and DickeyMiller & Faustman, 1996). Twenty-five BPRS interviews were carried out jointly between interviewers and the project coordinator to assess interrater reliability. Interrater reliability was very good, with intraclass correlation coefficients ranging between 0.96 for the negative symptom subscale and 0.98 for the positive symptom and depression anxiety subscales, and reaching 0.99 for the total score.
Overall functioning was evaluated through interviews with clinicians rather than clients, using the Global Assessment of Functioning (GAF) scale (American Psychiatric Association, 1994) as well as the Multnomah Community Ability Scale (MCAS; Reference Barker and BarronBarker & Barran, 1997; Reference Corbière, Crocker and LesageCorbiére et al, 2002). The GAF is a one-dimensional scale ranging between 0 and 100, with higher scores reflecting better adjustment. The MCAS has 17 items, which yield 4 subscales: interference with functioning, adjustment to living, social competence and behavioural problems. To assess substance misuse, clinicians also completed with the interviewer two one-dimensional rating scales: the Alcohol Use Scale (AUS) and Drug Use Scale (DUS) (Reference Drake, Mueser, McHugo, Sederer and DickeyDrake et al, 1996b ). Each of these scales has five levels, based on the DSM criteria for alcohol and drug use disorders, ranging from abstinence to severe dependence (dependence with institutionalisation) (Reference Drake, Mueser, McHugo, Sederer and DickeyDrake et al, 1996b ).
Programme fidelity
This was assessed on two occasions using the Supported Employment Fidelity Scale (Reference BondBond, 1997), 11 months and 2.5 years after the programme was initiated. The two ratings were consensus ratings between two different pairs of investigators (D.B. and E.L.; N.L. and E.L.), and were based on information from the programme manager and direct observation of programme functioning. Both ratings indicated good implementation of individual placement and support (72 and 70.5, out of 75).
Research attrition, programme exposure and programme retention
Figure 1 shows the flow of study participants through the trial. Interview completion rates were not significantly different at either 6 months or 12 months.
We assessed programme exposure using administrative hospital data. Individuals were considered to have received supported employment services (or usual services) if they were recorded as having had at least one contact with supported employment staff (or usual services staff) over the first and the second 3-month follow-up periods. Using this criterion, 68 study participants were exposed to supported employment (91%), and only 22 (30%) to usual services.
Retention in supported employment was determined according to whether clients received any services from their employment specialist during a 6-month period. Retention in supported employment was 100% during the first 6 months, and 92% during the final 6 months. Since usual vocational services consisted of people accessing whatever services they were interested in, we considered that all individuals who were assigned to usual services remained in them until the end, unless they moved from the geographical area served by the Douglas hospital. Using this criterion, retention in usual services was 97% over the first year.
Statistical analyses
Baseline comparisons
In order to assess baseline equivalence of the groups, proportions of categorical variables at baseline were compared according to initial group assignment using χ2-tests. Values of continuous measures were compared using either the t-test or, for non-normally distributed variables, the (non-parametric) Mann-Whitney U-test.
Vocational outcomes
Intention-to-treat analyses were conducted first, including all individuals and time periods for which we had data. Groups were compared on measures pertaining to any paid work (competitive or not) and competitive work only.
In order to compare employment trends, we estimated generalised linear mixed models, treating employment as a dichotomous variable (either some paid employment or none during successive 1-month periods, either some competitive employment or none), using a logit link function with a binomial distribution. These models were estimated using maximum likelihood methods, with the likelihood based on available data. Observations with one or more missing values were then kept in the analysis, but there was no imputation of missing values (Reference VoneshVonesh, 1992; Littel et al, 1996).
Progress through stages of the trial. R, indicates randomisation. Estimate for introductory session was based on numbers collected during recruitment of one-third of participants.
The models were estimated with and without a set of adjusters, including standard demographic variables, diagnosis, the two stratification variables and self-esteem at baseline. The latter variable was included because, as indicated below, it differed significantly between the two groups.
Although research attrition was not significantly different between the two groups, it was somewhat greater in the supported employment group (Fig. 1). It is reasonable to suppose that individuals who left supported employment during the course of the study had relatively poor subsequent employment experiences. Accordingly, as a sensitivity analysis, we re-estimated the models assuming that missing values for employment status were all zero.
The models were then re-estimated, including only individuals who had some exposure (as described above) to the supported employment model or usual services.
Because of the large number of zeroes and the positive skews of the distributions, we used Mann-Whitney U-tests to compare total hours worked in employment (paid or competitive), and total earnings (paid or competitive). Finally, we used a χ2-test to compare the distributions of jobs according to type of job, as classified using the first digit of the Dictionary of Occupational Titles codes, and Mann-Whitney U-tests to compare job duration by job type.
Non-vocational outcomes
We also estimated mixed effects regression models to evaluate the effect of group assignment on non-vocational outcomes. Group assignment, time (treated as a continuous variable) and a time × group interaction term were used to predict repeated measures of the non-vocational variables. These variables enabled us to estimate, respectively, the average effect over time of group assignment on each vocational variable, the time trend for each group following baseline, and any difference in the time trends between the two groups. These models were also estimated with and without adjusters, using the same set of adjusters as before, plus the baseline value of the dependent variable.
Analyses were carried out using the Statistical Package for the Social Sciences for Windows, version 11.5, except for estimation of mixed-effects models, which relied on the PROC GLINMIX routine in, SAS for Windows, version 9.1.
Ethical approval
The research protocol and consent forms were approved by the Douglas Hospital Research Ethics Board.
RESULTS
Table 1 compares the two groups at baseline in terms of sociodemographic and other measures. Using Simes' correction for comparisons of multiple outcomes, which is less conservative but more accurate than the Bonferroni correction (Reference SimesSimes, 1986; Reference Samuel-CahnSamuel-Cahn, 1996), only self-esteem emerges as statistically different between the two groups: it is markedly lower in the experimental group. Results not shown indicate that this difference arose from many observations, not a few outliers, and that the difference continued to manifest itself throughout the recruitment period.
Baseline comparability of the supported employment (n=75) and control (n=74) groups
| Measure | Usual services | Supported employment | P |
|---|---|---|---|
| Gender, % male | 60.8 | 62.7 | 0.82 |
| Age (years), mean (s.d.) | 40.6 (11.0) | 39.9 (9.0) | 0.54 |
| Ethnicity, % White | 79.7 | 84.0 | 0.50 |
| Education > 12 years, % | 47.3 | 38.7 | 0.29 |
| Single, never married, % | 78.4 | 80.0 | 0.81 |
| Diagnosis, % | 0.12 | ||
| Schizoaffective | 13.5 | 20.0 | |
| Other schizophrenia spectrum | 54.1 | 64.0 | |
| Bipolar disorder | 27.0 | 13.3 | |
| Other | 5.4 | 2.7 | |
| Clinical service, % | 1.0 | ||
| Satellite out-patient | 47.3 | 45.3 | |
| (less-intensive services) | |||
| Assertive community treatment | 9.5 | 12.0 | |
| Residential services | 21.6 | 21.3 | |
| Main out-patient | 6.8 | 6.7 | |
| Other case management | 14.8 | 14.7 | |
| Quality of life score, mean (s.d.) | 0.89 (0.92) | 0.74 (0.82) | 0.28 |
| Social provision scale global score, mean (s.d.) | 72.0 (9.0) | 69.0 (8.9) | 0.04 |
| Self-esteem rating score, mean (s.d.) | 48.2 (37.2) | 25.6 (30.8) | <0.001 |
| BPRS global score, mean (s.d.) | 43.3 (15.5) 1 | 41.3 (12.3) 2 | 0.74 |
| MCAS global score, mean (s.d.) | 68.8 (9.8) 3 | 71.2 (8.11) 4 | 0.22 |
| GAF score, mean (s.d.) | 58.6 (12.9) 5 | 62.0 (12.4) 4 | 0.14 |
| Alcohol Use Scale, % | 0.66 | ||
| Abstinent | 65.7 6 | 75.4 4 | |
| Use without impairment | 29.9 6 | 21.1 4 | |
| Misuse | 1.5 6 | 1.6 4 | |
| Dependence | 3.0 6 | 1.6 4 | |
| Drug use scale, % | 0.85 | ||
| Abstinent | 77.6 6 | 79.0 7 | |
| Use without impairment | 10.4 6 | 12.9 7 | |
| Misuse | 9.0 6 | 6.4 7 | |
| Dependence | 1.5 6 | 1.6 7 | |
| Severe dependence | 1.5 6 | 0 7 | |
| Competitive work in past 5 years, % | 33.8 | 28.0 | 0.44 |
| Paid non-competitive work in past 5 years, % | 35.1 | 48.0 | 0.11 |
BPRS, Brief Psychiatric Rating Scale; MCAS, Multnomah Community Ability Scale; GAF, Global Assessment of Functioning Scale
1. Data missing for 1 participant
2. Data missing for 4 participants
3. Data missing for 8 participants
4. Data missing for 14 participants
5. Data missing for 9 participants
6. Data missing for 7 participants
7. Data missing for 13 participants
Figure 2a shows actual percentages of clients in each group who had at least some paid employment during each successive month of the study, and Fig. 2b illustrates the same for competitive employment only. The figures also show 95% confidence intervals, calculated independently for each month, using the method of Agresti & Coull (Reference Agresti and Coull1998). (Actual percentages at each month are shown; Agresti & Coull intervals are not quite symmetrical around the actual percentages.) Although there is considerable overlap in the confidence intervals for any paid work, and some overlap for competitive employment, supported employment clients exhibited higher employment rates in each of the 12 months. The difference is much more pronounced for competitive employment.
Table 2 compares the two groups on employment outcomes. The differences on competitive employment outcomes clearly favour supported employment; those for any paid work favour supported employment but are not statistically significant.
Differences in outcomes for any paid employment between usual services and supported employment
| Variable | Usual services (n=74) | Supported employment (n=75) | P |
|---|---|---|---|
| Any paid job over 12 months, % | 52.7 | 68.0 | 0.06 |
| Worked more than 250 h over | 29.7 | 33.3 | 0.63 |
| 12 months, % | |||
| Worked 20 h or more for at least 4 weeks, % | 17.6 | 24.0 | 0.33 |
| Total hours worked, per person, mean (s.d.) | 223.9 (371.7) | 287.4 (406.9) | 0.11 |
| Total wages earned (Can$), per person, mean (s.d.) 1 | 1022.8 (2434.0) | 1296.8 (2575.9) | 0.06 |
| Total weeks worked, per person, mean (s.d.) | 14.1 (18.7) | 17.0 (19.9) | 0.11 |
| Tenure (average weeks per job), mean (s.d.) | 18.0 (16.9) | 15.8 (18.0) | 0.32 |
| Weeks at longest job, per person, mean (s.d.) | 12.7 (17.3) | 14.6 (17.9) | 0.18 |
| Days to first job for workers (n=90), mean (s.d.) | 84.1 (103.6) | 89.4 (116.1) | 0.66 |
1. Data missing for 1 participant in each group
Differences in outcomes for competitive employment between usual services and supported employment
| Variable | Usual service (n=74) | Supported employment (n=75) | P |
|---|---|---|---|
| Any competitive job over 12 months, % | 18.9 | 46.7 | <0.001 |
| Worked more than 250 h over 12 months, % | 8.1 | 13.3 | 0.30 |
| Worked 20 h or more for at least 4 weeks, % | 8.1 | 16.0 | 0.14 |
| Total competitive hours worked, per person, mean (s.d.) | 72.5 (251.6) | 126.4 (266.8) | <0.001 |
| Total competitive wages earned, per person (Can$), mean (s.d.) 1 | 520.8 (1901.0) | 961.7 (2162.9) | <0.001 |
| Total competitive weeks worked, per person, mean (s.d.) | 2.9 (8.5) | 6.9 (12.7) | 0.001 |
| Tenure (average weeks per competitive job), mean (s.d.) | 11.1 (13.5) | 8.9 (12.2) | 0.29 |
| Weeks at longest competitive job, mean (s.d.) | 2.8 (8.4) | 5.8 (10.9) | 0.001 |
| Days to first competitive job for workers (n=52), mean (s.d.) | 152.9 (123.3) | 126.3 (95.6) | 0.61 |
1. n=73 for both groups
Odds ratios from generalised linear mixed models, with and without covariates, for any paid employment as well as competitive employment, are shown in Table 3. Whether covariates are included or not, supported employment does not have a statistically significant independent effect on the likelihood of having paid employment in a given month. None of the covariates is statistically significant (all of the 95% confidence intervals include the value 1). In contrast, the results for competitive employment, consistent with Fig. 2b, reflect higher rates of competitive employment in the supported employment group, with an initially sharper rate of increase showing a greater decelerating trend towards the end of the year. Among the covariates, only age is statistically significant, with increasing age associated with lower rates of success in obtaining competitive employment.
Odds ratios (95%, CIs) from estimated generalised mixed linear models to predict monthly employment status, any paid and competitive employment (n=149, 12 months)
| Measure | Any employment | Competitive employment | ||||
|---|---|---|---|---|---|---|
| Supported employment 1 | 2.40 (0.74–7.83) | 2.61 (0.73–9.38) | 3.23 (0.76–13.7) | 4.94 (1.03–23.6) * | ||
| Time 2 | 1.14 (1.08–1.20) * | 1.14 (1.09–1.20) * | 1.44 (1.06–1.96) * | 1.45 (1.07–1.98) * | ||
| Supported employment × time squared | 1.04 (0.96–1.12) | 1.04 (0.96–1.12) | 1.52 (1.04–2.21) * | 1.53 (1.05–2.24) * | ||
| Time 3 | 0.98 (0.96–1.01) | 0.98 (0.96–1.01) | ||||
| Supported employment × time squared | 0.97 (0.94–1.00) * | 0.97 (0.94–1.00) * | ||||
| Age (years) | 0.94 (0.88–1.00) | 0.92 (0.86–0.98) * | ||||
| Female gender | 1.53 (0.44–5.37) | 0.94 (0.28–3.11) | ||||
| Education (years) | 1.16 (0.92–1.46) | 1.22 (0.97–1.53) | ||||
| Ever married | 1.64 (0.32–8.39) | 4.30 (0.92–20.1) | ||||
| White ethnicity | 1.47 (0.31–6.86) | 0.33 (0.07–1.57) | ||||
| Diagnosis–bipolar illness | 1.07 (0.23–5.00) | 1.54 (0.35–6.82) | ||||
| Clinical site with less intensive services | 0.56 (0.15–2.05) | 1.14 (0.33–3.99) | ||||
| > 1 year previous work experience | 0.65 (0.19–2.15) | 0.78 (0.25–2.41) | ||||
| Self-esteem at baseline | 1.00 (0.99–1.02) | 1.02 (1.00–1.03) | ||||
* P < 0.05
1. Intercepts not shown
2. Coefficients (odds ratios) for time in months show the trend or change in employment rate over time
3. Individual placement and support × time denotes interaction between group assignment and time
Results not shown indicate that, if missing values are replaced with zeroes in the analysis comparing monthly competitive employment rates, the P-value of the supported employment group × time interaction term rises to 0.07. Also, if the six individuals with almost no exposure to supported employment and the 53 individuals with almost no exposure to usual services, as defined above, are removed from the analysis, the effect of the supported employment model is no longer statistically significant.
Mixed-effects models reveal no effect of group assignment on symptoms, quality of life, social support, measures of functioning (GAF and MCAS) or the substance misuse measures (results not shown). They do, however, indicate a statistically significant significant time × group interaction for self-esteem (P < 0.01); this variable, initially lower in the supported employment group, rises with time (means: 25.6, 31.7, 47.2), whereas it shows a non-significant declining trend in the usual services group (48.2, 46.2, 42.3). There is, however, within the supported employment group no statistically significant correlation between, on the one hand, the change in self-esteem score between baseline and 12 months, or on the other hand, either hours in any paid employment (n=54, Spearman's ρ=-0.03, P=0.80), hours of competitive employment (n=54, Spearman's ρ=0.08, P=0.55) or number of contacts with supported employment staff over 12 months (n=54, Spearman's ρ=0.08, P=0.56).
Finally, Table 4 compares the distributions of competitive jobs and hours worked across occupational categories. The distribution of types of jobs is quite similar between the two groups, with slightly more than one-third of jobs in clerical and sales occupations, and a similar proportion in services occupations (e.g. cleaning, waiting on tables). More dissimilarity is apparent in the distribution of hours: 17% of hours that are in service jobs in the control group were replaced in the supported employment group with hours in other occupational categories. The more dissimilar distribution of hours than of jobs across occupational categories arises, mathematically, from the proportionally shorter duration of service jobs in the supported employment group. The differences between groups in hours per job, however, are not statistically significant in any of the occupational categories.
Number and duration of competitive jobs according to type of work and according to group assignment
| Type of work | Usual-services group Duration (h) | Supported employment group Duration (h) | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n (%) | Total (%) | Mean | Minimum | Maximum | n (%) | Total (%) | Mean | Minimum | Maximum | |||||||
| Professional, technical and managerial | 2 (10.5) | 35 (0.6) | 17.5 | 14 | 21 | 2 (3.2) | 273.6 (2.9) | 136.8 | 109.3 | 164.4 | ||||||
| Clerical and sales 1 | 7 (36.8) | 1553.3 (29.0) | 258.9 | 15 | 595 | 24 (38.7) | 3045.4 (32.2) | 132.4 | 2 | 780 | ||||||
| Services (1325.7) | 7 (36.8) | 3576 (66.7) | 510.9 | 16.2 | 1345 | 24 (38.7) | 4707.5 (49.7) | 196.1 | 1 | 1325.7 | ||||||
| Other production 2 | 1 (5.2) | 40 (0.7) | 40 | 40 | 40 | 7 (11.3) | 687.2 (7.3) | 83.8 | 7 | 222.9 | ||||||
| Miscellaneous (419.1) | 2 (10.5) | 158.6 (2.9) | 79.5 | 65.7 | 92.9 | 5 (8.1) | 755.6 (8.0) | 151.1 | 7 | 419.1 | ||||||
| Total 3 (1325.7) | 19 (100) | 5362.9 (100) | 297.9 | 14 | 1345 | 62 (100) | 9469.3 (100) | 155.2 | 1 | 1325.7 | ||||||
1. The numbers of jobs indicated for this category include, for each of the groups, one job for which data on hours worked were missing. The other statistics are calculated on the basis of non-missing data (i.e. missings not replaced by zeroes in calculation of means)
2. The categories: agricultural, fishery, forestry, processing, machine trades and benchwork are grouped together because of small numbers
3. The distributions of jobs by type are not significantly different between the usual-services and supported employment groups; neither are the average numbers of hours, for any of the occupational categories
Month-by-month employment rates with 95% confidence intervals, for (a) any paid work, (b) competitive employment only. Unbroken lines, supported employment group; broken lines, usual services.
DISCUSSION
Main study findings
Consistent with previous studies in the USA the present trial found that the individual placement and support model yields significantly better competitive employment outcomes than the alternative it was compared with, in this case a wide array of traditional vocational services. People assigned to the supported employment condition were much more likely to engage in vocational activities, and significantly more of them achieved at least some competitive employment over their first 12 months in the programme.
As in previous studies, however, a significant proportion (here, 53%) of the supported employment clients were unable to achieve any competitive employment over their first year in the programme. Only a small minority (13%) were able to average even 5 h work per week over the 1-year follow-up period. These findings underscore the importance of ongoing studies of potential enhancements to the model, such as the incorporation of cognitive remediation (Reference McGurk, Mueser and PascarisMcGurk et al, 2005; Reference Nuechterlein, Subotnik and VenturaNuechterlein et al, 2005).
Again as in previous studies, the large number of people who did not work or worked only to a minimal extent in each study condition probably helps to explain why no differences emerged between the groups on any of the non-vocational measures we examined, with the singular exception of self-esteem. The use of Nugent and Thomas' (Reference Nugent and Thomas1993) Self-Esteem Rating Scale, as opposed to Rosenberg's (1965) Self-Esteem Scale, which has been used in several previous studies of this model but has been reported as relatively insensitive to change (Reference Torrey, Mueser and McHugoTorrey et al, 2000), may partly account for this finding. The markedly lower self-esteem at baseline of the experimental group complicates interpretation, however. The observed rise in self-esteem in the experimental group could be due in part to regression to the mean, a hypothesis consistent with the absence of correlation between change in self-esteem and hours of work (any paid work or competitive work) or number of contacts with support staff.
Implications for generalisability of supported employment
Evidence on the effectiveness of individual placement and support outside the USA has until now been very limited. One recent British study, using a pre-post design, found that integrating a high-fidelity individual placement and support component into an early intervention service for young people with first-episode psychosis increased the open employment rate from 10% to 28% in 6 months (Reference Rinaldi, McNeil and FirnRinaldi et al, 2004). The present study, however, is the first randomised trial of the model to be carried out outside the USA. The province of Québec's somewhat European-style approach to social policy provides an environment for vocational rehabilitation that differs from that in the USA in several respects. On the one hand, study participants had no reason to fear losing medical insurance, including coverage of most hospital and physician services as well as medications. On the other hand, however, there are also at least three important differences that would be expected to reduce the comparative effectiveness of the model. First, unlike in the USA, people with disabilities in Québec have no economic incentive to work more than a few hours per week, with monthly earnings above Can$100 subtracted dollar-for-dollar from their monthly disability cheque. Furthermore, free public transportation passes are available only to those who engage in sheltered programmes but not competitive work or work in the adapted businesses mentioned earlier. Second, many alternative vocational programmes are available, including adapted businesses as well as social firms, which offer real jobs that pay at the minimum wage or better, but where most employees have disabilities. Third, as in many European countries, unemployment rates are chronically high, remaining at about 8% throughout the study period.
The fact that supported employment proved significantly more effective in such an institutional environment suggests that it may well prove more effective than traditional programmes at helping clients achieve competitive employment in countries outside the USA.
The impact of supported employment on competitive employment outcomes reported here, however, is not as large as that found in most previous studies. In a systematic review summarising results from five trials comparing supported employment with pre-vocational training, Crowther et al (Reference Crowther, Marshall and Bond2001) reported cumulative rates of competitive employment of 70% (130 out of 187) at 6 months and 65% (165 out of 252) at 12 months, compared with 36% (27 out of 75) and 47% (35 out of 75) in our study (P < 0.005 for both differences). They also report that, during the 12th month, 34% of clients in supported employment were employed, consistent with another recently published randomised trial conducted in Hartford, Connecticut (Reference Mueser, Clark and HainesMueser et al, 2004). In contrast we observed 22% employed in competitive settings at 12 months; in no month did more than 27% of clients for whom we had data spend at least some time in competitive employment. Among published randomised trials of supported employment only one reports competitive employment rates in the supported employment condition quite similar to ours (Reference Cook, Leff and BlylerCook et al, 2005), and another reports lower rates: Lehman et al (Reference Lehman, Goldberg and Dixon2002) show a competitive employment rate of about 11% in the 12th month of their study, in inner-city Baltimore, Maryland.
At least two types of factors may account for the somewhat lower rates of competitive employment reported here: differences in institutional environments and implementation issues.
Differences in institutional environments, which could on balance have reduced the effectiveness of the model in comparison with that in the USA, have already been described. With regard to implementation, although the supported employment programme quickly achieved and maintained a high score on the Supported Employment Fidelity Scale, the programme was launched at the same time as the study and thus, for early study participants, supported employment staff may have been less effective than they later became. In addition, supported employment staff were open to helping clients who so desired to find non-competitive work - as reflected in much higher rates for any paid than for competitive employment even in the supported employment group. Indeed, the rich array of available alternatives to competitive employment appears to have somewhat diminished the relative attractiveness of competitive employment for clients and even, particularly at the beginning, for the supported employment staff. Further, the staff report that many clients appeared not to have understood the difference between competitive and non-competitive employment when they agreed to participate in the study, despite the explanations given during the recruitment process.
One likely implication of the comparatively low effectiveness of the supported employment model in our setting is that its cost-effectiveness will be reduced. A report on this question is in preparation.
Study limitations
Four methodological limitations should be noted. Interviewers were aware, after the baseline interview, of the group assignment of at least some clients. However, employment data are objective in nature and were collected in the same way for both groups. Most of the non-vocational measures were based on questionnaire responses by clients and involved no interviewer judgement. Interrater reliability ratings for the BPRS were excellent. Second, somewhat more participants in the supported employment group dropped out of the study, although the difference is not statistically significant. We were able to verify, however, that our results are reasonably robust to a worst-case scenario in which all missing observations on employment correspond to zero values. Third, there was a significant difference in self-esteem scores between the two groups at baseline. We can only attribute this difference to chance, as neither the interviewer nor the client was aware of group assignment during the interview. Adjusting for this difference did not alter the results. Finally, the supported employment programme was started at the same time as the study. A more mature programme might well have produced a larger difference between the groups.
In conclusion, we found that supported employment remains considerably more effective than traditional vocational services at helping clients obtain competitive employment, even in a setting more similar to those found in many European countries than to those in the USA. The supported employment model may then prove more effective than traditional vocational programmes at promoting competitive employment for people with severe mental illness in countries outside the USA.
Acknowledgements
This study was funded by the Canadian Institutes of Health Research, the Québec Health Research Fund and AETMIS.
eLetters
No eLetters have been published for this article.