It is estimated that in high-income countries 5-20% of children and adolescents require mental health services and, in Europe, provision of services to those in need can be as low as 20%. 1 There is growing evidence that barriers to seeking help and achieving recovery for mental health problems include the stigma around mental illness, Reference Corrigan, Kerr and Knudsen2,Reference Gulliver, Griffiths and Christensen3 and that stigmatising attitudes start young. Reference Wahl4 Stigmatising attitudes to mental illness are widespread. Reference Thornicroft, Brohan, Rose, Sartorius, Leese and Group5 There is national and international recognition that this issue should be prioritised. 6,7 There is mixed evidence as to whether national initiatives can change such attitudes. Reference Paykel, Hart and Priest8-Reference Mehta, Kassam, Leese, Butler and Thornicroft11 Attempts to research and implement school-based mental health promotion in the UK have largely focused on making the whole-school environment more emotionally aware. 12-15 However, there are also many initiatives, in the UK and other countries, taking anti-stigma programmes into the classroom (for example Rethink, Royal College of Psychiatrists). One review concluded that educational interventions in schools provide positive outcomes on pupils' attitudes to mental illness, Reference Sakellari, Leino-Kilpi and Kalokerinou-Anagnostopoulou16 a belief that seems widespread in the literature. However, it reports positive findings of the studies reviewed without clearly describing the quality of the studies, making the findings difficult to interpret. Its conclusions are considerably more optimistic than an earlier review that questioned the reliability and validity of all studies reviewed. Reference Schachter, Girardi, Ly, Lacroix, Lumb and van Berkom17 However, there have been many published studies since 2006 (the limit of Schachter's et al's meticulous report Reference Schachter, Girardi, Ly, Lacroix, Lumb and van Berkom17 ), which may explain the discrepancy. This systematic review addresses two specific questions: (a) what current evidence is there to justify the growing optimism as to the effectiveness of school-based anti-stigma programmes and (b) what evidence is there to inform future successful programme design?
Method
Inclusion criteria
The types of studies included (using Cochrane Effective Practice and Organisation of Care (EPOC) group definitions) were randomised controlled trials (RCTs), cluster RCTs, non-randomised controlled trials (NRTs), or controlled before-after studies (CBA). Participants were children or adolescents attending primary or secondary school. School-based interventions targeting attitudes and stigma about mental illness were included. Studies were included if they measured outcomes of: knowledge/beliefs and attitudes towards mental illness, behavioural intentions, stigmatising behaviour or affect. The analysis of help-seeking outcomes is not covered in this review, because help-seeking is not directly associated with stigmatising attitudes/behaviour. Level of knowledge is also not directly associated with stigmatising attitudes but these outcomes are included as many of the ‘knowledge’ measures contain some belief and attitude statements. Known reliability/validity of the instruments was not an inclusion criterion, but will be commented on within the results.
Search methods and study selection
The following search engines were used: Medline, CINAHL and PsycINFO (1990-2013, articles in English) on 12 June 2013, using the keywords (stigma* OR attitude* OR awareness) AND (school or adolesc*) AND (educat* OR train* OR program*) AND (mental OR schizophreni* OR psychiatri*). The references lists cited in relevant reviews were also checked. Reference Sakellari, Leino-Kilpi and Kalokerinou-Anagnostopoulou16-Reference Corrigan, Morris, Michaels, Rafacz and Rüsch21 Studies were selected for inclusion by screening titles, abstracts and when necessary full texts, against the inclusion criteria.
Data extraction and critical appraisal
A data-extraction form based on the Cochrane EPOC group's data-collection checklist was used to record details about study characteristics, intervention design, outcome measures and results. Following this process the group's recently updated ‘suggested risk of bias criteria for EPOC reviews’ 22 was used to make judgements on the risk of bias (high, low or unclear) in each study in each of the domains suggested by the document. The domains assessed were: allocation sequence generation and concealment, baseline outcome measures and characteristics, comparison of site profiles (if applicable), protection against contamination, masking, completeness of outcome data, and outcome reporting (were data for each outcome, group and time point fully presented). In addition to this, the reliability and validity of the instruments used, as documented in the study reports, was noted.
Data synthesis
The review looked at the intervention effect of each study by comparing before and after outcome scores in the intervention and control groups. First, studies that provided follow-up data (rather than simply immediate post-test data) were reviewed. Of these, studies that reported a positive result (a statistically significant, P<0.05, change in any outcome measure compared with control) after the intervention were selected. These studies were reviewed for study quality, as judged by study design and risk of bias criteria. Studies with positive results at immediate post-intervention only were then reviewed for study quality. Positive results based on the use of specifically developed outcome measures with low reliability were excluded. To answer the second review question the intervention design features (such as duration, contact or non-contact, delivery) of those studies showing positive results were tabulated and compared.
Selection of studies using Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA).
Results
Of the 1261 studies identified in the initial search, 17 met the above criteria (Fig. 1). Reference Moher, Liberati, Tetziaff and Altman23
Intervention and study characteristics
The interventions varied in content and delivery methods (online Table DS1). Nine were education-only, Reference Rahman, Mubbashar, Gater and Goldberg24-Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 whereas eight had indirect Reference Chan, Mak and Law33,Reference Saporito, Ryan, Bethany and Teachman34 or direct Reference Ng and Chan35-Reference Pinto-Foltz, Logsdon and Myers40 contact with someone with lived experience. Fifteen studies targeted secondary-school pupils, two targeted primary school pupils. Reference Pitre, Stewart, Adams, Bedard and Landry28,Reference Ventieri, Clarke and Hay31 One included a few individuals over 18. Reference Ng and Chan35 The duration ranged from one-off interventions lasting 30-120 min to multiple sessions over a period of up to 4 months. The focus of the interventions was mental illness in 11 studies, schizophrenia in 3 and depression in 3. Five studies investigated the impact of already established interventions. Reference Wahl, Susin, Kaplan, Lax and Zatina30,Reference Schulze, Richter-Werling, Matschinger and Angermeyer36-Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38,Reference Pinto-Foltz, Logsdon and Myers40 The number of participants varied from 40 to 616. The follow-up time ranged from immediately post intervention only, up to 12 months. The outcome measures were secondary outcomes in two studies, Reference O'Kearney, Gibson, Christensen and Griffiths26,Reference O'Kearney27 which are shown at the end of Table DS1. One study was an RCT. Reference Saporito, Ryan, Bethany and Teachman34 Five studies were cluster RCTs, two using cluster randomisation at the school level, Reference Rahman, Mubbashar, Gater and Goldberg24,Reference Pitre, Stewart, Adams, Bedard and Landry28 three using cluster randomisation at the class/year level within selected schools. Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32,Reference Chan, Mak and Law33,Reference Pinto-Foltz, Logsdon and Myers40 Four studies were NRTs, six were CBA trials. It is unclear whether one study was an NRT or CBA. Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38
The comparison groups, other than Chan et al's, Reference Chan, Mak and Law33 which compared three intervention conditions, had normal lessons (no intervention) in 14 of the studies, a talk about healthy living from external speakers in 1 Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 and a video presentation about smoking in another. Reference Saporito, Ryan, Bethany and Teachman34 The vast majority therefore did not control for the effect of a novel programme, in many cases with outside speakers.
Outcome measures
Table DS1 shows all outcome measures used within the studies. Results from two additional scales were excluded as irrelevant to the review question (the Self-Efficacy Scale Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38 and the Strengths and Difficulties Questionnaire Reference Naylor, Cowie, Walters, Talamelli and Dawkins29 ). Of the remaining 31 outcome measures used (and reviewed here), most measures were ‘stigma’ measures: attitudes, behavioural intentions and in one study an affect measure. Reference Pinto-Foltz, Logsdon and Myers40 In addition, several studies tested factual knowledge gained. No studies measured actual behaviour. All measurements were self-report Yes/No, True/False or Likert-style questionnaires, except for the Implicit Association Test (IAT), Reference Saporito, Ryan, Bethany and Teachman34 where participants categorise words as quickly as possible.
For 14 of the instruments reliability was reported as good, in all but one of these cases the studies chose to report internal consistency (Cronbach's alpha) to back up that claim. The instruments' validity was usually not mentioned in the report, although some studies used previously well-tested instruments.
A total of 13 of the instruments were designed for the intervention or study; 6 of these had poor (or untested) reliability, casting doubt also on their validity Reference Rahman, Mubbashar, Gater and Goldberg24,Reference Naylor, Cowie, Walters, Talamelli and Dawkins29-Reference Ventieri, Clarke and Hay31,Reference Rickwood, Cavanagh, Curtis and Sakrouge37,Reference Pinto-Foltz, Logsdon and Myers40 and therefore on the results that they provide. These six were all knowledge measures. The other seven were piloted and/or internally consistent. Reference Wahl, Susin, Kaplan, Lax and Zatina30,Reference Ventieri, Clarke and Hay31,Reference Chan, Mak and Law33,Reference Schulze, Richter-Werling, Matschinger and Angermeyer36
Study quality
Details of study quality are provided in Table 1. Only one study, a cluster RCT, adequately described randomisation and allocation concealment. Reference Rahman, Mubbashar, Gater and Goldberg24 Baseline outcome measures and baseline group characteristics were clearly compared and similar in nine (in addition, one study showed similarity in one but not the other outcome Reference Rickwood, Cavanagh, Curtis and Sakrouge37 ) and six studies respectively. Four studies used different sites as their control and intervention groups and none of these studies clearly compared (with measures of significance) the sites' profiles. These four studies only were able to clearly protect against contamination. Due to using self-report questionnaires none of the outcome measures were masked or objective (the IAT is ‘self-report’ but aims to assess automatic memory associations and therefore is less open to bias Reference Saporito, Ryan, Bethany and Teachman34 ). There was the potential of attrition bias being introduced because of incomplete data in 13 studies. Most of the studies did not omit important outcome data in their reports. Four studies mention a power calculation. One was underpowered, Reference Pinto-Foltz, Logsdon and Myers40 the other three report having sufficient sample size. Reference O'Kearney, Gibson, Christensen and Griffiths26,Reference O'Kearney27,Reference Naylor, Cowie, Walters, Talamelli and Dawkins29
Various methods were used to enhance consistency of delivery. In two studies the presenters were trained and sessions monitored for fidelity Reference Naylor, Cowie, Walters, Talamelli and Dawkins29,Reference Pinto-Foltz, Logsdon and Myers40 and two interventions used a computer program. Reference O'Kearney, Gibson, Christensen and Griffiths26,Reference O'Kearney27 Five others mention training the presenters, Reference Rahman, Mubbashar, Gater and Goldberg24,Reference Chan, Mak and Law33,Reference Saporito, Ryan, Bethany and Teachman34,Reference Rickwood, Cavanagh, Curtis and Sakrouge37,Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38 the remaining eight provided material for the presenters to follow.
Intervention effects
To answer the first review question it is helpful to look at whether the studies with positive changes in stigma (and knowledge) outcomes are of high enough quality to give confidence in their findings. The final two rows of Table 1 show which studies reported statistically significant results at follow-up (for knowledge and stigma outcomes). Results of each outcome measure are tabulated as either reporting a significant positive change (a tick) or no significant change (a cross).
Table 2 gives an overview of the results reported in the studies at post-test and follow-up, and indicates whether the outcomes measured changed significantly (a tick) or not (a cross). Results from the six outcome measures developed specifically for the interventions they were testing, with reliability not measured (or α < 0.7), are not included in this section. Table 2 shows which outcome measures this applies to (represented by /). For two studies, where this involved the only instrument used, Reference Rahman, Mubbashar, Gater and Goldberg24,Reference Naylor, Cowie, Walters, Talamelli and Dawkins29 there are therefore limited conclusions that can be drawn here despite the fact that they did otherwise have relatively good methodology, according to the risk of bias table.
Studies with positive results at follow-up
Twelve studies collected information at follow-up. Of these, seven studies showed some statistically significant positive changes at follow-up, Reference Esters, Cooker and Ittenbach25,Reference Wahl, Susin, Kaplan, Lax and Zatina30-Reference Chan, Mak and Law33,Reference Ng and Chan35,Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 and these are summarised here. All were at high risk of selection bias except for the two cluster RCTs, which did not, however, have a clearly described method of randomisation. All had high-risk levels of attrition or an unclear description of individuals who dropped out, except for Economou et al. Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32
Economou et al's Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 cluster RCT compared change in mean score per item on their belief/attitude questionnaire and reported that 8/10 items were answered significantly better at follow-up than baseline in the intervention group. They report no significant change in the control group but do not present these data. There was no significant improvement in social distance scores at follow-up. Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 Chan et al's Reference Chan, Mak and Law33 cluster RCT showed significant positive change in knowledge and social distance but not stigma at follow-up. This study discarded 35% of their data (because of absenteeism or returning incomplete measures) and it was not clear from which group(s) these missing data were from.
Ventieri et al's Reference Ventieri, Clarke and Hay31 study in a primary school used Schulze et al's Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 social distance scale and a novel instrument to test ‘benevolence’ and ‘unkindliness’, piloted on a group of pre-adolescents and tested for reliability. The intervention group showed positive change compared with the controls in all three measures. Schools were invited into the study based on assignment (to control or intervention). Reference Ventieri, Clarke and Hay31 In Wahl et al's Reference Wahl, Susin, Kaplan, Lax and Zatina30 study, mean total score in knowledge, attitudes and social distance (on scales developed for the study) improved slightly but significantly. Only 47% of eligible pupils were included in the analysis (those who took part in the three-session programme and completed all questionnaires).
Study design, quality and outcomes
| Saporito et al (2011) Reference Saporito, Ryan, Bethany and Teachman34 | Economou et al (2011) Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 | Pinto-Foltz et al (2011) Reference Pinto-Foltz, Logsdon and Myers40 | Chan et al (2009) Reference Chan, Mak and Law33 | Pitre et al (2007) Reference Pitre, Stewart, Adams, Bedard and Landry28 | Rahman et al (1998) Reference Rahman, Mubbashar, Gater and Goldberg24 | Ventieri et al (2011) Reference Ventieri, Clarke and Hay31 | Wahl et al (2011) Reference Wahl, Susin, Kaplan, Lax and Zatina30 | Robinson et al (2010) Reference Robinson, Gook, Yuen, Hughes, Dodd and Bapat39 | Naylor et al (2009) Reference Naylor, Cowie, Walters, Talamelli and Dawkins29 | Conrad et al (2009) Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38 | O'Kearney (2009) Reference O'Kearney27 | O'Kearney et al (2006) Reference O'Kearney, Gibson, Christensen and Griffiths26 | Rickwood et al (2004) Reference Rickwood, Cavanagh, Curtis and Sakrouge37 | Schulze et al (2003) Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 | Ng & Chan (2002) Reference Ng and Chan35 ,Footnote d | Esters et al (1998) Reference Esters, Cooker and Ittenbach25 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Design | RCT | Cl-RCT | Cl-RCT | Cl-RCT | Cl-RCT | Cl-RCT | NRT | NRT | NRT | NRT | ? NRT | CBA | CBA | CBA | CBA | CBA | CBA |
| InterventionFootnote a | I, Ed | Ed | C, Ed | I, Ed | Ed | Ed | Ed | Ed | C, Ed | Ed | C, Ed | Ed | Ed | C, Ed | C, Ed | C, Ed | Ed |
| DurationFootnote b | 1 | 1 | 1 | 1 | ? | 3 | 2 | 2 | 1 | 3 | 2 | 3 | 3 | 1 | 2 | 3 | 2 |
| Risk of biasFootnote c | |||||||||||||||||
| 1 | ? | ? | ? | ? | ? | ✓ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ |
| 2 | ? | ? | ? | ? | ? | ✓ | ˟ | ? | ? | ? | ? | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ |
| 3 | ? | ✓ | ✓ | ? | ✓ | ✓ | ✓ | ? | ? | ✓ | ? | ✓ | ✓ | [6 | ˟ | ✓ | ? |
| 4 | ✓ | ✓ | ✓ | ✓ | ✓ | ˟ | ˟ | ? | ˟ | ✓ | ? | ˟ | ˟ | ˟ | ˟ | ? | ˟ |
| 5 | ˟ | ? | ˟ | ? | |||||||||||||
| 6 | ? | ˟ | ? | ˟ | ✓ | ✓ | ✓ | ? | ˟ | ✓ | ? | ˟ | ˟ | ? | ˟ | ? | ˟ |
| 7 | ? | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ | ˟ |
| 8 | ? | ✓ | ✓ | ? | ? | ✓ | ? | ˟ | ? | ✓ | ? | ˟ | ˟ | ? | ? | ˟ | ? |
| 9 | ✓ | ˟ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ˟ | ✓ | ˟ | ˟ | ˟ | ✓ | ✓ |
| Results of follow-upFootnote e | |||||||||||||||||
| Knowledge | n/a | - | / | ✓ | n/a | n/a | / | / | n/a | / | - | ˟ | - | n/a | - | - | - |
| Stigma | n/a | ✓˟ | ˟ | ˟✓ | n/a | n/a | ✓✓✓ | ✓✓ | n/a | - | ˟ | ˟ | ˟ | n/a | ˟✓ | ˟˟✓Footnote d | ✓ |
RCT, randomised controlled trial; Cl-RCT, cluster RCT; NRT, non-RCT; CBA, controlled before-after studies.
a. Intervention type: C, direct contact; I, indirect contact; Ed, education.
b. Duration: 1, one session; 2, >1 session but within 1 week; 3, weekly sessions for 2 weeks or more.
c. Risk of bias: ✓, low risk; ˟, high risk; ?, unclear risk. 1,Was the allocation sequence adequately generated?; 2,Was the allocation sequence adequately concealed?; 3 Were baseline outcome measurements similar (for outcomes included in the review)?; 4,Were baseline characteristics similar?; 5,Were site profiles compared if different sites used as control/intervention?; 6,Was study adequately protected against contamination?; 7,Was knowledge of the allocated interventions adequately prevented during the study (masking)?; 8, Was incomplete outcome data adequately addressed?; 9, Was the study free from selective outcome reporting?
d. Shows the proportion of questionnaire items that did or did not show significant change (i.e. if results in the study are reported by change in individual questionnaire items, or for Opinions about Mental Illness when the six factors are reported separately).
e. Results at follow-up for ‘stigma’ (attitudes, behavioural intentions or affect) and knowledge measures: ✓, statistically significant change in intervention group in stigma-relevant or knowledge outcome measure (summed scores); ˟, no statistically significant difference;-, outcome not measured in study; n/a, outcome not measured in study at this time point. More than one tick or cross in a cell indicates that more than one outcome measure was evaluated. Results from the six outcome measures developed specifically for the interventions they were testing with reliability not measured (or α < 0.7) are not included. If this leaves no results at a time point, this is represented by /.
Results
| Saporito et al (2011) Reference Saporito, Ryan, Bethany and Teachman34 | Economou et al (2011) Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 | Pinto-Foltz et al (2011) Reference Pinto-Foltz, Logsdon and Myers40 | Chan et al (2009) Reference Chan, Mak and Law33 | Pitre et al (2007) Reference Pitre, Stewart, Adams, Bedard and Landry28 | Rahman et al (1998) Reference Rahman, Mubbashar, Gater and Goldberg24 | Ventieri et al (2011) Reference Ventieri, Clarke and Hay31 | Wahl et al (2011) Reference Wahl, Susin, Kaplan, Lax and Zatina30 | Robinson et al (2010) Reference Robinson, Gook, Yuen, Hughes, Dodd and Bapat39 | Naylor et al (2009) Reference Naylor, Cowie, Walters, Talamelli and Dawkins29 | Conrad et al (2009) Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38 | O'Kearney (2009) Reference O'Kearney27 | O'Kearney et al (2006) Reference O'Kearney, Gibson, Christensen and Griffiths26 | Rickwood et al (2004) Reference Rickwood, Cavanagh, Curtis and Sakrouge37 | Schulze et al (2003) Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 | Ng & Chan (2002) Reference Ng and Chan35 | Esters et al (1998) Reference Esters, Cooker and Ittenbach25 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Design | RCT | Cl-RCT | Cl-RCT | Cl-RCT | Cl-RCT | Cl-RCT | NRT | NRT | NRT | NRT | ? NRT | CBA | CBA | CBA | CBA | CBA | CBA |
| Post-test results | |||||||||||||||||
| Significant effect reported? | |||||||||||||||||
| Knowledge | - | - | ˟ | ✓ | - | ✓ | ✓ | ✓ | - | n/a | - | ˟ | - | ✓ | - | - | - |
| Stigma | ˟˟ | ✓✓ | ˟ | ✓✓ | ˟✓Footnote a | - | ✓✓✓ | ✓✓ | ✓✓ | n/a | ✓ | ˟ | ˟ | ✓ | ˟✓ | ˟˟✓Footnote a | ✓ |
| Results considered in reviewFootnote b | |||||||||||||||||
| Knowledge | - | - | / | ✓ | - | / | / | / | - | n/a | - | ˟ | - | / | - | - | - |
| Stigma | ˟˟ | ✓✓ | ˟ | ✓✓ | ˟✓Footnote a | - | ✓✓✓ | ✓✓ | ✓✓ | n/a | ✓ | ˟ | ˟ | ✓ | ˟✓ | ˟˟✓ | ✓ |
| Follow-up results | |||||||||||||||||
| Significant effect reported? | |||||||||||||||||
| Knowledge | n/a | - | ✓ | ✓ | n/a | n/a | ✓ | ✓ | n/a | ˟✓Footnote a | - | ˟ | - | n/a | - | - | - |
| Stigma | n/a | ✓˟ | ˟ | ˟✓ | n/a | n/a | ✓✓✓ | ✓✓ | n/a | - | ˟ | ˟ | ˟ | n/a | ˟✓ | ˟˟✓Footnote a | ✓ |
| Results considered in reviewFootnote b | |||||||||||||||||
| Knowledge | n/a | - | / | ✓ | n/a | n/a | / | / | n/a | / | - | ˟ | - | n/a | - | - | - |
| Stigma | n/a | ✓˟ | ˟ | ˟✓ | n/a | n/a | ✓✓✓ | ✓✓ | n/a | - | ˟ | ˟ | ˟ | n/a | ˟✓ | ˟˟✓Footnote a | ✓ |
RCT, randomised controlled trial; CI-RCT, cluster RCT; NRT, non-RCT; CBA, controlled before-after studies. ✓, statistically significant change in intervention group in stigma-relevant or knowledge outcome measure (summed scores); ˟, no statistically significant difference; -, outcome not measured in study; n/a, outcome not measured in study at this time point. More than one tick or cross in a cell indicates that more than one outcome measure was evaluated.
a. Shows the proportion of questionnaire items that did or did not show significant change (i.e. if results in the study are reported by change in individual questionnaire items, or for Opinions about Mental Illness when the six factors are reported separately).
b. Do not include results from the six outcome measures developed specifically for the interventions they were testing with reliability not measured (or a50.7). If this leaves no results at a time point, this is represented by /.
Schulze and colleagues summed the amount of positive responses from each student on their novel instrument testing for stereotypes and social distance. Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 Stereotypes, but not social distance, changed more positively in the intervention than in the control group. This study reported significant differences in baseline outcome measures and baseline characteristics, likely related to the fact their intervention group chose to take part in the mental illness module. Ng & Chan Reference Ng and Chan35 report a significant improvement in 2/6 Opinion about Mental Illness in Chinese Community (OMICC) factors (benevolence and stigmatisation) between the intervention and control groups, but a significant worsening in both groups in attitudes to restrictiveness. Esters et al's small study (n = 40) reported statistically significant positive change on a well-validated scale measuring opinions about mental illness. Reference Esters, Cooker and Ittenbach25
Studies with positive results at post-test but not follow-up
There were a further four studies that report statistically significant positive results at immediate post-test only (Table 2). They all have high or unclear risk of selection and attrition bias. Pitre et al's three-session puppet show in a primary school reports positive change for the intervention group on the adapted Opinions about Mental Illness (OMI) scale, in 3/6 factors. Reference Pitre, Stewart, Adams, Bedard and Landry28 Robinson et al's study reports significant changes (compared with baseline and control) after their 2 h session on stigma and attitudes. Reference Robinson, Gook, Yuen, Hughes, Dodd and Bapat39 The studies of Rickwood et al Reference Rickwood, Cavanagh, Curtis and Sakrouge37 (one session intervention) and Conrad et al Reference Conrad, Dietrich, Heider, Blume, Angermeyer and Riedel-Heller38 (1-day intervention), do not present any data other than regression statistics, making their findings hard to assess.
No positive results
Some studies showed no significant changes at either post-test or follow-up. Saporito et al Reference Saporito, Ryan, Bethany and Teachman34 was the only RCT, randomising at pupil level, although it is not clear what method of randomisation they used. There was no significant improvement in implicit or explicit attitudes to mentally ill people. Pinto-Foltz et al Reference Pinto-Foltz, Logsdon and Myers40 carried out a cluster RCT with more low-risk scores than most of the other studies reviewed. They provided a one-session intervention and found no post-intervention difference in stigmatising attitudes. O'Kearney et al's Reference O'Kearney, Gibson, Christensen and Griffiths26 and O'Kearney's Reference O'Kearney27 studies of five online sessions (one in males, one in females) recorded results at 5 months. Attitudes (and depression literacy in the later study) were measured as secondary outcomes but showed no significant change. 7
Effective intervention design
To answer the second review question it is necessary to see whether there are any consistent features in the intervention programmes in those studies that show positive results. However, the comparison between the results of studies describing such different interventions and methodology is difficult. Chan et al Reference Chan, Mak and Law33 is the only example of a study investigating which aspect of a one-off session might offer the most benefit. The most improvement was seen in the group that had education (a 30 min lecture) followed by a 15 min video (rather than vice versa, or purely education).
Of the studies with positive results at follow-up there is no obvious pattern about what makes a successful intervention. These seven studies include two interventions of only one session and one of the longest interventions (over 10 weeks). Four had no element of contact, two direct, one indirect contact. The follow-up time at which the positive results were recorded ranged from 1 to 12 months. One study was in a primary school.
Discussion
Within the literature there are frequent references to the existing evidence for the effectiveness of school-based interventions in reducing stigma of mental illness in young people. This systematic review of available evidence does not support those statements. Showing a significant difference in self-report questionnaires immediately after an intervention seems unsurprising and, if that is the limit of the effect of the programme, seems insufficient grounds for rolling out the programme more widely. It is proposed here that a successful programme would show a positive change in outcomes at follow-up, which was the case in seven studies Reference Esters, Cooker and Ittenbach25,Reference Wahl, Susin, Kaplan, Lax and Zatina30-Reference Chan, Mak and Law33,Reference Ng and Chan35,Reference Schulze, Richter-Werling, Matschinger and Angermeyer36 However, the potential for selection and attrition bias, which can exaggerate intervention effect, are common themes in all but one (Economou et al Reference Economou, Louki, Peppou, Gramandani, Yotis and Stefanis32 ) of these studies.
There is one RCT and five cluster RCTs within this body of evidence. Only two of these showed statistically significant improvements in outcome measures at follow-up. Only one of the RCTs clearly described their randomisation process, making it difficult to judge the risk of selection bias in the others. Of the other study designs, Naylor et al's Reference Naylor, Cowie, Walters, Talamelli and Dawkins29 study stands out as having a greater number of low-risk scores. Small positive changes were seen in their knowledge measure but the validity of the tool used remains doubtful.
There is insufficient data to answer the review question concerning how one might design a successful intervention. Unfortunately, no elements were found to be consistent between the studies with positive results. In the absence of this evidence it is tempting to extrapolate from similar adult studies (summarised in a review as showing positive results Reference Dalky19 ). However, two papers present findings that caution against this. ‘In our own voice’ had positive results in adults but ‘disappointing’ results in adolescents Reference Pinto-Foltz, Logsdon and Myers40 and a more recent meta-analysis of anti-stigma approaches reported that although ‘contact’ was better than education at reducing stigma in adults the reverse was true in adolescents. Reference Corrigan, Morris, Michaels, Rafacz and Rüsch21
Results from studies to date leave uncertainty as to whether interventions to reduce stigma in schools are not effective, whether interventions have been unsuccessful because they have not contained the right combination of elements or whether the studies have not been designed in such a way as to demonstrate efficacy.
Challenges in developing interventions include the need to assess different elements of programme content (contact, educational, etc.) and delivery style against each other. Information is also needed about whether targeting certain groups of children is more successful than universal provision. Indeed, not all students may need an intervention of this type. Only a third of pupils in a Scottish study reported moderate-high levels of stigma. Reference Chandra and Minkovitz41 It is also unclear whether the primary-school age child would be more open to anti-stigma messages, as very few studies have been carried out in this age group.
It is proposed here that the starting assumption when developing an intervention is that it should be long enough and intensive enough to provide some effect at follow-up. The studies reviewed here do not agree on how long a successful intervention should be or at what interval to assess follow-up.
There are daunting issues for study designers to contend with in this field. Stigma is a multifaceted concept, and even well-established measures have their limitations (for example social distance scales not being validated against discriminatory or supportive behaviour Reference Jorm and Oh42 ). These measures are self-report questionnaires, which are at risk of social desirability bias (particularly, it could be argued, if done after an anti-stigma intervention). The absence of measures to examine change in behaviour after anti-stigma programmes has been recently commented on in a meta-analysis as regrettable. Reference Corrigan, Morris, Michaels, Rafacz and Rüsch21 Maybe resources need to be first directed towards refining age-appropriate measures more closely linked to actual behavioural outcomes. Adverse effects of an intervention also need to be monitored. Recruiting pupils within a school environment is also challenging. Recruitment difficulties in some of the studies described led to a need to actively recruit volunteers to the intervention arm, leading to problems with selection bias. It is also resource heavy to expose the control group to a different type of intervention - hence most of the controls in these studies were simply exposed to ‘normal lessons’.
The protocol of a proposed UK-based, feasibility trial Reference Chisholm, Patterson, Torgerson, Turner and Birchwood43 tackles many of these methodological issues. This well-powered study plans to have an active and randomised control (describing adequate sequence generation and allocation), comparing education with education and contact, carefully prepared material already piloted, 2-week and 6-month follow-up, and will compare the intervention effect by baseline characteristics. Reference Chisholm, Patterson, Torgerson, Turner and Birchwood43 If this trial does not suffer from significant implementation and reporting difficulties the results will be the most definitive to date. This review shows that, although it is inherently attractive to believe that school-based interventions reduce stigma to mental illness in young people, there is currently no strong evidence to support this conclusion.
Limitations
The limitations of this review include not searching the grey literature and the exclusion of studies written in foreign languages. There is a risk of bias in study selection and data extraction as one author performed these processes. Also, the authors of studies were not contacted for information that could not be gleaned from the published study papers.
Open access
eLetters
No eLetters have been published for this article.