Hostname: page-component-7479d7b7d-t6hkb Total loading time: 0 Render date: 2024-07-16T02:03:14.578Z Has data issue: false hasContentIssue false

Economic evaluation of the anti-stigma social marketing campaign in England 2009-2011

Published online by Cambridge University Press:  02 January 2018

Sara Evans-Lacko
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, London, UK
Claire Henderson*
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, London, UK
Graham Thornicroft
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, London, UK
Paul McCrone
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, London, UK
*
Dr Claire Henderson, Health Service and Population Research Department, Institute of Psychiatry, De Crespigny Park, London SE5 8AF, UK. Email: claire.1.henderson@kcl.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Background

Evidence on the economic impact of social marketing antistigma campaigns in relation to people with mental illness is limited.

Aims

To describe the economic impact of the Time to Change (TTC) anti-stigma social marketing campaign, including the potential effects on the wider economy.

Method

Data collected for the evaluation of TTC were combined with the social marketing campaign expenditure data to investigate differences in knowledge, attitudes and behaviour in relation to campaign awareness. To evaluate the return on investment, we applied a decision model that estimated the impact on employment for people with depression.

Results

Based on average national social marketing campaign costs, the economic benefits outweighed costs even if the campaign resulted in only 1% more people with depression accessing services and gaining employment if they experienced a health improvement The cost per person with improved intended behaviour was at most £4 if we assume the campaign was responsible for 50% of the change. Costs associated with improved knowledge and attitudes, however, were more variable.

Conclusions

The findings suggest that the TTC anti-stigma social marketing campaign is a potentially cost-effective and low-cost intervention for reducing the impact of stigma on people with mental health problems.

Type
Papers
Copyright
Copyright © Royal College of Psychiatrists, 2013

Evidence of the economic impact of social marketing anti-stigma campaigns in relation to people with mental illness is limited. Reference McCrone, Knapp, Henri and McDaid1,Reference McDaid2 Some evidence does exist for the cost-effectiveness of social marketing programmes for other behavioural health issues such as alcohol misuse and smoking, Reference Anderson, Chisholm and Fuhr3,Reference Holtgrave, Wunderink, Vallone and Healton4 but economic evaluations of social marketing campaigns are generally lacking. Social marketing campaigns have the potential to reach large audiences; however, their effectiveness might be reduced by lack of audience insight, an overcrowded media market or ineffective message delivery. Reference Andreasen5,Reference Wakefield, Loken and Hornik6 Moreover, the sustainability of social marketing campaign effects is questionable. Given the high costs of social marketing and advertising, evaluation of the cost in relation to outcomes is fundamental. Reference Wakefield, Loken and Hornik6,Reference Evans-Lacko, London, Little, Henderson and Thornicroft7 It is possible that reductions in stigma and discrimination might lead to better and more timely access to services in addition to increased rates of employment, Reference Brohan, Henderson, Wheat, Malcolm, Clement and Barley8-Reference Schomerus and Angermeyer11 which could translate to monetary benefits. Additionally, better attitudes among the general public are associated with lower self-stigma and perceived discrimination and higher empowerment among people with mental health problems. Reference Evans-Lacko, Brohan, Mojtabai and Thornicroft12 Funding for mental health services has lagged behind that for other health conditions, and it is especially important that this limited investment has clear economic benefits. Reference Corrigan and Watson13,Reference Evans-Lacko, Baum, Danis, Biddle and Goold14 Operationalisation of the impact of anti-stigma social marketing campaigns in relation to mental health may hinder evaluation, as definition and measurement of outcome targets are challenging in comparison with campaigns that address more defined behaviours such as smoking. Research into the costs associated with stigma and discrimination against people with mental health problems, however, is emerging, Reference Sharac, McCrone, Clement and Thornicroft15 and McCrone et al have developed an innovative model to assess the cost-effectiveness of anti-stigma campaigns for people with mental health problems. Reference McCrone, Knapp, Henri and McDaid1 We applied this newly developed model in conjunction with the Time to Change (TTC) social marketing campaign evaluation data to investigate the economic impact of the campaign, including the potential effects on the wider economy.

Method

Data sources

Data collected for the evaluation of the TTC social marketing campaign (measures and data collection methods for the evaluation of the anti-stigma social marketing campaign are described in detail elsewhere; see Evans-Lacko et al, Reference Evans-Lacko, Malcolm, West, Rose, London and Rüsch16 this supplement) were combined with the social marketing campaign expenditure data to investigate differences in knowledge, attitudes and behaviour in relation to campaign awareness. Data on knowledge, attitude and intended behaviour outcomes in addition to campaign awareness were collected before and after each burst of marketing activity (including six bursts in total) by an online market research panel. Reference Henderson, Evans-Lacko, Flach and Thornicroft17 Approximately 1000 participants, restricted to the campaign target population (i.e. residents of England, aged 25-45 years and of middle-income socioeconomic groups), were recruited for each burst. Quota sampling ensured equal distributions of age, gender and socioeconomic group across time points and that the sample was geographically representative of the population in England.

Measures

Campaign awareness

Prompted campaign awareness was assessed for each type of media and/or activity used by TTC. Individuals who reported seeing any of the advertisements were categorised as ‘campaign aware’ whereas those who responded ‘no’ or ‘don't know’ were categorised as ‘not campaign aware’. Campaign awareness associated with the post-burst stage pertained to awareness of the specific media activity immediately preceding the survey, whereas awareness during the pre-burst stage referred to recall of the media used in the previous campaign burst.

Knowledge

Mental health-related knowledge was measured by the Mental Health Knowledge Schedule (MAKS). Reference Evans-Lacko, Little, Meltzer, Rose, Rhydderch and Henderson18 This comprises six items covering stigma-related mental health knowledge areas (help-seeking, recognition, support, employment, treatment and recovery) and six items that enquire about classification of various conditions as mental illnesses. Reference Rusch, Evans-Lacko and Thornicroft19 Overall test-retest reliability of the MAKS is 0.71 and the overall internal consistency among items is 0.65. Reference Evans-Lacko, Little, Meltzer, Rose, Rhydderch and Henderson18

Attitudes

Mental health-related attitudes were assessed based on three items from the Community Attitudes toward the Mentally Ill (CAMI) scale. Reference Taylor and Dear20 These items were chosen a priori in collaboration with the campaign developers and approved by an expert panel including service users and international stigma experts, and allowed the evaluation results to be directly compared with the concurrent Department of Health national surveys on mental illness, which have used the CAMI since 1993 (for additional information on selection of items see Evans-Lacko et al, Reference Evans-Lacko, Malcolm, West, Rose, London and Rüsch16 this supplement). These items assessed attitudes regarding commonality (‘Virtually anyone can become mentally ill’); responsibility (‘People with mental health problems should not be given any responsibility’) and dangerousness (‘People with mental health problems are far less of a danger than most people suppose’). All items addressed messages that were explicitly included in the campaign: for example the ‘1 in 4’ message was a part of the myths/facts campaign and addressed commonality; the ‘Schizo movie’ advert aimed to challenge stereotypes about mental illness and dangerousness (www.youtube.com/watch?v=6IBgkks_jLw); and the personal testimonials of people with mental illness included in the campaign demonstrated real stories of empowerment.

Intended behaviour

Mental health-related intended behaviour (the level of intended future contact with people with mental health problems) was measured by the Reported and Intended Behaviour Scale (RIBS). Reference Evans-Lacko, Rose, Little, Flach, Rhydderch and Henderson21 We assessed changes in four intended behaviour outcomes (domains comprised living with, working with, living nearby and continuing a relationship with someone with a mental health problem). Assessment of intended behaviour allowed for brief and feasible evaluation at the population level. Overall test-retest reliability of the RIBS was 0.75 and the overall internal consistency of the scale was 0.85.

Cost analysis

To estimate costs associated with changes in knowledge, attitudes and behaviour associated with the TTC campaign, we first determined the percentage agreement with each knowledge, attitude and intended behaviour item among those aware v. not aware of the TTC campaign. To improve generalisability of the estimate to the English population, survey sampling weights were developed to match characteristics of the recruited sample to the target population in England according to prevalence rates of ethnicity within geographic region reported by the Office for National Statistics. Additionally, to minimise bias associated with campaign awareness (i.e. potential endogenous confounders associated with campaign awareness and outcomes), we applied stabilised inverse probability weights. Reference Zanutto22 These allow for modelling of the independent relationship between exposure to the campaign and stigma outcomes and minimise potential confounding associated with campaign exposure. Reference Curtis, Hammill, Eisenstein, Kramer and Anstrom23-Reference Rubin25 To calculate these weights the conditional probability of campaign awareness or propensity for campaign exposure was estimated by fitting a logistic regression model of awareness using all potential confounders (gender, socioeconomic group, ethnic group and familiarity/knowing someone with a mental health problem). Weighting observations by the inverse probability of exposure can introduce excessive variability to the estimates, thus we used stabilised weights that normalised the conditional probability of awareness by the marginal predicted probability of awareness. Reference Poon, Aldous, Mathews, Kitahata, Kahn and Saag26,Reference Robins, Hernan and Brumback27 The marginal predicted probability of awareness was estimated by fitting a simple logistic regression model containing only the intercept term. Finally, a separate stabilised inverse probability weight (conditional probability of awareness/marginal probability of awareness) was applied to each observation. The stabilised inverse probability weights allow for better estimation of standard errors because the distribution of weights is shrunk around 1. All stigma outcomes were subsequently calculated using a combination of inverse probability weights and sampling weights.

The difference in agreement with each statement among those who were campaign aware v. not aware was calculated and the percentage difference was then extrapolated to the number of individuals in the target population who were campaign aware. Based on the campaign awareness level of 44.1% (average awareness across bursts) this translates to 6 348 711 being campaign aware, assuming a target population of 14 396 171 (number of individuals in England aged 25-45 years). It might be that the campaign effects would be limited to the target group and would only include those in socioeconomic groups B, C1 and C2, in which case the population would be smaller and the subsequent costs per person with changed knowledge, attitudes and intended behaviour would be higher. As it is unclear how much of the change in stigma outcomes could be attributed to the TTC campaign, we performed a sensitivity analysis for each outcome with success rates ranging from 10% to 100%. These figures were then combined with data on the costs of the campaign in the form of a cost-effectiveness ratio.

Return on investment analysis

To evaluate the return on investment for the TTC campaign we applied a decision model previously described by McCrone et al which estimates the impact that reduced stigma/discrimination has on employment for people with depression. Reference McCrone, Knapp, Henri and McDaid1 The model assumes that some people with depression will not seek help because of the stigma that they experience or anticipate, and that employment opportunities for people with depression are diminished because of discrimination by potential employers. Based on findings from Scotland's ‘see me’ campaign, McCrone et al assumed a 9% point reduction in those not seeking treatment owing to stigma/discrimination in the presence of a campaign and a similar impact on employment rates. We need to be cautious when generalising from one campaign to another, thus these analyses explored a number of scenarios in relation to TTC. These are that first, the TTC campaign resulted in increased service use of between 1 and 10 percentage points but had no separate impact on employers; second, the TTC campaign did not result in increased service use but did increase employment rates for those with depression by 1-10%; and third, the campaign resulted in both increased service use and employment rates (again by 1-10%). The economic value of increased work is estimated to be £9446, based on the average increase in wages following increased work time. Reference Layard, Clark, Knapp and Mayraz28 This is assuming a low level of annual earnings and as such the results here may be underestimates. All statistical analyses were carried out using SAS version 9.1 for Windows 7.

Results

Campaign expenditure

Campaign expenditure varied significantly, with burst 6 costing the least at £520 000 and burst 5 costing the most at £1.31 million. This resulted in an average campaign burst expenditure of £748 000.

Cost of change in public knowledge

For the MAKS knowledge items, differences in agreement between campaign aware v. not aware individuals ranged from −0.3% (‘Most people with mental health problems go to a healthcare professional to get help’) to 8.0% (‘Most people with mental health problems want to have paid employment’) (Table 1). Although we were able to minimise confounding associated with campaign awareness through the use of inverse probability score weighting, it is still unclear how much of the difference between groups could be attributed to the campaign. Thus, Fig. 1 presents a sensitivity analysis of the cost per person with changed knowledge based on a 10-100% success rate (see also online Figs DS1 and DS2). The higher the rate of success the greater the proportion of people with a change in knowledge due to the campaign, and therefore the cost per person declines as success rate increases. If the campaign is responsible for 100% of the change then the cost per person with changed knowledge ranges from £1.47 for the MAKS employment item to £4.28 for the medication item. If we use a more conservative estimate, that the campaign is responsible for 50% of the change, then the costs range from £2.95 (employment) to £8.56 (medication). As the item about seeking help from a professional was not associated with an increase, this was not analysed.

Fig. 1 Cost per person with changed knowledge (Mental Health Knowledge Schedule items) associated with the anti-stigma marketing campaign.

The estimate for the item ‘go to a professional’ was negative, implying that the campaign would increase costs and have worse outcomes. For the sake of clarity we have only included lines showing the cost per unit change in attitude if these improved.

Table 1 Agreement with statements about mental illness among those aware and unaware of the Time to Change mass-media campaign (applying survey weights and inverse probability weights)

Percentage agreeing with statementFootnote a Implied
number improving
n (95% CI)
Campaign-aware
group % (95% CI)
Campaign-unaware
group % (95% CI)
Difference
% (95% CI)
Knowledge (MAKS items)
Most people with mental health problems want to have
paid employment
70.5 (68.0 to 72.9) 62.5 (60.6 to 64.5) 8.0 (6.1 to 10.4) 507 897 (387 271 to 660 266)
If a friend had a mental health problem, I know what
advice to give them to get professional help
46.3 (43.6 to 48.9) 41.2 (39.2 to 43.0) 5.1 (3.1 to 7.7) 323 784 (196 810 to 488 851)
Medication can be an effective treatment for people
with mental health problems
72.3 (69.9 to 74.8) 70.0 (68.1 to 71.8) 2.3 (0.4 to 4.8) 146 020 (25 395 to 304 738)
Psychotherapy (e.g. talking therapy or counselling) can
be an effective treatment for people with mental health
problems
79.3 (77.1 to 81.6) 74.9 (73.1 to 76.6) 4.4 (2.6 to 6.7) 279 343 (165 066 to 425 364)
People with severe mental health problems can fully
recover
48.0 (45.3 to 50.7) 41.6 (39.6 to 43.7) 6.4 (4.4 to 9.1) 406 318 (279 343 to 577 733)
Most people with mental health problems go to
a healthcare professional to get help
26.5 (24.1 to 28.8) 26.8 (25.0 to 28.6) –0.3 (–2.1 to 2.0) –19 046 (–133 333 to 126 974)
Attitudes (CAMI items)
Virtually anyone can become mentally ill 88.9 (87.1 to 90.6) 85.7 (84.3 to 87.0) 3.2 (1.7 to 4.6) 203 159 (107 928 to 292 041)
People with mental health problems should not be
given any responsibility
63.2 (60.6 to 65.8) 61.4 (59.4 to 63.3) 1.8 (–0.2 to 4.4) 114 277 (–12 697 to 279 343)
People with mental health problems are far less of a
danger than most people suppose
57.4 (54.8 to 60.1) 49.5 (47.4 to 51.5) 7.9 (5.8 to 10.6) 501 548 (368 225 to 672 963)
Intended behaviour (RIBS items)
In the future, I would be willing to live with someone with
a mental health problem
43.9 (41.2 to 46.5) 36.2 (34.2 to 38.2) 7.7 (5.7 to 10.3) 488 851 (361 877 to 653 917)
In the future I would be willing to work with someone with
a mental health problem
69.8 (67.3 to 72.3) 61.0 (59.0 to 62.9) 8.8 (6.8 to 11.3) 558 687 (431 712 to 717 404)
In the future I would be willing to live nearby to someone
with a mental health problem
68.0 (65.5 to 70.6) 61.3 (59.3 to 63.3) 6.7 (4.7 to 9.3) 425 364 (298 389 to 590 430)
In the future I would be willing to continue a relationship
with a friend who developed a mental health problem
79.1 (76.9 to 81.4) 74.0 (72.3 to 75.8) 5.1 (3.4 to 7.4) 323 784 (215 856 to 469 805)

CAMI, Community Attitudes toward the Mental Ill; MAKS, Mental Health Knowledge Schedule; RIBS, Reported and Intended Behaviour Scale.

a. All items were coded so that agreement summarised a less stigmatising response.

Cost of change in public attitudes

For the CAMI attitude items, differences in agreement between campaign aware v. not aware individuals ranged from 1.8% (‘People with mental health problems should not be given any responsibility’) to 7.9% (‘People with mental health problems are far less of a danger than most people suppose’) (see Table 1). Figure 2 demonstrates that if the campaign is responsible for 100% of the change then the cost per person with improved attitudes ranges from £1.25 for the dangerousness item to £5.47 for the responsibility item (see also online Figs DS3 and DS4). If we use the more conservative estimate that the campaign is responsible for 50% of the change the costs range from £2.50 (dangerousness) to £10.94 (responsibility).

Fig. 2 Cost per person with changed attitude (Community Attitudes toward the Mentally Ill items) associated with the anti-stigma marketing campaign.

Cost of change in public intended behaviour

For the RIBS intended behaviour items, differences in agreement between campaign aware v. not aware individuals ranged from 5.1% (‘In the future I would be willing to continue a relationship with a friend who developed a mental health problem’) to 8.8% (‘In the future I would be willing to work with someone with a mental health problem’) (see Table 1). Figure 3 demonstrates that if the campaign is responsible for 100% of the change then the cost per person with improved intended behaviour ranges from £1.93 (continue a relationship) to £1.12 (work with someone) (see also online Figs DS5 and DS6). If we use the more conservative estimate that the campaign is responsible for 50% of the change the costs range from £2.24 (work with someone) to £3.86 (continue a relationship).

Fig. 3 Cost per person with changed intended behaviour (Reported and Intended Behaviour Scale items) associated with the anti-stigma marketing campaign.

Return on investment

Tables 2, 3, 4 describe the economic gain as a result of the TTC anti-stigma social marketing campaign for a hypothetical individual with depression. The gain consists of increased work (with wage rates used as a proxy for extra production) minus the extra service costs incurred through increased uptake. Table 2 suggests that compared with a situation in which 50% of people with depression do not access services owing to stigma and discrimination, the TTC anti-stigma social marketing campaign resulting in 49% not accessing services owing to stigma or discrimination (1% success) would result in more people experiencing a health gain and therefore more also having increased work time. Averaged across all English adults with depression, this results in £3 per person of economic gain. When we extrapolate this rate to adults in the target age range of 25-44 years the return on investment is estimated to be between 0.7 and 1.9, depending on campaign cost. If we assume a 10% success rate the return on investment increases to between 6.7 and 18.5.

Table 2 Return on investment from anti-stigma campaign if access to services is improved but employment rates are unchanged

Gain per person
with depression, £
Gain for people with depression
aged 25-44 years, £
Return on investment for different campaign costs, UK£
Success rate, % £725 000 £1 m £1.5 m £2 m
   0 0 0 0 0 0
   1 3 1 342 305 1.9 1.3 0.9 0.7
   2 7 2 684 609 3.7 2.7 1.8 1.3
   3 10 4 026 914 5.6 4.0 2.7 2.0
   4 14 5 369 219 7.4 5.4 3.6 2.7
   5 17 6 711 523 9.3 6.7 4.5 3.4
   6 20 8 053 828 11.1 8.1 5.4 4.0
   7 24 9 396 132 13.0 9.4 6.3 4.7
   8 27 10 738 437 14.8 10.7 7.2 5.4
   9 30 12 080 742 16.7 12.1 8.1 6.0
10 34 13 423 046 18.5 13.4 8.9 6.7

Table 3 Return on investment from anti-stigma campaign if access to services is unchanged but employment rates are increased

Gain per person
with depression, £
Gain for people with depression
aged 25-44 years, £
Return on investment for different campaign costs, UK£
Success rate, % £725 000 £1 m £1.5 m £2 m
   0 0 0 0 0 0 0
   1 41 16 200 268 22.3 16.2 10.8 8.1
   2 82 32 400 536 44.7 32.4 21.6 16.2
   3 122 48 600 804 67.0 48.6 32.4 24.3
   4 163 64 801 071 89.4 64.8 43.2 32.4
   5 204 81 001 339 111.7 81.0 54.0 40.5
   6 245 97 201 607 134.1 97.2 64.8 48.6
   7 286 113 401 875 156.4 113.4 75.6 56.7
   8 326 129 602 143 178.8 129.6 86.4 64.8
   9 367 145 802 411 201.1 145.8 97.2 72.9
10 408 162 002 678 223.5 162.0 108.0 81.0

Table 4 Return on investment in anti-stigma campaigns if both access to services and employment rates are improved

Gain per person
with depression, £
Gain for people with depression
aged 25-44 years, £
Return on investment for different campaign costs, UK£
Success rate, % £725 000 £1 m £1.5 m £2 m
   0 0 0 0 0 0 0
   1 44 17 655 074 24.4 17.7 11.8 8.8
   2 90 35 535 152 49.0 35.5 23.7 17.8
   3 135 53 640 234 74.0 53.6 35.8 26.8
   4 181 71 970 320 99.3 72.0 48.0 36.0
   5 228 90 525 409 124.9 90.5 60.4 45.3
   6 275 109 305 502 150.8 109.3 72.9 54.7
   7 323 128 310 599 177.0 128.3 85.5 64.2
   8 372 147 540 699 203.5 147.5 98.4 73.8
   9 421 166 995 803 230.3 167.0 111.3 83.5
10 470 186 675 911 257.5 186.7 124.5 93.3

Taking together all of the 40 campaign cost-success rate combinations the return is above 1 for all but two cases (where campaign costs are very high and the success rate is low). If the campaign had no effect on service use or employment, the return on investment would be zero and there would be a loss equal to the value of the campaign. If the campaign affects employment rates but does not separately affect uptake of services then the return on investment ranges from £8.1 million (high cost and low success rate) to £223.5 million (low cost and high success rate) (Table 3). This is because there is an employment effect for those with health improvements regardless of whether they are accessing services because of reduced stigma/discrimination. In all campaign cost-success rate combinations the return on investment is well above 1. The worst scenario is that with a campaign cost of £2 million and only a 1% success rate the return is 8 times the investment.

Table 4 demonstrates that the largest returns on investment follow a campaign that influences both service use and employment rates. In this scenario the figures range from £8.8 million to £257.5 million. Notably, it is also possible that there is no effect on either service use or employment, and that the return on investment would therefore be zero.

Overall TTC programme cost

The total TTC programme cost £20.5 million. The ratio between this and the total cost of mental health problems is lower than the ratio between campaign costs for stroke and obesity and the respective costs of these clinical areas (Table 5).

Table 5 Comparative cost of public health programmes in the UK

Campaign Clinical area Total cost of programme £ Cost of condition(s) £ Campaign cost as % of condition cost %
Time to Change Mental health 20 million 48.6 billionFootnote a 0.04
Change4Life Obesity 75 million 20.2 billion 0.37
Alcohol Effects Alcohol misuse 6 million 15.4 billion 0.04
Stroke Act F.A.S.T. Stroke 12 millionFootnote b 7.0 billion 0.17

a. One billion is 1×109.

b. Approximate value.

Discussion

Our findings suggest that the Time to Change anti-stigma social marketing campaign is a potentially cost-effective and low-cost intervention for improving intended behaviour among the general public in addition to some (but not all) types of mental health-related knowledge and attitudes. Moreover, when taking into account potential effects on the wider economy such as service utilisation and employment, the economic benefits outweigh the financial costs of the campaign for almost every scenario. Finally, when comparing the costs of the entire TTC programme in relation to other types of public health campaigns, the TTC costs are relatively low when considering the global burden of disease consequent upon neuropsychiatric disorders. Reference Collins, Patel, Joestl, March, Insel and Daar29-31

We estimated relatively low costs for changes in intended behaviour and varied costs associated with knowledge and attitude outcomes. If the campaign is responsible for 50% of the intended behaviour change, the cost per person with improved intended behaviour is at most £4. The higher relative impact on behavioural outcome in relation to knowledge and attitudes is consistent with other findings in relation to the TTC programme (see Evans-Lacko et al Reference Evans-Lacko, Malcolm, West, Rose, London and Rüsch16 and Thornicroft et al, Reference Thornicroft, Goulden, Shefer, Rhydderch, Rose and Williams32 this supplement). The cost associated with the three attitude items varied, with the responsibility item being associated with the highest cost. It is notable that the dangerousness item was associated with greatest improvement - and thus lowest cost - among the three attitude items (i.e. £2.50 if we assume a 50% success rate). Previous work suggests that this belief is deeply rooted and persists over time. Reference Pescosolido, Martin, Long, Medina, Phelan and Link33,Reference Schomerus, Schwahn, Holzinger, Corrigan, Grabe and Carta34 Moreover, it has been suggested that a weakness of anti-stigma campaigns is that they would not be successful at addressing entrenched attitudes about difficult topics such as dangerousness. Reference Thompson, Stuart, Bland, Arboleda-Florez, Warner and Dickson35 In response, the TTC anti-stigma social marketing campaign developed the ‘Schizo film’ advert which specifically addressed this stereotype. These findings suggest that targeted media addressing the dangerousness and mental illness stereotype may be an effective and low-cost approach, although its effectiveness will be influenced by the extent of news media reporting that supports this stereotype (see Thornicroft et al, this supplement Reference Thornicroft, Goulden, Shefer, Rhydderch, Rose and Williams32 ).

The return on investment figures demonstrate positive economic benefits in relation to campaign costs of up to £2 million. These figures are likely to be underestimates because the analysis focused only on people with depression (because that is where there is most evidence about employment effects following treatment). The TTC campaign, however, aims to reduce stigma and discrimination across all mental health conditions. In addition, we have assumed employment gains only for people with health improvements, whereas a successful campaign might result in employment gains for those with no health improvement also.

Strengths and limitations

Our assessment of campaign awareness included a detailed questioning process to assess prompted and unprompted campaign awareness for each type of media. However, we could only assess self-reported campaign awareness and not actual awareness. Additionally, although we were able to minimise the endogeneity bias associated with campaign awareness through the use of inverse propensity weights, our model may not have accounted for this completely. Additionally, although the effects of the campaign may increase over time and may vary according to the campaign message, our data did not allow us to measure serial awareness and thus our analysis was based on average estimates across all bursts. A related limitation is that it is impossible to know the impact on knowledge, attitudes and intended behaviour that can be attributed to the TTC campaign rather than to other contemporaneous influences present in England. Additionally, as the campaign has been running from 2009 to 2012, it is unclear how sustainable these effects would be over the long term without the presence of the campaign. Finally, our evaluation includes assessment of knowledge, attitude and intended behaviour, and thus the economic effects depend on these changes being reflected in actual behaviour. Thus, we must also consider the possibility that the campaign has no effect on behaviour and hence no economic benefit. Evaluation of discrimination experienced by people using mental health services over the same period (see Corker et al, this supplement Reference Corker, Hamilton, Henderson, Weeks, Pinfold and Rose36 ) suggests that discriminatory behaviour against people with mental illness is decreasing; however, we cannot be certain how much of these effects are directly related to the TTC campaign or to changes in public stigma.

For this evaluation we focused only on certain effects, and given that the campaign will have had other impacts, the findings here are conservative. Moreover, knowledge, attitude and intended behaviour change are not mutually exclusive outcomes. That is to say, if spending on the campaign results in views about the psychotherapy statement changing, then views about the other statements are likely to change also.

Implications

This study suggests that mass media campaigns can produce positive changes associated with economic benefits across a large target group. As the media, delivery and specific campaign message used by different anti-stigma campaigns are likely to vary according to population needs, budget and organisations involved in the development, Reference Clement, Lassman, Barley, Evans-Lacko, Williams and Pagdin37,Reference Clement, Jarrett, Henderson and Thornicroft38 the findings associated with these specific items may not generalise to all anti-stigma social marketing campaigns. Reference Corrigan39,Reference Evans-Lacko, Henderson and Thornicroft40 Additionally, it is possible that targeting anti-stigma efforts towards specific ‘high-impact’ subgroups (e.g. campaigns focused on employers, teachers or healthcare staff) might be more cost-effective. Thus, the opportunity costs of working with broad rather than specific target groups or employing other types of anti-stigma interventions should be considered and investigated in future research. As funding in mental health generally has lagged behind that for other health conditions, it is especially important to demonstrate economic impact for mental health-related programmes. Reference Corrigan and Watson13,Reference Evans-Lacko, Baum, Danis, Biddle and Goold14 To demonstrate value an intervention must be effective, and therefore it is important to understand the effect that spending money on TTC has on the lives of people with mental illness, as well as potential savings in some areas. Additional research has begun to make this case (see Corker et al, this supplement Reference Corker, Hamilton, Henderson, Weeks, Pinfold and Rose36 ); however, a better understanding of the potential range of effects is needed. These initial findings, however, should give policy makers more confidence around investment in anti-stigma social marketing interventions such as the TTC campaign.

Funding

The evaluation of Time to Change 2007-11 was funded by the Big Lottery Fund, Comic Relief, and SHiFT.

Acknowledgements

We thank Sue Baker (Time to Change), Maggie Gibbons (Mental Health Media), Paul Farmer (MIND), Paul Corry (Rethink Mental Illness) for their collaboration on the evaluation. CH, PM and GT are supported by a National Institute for Health Research (NIHR) Applied Programme grant awarded to the South London and Maudsley NHS Foundation Trust. GT is also supported in relation to the NIHR Specialist Mental Health Biomedical Research Centre at the Institute of Psychiatry, King's College London and the South London and Maudsley NHS Foundation Trust (GT). CH is also supported by grants from Guy's and St Thomas's Charity and the Maudsley Charity. All opinions expressed here are solely those of the authors.

Footnotes

Declaration of interest

G.T. has received grants for stigma-related research in the past 5 years from Lundbeck UK and from the National institute for Health Research, and has acted as a consultant to the UK Office of the Chief Scientist.

References

1 McCrone, P, Knapp, M, Henri, M, McDaid, D. The economic impact of initiatives to reduce stigma: demonstration of a modelling approach. Epidemiol Psichiatr Soc 2010; 19: 131–9.Google Scholar
2 McDaid, D. Countering the Stigmatisation and Discrimination of People With Mental Health Problems in Europe. European Commission, 2008.Google Scholar
3 Anderson, P, Chisholm, D, Fuhr, DC. Effectiveness and cost-effectiveness of policies and programmes to reduce the harm caused by alcohol. Lancet 2009; 373: 2234–46.Google Scholar
4 Holtgrave, DR, Wunderink, KA, Vallone, DM, Healton, CG. Cost-utility analysis of the national truth campaign to prevent youth smoking. Am J Prev Med 2009; 36: 385–8.Google Scholar
5 Andreasen, A. Social Marketing in the 21st Century. Sage, 2006.Google Scholar
6 Wakefield, MA, Loken, B, Hornik, RC. Use of mass media campaigns to change health behaviour. Lancet 2010; 376: 1261–71.Google Scholar
7 Evans-Lacko, S, London, J, Little, K, Henderson, C, Thornicroft, G. Evaluation of a brief anti-stigma campaign in Cambridge: do short-term campaigns work? BMC Public Health 2010; 10: 339.Google Scholar
8 Brohan, E, Henderson, C, Wheat, K, Malcolm, E, Clement, S, Barley, EA, et al Systematic review of beliefs, behaviours and influencing factors associated with disclosure of a mental health problem in the workplace. BMC Psychiatry 2012; 12: 11.Google Scholar
9 Evans-Lacko, S, Thornicroft, G. Stigma among people with dual diagnosis and implications for health services. Adv Dual Diagn 2010; 3: 47.Google Scholar
10 Rusch, N, Corrigan, PW, Wassel, A, Michaels, P, Larson, JE, Olschewski, M, et al Self-stigma, group identification, perceived legitimacy of discrimination and mental health service use. Br J Psychiatry 2009; 195: 551–2.Google Scholar
11 Schomerus, G, Angermeyer, MC. Stigma and its impact on help-seeking for mental disorders: what do we know? Epidemiol Psichiatr Soc 2008; 17: 31–7.Google Scholar
12 Evans-Lacko, S, Brohan, E, Mojtabai, R, Thornicroft, G. Association between public views of mental illness and self-stigma among individuals with mental illness in 14 European countries. Psychol Med 2012; 42: 1741–52.Google Scholar
13 Corrigan, PW, Watson, AC. Factors that explain how policy makers distribute resources to mental health services. Psychiatr Serv 2003; 54: 501–7.Google Scholar
14 Evans-Lacko, SE, Baum, N, Danis, M, Biddle, A, Goold, S. Laypersons' choices and deliberations for mental health coverage. Adm Policy Ment Health 2012; 39: 158–69.Google Scholar
15 Sharac, J, McCrone, P, Clement, S, Thornicroft, G. The economic impact of mental health stigma and discrimination: a systematic review. Epidemiol Psichiatr Soc 2010; 19: 223–32.Google Scholar
16 Evans-Lacko, S, Malcolm, E, West, K, Rose, D, London, J, Rüsch, N, et al Influence of Time to Change's social marketing interventions on stigma in England 2009-2011. Br J Psychiatry 2013; 202 (suppl 55): s7788.Google Scholar
17 Henderson, C, Evans-Lacko, S, Flach, C, Thornicroft, G. Responses to mental health stigma questions: the importance of social desirability and data collection method. Can J Psychiatry 2012; 57: 152–60.CrossRefGoogle ScholarPubMed
18 Evans-Lacko, S, Little, K, Meltzer, H, Rose, D, Rhydderch, D, Henderson, C, et al Development and psychometric properties of the Mental Health Knowledge Schedule. Can J Psychiatry 2010; 55: 440–8.Google Scholar
19 Rusch, N, Evans-Lacko, S, Thornicroft, G. What is a mental illness? Public views and their effects on attitudes and disclosure. Aust N Z J Psychiatry 2012; 46: 641–50.Google Scholar
20 Taylor, SM, Dear, MJ. Scaling community attitudes toward the mentally ill. Schizophr Bull 1981; 7: 225–40.Google Scholar
21 Evans-Lacko, S, Rose, D, Little, K, Flach, C, Rhydderch, D, Henderson, C, et al Development and psychometric properties of the reported and intended behaviour scale (RIBS): a stigma-related behaviour measure. Epidemiol Psychiatr Sci 2011; 20: 263–71.Google Scholar
22 Zanutto, EL. A comparison of propensity score and linear regression analysis of complex survey data. J Data Sci 2006; 4: 6791.Google Scholar
23 Curtis, LH, Hammill, BG, Eisenstein, EL, Kramer, JM, Anstrom, KJ. Using inverse probability-weighted estimators in comparative effectiveness analyses with observational databases. Med Care 2007; 45 (suppl 2): s1037.Google Scholar
24 Hirano, K, Imbens, G. Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Serv Outcomes Res Methodol 2001; 2: 259–78.Google Scholar
25 Rubin, D. Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv Outcomes Res Methodol 2001; 2: 169–88.CrossRefGoogle Scholar
26 Poon, AF, Aldous, JL, Mathews, WC, Kitahata, M, Kahn, JS, Saag, MS, et al Transmitted drug resistance in the CFAR network of integrated clinical systems cohort: prevalence and effects on pre-therapy CD4 and viral load. PLoS One 2011; 6: e21189.Google Scholar
27 Robins, JM, Hernan, MA, Brumback, B. Marginal structural models and causal inference in epidemiology. Epidemiology 2000; 11: 550–60.Google Scholar
28 Layard, R, Clark, D, Knapp, M, Mayraz, G. Implementing the NICE Guidelines for Depression and Anxiety: A Cost-Benefit Analysis. LSE Centre for Economic Performance, 2006.Google Scholar
29 Collins, PY, Patel, V, Joestl, SS, March, D, Insel, TR, Daar, AS, et al Grand challenges in global mental health. Nature 2011; 475: 2730.Google Scholar
30 World Economic Forum. The Global Economic Burden of Non-Communicable Diseases. World Economic Forum, 2011.Google Scholar
31 World Health Organization. Global Burden of Mental Disorders and the Need for Comprehensive, Coordinated Response from Health and Social Sectors at the Country Level. Document EB130.R8. WHO, 2012.Google Scholar
32 Thornicroft, A, Goulden, R, Shefer, G, Rhydderch, D, Rose, D, Williams, P, et al Newspaper coverage of mental illness in England 2008-2011. Br J Psychiatry 2013; 202 (suppl 55): s649.Google Scholar
33 Pescosolido, BA, Martin, JK, Long, JS, Medina, TR, Phelan, JC, Link, BG. ‘A disease like any other’? A decade of change in public reactions to schizophrenia, depression, and alcohol dependence. Am J Psychiatry 2010; 167: 1321–30.Google Scholar
34 Schomerus, G, Schwahn, C, Holzinger, A, Corrigan, PW, Grabe, HJ, Carta, MG, et al Evolution of public attitudes about mental illness: a systematic review and meta-analysis. Acta Psychiatr Scand 2012; 125: 440–52.Google Scholar
35 Thompson, AH, Stuart, H, Bland, RC, Arboleda-Florez, J, Warner, R, Dickson, RA, et al Attitudes about schizophrenia from the pilot site of the WPA worldwide campaign against the stigma of schizophrenia. Soc Psychiatry Psychiatr Epidemiol 2002; 37: 475–82.Google Scholar
36 Corker, E, Hamilton, S, Henderson, C, Weeks, C, Pinfold, V, Rose, D, et al Experiences of discrimination among people using mental health services in England 2008-2011. Br J Psychiatry 2013; 202 (suppl 55): s5863.Google Scholar
37 Clement, S, Lassman, F, Barley, E, Evans-Lacko, S, Williams, P, Pagdin, R, et al Mass media interventions for reducing mental health-related stigma. Cochrane Protocol. Cochrane Database Syst Rev 2011; issue 12: CD009453.Google Scholar
38 Clement, S, Jarrett, M, Henderson, C, Thornicroft, G. Messages to use in population-level campaigns to reduce mental health-related stigma: consensus development study. Epidemiol Psichiatr Soc 2010; 19: 72–9.Google Scholar
39 Corrigan, PW. Where is the evidence supporting public service announcements to eliminate mental illness stigma? Psychiatr Serv 2012; 63: 7982.Google Scholar
40 Evans-Lacko, S, Henderson, C, Thornicroft, G. Strengthening the evidence base to support PSAs. Psychiatr Serv 2012; 63: 3.Google Scholar
Figure 0

Fig. 1 Cost per person with changed knowledge (Mental Health Knowledge Schedule items) associated with the anti-stigma marketing campaign.The estimate for the item ‘go to a professional’ was negative, implying that the campaign would increase costs and have worse outcomes. For the sake of clarity we have only included lines showing the cost per unit change in attitude if these improved.

Figure 1

Table 1 Agreement with statements about mental illness among those aware and unaware of the Time to Change mass-media campaign (applying survey weights and inverse probability weights)

Figure 2

Fig. 2 Cost per person with changed attitude (Community Attitudes toward the Mentally Ill items) associated with the anti-stigma marketing campaign.

Figure 3

Fig. 3 Cost per person with changed intended behaviour (Reported and Intended Behaviour Scale items) associated with the anti-stigma marketing campaign.

Figure 4

Table 2 Return on investment from anti-stigma campaign if access to services is improved but employment rates are unchanged

Figure 5

Table 3 Return on investment from anti-stigma campaign if access to services is unchanged but employment rates are increased

Figure 6

Table 4 Return on investment in anti-stigma campaigns if both access to services and employment rates are improved

Figure 7

Table 5 Comparative cost of public health programmes in the UK

Submit a response

eLetters

No eLetters have been published for this article.