Selecting indicators of change

First, which indicators should be used to assess the success of a campaign? In TTC it was decided that progress would be measured by assessing whether the goals set at the beginning of the campaign had been reached. This way of measuring effects has been promoted and used by many: the two main problems with this approach are that goals can be set so low that success is inevitable (or so high that success is impossible), and that it is often difficult to say whether the goals are relevant to the desirable meta-effects of anti-stigma programmes (for example, long-term improvement in the quality of life of people who have a mental disorder). Dealing with the effects of confounding factors - e.g. events that were not foreseen but might have had an impact on the quality of life of people with mental disorders - presents yet another set of problems, whose solution is made difficult by our ignorance about the effects that such confounding factors may have, and about their identity.

Assessing the relevance of goals

A second set of questions relates to the relevance of success of the campaign. Accepting the limitations of using a comparison of initially set goals with achievements after the campaign, it is necessary to establish whether the achievement of these goals matters to anyone but those who designed the programme. Does it really matter whether the newspapers (which are losing readership at a rapid rate) write in more positive or less positive ways about mental illness? Will the negative reports in newspapers affect the politicians involved in developing legislation concerning people who are mentally ill?

Related to the relevance of the goals of an anti-stigma programme to the objectives of mental health programmes is the determination of the necessary size of impact that the anti-stigma activity will have. How big should a reduction in the number of negative reports in newspapers be before it can be considered relevant and causing change? What criteria should be used in answering this question? How long should the newspaper reports be negative before they produce a demonstrable impact on politicians? Is the time to impact different for different people - for instance, are teachers influenced as easily as employers or members of parliament? What is the minimum desirable change of attitudes that is likely to produce a change of behaviour? The complexity of the pathogenesis of stigmatisation and of its effects has so far apparently defeated research that would be necessary to answer the above questions, which makes the results of the evaluation of anti-stigma campaigns difficult to interpret.

Measuring long-term effects

Other questions arise as well. Among them is the uncertainty about the long-term effects of relatively minor changes. Would a change in attitudes of school children (such as that produced by a school anti-stigma programme) affect the behaviour of the children when they grow up? This question cannot be answered by a survey measuring the duration of a change of attitudes, for at least two reasons. First, because even if the change of attitudes cannot be demonstrated to persist, some of the decisions of children and their parents (e.g. concerning a change of school) might have been affected by the change of attitudes due to the programme - which, in turn, might lead to a variety of consequences; and second, because most attitudes change their relevance and effects on behaviour over time.

Yet another group of questions concerns the usefulness of results of evaluative research in cultures different from that in which the anti-stigma programme was originally done. In a rapidly globalising world this question is relevant because it can be expected that programmes undertaken in one country will inspire similar efforts elsewhere. It can be argued that the processes of globalisation and ever-increasing rates of migration will diminish the differences between cultures and that exploration of the appropriateness of methods used in the evaluation and development of programmes is therefore losing its importance; but this thesis has also to be approached by relevant research before we accept it.

Patients' views

The assessment of the effects of anti-stigma campaigns by people with mental disorders and their families is clearly a central element in the process of evaluation of such efforts. The TTC evaluative team was exemplary in its design to obtain information about the impact of the campaign seen through the eyes of people with mental disorders. It opted to use a well-tested assessment instrument that has shown its value in a variety of countries (the Discrimination and Stigma Scale) in order to explore experienced and anticipated discrimination and changes in such discrimination related to the campaign. ^{Reference Corker, Hamilton, Henderson, Weeks, Pinfold and Rose2} The results were interesting and encouraging, showing that the campaign had positive effects; what was somewhat disappointing was that the proportion of people with mental disorders who were approached and agreed to participate in the evaluation remained low despite an added letter of invitation by the local authorities and the offer of a modest (monetary) incentive. In addition the evaluation team tried to increase the proportion of responders - which was in keeping with usual rates of response for telephone surveys - by employing as interviewers people who had experienced mental illness, but this did not make a difference. It is possible to think of explanations for this finding, including a mistrust of authority, which people with mental illness (often rightly) do not consider their ally. Undoubtedly future evaluations will have to think of ways of enhancing the participation of people using mental health services in the evaluation of anti-stigma activities.

The addition of a significant evaluative component to the TTC programme has been an important step, and the results of the evaluation reported in this supplement are immensely useful; not only because they answer some of the questions about TTC and other anti-stigma programmes, but also because they remind us of the need to answer other questions and develop other methods of evaluation that can be put at the disposal of those who will - we hope - develop anti-stigma programmes in the near future.