For 13 years the Royal College of Psychiatrists’ Public Education Committee has been producing award-winning public information leaflets (Fig. 1). There are now over 15 titles in the ‘Help is at Hand’ series (http://www.rcpsych.ac.uk/info/help/index.htm). These cover core disorders, such as schizophrenia, and more general topics, such as bereavement. Over 5 million leaflets have been distributed free of charge to the public, family practitioners, psychiatric units, libraries - even funeral directors. General practices and clinics distribute some, but most have been supplied directly by the College. Some have been translated into other languages such as Chinese and Spanish, although the majority of our materials are still available only in English. More recently it has become clear that:

1. • The leaflets had a traditional, prescriptive, approach; they needed to move towards a more dispassionate provision of information, to serve as a tool to support clients to make better-informed choices.

2. • To ensure that our information is perceived to be unbiased, we need to move away from the pharmaceutical sponsorship which has funded the printing of the leaflets.

Fig. 1.

Some of the ‘Help is at Hand’ leaflets.

In 2002 the Editorial Sub-Committee of the Public Education Committee was established to expedite these developments. It consists of three College members and five members of staff. The Sub-Committee's activities are summarised in Box 1.

Box 1.

Role of the Editorial Sub-Committee of the Public Education Committee in leaflet development

The Editorial Sub-Committee:

• identifies areas to be covered by the series • identifies suitable partner organisations • commissions experts to provide the necessary information and advice • writes the text • scores the draft using DISCERN criteria • obtains feedback • identifies sponsors and partners • considers distribution • involves service users and advocates • ensures regular review of existing publications

Although the ‘Help is Hand’ leaflets have always been popular, there are some areas of concern. The Patients’ and Carers’ Liaison Group has vetted our leaflets, but there has been no other direct user or carer input. We have probably omitted significant issues of importance to patients, carers and relatives. Informal feedback in the form of letters to the College has generally been positive, but there has been no structured evaluation. Our ability to revise leaflets has been determined not by the need for updating, but by exhaustion of the stock and the vagaries of sponsorship.

Leaflet ratings

Some limited independent assessment of the ‘Help is at Hand’ leaflets has been carried out. Currie et al (Reference Currie, Dimambro and Joice2002) used the DISCERN criteria to evaluate a number of different leaflets on schizophrenia (DISCERN is an instrument that rates several areas, including relevance to the target audience, the ground covered, the presence of references, and even-handed coverage of alternative views or treatments [http://www.discern.org.uk]; Reference Charnock, Shepperd and NeedhamCharnock et al, 1999). The ‘Help is at Hand’ leaflet was rated as only ‘fair’ in this assessment. The committee now uses the DISCERN criteria to screen all our leaflets at the first draft.

Coulter et al (Reference Coulter, Entwistle and Gilbert1998) assessed seven leaflets on depression using two focus groups, one of people who had had significant depressive episodes and another of specialist experts. The College leaflet was rated top by the specialist group, which felt that it ‘provided the most useful information for patients who wanted to be informed participants in decisions about their care’. The patients group, on the other hand, appreciated the information but did not like the small print size and tone, so ranked it sixth out of the seven publications.

Humfress & Schmidt (Reference Humfress and Schmidt1999) used the Gunning FOG Index. This tests readability by scoring the average sentence length and the number of long words in a piece. The College leaflets had a median score of 13, compared with 15 for the Mind series of leaflets. A score of 15 is roughly equivalent to the level of a BMJ article. To make sure that most people can read our leaflets we should be aiming for a score of around 10. This represents the reading age of 12-13 years, the level at which most tabloid newspapers are written.

Online feedback

Useful though they have been, none of the above assessments has explored how ‘live’ readers of the leaflets were responding to them. The establishment of the College website has presented us with a way of doing this. In fact, ‘Mental Health Information’ has become the most popular part of the College website. The most viewed leaflets get about 160 true visits (as opposed to ‘hits’) a day. On average each visitor spends about 6 min on a leaflet page. We can now receive direct feedback from readers. Each leaflet has an online feedback form and we are currently receiving around 100 responses every week.

Survey of online responses

We analysed all the feedback forms completed online between 18 December 2002 and 7 October 2003. These forms were the original questionnaire, consisting of five forced-choice scales (Fig. 2) and a free text area. An overall score for each leaflet was obtained by awarding points for each response (5, agree strongly; 4, agree; 3, neutral; 2, disagree; 1, strongly disagree); in the case of the statement ‘This leaflet talks down to me’ the points were reversed, so that high scores consistently indicated a favourable ranking.

Fig. 2.

Online forced-choice feedback questionnaire.

The overall scores were high (Table 1), although the ‘This leaflet talks down to me’ item performed poorly relative to the other categories. This low score may have been due to the tone of the leaflets. However, it might also have been influenced by the fact that the scoring for this item worked in the opposite direction to other items on the screen. The feedback screen has now been changed so that all items score in the same direction, and we look forward to seeing if there is any overall change in the scores for this item.

Table 1.

Online ratings of ‘Help is at Hand’ leaflets

		Feedback scores: mean (s.d.)
Leaflet1	Responses (n)	Readable	Useful	Talks down	Well designed	Recommended	Overall
Manic Depression	732	4.5 (1.1)	4.5 (1.1)	3.1 (2.9)	4.1 (1.2)	4.2 (1.2)	4.3 (0.5)
Depression	149	4.4 (1.2)	4.4 (1.1)	3.7 (2.9)	4.2 (1.0)	4.2 (1.3)	4.2 (0.3)
Bereavement	298	4.6 (0.9)	4.5 (0.8)	3.3 (3.0)	4.2 (1.0)	4.3 (1.0)	4.2 (0.5)
Post Natal Depression	397	4.6 (1.0)	4.6 (1.0)	3.1 (2.9)	4.2 (1.0)	4.5 (1.0)	4.2 (0.6)
Memory & Dementia	150	4.6 (1.0)	4.6 (0.9)	3.1 (2.6)	4.1 (1.2)	4.4 (1.1)	4.2 (0.6)
Alcohol & Depression	101	4.6 (1.0)	4.5 (1.0)	3.1 (2.9)	4.0 (1.3)	4.0 (1.5)	4.0 (0.5)
Men Behaving Sadly	113	4.5 (1.0)	4.4 (0.9)	2.9 (2.7)	4.1 (1.0)	4.0 (1.3)	4.0 (0.6)
Depression in People with Learning Disability	47	4.3 (1.4)	4.3 (1.4)	2.8 (2.7)	4.0 (1.4)	4.3 (1.0)	4.0 (0.6)
Schizophrenia	182	4.4 (1.2)	4.4 (1.2)	2.7 (2.6)	4.0 (1.3)	4.2 (1.4)	3.9 (0.6)
Sleeping Well	117	4.2 (1.4)	4.1 (1.3)	2.7 (2.9)	3.9 (1.3)	4.0 (1.4)	3.8 (0.5)
Social Phobias	242	4.2 (1.4)	4.0 (1.4)	2.9 (2.7)	3.9 (1.3)	3.9 (1.6)	3.8 (0.5)
Anxiety & Phobias	443	4.2 (1.4)	4.1 (1.4)	2.8 (2.7)	3.9 (1.3)	3.9 (1.4)	3.8 (0.5)
Surviving Adolescence	193	4.1 (1.3)	4.0 (1.3)	2.7 (2.7)	3.7 (1.4)	3.7 (1.6)	3.7 (0.5)
Anorexia & Bulimia	1070	4.1 (1.4)	4.0 (1.4)	2.5 (2.5)	3.5 (1.5)	3.8 (1.6)	3.6 (0.6)
Total	4234

If this item is omitted, approval ratings for all categories were over 70%. This held even for the leaflets on schizophrenia and eating disorders, where individuals more often find themselves in conflict with psychiatric notions about their condition. This information is reassuring, but also highlights areas for review. For example, since obtaining this feedback, we have rewritten the adolescence leaflet and retitled it explicitly as A toolkit for parents. We hope that this will reduce the ‘talks down’ score, and are looking at ways of developing a leaflet exclusively for adolescents.

Discussion

Overall, all the leaflets in the series did well. The feedback has allowed us to identify those that are performing less well, and these now have priority for rewriting. The feedback forms seem acceptable to our readers and will be attached to all leaflets produced by the College. We will also add further items:

1. • to identify more clearly who the reader is - for example, a service user, a carer, a student or a member of the public;

2. • to identify the national location of the reader, to help identify and quantify our global audience.

The webpage feedback has demonstrated that it is possible to make available a wide range of understandable and useful mental health information directly to the public. Leaflets can now be revised and published on the website without the delays inherent in paper publication. However, paper is still an important medium, especially for use in the context of a consultation. Such information has historically not been available in specialist mental health settings. Having established the broad acceptability of our information products over the web, we are now marketing paper versions of the leaflets to all mental health providers in the UK. Unlike content published on the web, we cannot guarantee that paper leaflets reach their intended readership. So, the next step in our evaluation will be to ascertain the penetration of leaflets purchased by health organisations into settings where they should be available to workers, clients and carers.

Declaration of interest

The authors have all been involved in the writing and editing of the ‘ Help is at Hand’ leaflets, but have received no payment for this activity.