The internet is increasingly used as a source of health-related information. Reference Powell and Clark1 However, the content and quality of many health-related websites has often been characterised as poor. Reference Eysenbach, Powell and Kuss2 For example, one review of 79 studies, Reference Eysenbach3 in which 5941 websites were evaluated, found that the quality of the material was ‘deemed to be a problem’ in 70% of the websites examined. Website quality is largely determined by tools or instruments that measure the extent to which the online information complies with designated standards or criteria. Reference Gagliardi and Jadad4,Reference Bernstam, Shelton, Walji and Bernstam5 Any information on the website detailing clinical knowledge should be based on recognised policy guidelines or current evidence. Websites containing material that concurs with current practice therefore receive higher scores than websites displaying material that does not. The principles for publication in peer-reviewed literature also apply in that the more transparent the information, the higher the score assigned. Reference Silberg, Lundberg and Musacchio6
The physical characteristics of a website, such as its design, layout and readability, are also evaluated and are often standardised across disease states. Kisely et al in their review of mainly American and Australian schizophrenia websites, Reference Kisely, Ong and Takyar7 found that agreement with evidence-based practice (i.e. guidelines produced by the American Psychiatric Association and Western Australian Psychotropic Drugs Committee) ranged from 2 to 55% depending on the recommendation under investigation. For example, just over half of their sample (53%) made a reference to atypical antipsychotics and only 5% mentioned ‘that clozapine is indicated after non response to 2 other antipsychotics’. Furthermore, the majority of websites required a ‘high’ reading level to understand the material, despite evidence that individuals with schizophrenia often struggle with reading and writing. Reference Condray, Steinhauer and Goldstein8,Reference Morice and McNicol9 It would therefore be reasonable to expect that websites offering information about the care and treatment of schizophrenia be written in an easy-to-read manner. A recent analysis of the readability of 20 UK-based schizophrenia websites Reference Kalk and Pothier10 also confirmed Kisely et al's findings that the readability of schizophrenia websites is poor, implying that the material is difficult to read or comprehend. The authors classified the readability of 40% of their websites as ‘very difficult’ and 55% as ‘difficult’ when based upon scores calculated by the Flesch Reading Ease Score (FRES). Reference Flesch11 The FRES is based on the number of syllables per word and the number of words per sentence. A FRES >60 is considered appropriate for consumer-orientated information. Reference Clement, Wilson and Bingham12
Knowledge of schizophrenia is generally poor among the British public. Reference Luty, Fekadu and Dhandayudham13 It is therefore possible that the families, friends and other associates of people with schizophrenia will look to the internet for information about the condition. There are currently no published studies that describe the contents or quality of UK schizophrenia websites. This dearth of knowledge, particularly on the status of information about antipsychotic medication, is concerning because some individuals may make decisions affecting their treatment based on information they have found on the internet. Clinicians may therefore find it useful to know about the quality of information on antipsychotic medication that is available on non-specialist UK schizophrenia websites.
Method
A pro forma, the Strathclyde Website Evaluation Form (SWEF) was originally developed to characterise and evaluate the contents of websites related to the management, in particular the pharmacological treatment, of attention-deficit hyperactivity disorder. A detailed description of the design process has been published elsewhere. Reference Akram, Thomson, Boyter and Morton14 For the present study, Section 1 of the SWEF, which examines the clinical content of the website, was adapted to reflect issues pertaining primarily to the pharmacological treatment of schizophrenia. Attributes under investigation were concerned with descriptions of disease aetiology, symptomatology, diagnosis, the different types of antipsychotic drugs and their side-effect profile. Since the SWEF is biased towards evaluating information on medication, other therapies associated with the treatment of schizophrenia were not considered. Section 2 of the SWEF was concerned with the physical properties of the website, including its layout and accountability. Readability was determined by the FRES. Each criterion (or attribute) under investigation was given a score between 0 and 2. A score of 2 was awarded if the website fully satisfied the criterion under investigation, 1 if it was partially fulfilled and 0 if it was unfulfilled. The maximum achievable score for a website was 52 (32 for website content; 20 for physical properties). The SWEF was piloted on a sample of North American schizophrenia websites. Interrater reliability was determined by G.A. and A.O. on a sample of Canadian schizophrenia websites. Their scores were compared and the resulting reliability coefficient (kappa) was found to be 0.73.
For the main study, the phrase ‘schizophrenia’ was entered into Google, Yahoo, Ask and MSN search engines on 17 December 2008. (This approach was taken because most individuals tend to initiate searches by entering the name of the disease or condition directly into a generic search engines. Reference Peterson, Aslani and Williams15 The lay public are also unlikely to be familiar with the term ‘antipsychotic’ and the names of individual drugs.) The first 100 URLs from each search engine were visited. Websites that were inaccessible, duplicates or not specifically about schizophrenia were excluded and the reminder stored for subsequent evaluation. The final sample of websites was split into four categories based on the characteristics of the host organisation. These were: charities and support groups; commercial organisations; government and professional bodies; and a miscellaneous category that included web-based ‘Ask the Doctor’ type organisations.
Data were analysed using SPSS version 16 for Windows. Differences in mean scores among host categories were compared by analysis of variance using Bonferroni correction for multiple tests. Statistical significance was defined as P<0.05.
Results
A total of 33 websites were identified and evaluated using the SWEF. The commercial and government/professional categories contained 9 websites each; the charity/support groups 8 websites; and the miscellaneous category contained 7 websites. The total SWEF scores for the sample were normally distributed with a mean total score of 26 (s.d. = 8) out of a possible 52. The miscellaneous websites scored highest with a mean value of 30 (s.d. = 9.3). No significant difference was found in mean scores among the different host categories (F (3,29) =1.44, P = 0.25). Table 1 summarises the mean scores per host category. Online Table DS1 details the criteria used to asses the websites and the subsequent scores.
Strathclyde Website Evaluation Form (SWEF) scores per website category
| SWEF, mean (s.d.) | |||
|---|---|---|---|
| Website category | Total score (max. 52) | Content score (max. 32) | Physical score (max. 20) |
| Charities/support groups, n = 8 | 23.6 (6.5) | 12.6 (6.2) | 11.0 (1.7) |
| Commercial organisations, n = 9 | 23.0 (7.4) | 13.4 (5.8) | 9.6 (2.9) |
| Government/profession, n = 9 | 27.9 (8.8) | 15.4 (5.7) | 12.4 (3.3) |
| Miscellaneous, n = 7 | 30.1 (9.3) | 20.4 (7.3) | 9.7 (2.7) |
| Total, n = 33 | 26.0 (8.2) | 15.3 (6.6) | 10.7 (2.9) |
The majority of websites provided detailed information about the UK prevalence and incidence of schizophrenia and how it differs between genders. Just under half of the sample (48.5%) provided a detailed explanation of the multifactorial nature of the causes of schizophrenia. The prognosis was poorly described, with only nine websites mentioning that the earlier the age of disease onset the worse the outcome. All websites described the positive symptoms, but 16 of these also described negative symptoms and the effects on cognition. Most websites (n = 23) mentioned antipsychotic medication but only a few described the differences between the atypical and typical classes in any detail. Ten websites mentioned at least two or more side-effects expected from typical antipsychotics and provided an explanation for the potential co-prescription or use of anticholinergic medication if side-effects were to appear. Side-effects from atypical antipsychotics were less widely discussed, with only four websites mentioning four or more recognised side-effects (including anxiety, agitation, constipation, diabetes, drowsiness, dry mouth, gastrointestinal disturbance, increased appetite, weight gain). Clozapine-specific side-effects were described in even less detail, with only five websites mentioning the risk of seizures and agranulocytosis. The use of clozapine for treatment-resistant schizophrenia was mentioned on nine websites but only two detailed how its supply (prescription) is linked to the outcome of regular blood monitoring that is further characterised according to levels of risk. The majority (79%) of websites did not mention injectable formulations and only three mentioned both risperidone long-acting injection and conventional depots with descriptions of how they differed. In total, 25 websites (76%) did allude to the long-term nature of antipsychotic therapy, with six websites providing additional information about the need for good adherence to medication and its importance in avoiding relapse.
The majority of websites studied (79%) did not provide authorship details and only two provided the author's name, professional credentials and contact details. Currency was reasonable: 20 websites had been updated or created within the past 3 years. Almost all the websites were easy to navigate and over half provided a search facility on the main or home page. A large number of websites (n = 23) had links to external websites, most notably to the Royal College of Psychiatrists mental health information page and NHS Direct. Advertising was generally absent or extremely limited, although there were three websites on which it was found to be rather obtrusive. One instance was on the GMTV website, which displayed various pop-ups about other ‘interesting topics’ seen on the programme. The other two instances occurred on the online diagnosis or ‘Dear Dr’ websites in the miscellaneous category. Picture images or graphic representations were largely absent, with the majority of information provided as written text. A total of 20 websites displayed both a disclaimer and advice to consult a health professional, but 10 sites did not have any such information visible. Four websites recorded a FRES >60. These were www.lilly.co.uk, www.bbc.co.uk, www.makingspace.co.uk (a support group) and www.surgerydoor.co.uk (an internet-based health site). The URLs of the 10 highest scoring websites are given in Table 2. Total scores ranged from 30 to 43. Several websites share final positions because of identical total scores. The URLs of all 33 websites studied are provided in the online supplement.
Top ten schizophrenia websites according to their total Strathclyde Website Evaluation Form score
| Position | Website URL | Total score |
|---|---|---|
| 1 | http://www.rcpsych.ac.uk/mentalhealthinformation/mentalhealthproblems/schizophrenia.aspx | 43 |
| 2 | http://www.nhs.uk/Conditions/Schizophrenia/Pages/Introduction.aspx?url=Pages/what-is-it.aspx&r=1&rtitle=Schizophrenia+-+Introduction | 40 |
| 2 | http://www.netdoctor.co.uk/diseases/facts/schizophrenia.htm | 40 |
| 4 | http://en.wikipedia.org/wiki/Paranoid_schizophrenia | 39 |
| 5 | http://www.thesite.org/healthandwellbeing/mentalhealth/otherconditions/schizophrenia | 35 |
| 6 | http://www.makingspace.co.uk/ | 32 |
| 6 | http://www.patient.co.uk/showdoc/23069111/ | 32 |
| 8 | http://www.psychiatry24x7.com/bgdisplay.jhtml?itemname=schizophrenia_about | 30 |
| 8 | http://www.oneinonehundred.co.uk/ | 30 |
| 8 | http://mentalhealthcare.org.uk/content/?id=188 | 30 |
Discussion
Main findings
Information about atypical antipsychotic drugs on UK schizophrenia websites was lacking and superficial. Nowadays, atypical antipsychotics tend to be used as first-line treatment in schizophrenia. However, the websites under investigation gave the impression that typical antipsychotics were the main form of treatment in the UK by placing a disproportionate emphasis on these compounds. (The latest guidance from the National Institute for Health and Clinical Excellence (NICE) 16 no longer makes a distinction between the two classes. However, the document was not published before the websites were investigated). Where atypical antipsychotics are mentioned, relatively few websites detail the differences between the two classes.
Information about medication, in particular ‘what it does’ and its side-effects are found to be the main reasons for online health-seeking behaviour. Reference Peterson, Aslani and Williams15,Reference Raynor, Savage, Knapp and Henley17 Furthermore, exposure to information about side-effects can have a positive effect towards medication adherence. Reference Keown, Slovic and Lichenstein18,Reference Howland, Baker and Poe19 However, information about antipsychotic drug side-effects, particularly those arising from atypical antipsychotics, was found to be rather vague. Most websites did list weight gain as a potential side-effect, however only a few, if any, quantified the likelihood or extent (in kilograms) of potential weight gain. This is important as the risk and extent of weight gain varies substantially between antipsychotics. Reference Taylor and McAskill20
It is possible that the typical group of antipsychotics continue to dominate information sources because their side-effects (akathesia and tardive dyskinesia) are thought to be more serious because they are more noticeable and can be irreversible. However, side-effects from the atypical antipsychotics can be equally serious (e.g. weight gain, metabolic syndrome, agranulocytosis) and there appears to be no justifiable reason for this lack of information about their side-effects. In a survey conducted by the National Schizophrenia Fellowship, 21 half the sample (n = 260) reported that they had not been given any information about antipsychotic drug-induced side-effects from their doctor or psychiatrist. It is therefore quite likely that individuals in this position will turn to the internet to seek the required information.
A significant proportion of individuals with schizophrenia may not respond to conventional antipsychotics and hence are likely to become possible candidates for clozapine. Due to the risk of agranulocytosis, clozapine can only be used if the patient undergoes regular blood monitoring. The monitoring procedure and dispensing arrangements of clozapine can become fairly complicated; however, if this information were more widely available to people with schizophrenia and their carers, it could be beneficial. This was not the case in the majority of websites. One reason for the lack of clozapine-specific information may be because none of the manufacturers and UK distributors of clozapine appeared in the original search results. Perhaps if the phrase ‘clozapine’ had been used as a search term, clozapine-specific websites would have been identified. However, medication-specific searching would have resulted in a deviation from the predefined study methodology and so was not utilised.
Details about the injectable antipsychotics were also limited. Administration of injectable antipsychotic medication is often associated by the general public with degrading practices, whereas it can offer a convenient alternative to regular oral medication. Furthermore, unlike the traditional depot injections, the formulation of risperidone long-acting injection demands that the individual must adhere to the fortnightly cycle to avoid periods without antipsychotic cover. Reference Janssen-Cilag22 This important, albeit technical, information was not mentioned on any website. People with schizophrenia or their carers may not appreciate the reasons for such rigid adherence to the fortnightly cycle and may mistakenly assume that being a couple of days ‘out of synch’ will have no real consequence.
The highest scoring website was hosted by the Royal College of Psychiatrists. The College has in recent years, made a concerted effort to address discrepancies in information provision in mental health, for example, the ‘Defeat Depression’ and ‘Changing Minds’ campaigns. 23,24 Unfortunately, the profession is not always viewed as a neutral source of information Reference Nasser25,Reference Breggin26 and some individuals may view information on the Royal College of Psychiatrists' website with suspicion. In contrast, health information provided on websites developed by governments tend to have a more positive reception. 16 However, the government/professional category was not the highest scoring category in the present study. One explanation for this finding is that some of the websites belonged to government agencies unrelated to health, for example, the Department for Work and Pensions. This website was aimed at people who were claiming social security benefits and although it contained some information about schizophrenia and its treatment, detailed descriptions of medications were absent. A number of health professional organisations that specialised in counselling and other psychological therapies also appeared in the original search results and so had been evaluated. The overall mean score for the government/professional category was reduced by these websites because they did not focus on antipsychotic treatment.
Strengths and limitations
The methodology used in the evaluation process may have influenced the poor scores observed for the charities and support groups. For example, some of the charities, particularly Mind and Rethink, have entire web pages (or pdf copies of factsheets) devoted to antipsychotics, but these pages are not directly linked to the main website that was evaluated. If there was hyperlinked-text embedded within the main page that linked directly to a page on medication, then that link was followed and the subsequent page evaluated. However, this was only done if the page was accessible in one click and formed part of the original website. Such an approach is likely to have disadvantaged some organisations and it is possible that the final scores and the positions of websites in the top ten would have been different if we had allowed a more flexible approach to identifying additional information. This explains why some websites from reputable mental health organisations performed poorly compared with the generalised health and beauty websites that comprised the miscellaneous category.
The physical components of the websites generally scored well. Most websites were relatively up to date when compared against an arbitrary 3 years. Unfortunately, we did not record actual dates so cannot differentiate between those that were only 12 months old compared with those that may have been 24-36 months old. Nevertheless, nine websites did not display any details about their date of origin or last update. This makes it difficult to assign credibility to the information as one cannot be sure of its relevance. In keeping with the literature, the websites chosen also scored poorly when judged against the FRES. However, the FRES was calculated on text in the medication section, which may contain slightly more scientific language or polysyllabic words that could unduly influence the score. In contrast to the sample described by Kalk & Pothier, Reference Kalk and Pothier10 only 4 of the websites scored above 60 in the present study. As the URLs and web addresses of the Kalk & Pothier websites are unknown, it is difficult to compare results or comment on why this discrepancy arose. Nevertheless, it is unfortunate that so few websites were written in an easy to read manner.
It is also possible that had a different search strategy been used, different websites would have been identified; therefore the reproducibility of this study might be questioned. However, it was our intention to mimic the strategy most likely to be used by the average lay person who only visits about two or five websites in one sitting and rarely uses a defined search strategy. Reference Fox and Raine27 Few individuals go beyond the first page of their search results and websites where the key words appear on the website description of URLs are generally preferred. Reference Peterson, Aslani and Williams15,Reference Adams, de Bont and Berg28 Our search strategy, however simplistic, was therefore similar to one that would be applied by the average person.
This study was intended to provide guidance for clinicians about the nature of information about anti-psychotics on UK schizophrenia websites that are likely to be accessed by the general public. Although there are some issues concerning the completeness of the information, it is reassuring that no instances of false or misleading information were identified.
Acknowledgements
G.A. would like to thank Adebunmi Ogunnowo who acted as an independent rater during the design process.
Open access
eLetters
No eLetters have been published for this article.