Mark Taylor, co-chair of the Scottish Intercollegiate Guidelines Network (SIGN) guideline on schizophrenia published in 2013 1 (referred to hereafter as SIGN 131), wrote an editorial for this journal (co-authored by Udayanga Perera) claiming that the National Institute for Health and Care Excellence (NICE) guideline on psychosis and schizophrenia 2 (referred to hereafter as CG178) was open to a critique of bias, whereas SIGN 131 was unbiased and evidence-based. Reference Taylor and Perera3 They claimed that CG178 showed positive bias to psychosocial interventions, in particular cognitive–behavioural therapy for psychosis (CBTp) and arts therapies, and showed negative bias to drug treatment.
Given the status that evidence-based guidelines in mental health now have in psychiatric practice, it is important to understand how one set of evidence can lead two guideline developers to two seemingly divergent views. It is important, and not mentioned in Taylor & Perera's editorial, that both SIGN 131 and CG178 are updates of the 2009 NICE guideline on schizophrenia 4 (referred to here as CG82). We will describe the institutions, processes and methodologies used for developing the NICE and SIGN guidelines, and then examine the claims made by Taylor & Perera about CG178.
NICE guideline production and the National Collaborating Centres
NICE began providing evidence-based guidance for the National Health Service in England in 1999 and has rapidly become one of the most important innovations in health and social care in the past 50 years. The clinical guidelines programme is run by the National Collaborating Centres, including the National Collaborating Centre for Mental Health (NCCMH) which produced the very first NICE guideline (on schizophrenia) in 2002. The World Health Organization evaluated this as the world's best evidence-based guideline on schizophrenia, compared with 25 other national guidelines. Reference Gaebel, Weinmann, Sartorius, Rutz and McIntyre5 The NCCMH has updated the original NICE guideline on schizophrenia twice (2009 and 2014).
Methodology, quality assurance and conflicts of interest
Each NICE guideline takes just over 2 years to produce by a guideline development group (GDG), consisting of about 15 experts recruited through public adverts, including researchers, professionals, patients and carers, supported by a technical team, including systematic reviewers, health economists, information scientists, project managers and research assistants. The GDG is an independent advisory committee that strictly follows NICE's published methodology (CG178 used the 2012 version of the guidelines manual; 6 there was a major revision of the manual in October 2014 which unified methodologies across all NICE programmes). All processes and products are continuously monitored and quality assured by NICE's own commissioning managers, editors and technical support unit, and subjected to several stages of validation (including being critiqued by extensive stakeholder and external expert consultation and evaluation).
NICE takes the issue of conflicts of interest very seriously because of the potential threat to the integrity and transparency of the guidelines, and has a well-defined process for recording any interests and dealing with conflicts. If there are conflicts of interest related to certain topics, a GDG member would be asked to leave the GDG meeting for the period of time that the topic of interest was being discussed. If a GDG member discloses an interest that significantly conflicts with their role as a GDG member, they would be asked to leave the GDG entirely. Declarations of interest are included in the final guideline, and are in the public domain during the consultation of the guideline. Finally, and an important difference between CG178 and SIGN 131, is that a person cannot be appointed as a NICE guideline chair if they have a personal pecuniary conflict of interest. This appears not to be the case for SIGN 131.
NICE and SIGN on schizophrenia: different scopes and different methods
For CG178, the evidence reviews for pharmacological and psychological interventions were not updated since NICE's view (based on literature surveillance and expert consultation, including psychopharmacology experts) was that there had been insufficient new evidence since the publication of CG82 in 2009. Instead, NICE asked the NCCMH to expand the areas within the guideline to include self-management, carer experience, carer interventions and peer support, and to update service-level interventions such as early intervention and assertive community treatment, areas that did have new evidence that might change guideline recommendations. Most of the reviews undertaken were new reviews conducted by the NCCMH guided by the GDG. All processes and methods were subject to the usual, extensive quality assurance, expert review and stakeholder consultation. Furthermore, in addition to making available the full guideline and appendices, NICE publishes minutes of the GDG meetings and documents from both consultations (scope and draft recommendations). 2 As far as we can tell, none of this detailed information underpinning guideline development is publically available for SIGN guidelines.
For SIGN 131, the scope was extended to include psychosis with coexisting substance misuse and perinatal issues (for both of these areas, NICE has whole guidelines – see CG120 7 and CG192 8 ). SIGN 131 was largely based on CG82 and undertook a narrative synthesis of randomised controlled trials and other studies published between 2008 and 2011. The additional syntheses were undertaken by SIGN reviewers in conjunction with their GDG. The guideline did not undertake any de novo meta-analyses or update meta-analyses from CG82. SIGN 131 underwent consultation and peer review, but documents from this process are not made routinely available.
NICE and SIGN: how are they different?
The NICE and SIGN guidelines have a number of similarities, for example recommending the use of antipsychotics (including clozapine), family intervention, early interventions, assertive community treatment and CBTp. However, there are also many differences, ones which Taylor & Perera bring to the fore. We will deal with these in turn.
Psychosis and schizophrenia v. schizophrenia
Taylor & Perera criticise CG178 for including the term psychosis in the title as potentially ambiguous. The title change from ‘schizophrenia’ to ‘psychosis and schizophrenia’ came about through consultation with patient and professional groups who expressed the view that the guideline should update early intervention in psychosis services, which include people with early psychosis. It therefore made sense that the title reflected the content. The Schizophrenia Commission (independent of NICE) recommended exercising ‘extreme caution in making a diagnosis of schizophrenia as it can generate stigma and unwarranted pessimism’ and suggest ‘the more general term “psychosis” is preferable, at least in the early stages’. 9
Psychological v. pharmacological interventions
Taylor & Perera also suggest that, on a simple count of recommendations on psychological and pharmacological interventions, they have discovered bias in the NICE guideline. They assert that SIGN 131 is less biased because 60% of all its recommendations refer directly to drug treatments, whereas a mere 24% of recommendations in CG178 refer to drugs and most of those are in combination with psychosocial interventions. Given the very different scopes and total recommendations of the two guidelines, this assertion is meaningless. For the record, CG178 includes 110 recommendations, of which 24 (22%) are about psychological treatment and 31 (28%) are about antipsychotic medication. Only four recommendations are about both psychological and antipsychotic treatment. Counting recommendations on drugs and on psychological treatments has no grounding in evidence. The suggestion that these percentages mean that CG178 is biased against drug treatments and in favour of CBTp is, in our view, unfounded.
CBT as a panacea
Taylor & Perera also imply that CBTp is presented as a panacea. Certainly CG178 recommends that everyone with psychosis or schizophrenia should be offered CBTp on the basis of the systematic review and meta-analysis from 2009. The suggestion that a more recent, less favourable review Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws10 would have altered this recommendation ignores the fact that there were another four reviews of CBTp published last year, Reference Burns, Erickson and Brenner11–Reference Velthorst, Koeter, van der Gaag, Nieman, Fett and Smit14 and four of the five (including Jauhar et al Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws10 ) concluded that there were significant benefits to CBTp compared with treatment as usual or active control comparators. Additionally, their cited review Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws10 did not include any consideration of effects at follow-up, had not pre-registered their protocol and has been criticised Reference Peters15 for idiosyncratic inclusion criteria and drawing conclusions unjustified by the evidence.
Supposed bias in trials of CBT
Taylor & Perera also cite the importance of masking in studies of CBTp, with effect sizes being lower in blinded trials. Masking is an acknowledged problem in psychological treatment trials, which have the disadvantage that patients will know whether they receive the treatment or a comparator. Masked assessors will not know, however, and more recent CBTp trials include these. These sources of potential bias are also accounted for in NICE processes. Although double-blind drug treatment trials have the apparent advantage that patients do not know whether they receive the drug or its comparator, because side-effects such as weight gain or extrapyramidal effects are noticeable, both patients and assessors may be able to guess. These issues are rarely discussed in drug trials.
However, it is unfortunate that Taylor & Perera make no mention of another bias, predominantly associated with the drug industry, of selective publication of studies. NICE guidelines have played a leading role in reducing the impact of bias across the board, including selective publishing. Reference Kendall, Glover, Taylor and Pilling16 The NICE recommendation that CBTp be offered is made on the basis that some people respond and others will not, but we are currently unable to identify who is likely to benefit at an individual level. This is also the case for antipsychotics.
Taylor & Perera further propose that CBTp may be associated with specific adverse effects; although this is a possibility, current evidence, including from the very trials they cite, have shown fewer deteriorations Reference Morrison, Turkington, Pyle, Spencer, Brabban and Dunn17 and significant improvements in internalised stigma Reference Morrison, French, Stewart, Birchwood, Fowler and Gumley18 relative to comparators. Finally, in Taylor & Perera's conclusions, they assert that CG178 makes strong recommendations based on no evidence at all, for instance that the dose of CBT should be at least 16 planned sessions. This is untrue; justification is provided in the relevant section of the full guideline (see section 9.4.9), 2 and SIGN 131 contains the same recommendation.
CBT for at-risk mental states
For CG178, a meta-analysis of drug and psychological treatment trials for people thought to be at risk of psychosis (at-risk mental states; ARMS) was conducted. SIGN 131 did not examine treatments for ARMS. ARMS were originally evaluated in the NICE guideline on psychosis and schizophrenia in children and young people (CG155, 19 published in January 2013) and it was recommended that CBT should be considered as there was evidence of benefit, whereas treatment with antipsychotics appeared to show no benefit. For adults, CG178 strengthened this recommendation to ‘offer CBT’ for people with ARMS. This change occurred on the basis of inclusion of an additional trial in the meta-analysis that was conducted by the GDG. Reference Stafford, Jackson, Mayo-Wilson, Morrison and Kendall20 The conclusion that CBT could prevent transition to psychosis at 12 months in some people was also replicated by two independent meta-analyses Reference Hutton and Taylor21,Reference van der Gaag, Smit, Bechdolf, French, Linszen and Yung22 published in the period between CG155 and CG178.
CBT alone for first-episode psychosis
CG178 states that, to promote consistency with CG155, ‘[for people with first episode psychosis] the GDG saw the value in advising practitioners of the equivocal evidence regarding psychological interventions when compared with antipsychotic medication and recommended that if a person wished to try a psychological intervention alone, this could be trialled over the course of 1 month or less’ (p. 240). 2 A Cochrane review examining the effectiveness of antipsychotics v. placebo or psychosocial interventions in early schizophrenia concluded that the data are too sparse to assess the outcomes. Reference Bola, Kao and Soydan23 As Taylor & Perera observe, absence of evidence is not evidence of absence of an effect, and we have no evidence to suggest that medication must be the first-line intervention for people who retain decision-making capacity. The recommendation to allow choice underlines the commitment to shared decision-making and collaborative care that is emphasised in the NICE service user experience guideline, 24 and also reflects the inclusion of people with lived experience as core members of the GDG, which is a strength rather than a weakness, since guidelines that do not take account of the wider context (including human rights issues) could be harmful. Although the GDG concluded that adding an additional delay of 1 month to the duration of untreated psychosis was highly unlikely to have a deleterious effect on long-term outcomes, CG178 nevertheless recommends: ‘advise people who want to try psychological interventions alone that these are more effective when delivered in conjunction with antipsychotic medication’ (section 9.4.10.2). 2
Arts therapies
CG178 included recommendations from CG82, which were based on a systematic review of a range of different psychosocial interventions. Apart from CBTp and family intervention, no other psychosocial intervention except for arts therapies was recommended in 2009. There were sufficient trials to undertake a meta-analysis of arts therapies, including art therapy, music therapy and body-dance movement therapy. SIGN 131 did not undertake any review of arts therapies and yet Taylor & Perera criticise CG178 for giving a tentative recommendation for arts therapies in the treatment of negative symptoms. CG178 recommends that arts therapies may be considered for negative symptoms because early data suggested that arts therapies had an effect where drug treatments appear not to for negative symptoms. Moreover, the effect size for arts therapies in targeting negative symptoms was slightly larger than for CBT. As the intervention with the largest effect size, the GDG was justified in recommending this as a possible treatment. In doing so the GDG also increased the treatment options available to people with psychosis and schizophrenia. Updating these recommendations was not in the scope of CG178, and the more recent Matisse trial, not considered in CG82, is unlikely to change the recommendations. Reference Kendall25
Antipsychotics
CG178 included recommendations from CG82 for the use of antipsychotic medication, with some amendments for clarity and for consistency with CG155. These recommendations were based on several systematic reviews covering:
-
(a) initial treatment of people with first-episode or early schizophrenia
-
(b) oral antipsychotic medication in the treatment of the acute episode
-
(c) promoting recovery in people with schizophrenia that is in remission
-
(d) promoting recovery in people with schizophrenia whose illness has not responded adequately to treatment
-
(e) treatment with depot/long-acting injectable antipsychotic medication
-
(f) side-effects of antipsychotic medication
-
(g) effectiveness of antipsychotic medication based on pragmatic clinical trials
-
(h) health economic evidence.
Taylor & Perera make a number of factually inaccurate claims about these recommendations. First, they claim there was an absence of a relevant expert on the GDG, and this led to ‘non-specific and vague’ recommendations about antipsychotic pharmacotherapy. However, CG82, which developed these recommendations, included several experts in psychopharmacology. Second, they state the recommendations do not reflect the evidence that there are efficacy differences between antipsychotics. However, as can be seen in section 10.10 of the full guideline, the GDG considered this issue. The recommendations reflect their view that treatment with antipsychotics should be considered an explicit individual therapeutic trial, with a collaborative choice of antipsychotic made by patients and professionals together. Third, they claim that recommendation 10.11.1.11, 2 which recommends not using a loading dose, illustrates that the guideline has ‘overlooked that long-acting injectable paliperidone palmitate requires a loading dose’. However, this recommendation is in a section specific to use of oral antipsychotics. In the section on using long-acting antipsychotics, it is recommended that prescribers follow the British National Formulary or Summary of Product Characteristics. Finally, they suggest that CG178 does not include recommendations for ‘treatment-resistant schizophrenia and negative symptoms’ when these are, in fact, contained in sections 9.3.8.1 and 10.5. In our view, these assertions are both careless and irresponsible.
What could explain the differences between the NICE and SIGN guidelines?
The differences between the two guidelines in scope, methodology and rigour explain most, but not all, of the differences between SIGN and NICE on psychosis and schizophrenia. NICE probably does undertake a more exacting and reliable approach and has covered much more ground in much greater depth in psychosis and schizophrenia than SIGN have to date (NICE have five guidelines of direct relevance). However, the differences between NICE (CG178) and Taylor & Perera's views are much greater than between the content of CG178 and SIGN 131. It is important to restate that the NICE (CG178) and the SIGN (SIGN 131) guidelines are both based on the 2009 NICE guideline – CG82 – a fact not even acknowledged in Taylor & Perera's editorial. Indeed, their editorial uses the evidence, and guideline recommendations, selectively, and in so doing demonstrates surprisingly limited knowledge of both the 2009 (CG82) and the 2014 (CG178) NICE guidelines and the evidence on which these guidelines have been developed. In misrepresenting CG178, Taylor & Perera may themselves be guilty of bias.
In our view, disagreement, analysis and debate are essential aspects of an intellectual culture rooted in evidence-based medicine, whereas ill-supported accusations of bias and going beyond the evidence is not. This is why NICE has such rigorous methodologies and structures underpinning the production of their guidelines.
eLetters
No eLetters have been published for this article.