How to appraise clinical guidelines

Results We unable to identifyrecent guidelines. Theappraisal tool we usedfailed to identify some of the difficulties now apparent when considering the validity and utility of guidelines. Clinicalimplications Criticalappraisaltoolsare useful in providing a framework for assessing published material, but runthe riskof blinding uswith theirsimplicity.

The principles of critical appraisal that under pin evidence-based medicine (EBM) can be applied to a diverse range of papers. Although the evidence-based approach is frequently used for papers on therapy and diagnostic tests, many other types of publication lend them selves well to the EBM process. Practice guide lines are "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" (Anonymous, 1994). The implementation of guidelines has been shown to improve the process of health care delivery, although the effect on patient outcome is less compelling. Recent years have seen an increase in the number of clinical guidelines published, as well as some debate about the legal standing of these documents. Like most other publications, guidelines should not be taken to be valid and useful without a critical appraisal.

Use of clinical guidelines for antidepressant prescribing
Vignette A middle-aged woman with a six-month history of a depressive disorder was referred by her general practitioner for advice about further management. The case raised some questions about the pharmacological management of de pression. Given the variety of antidepressants now available, we felt that the choice of agent had become a complex issue. We sought published guidelines to aid us with the decision of which antidepressant to prescribe.

Question
In patients with depressive disorder are there adequate practice guidelines concerning the appropriate pharmacological treatment?

Literature search
The literature review was initially undertaken in Embase rather than Mediine, as Embase tends to be better at identifying pharmacologically-based papers. Using the headings 'antidepressant agent' and 'practice guidelines', three papers were identified in 1996-97 but none of these appeared relevant. A further three were identified in 1994-95, but again these were not pertinent. Taking the search back to 1992-93 revealed no papers as 'guidelines', probably because this was not used as a subject heading term at that time. Next we did a textword search for 1992-93 using 'guidelines', which revealed 4323 papers. When combined with the 1510 papers on 'anti-depres sant agent', 27 papers emerged including one British-based paper that appeared to address our question: 'Guidelines for treating depressive ill ness with anti-depressants. The statement from the British Association for Psychopharmacology' (Montgomery et al, 1993).
No further articles of interest were identified when the search was repeated on Mediine using the medical subject heading 'depressive disor der', and limiting the articles to 'guidelines', or by combining 'depressive disorder' with a textword search on 'guidelines'.
Brie/"outline of the article The paper began with a brief discussion of terminology of depression. The authors dis cussed the shifting terminology in this field, and the article focused on the treatment of 'depression' as understood by the clinician, rather than cases defined by diagnostic criteria. There followedan overviewof the epidemiology of depression, suggesting that in the UK some 375 000 people a year have untreated "moderately severe depression". The authors stated that the response to antidepressants in controlled evaluations Is approximately twice that of placebo over four to six weeks and between 70 and 80% of patients on antidepres sants will ultimately improve. The efficacy of new antidepressants selective serotonin reuptake inhibitors (SSRIs) had been established in placebo controlled trials and in comparison with tricyclic antidepressants.
The guidelines stated that long-term efficacy of SSRIs in reducing subsequent episodes of depression has also been established. Less cardio-toxic antidepressants should be used in those who are thought to be of suicide risk and sub-therapeutic doses should be avoided (although dose guidelines are not given). The authors conclude that 'new' antidepressants have real advantages in practical terms in the treatment of depression. They are safer and their better tolerability makes them easier to reach an effective dose and because they are better tolerated patients are more willing to continue taking medication, improving their chances of response. The guidelines suggested that treatment should continue for at least four months at full antidepressant dose. Sackett et al (1997) define clinical guidelines as "user friendly statements that bring together the best external evidence and other knowledge necessary for decision making about a specific health problem". We appraised the guidelines here using the suggestions of the EBM working group (Haywood et al 1995;Wilson et al 1995).

Are the recommendations valid?
Were all important options and outcomes consider ed? The authors discuss side-effects, compli ance, toxlcity and safety in overdose of the various antidepressants.
They suggest caution with re gard to the SSRIs because of the paucity of longterm clinical experience. Other pharmacotherapeutic options such as lithium and other mood stabilisers were not considered in this paper.
Was an explicit and sensible process used to identify selecting and combining evidence? No. There were only 15 references altogether in these guidelines and the authors did not appear to have systematically appraised all the evidence. To a certain extent there is conflict between making a series of guidelines user friendly by not overwhelming the reader with the evidence underpinning the guidelines and on the other hand providing the evidence to enable the reader to make a judgement as to how good the guidelines are.
Was an explicit and sensible process used to consider the relative value of different out comes? Not really. Although certain outcomes can be easily measured (e.g. the improvement in mood state in antidepressant treatment), the utility of other outcomes (i.e. the meaning that has for the patient) varies considerably from individual to individual. For example, some individuals may not view the presence of anticholinergic side-effects as important as recover ing from depression, others may. The guidelines reviewed here did not explore the relative value of different outcomes.
is the guideline likely to account for important recent developments?
There is a paucity of cited evidence in this publication: only one reference is published after 1990. However, the paper does make reference to paroxetine, fluvoxamine and sertraline, so given that it was published in 1993 it was relatively up-to-date. Since the publication of these guidelines there have been many more antidepressants introduced, including venlafaxine, reboxetine and mirtazapine. In summary, therefore, the guidelines probably were reason ably up-to-date at the time of publication but not now.
Has the guideline been subject to peer review and testing? The Journal of Psychopharmacology is a peer reviewed journal and many of the authors are established and respected authorities In psychopharmacology.
One would assume that the paper had been subjected to external peer review.
What are the recommendations? Are practical clinically important recommend ations made? Yes. The main message of this paper is to underline the fact that patients with depression need treatment, they need it at adequate doses and for a long enough period of time.

How strong are the recommendations?
The recommendations are reasonably strong without being didactic. The reader is left with the impression that SSRIs may be preferable to the tricyclic antidepressants but the paper is not overbearing in its recommendations. Are the recommendations applicable to my patient? Yes. In this case we may feel confident in proceeding with a course of either tricyclic antidepressants or SSRIs at the full dose.

Comment
The guidelines appraised here were a readable and helpful document on the treatment of depressive illness with antidepressants. They are now out-of-date as there have been many new developments in antidepressant treatments since they were published and a lot more experience has been gained with SSRIs. Although, these guidelines suggest SSRIs are better tolerated than tricyclic antidepressants, meta-analyses comparing discontinuation rates of these compounds have had differing conclu sions (Song et al 1993;Anderson & Tomenson. 1995;Hotopf et al 1997). Examination of these meta-analyses would be another interesting EBM exercise. The other relevant guideline on the treatment of depressive disorder in adults identified in our literature search was published by the American Psychiatric Association (1993). Given that our literature search failed to identify a more recent publication, the scope for answer ing our original question was limited.
Our appraisal tool could also be considered out-of-date and too simplistic. Guidelines pub lished now should include a statement of the shelf-life, beyond which the recommendations should be revised or disregarded. The target populations for the guidelines should be speci fied. Furthermore, any conflict of interest of the authors and a statement of funding source for the guidelines should be made explicit.
The guidelines assessed here do not meet all the rigorous tests of validity outlined in the papers published by the EBM working party. This does not mean that the guidelines are not helpful. The past few years has seen an expan sion of the knowledge and understanding of what constitutes a good publication. The main criti cism of these guidelines is that they appear to represent a consensus statement of a group of psychopharmacologists rather than appraisal of all information at hand. The views of clinicians in primary care and patients are not considered.
If these guidelines were to be rewritten now, almost certainly they would be much longer, not only because of the expansion of knowledge but also because of the impact of evidence-based practices. Whether such a publication would be 'better' is perhaps more difficult to decide. Overall, we felt this exercise was helpful in learning how to appraise a set of guidelines, although the guidelines themselves were of limited use.