Evidence-based psychiatry : pride and prejudice

to medical training, but also to clinical practice, research activity, health care management, pur chasing and policy making. The proponents of EBM point towards an ever larger gap between research findings and clinical practice, as doctors are increasingly unable to keep up with vast amounts of new information. New technology can be used to bridge that gap. Instead of advocating a form of treatment because of tradition, clinical intuition or deference to a senior colleague, a doctor may go to a computer and look up the available research evidence. The Idea is very attractive. Doctors are kept up-todate in their practice, they continuously educate themselves, and patients receive the best treat ments. Similarly managers, purchasers and health politicians can make informed decisions based on an increasingly sophisticated database. The development of guidelines and protocols, enforced via the lever of contracting, may be seen as "bringing order into the chaos of widespread variations" in clinical practice (McKee & Clarke,

The term 'evidence-based medicine' (EBM) was coined to describe a learning strategy for medical students that emphasises the "process of system atically finding, appraising and using con temporaneous research findings as the basis for clinical decisions" (Evidence-Based Medicine Working Group, 1992: Rosenberg & Donald, 1995).We have now moved on to an era of "evidence-based everything" (Smith, 1995) with the term 'evidence-based' being attached not only to medical training, but also to clinical practice, research activity, health care management, pur chasing and policy making.
The proponents of EBM point towards an ever larger gap between research findings and clinical practice, as doctors are increasingly unable to keep up with vast amounts of new information.New technology can be used to bridge that gap.Instead of advocating a form of treatment because of tradition, clinical intuition or deference to a senior colleague, a doctor may go to a computer and look up the available research evidence.The Idea is very attractive.Doctors are kept up-todate in their practice, they continuously educate themselves, and patients receive the best treat ments.Similarly managers, purchasers and health politicians can make informed decisions based on an increasingly sophisticated database.The development of guidelines and protocols, enforced via the lever of contracting, may be seen as "bringing order into the chaos of widespread variations" in clinical practice (McKee & Clarke, 1995).
Despite its apparent simplicity EBM has become rather controversial.Much hostility was evident in a recent debate on the topic in the Lancet in which proponents of EBM hailed the approach as a paradigm revolutionising medical practice and caricatured traditional medicine as authoritarian, "oligarchic and closed" (Marshall. 1995).They pointed towards many "inexcusable delays and inexplicable variations in the incor poration of evidence into traditional medical practice" (Sackett, 1995).By contrast, critics of EBM have pointed out that it is not a new idea and that good medicine has always attempted to incorporate the best available scientific evidence.Others have called the term "an example of newspeak" that "would have delighted George Orwell" (Fowler, 1995) and have criticised its exponents for "their arrogance, their jargon, and their penchant for denigrating others . . .The steps and recommendations of the evidence based medicine acolytes reek of obfuscation and platitudes" (Morgan, 1995).The tone of the debate clearly distracts from the underlying issues of the significance, benefits and limitations of EBM.

Psychiatry and EBM
As a discipline with diverse psychosocial as well as biomÃ©dical roots, psychiatry has perhaps been more hesitant than other branches of medicine in embracing the approach.However, the American Psychiatric Association in developing DSM-IV made an explicit commitment to a "formal evidence-based process for the Work Groups to follow", and a recent review of the topic has appeared (Goldner & Bilsker, 1995).

How mach of what we do is based on scientific evidence?
While earlier reports gave an estimate of only 10-20% of medical interventions being based on scientific evidence, a recent study on general medical in-patients found that in 53% of patients the primary treatment was supported by rando mised controlled trials (RCTs; Ellis et al 1995).A replication of this work in 40 psychiatric inpatients suggests that 65% of main interventions are based on RCTs (Geddes et al 1996).Summers & Kehoe (1996) in a case-note review of 158 psychiatric patients identified 160 decisions to initiate new treatments (pharmacological, psy chological or social).In 53% of these interventions supporting evidence from RCTs was identified.The most frequent were specific drug treatments for depression and psychotic symptoms.A further 10% of interventions concerned decisions for which a trial would have been unethical (e.g. the decision to put a suicidal patient on close observation in hospital).The remaining 37% of interventions included practical measures and supportive psychotherapy.The authors of both studies concede that they only examined a limited range of clinical decisions.Moreover, this kind of study depends on the assumption that diagnoses and assessments of severity were accurate and it does not establish that the evidence-based decisions "represented the most effective or the most acceptable or cost-effective treatment for each individual" (Summers & Kehoe, 1996).
Even enthusiasts of EBM accept that "many areas of clinical practice cannot, or will not, ever be adequately tested" (EBM Working Group, 1992) for ethical and other reasons.In psychiatry this includes such difficult areas as the interface between the law and psychiatry.Some areas of psychiatry are perhaps more straightforward to study than others.This may be reflected in the choice of psychiatric papers included in the recently launched journal Evidence-Based Medicine which so far mainly seems to include articles from psychiatry on pharmacological treatment or cognitive-behavioural treatment of neurotic and somatoform disorders.
How good are the available data?One eminent statistician (Alunan, 1994). in point ing towards the poor quality of much medical research, held responsible doctors' "general failure to appreciate the basic principles underlying scientific research, coupled with the 'publish or perish' climate".Altaian goes on to point out that "huge sums of money are spent annually on research that is seriously flawed through the use of inappropriate designs, unrepresentative samples, small samples, incorrect methods of analysis, and faulty interpretation".Even In a high-impact journal like the British Journal of Psychiatry, the statistical error rate is 40% and there is no evidence that this is improving (McGuigan. 1995).Although not all of the errors are severe, some are serious enough to cast doubt on the conclusions drawn in the studies.
An additional problem is that a considerable proportion of medical research never gets pub lished, often because the results are "negative".There is evidence to suggest that unpublished trials may have systematically different results to those that are published (Chalmers et al, 1992).
The variable quality of individual studies, publication bias and a number of other problems will also bedevil systematic reviews and metaanalysis (Eysenck, 1994).The activities of the Cochrane Centre, established as part of the NHS's research and development programme to facilitate systematic reviews of randomised trials, are an attempt to overcome some of these difficulties (Chalmers et al 1992)."Cochrane groups" for schizophrenia and more recently for depression/neurosis have been formed.

In the borderland between EBM and the art of psychiatry
Many clinical trials recruit diagnostically clearcut and previously untreated cases, sufficiently motivated to comply with a particular study regime, in short the kind of patient we rarely meet in day-to-day practice.Many of our patients have significant comorbidity (substance abuse, personality disorder), which may reduce the applicability of trial results to them.Patients' preferences may conflict with what is the best treatment according to scientific evi dence.While our task as clinicians is to commun icate as clearly as possible about the relative benefits of different treatment options and the evidence for and against them, to go on insisting on "the one best option" may increase a patient's resistance to treatment and may disrupt the doctor-patient relationship.We may have to accept what is second best without being too incensed or disappointed to continue to work with a particular patient.
What are we to do if the available evidence on a particular clinical problem is scanty and flawed?One thoughtful recent review on the "grey zones of clinical practice" concluded that "where the evidence about risk-benefit ratios of competing clinical options is incomplete or contradictory . . .evidence-based medicine offers little help" (Naylor, 1995).Faced with such a situation some clinicians will "espouse minimalism whereas others will favour intervention based on inference and experience".Who is to say which of these two options is more appropriate in a given clinical case?Naylor concluded that ". . . the prudent application of evaluative sciences will affirm rather than obviate the need for the art of medicine".

Practical and skills-based hwdles to EBM
If we are to be serious about EBM there are considerable resource and training implications.Most NHS psychiatrists do not (yet) have ready access to office-based PCs linked to electronic databases.If not conducted properly, electronic searches may miss up to 50% of the available evidence (Chalmers et al 1992).A trip from a community team base to the nearest teaching hospital library to pull out articles identified in a computer search may be a time-consuming enterprise.Many psychiatrists will feel that they do not have the necessary skills to appraise research articles critically.Basic appraisal skills can be taught with the help of simple guidelines, but it is debatable how valuable these are in reaching a more in-depth understanding of the available evidence.The Research Training Courses run by the Royal College of Psychiatrists may help to bridge the gap, as may local initiatives like evidence-based journal clubs (see Gilbody, 1996).

EBM, purchasing and policy
What is declared a priority in research, develop ment, purchasing and policy is not value free and will be driven by political and economic expediency.This is perhaps particularly the case in psychiatry, where strong public opinions exist regarding different aspects of clinical practice, most recently concerning perceived failures of community care.Purchasers are influenced in their willingness to fund treatments by the way in which research evidence is presented to them (Fahey et al, 1995).One frequently voiced concern is that EBM will facilitate cost-cutting exercises, especially in areas where evidence is scanty.However, Sackett et al (1996) point out that EBM is as likely to reduce the cost of care in some areas as it is to raise it in others.

Concias ions
None of these concerns constitute reasons for dismissing EBM.However, they do serve to point out that a degree of scepticism is healthy and that a more explicit recognition of the limitations of EBM may inject a greater degree of reality into some of the current debates.