CHILD PSYCHIATRY: BEYOND RANDOMISATION?

In an important and challenging paper, Graham (Reference Graham2000, this issue) endeavours to bridge the gap between research findings and clinical practice in child psychiatry. He argues that the standards for defining the evidence base in child psychiatry should be different from those used in other branches of medicine. Although acknowledging the importance of randomised controlled trials (RCTs) in establishing the efficacy of some interventions, he points to their limitations as a guide to the evidence base for child psychiatry. Randomised controlled trials are often performed on unrepresentative samples and use outcome measures of doubtful clinical significance. He concludes that for “at least a very long time into the future, it is likely that RCTs will have only a limited place among the range of influences affecting treatment interventions”. Instead, he suggests that there should be greater emphasis on clinical experience, comparative audit and qualitative research methods.

The paper raises many questions for child psychiatrists, of which perhaps the most important is how far clinical practice should be determined by the results of RCTs. Professor Graham rightly points out that many (although by no means all) of the RCTs conducted thus far have been efficacy studies. That is, they have been concerned mainly with whether an intervention can work in the best possible circumstances. It is not clear whether the interventions studied in these trials will be applicable in the real world. Moreover, he suggests that much of the work performed by child psychiatrists is concerned with the management of the stresses associated with children's mental problems, and that such work is not amenable to evaluation in RCTs.

Should we, then, be giving more weight to other kinds of evidence such as qualitative research, comparative audit and clinical experience? These techniques will undoubtedly have an important role in ensuring how interventions are implemented. However, there are several reasons for thinking that RCTs should be the ‘gold standard’ for determining which interventions should be used by child psychiatrists. First, randomisation is the only way of ensuring that allocation to comparison groups is unbiased with regard to all factors relevant to treatment responsiveness and prognosis. When random allocation is not used, it is sometimes possible to control for differences between groups in other ways, such as by careful matching. However, this is only possible when all important prognostic factors are known and can be measured accurately. This is seldom possible in child psychiatry.

There are other problems that could arise from dismissing RCTs as being too blunt an instrument to evaluate child psychiatry. The main one is that in other parts of medicine RCTs are increasingly seen as the best way of establishing whether an intervention is effective. For example, the Interim Guidance from the National Institute for Clinical Excellence on the appraisal of new and existing technologies (National Institute for Clinical Excellence, 1999) states that “data to support claims for clinical effectiveness should, ideally, be derived from randomised controlled trials”. There are obvious dangers for child psychiatry if it goes in one direction while the rest of medicine goes in another. One wonders, for example, whether in the new evidence-based environment purchasers of child mental health services will be prepared to pay for interventions that they perceive as being of unproven benefit. Similarly, users of services are going to be much better informed than they have been in the past, and may demand treatments of proven effectiveness.

Randomised trials will continue, then, to provide an important assessment of child psychiatric interventions. However, if such trials are to be of value to clinicians, they will have to be designed in ways that deal with the criticisms that Graham makes of extant research. How can this be achieved? The first step will be the greater use of so-called pragmatic or effectiveness RCTs. Pragmatic trials aim to answer clinical questions in real-life clinical settings (Reference Hotopf, Churchill and LewisHotopf et al, 1999). Subjects are usually those who are seen in everyday practice and comorbid cases are not excluded. Outcome measures are clinically relevant and include not only assessments of the presenting problem (e.g. depression) but also of quality of life, client satisfaction and economic costs.

Pragmatic trials will not, however, be enough. As Graham points out, there are substantial barriers to the implementation of existing findings from treatment research. Future trials need to study treatments that, if they worked, could be implemented within the health service. Researchers and clinicians need to collaborate in the development of such treatments and on plans for dissemination.

In summary, randomised trials are only necessary when there is substantial uncertainty about whether a procedure is effective or not. It is obviously good practice, for example, to communicate with other agencies, to keep legible records, to give advice about social security benefits, and so on. Much of the day-to-day work of child psychiatrists therefore need not be evaluated in RCTs. However, when there is doubt about the value of an intervention, RCTs will continue to be one of the best ways of assessing its effectiveness.

Child psychiatry needs a new generation of large pragmatic RCTs. These trials must tackle clinically relevant questions, use interventions that are capable of implementation and be carried out under normal health service conditions.