Although typically associated with pharmaceuticals, randomised controlled trials (RCTs) are the gold standard for evaluating everything from medical devices and screening tools to behavioural interventions. In mental healthcare it is striking that one of the main treatments is pharmaceutical and therefore readily testable through trials, but the other, therapy,Footnote a is arguably less so. As a philosopher of psychiatry, I will ask whether it is appropriate to use a study design developed for one intervention to evaluate another quite different intervention.
Before we begin, the stakes: healthcare provisioning and research funding is decided almost exclusively on the basis of RCTs. Therapies with an RCT evidence base, most notably cognitive–behavioural therapy (CBT), have become the main treatment and, in doing so, have limited patient choice within public healthcare provision and private insurance coverage. For instance, in the National Health Service (NHS) Talking Therapies Programme, only 11% of high-intensity practitioners offered a non-CBT therapeutic modality. 1 Relatedly, professionals trained in other modalities are often ineligible for certain roles, which is concerning given global workforce shortages in the mental health professions. These problems are widespread, particularly in countries that model their therapy provision on the UK, such as Australia and Canada.
An ontological mismatch: lack of ‘fit’ between therapy and RCTs
Ontology is the field of philosophy that asks questions about existence and clarifies what something is – that is, its nature and characteristics. I am interested in two main ontological questions: (a) what sorts of characteristics do RCTs require of the interventions they evaluate? Or, what is an intervention? (b) What is therapy? Or, what can be understood about therapy from how it is practised by clinicians and engaged with by clients?
Further questions inevitably follow: what is mental disorder? What is progress or recovery in therapy? My aim is not to offer answers but to demonstrate an ontological mismatch where RCTs presuppose that the interventions they evaluate have a certain ‘shape’ and, although drugs readily fit this shape, therapy does not.
RCTs appear to have a number of requirements, namely that the intervention is:
-
(a) Standardised: the intervention must be if not identical then sufficiently similar among all participants in the active group so that outcomes can be compared. The intervention must also be sufficiently similar across multiple trials to aggregate them in systematic reviews and meta-analyses.
-
(b) Specific: the intervention must target a particular disease or condition.
-
(c) Time-limited: the intervention must act over a timescale that allows outcomes to be captured within the duration of a trial.
-
(d) Measurable: the intervention brings about specific desired outcomes such as the alleviation of particular symptoms or recovery from some condition, ideally according to some numeric scale.
Most drugs, including psychopharmaceuticals, meet these conditions. Drugs are readily standardised and target a particular diagnostic condition or cluster of symptoms. Although some drugs act cumulatively, take longer to show an effect or have a non-linear response, outcomes can be captured within the typical time frame of a RCT.
Therapy struggles with these conditions. It is typically understood as highly contextual and idiosyncratic, involving a dialogue and relationship that is unique to the two people involved and the specific issues to be addressed, often occurring in an open-ended manner, with nuanced outcomes, on a timescale that is not predetermined. Therapies have been adapted to meet the requirements of clinical trials. However, even with adaptations, I believe that the routine practice of therapy does not readily fit the shape that RCTs require of interventions.
To illustrate this mismatch I consider two fictional vignettes. Vignette 1 is intended to capture ‘evidence-based’ therapy adapted for RCTs, with vignette 2 showing therapy free from the constraints of RCTs. Some of the differences are subtle and are most apparent in the aims, timescale and trajectory of therapy.
Vignette 1
Rashid, aged 36, goes to his general practitioner (GP) with tightness in his chest, difficulty breathing and trouble concentrating at work. He feels tired and has little interest in maintaining relationships with his friends or pursuing his former hobbies. He has become consumed with worry about making mistakes at work and being fired. He believes he has a serious health issue. The GP runs tests to confirm that there is no obvious physical basis for his symptoms and suggests he try therapy, because Rashid may have depression or an anxiety disorder. The GP surgery is a recruitment site for a therapy trial and Rashid is randomised to the therapy group.
The therapist works with Rashid to identify which of his problems needs to be addressed first. They agree that feeling calmer may be helpful for his concentration. Rashid is taught techniques to calm himself (applied relaxation and mindfulness exercises) and encouraged to practise them daily.
During the fourth session, Rashid volunteers that his father was diagnosed with cancer and they discuss whether this might be related to the anxieties about his own health. By the end of eight sessions, Rashid sees a noticeable improvement with his breathing difficulties and feels more confident about work. He believes his health worries are related to the shock of his father’s diagnosis and he has been practising ‘reality testing’ (CBT) of these thoughts by reminding himself that the GP had already conducted tests.
After several months, Rashid returns to therapy when his difficulties recur.
Vignette 2
Rashid embarks on long-term therapy having tried brief therapy several times. The therapist asks what has been fruitful in the past and encourages Rashid to keep practising the exercises (applied relaxation, mindfulness). Over several sessions they discuss his father’s diagnosis, Rashid’s health anxieties, his family and work over several sessions. Rashid clarifies that he financially supports his parents, who live in Iran, sharing that he is terrified he will not be able to do so if he loses his job or becomes ill. The therapist asks Rashid about his relationship with his father and is told that they are close.
By session ten, Rashid feels that he no longer needs therapy because he understands how his fears over being unable to support his family are affecting him and is able to calm himself when distressed. The therapist suggests he think about whether he is ready for the therapy to come to an end while he is away visiting his family. During this session, Rashid mentions that he is stressed about the visit and, when asked why, he becomes distressed and hostile. He ends the session early and the therapist does not hear from him for several weeks after his return.
When Rashid returns he is sullen and silent, offering only brief, uninformative comments. In the next session, he apologises and says that he felt his family was being criticised. During subsequent sessions, Rashid is very talkative and confesses that he finds family visits difficult because they often demand that he move back, something he does not want to do. Over several months they explore his childhood and the pressure Rashid has felt as the oldest son in being responsible for his family. He admits that his father is quite authoritarian, which makes him dread the calls that have become more frequent since he has fallen ill. The therapist encourages Rashid to imagine his father is in the room to rehearse how they might discuss his feelings about moving back, and his childhood, topics he feels unable to raise directly (psychodrama, ‘chair work’).
By the time Rashid is ready to end the therapy, some months later, he is more consistent at work and has re-established contact with his friends; he is surprised to find that many of them have complicated relationships with their families. Rashid continues to feel tense, especially when he interacts with his parents. He worries about his father’s health and the possibility of moving back to Iran, resulting in sleepless nights. He does not believe he will feel better soon because the situation is ongoing, but expresses gratitude to his therapist in the final session for helping him. He states that he needs to discuss his future plans with his family but does not think this is a good time.
Standardised
The main adaptation of therapy for RCTs has been the development of manuals, standardising therapy within and across trials. Manuals are specific to particular therapeutic modalities; however, therapy is generally practised ‘eclectically’, where practitioners draw on techniques and theoretical understandings from a variety of modalities. In the UK, 87% of respondents in a survey claimed to draw on multiple modalities; Reference Hollanders and McLeod2 eclecticism is also widespread even in highly regulated environments such as the NHS. Reference Ratnayake and Dunn3 Eclecticism is apparent in both vignettes, with practitioners drawing on whatever seems most fruitful for Rashid’s progress. Specific techniques have been highlighted in the vignettes because typical practice is, on the surface, atheoretical – as the philosopher-cum-therapist Hanna Pickard once remarked in conversation, practitioners ‘hold theory lightly’.
Specific
Just as specific modalities are not obvious in the vignettes, neither feature discussion of diagnoses by therapists. Rather, the focus is on issues raised by the client. Certain schools of therapy, e.g. client-centred psychotherapy, deliberately eschew diagnoses, with most schools describing competencies and best practice in terms of the therapy being ‘client-led’, ‘non-directive’ or ‘collaborative’. In contrast, trials and manuals presuppose what is often described as ‘the medical model’ or a medical ontology, with psychiatric diagnoses (based on the DSM or ICD) targeted by specific interventions.
Measurable
Psychiatric diagnoses are defined in terms of symptoms and, consequently, the primary outcome in trials is symptom alleviation, tracked through outcome measures. These measures are often the same ones used in psychopharmaceutical trials (e.g. PHQ-9) or are similarly symptom focused. However, practitioners and clients seemingly understand progress and recovery differently. Fine-grained qualitative research with clients suggests complex goals such as better self-understanding viewed as ongoing processes rather than cut-off points on outcome measures. Reference Moltu, Stefansen, Nøtnes, Skjølberg and Veseth4 Other research shows practitioners favouring clinical judgement over measures to gauge client progress, even in the NHS where measures are standard. Reference Ratnayake and Dunn3
Rashid’s progress is unclear in both vignettes: in the first, his symptoms alleviate but recur; in the second, symptoms have arguably worsened, with additional insomnia. Nevertheless, Rashid’s life appears stable: his employment is more secure, and he has a greater support network and a better awareness of his distress and how to resolve it – types of progress that are difficult to measure with standard outcomes. The differing understandings of progress also shape the therapeutic approach. The first vignette focuses on symptom alleviation through resolving immediate problems whereas the case history in the second is wide-ranging, delving into the past and exploring interpersonal dynamics.
Time-limited
Therapy trials tend to be brief, with most offering fewer than 20 sessions. The danger of such studies is clear from the vignettes: Rashid appears to make early progress in both, but in the second the therapist encourages him to reflect on bringing the therapy to an end, leading to further progress following a rupture in the relationship. Therapy is generally non-linear, with initial reluctance to ‘open up’ Reference Kleiven, Hjeltnes, Råbu and Moltu5 and periods of rupture, Reference Talbot, Ostiguy-Pion, Painchaud, Lafrance and Descôteaux6 meaning that overall or long-term efficacy cannot be determined if a study ends prematurely. It has been suggested that brief therapy trials capture an initial improvement that is not always stable, as in the first vignette, resulting in re-referrals to therapy services or other secondary care. Reference Martin, Iqbal, Airey and Marks7
For the purposes of healthcare provisioning subject to economic pressures, it would be ideal for an intervention to demonstrate efficacy within a short time frame. This does not address whether a particular type of therapy is overall effective, as evidenced by the seminal 18-month Tavistock Adult Depression Study on psychoanalysis; nor does it address the vital clinical question of whether a therapy that is, on average, efficacious in a limited number of sessions will work for a particular client in the same time frame. Some clients, particularly those with entrenched mental health difficulties, may require longer-term therapy whereas others, such as Rashid, might require intermittent therapy of varying length as life circumstances change. As noted, therapy practised well is responsive to the needs of the client, necessitating flexible treatment plans.
This ontological mismatch brings us to epistemology, concerning knowledge and whether we are seeking it through the right methods.
Epistemology: the right tools for the job
The type of intervention therapy, and ideas around how it should be practised, make it a poor fit for RCTs that presuppose interventions that are standardised, specific, measurable and time-limited. To consider the analogy with pharmaceuticals, the discrepancy between therapy during trials versus routine practice is akin to a different drug being prescribed by pharmacies than the drug tested during trials. In short, we have an external validity problem.
At this point, an RCT proponent might demur, could we adapt trials to reflect how therapy is different from drugs? Trials could be longer, less focused on diagnoses and capture more expansive ideas of progress. Such adaptations have been made and are necessary if we are to continue including RCTs in our evidence base for therapy. However, trials are simply a tool for evaluating the efficacy of interventions: they are not an end in themselves. If the task of evaluating therapy continues to focus on adapting therapy for RCTs on the one hand and adapting RCTs for therapy on the other, we are unlikely to make progress on the more fundamental question of whether therapy as it is actually practised is effective.
Methodology: evidential pluralism for the evaluation of therapy
Rather than shoehorning therapy into the mould of RCTs, we require a more diverse, pluralistic evidence base within which RCTs will have their place but not a privileged position. The emphasis would be on research and study designs better suited for evaluating therapy. Three types seem to be indicated: mechanistic, practice-based and service user research.
Mechanistic research or process research, exploring how therapy works, is of vital importance. In fact, the preceding argument shows the unsuitability of RCTs given the mechanism through which therapy works: one that is longer-term, not targeted towards specific diagnoses or symptom alleviation, eclectic and client-specific rather than standardised.
A central mechanistic question is whether it is theories and techniques of specific schools, such as cognitive restructuring, that explain the efficacy of therapy or factors common to all schools of therapy, such as the empathy of the therapist. This unresolved question undergirds the policy issues above. If common factors play a key role in accounting for the efficacy of therapy, and many types of therapy are simply less amenable for testing via trials, then there is justification for offering other therapies via public healthcare systems and medical insurance; and similarly, for making the workforce open to practitioners of other modalities.
Practice-based research – that is, research conducted in routine settings, with all the accompanying difficulties: long waiting lists, overstretched staff, clients with complex presenting problems – may be a more reliable indicator of efficacy because it considers therapy in situ. Currently, even in cases where outcomes are monitored and large data-sets collected in routine settings – for instance, by NHS Digital – these data are not used by institutions such as the UK’s National Institute for Health and Care Excellence. Such data-sets show parity of outcomes across a number of therapeutic modalities, Reference Barkham and Saxon8 providing justification for expanding therapy provision. Sidelining practice-based research also presents a barrier to innovation. Novel approaches demonstrating efficacy within routine practice do not get developed further or disseminated because practitioners rarely have the resources to prove efficacy under the prevailing RCT standard.
Service user research can contribute to understanding efficacy therapy, namely with implementation and adaptation. RCT-backed therapies, including CBT, appear to be limited in certain populations such as Black, Asian and Minority Ethnic clients. Reference Naz, Gregory and Bahu9 Research such as focus groups with clients similar to Rashid (e.g. first-generation immigrants) exploring familial issues will help develop culturally appropriate therapies, effective for the diverse clients seeking treatment via public healthcare. More generally, understanding whether therapy achieves the nuanced outcomes to which clients aspire will involve service user research. Indeed, if the demands for treatments to be person-centred and for research to involve patients are to be met, service user research is the sine qua non of an acceptable evidence base.
We are in an age of methodolatory – RCTs are so revered and play so central a role in determining healthcare policy and research agendas that it seems more reasonable to force interventions to fit than to consider alternatives. There are, thankfully, dissenters, movements such as Evidence-Based Medicine+, philosophers of science and advocates for service user research calling for evidential plurality in wider medical research. It is time for psychiatry to join this disgruntled chorus. Where else is this mismatch so apparent as in the two main interventions for mental health?
Data availability
Data availability is not applicable to this article as no new data were created or analysed in this editorial.
Acknowledgements
The author thanks Trish Greenhalgh (University of Oxford, Nuffield Department of Primary Care Health Sciences) and Jon Williamson (University of Manchester, Department of Philosophy) for comments on an earlier draft.
Funding
This piece was supported by UK Research and Innovation Cross Research Council Responsive Mode (grant no. 25130 for Interdisciplinary Research). The funder had no role in the planning, research or writing of this piece.
Declaration of interest
None.
eLetters
No eLetters have been published for this article.