The rise of evidence-based medicine has facilitated wider provision of empirically supported therapies (American Psychiatric Association, 2004) for some specific disorders, such as cognitive–behavioural therapy (CBT) for anxiety and depression, but the emphasis on what works has overshadowed research on how therapies work throughout the UK (Scottish Executive, 2006; NHS Education Scotland, 2007; Department of Health, 2008). The empirically supported therapies strategy, based heavily on the ‘drug metaphor’ (Reference Stiles and ShapiroStiles & Shapiro, 1994), sets randomised controlled trials for specific disorders as the gold standard, and requires similar criteria to those that the Food and Drug Administration uses for medications, in order to recognise a therapy as a ‘well-established’ empirically supported therapy (Reference Wampould, Lichtenberg and WaehlerWampould et al, 2002). However, little attention has been given to how we measure and routinely monitor the quality of the therapy delivered or how we observe what therapists actually do (as opposed to what they originally were qualified to do) in day-to-day practice. This paper explores some key issues; given the Improving Access to Psychological Therapies' focus on CBT, we comment on research in this area, but other therapy studies are used to emphasise the complexity of trying to establish how therapist variables influence patient outcome.

Factors that influence therapy outcome

Research on different models of efficacious empirically supported therapies for specific disorders suggests that a number of non-specific factors account for a large proportion of the predicted variance in therapy outcome. These can be broadly classed as common therapy characteristics, patient factors, patient–therapist interactions and therapist factors. For example, despite different theoretical perspectives, the brief therapies that are most effective in treating chronic unipolar and bipolar mood disorders have common characteristics such as sharing with the patient a coherent stress vulnerability model, providing a clear rationale for the interventions made, plus highly structured therapy sessions with relevant ‘homework’ (Reference Scott and ColomScott & Colom, 2008). Patient characteristics and the severity, chronicity or complexity of problems may also act as moderators of response. To minimise ‘error variance’, efficacy randomised controlled trials try to avoid patient heterogeneity and reduce therapist variability. Therapists undertake pre-trial training in the use of a manualised therapy to standardise their pratice. Almost by default, this can obscure the importance of therapist–patient interactions and variability in therapist competence (Reference BeutlerBeutler, 2007), as the therapists in these trials are not necessarily typical of those in the real world, and process data are usually reported independently of patient outcomes in inadequately powered post hoc analyses. This is unfortunate, as research demonstrates that some therapists are consistently effective whereas others are consistently ineffective (e.g. Reference Lafferty, Beuder and CragoLafferty et al, 1989), and that therapists' ‘experience’ may be more significant when working with some patient groups than with others (Reference BeutlerBeutler, 2007). As a minimum, we therefore need to consider generic therapy skills, such as therapeutic alliance, and specific therapy skills, such as competence in delivery of interventions, and fidelity to a therapy model.

Generic therapy skills

Therapeutic alliance and therapist competence are not the same, but they are related. Building a therapeutic alliance is a necessary foundation of any therapy, especially in CBT, where the therapist cannot simply act as a passive vehicle for delivering the techniques from a manual (Reference BeutlerBeutler, 2007). Early research on the therapeutic alliance produced conflicting findings of its overall importance in predicting outcomes, but this may be an artefact of oversimplifying its nature. Recent studies demonstrate that a therapeutic alliance comprises linked constructs: the therapist has to build and maintain a good working relationship with the patient, and both patient and therapist need to be in agreement on both the goals of therapy and how these are to be achieved. This alliance between therapist and patient is not a unique predictor of therapy outcome; it is also important in medication studies (Reference Martin, Garske and DavisMartin et al, 2000). A therapist who lacks the ability and skill to form a strong collaborative relationship will be ineffective, no matter how many technical interventions they can perform, as alliance is necessary for engagement. However, naturalistic studies demonstrate that alliance is necessary but not sufficient, as once the patient is engaged, the therapist's level of competence is directly associated with outcome (Reference Trepka, Rees and ShapiroTrepka et al, 2004).

Specific therapy skills

When we assess therapist competence, we check whether the therapy is delivered in a technically skilful manner within a therapeutic model and whether appropriate techniques are employed in a timely manner. When assessing a therapist delivering CBT, we could use one of many measures developed that rate both the general qualities displayed by the therapist and their fidelity to specific aspects of CBT. For example, is there a clear conceptualisation of the patient's problems and does this guide the use of the interventions employed? Is the therapist focusing on key (distressing) thoughts most likely to be linked to core maladaptive beliefs? Does the therapist, for the most part, work to help the patient see new perspectives through guided discovery rather than employing persuasion or debate?

However, therapist's competence is not static – development of skilfulness and/or maintenance of current skills are crucial. Post-training on a CBT course, many individuals return to their previous clinical setting and, without supervision, no longer deliver the therapy in the way they were trained, and so what they do in practice no longer reflects what they learnt and appear qualified to do. For example, we trained two groups of multidisciplinary professionals to the same level of CBT practice; one group then continued with supervised CBT, the other with unsupervised CBT. Follow-up demonstrated that the first group continued to progress, but the second groups' skills declined and/or their practice became more idiosyncratic (Reference Mannix, Blackburn and GarlandMannix et al, 2006). As in other studies, therapist factors such as prior qualifications, professional background or training did not predict skills level (Reference Davidson, Scott and SchmidtDavidson et al, 2004).

Therapist competence and fidelity to the model may be predicted by their experience – but only if measured in a reliable and valid way. Experience is not a temporal concept (Reference KeislerKeisler, 1996) and ‘years since qualification as a therapist’ is not an accurate measure (post-qualification, you may spend 10 or 100% sessions practising therapy; but still you can be delivering it badly for many years). It may be more useful to replace the term ‘experience’ with ‘expertise’. The latter may then be further classified – for example, is the therapist's expertise only related to a specific therapy or a specific disorder?

Therapist competence and patient outcomes

Is a therapist's competence at delivering psychological therapy associated with patient improvement? The obvious answer is yes (e.g. Reference Crits-Christoph, Baranackie and KurciasCrits-Christoph et al, 1991), but clinical services pay virtually no attention to this. Patients are frequently referred to expert therapists having been declared a ‘CBT failure’, for the assessment to reveal that the patient had received interventions that have very little in common with any recognised CBT approach for their disorder. In more controlled clinical settings, such as therapy randomised controlled trials, therapist competence and fidelity to the model is usually assessed. However, relatively few studies examine the relationship between an individual therapist's competence and clinical outcomes for the patients treated by that therapist. Our own study examining the effect of individual therapist competence on outcome of a brief treatment, manualised cognitive therapy for repeated self-harm, showed that only patients treated by more competently rated therapists showed significant clinical improvements at 12-month follow-up compared with those patients who were treated by less competent therapists (Davidson et al, 2006).

Problems in process research

In pragmatic randomised controlled trials, we ask, ‘Is the therapy effective?’. To answer this, we use intention-to-treat analysis to evaluate the effect of randomisation on outcome. As noted above, the effect of number of sessions received, therapist competence or process measures such as therapeutic alliance on outcome has mainly been assessed post hoc using statistical measures of association or regression analysis: a strategy that may ignore hidden confounding and measurement error. We do not usually ask at the beginning of a randomised controlled trial of psychological therapy, ‘Is the therapy more (or less) effective with therapists of different levels of competence?’. If we did ask this, we would additionally randomise participants to more or less competent therapists. This is likely to be unacceptable to participants and to ethics committees and yet this probably represents what happens in reality. If we could do such a trial, incorporating ratings of therapist competence into our intention to treat analysis would still be problematic as several assumptions are being made here. For example, is therapist's level of competence a stable attribute? Are all therapists competent to the same degree with all clients? Is therapy being delivered in a highly uniform and predictable manner, moment by moment and session by session? We could go on listing assumptions here, but the point is, psychological therapies are more complex interventions than taking a pill – there is no uniform standard dose of the ‘right model’ for the appropriate length of time.

As such, researchers are now trying to develop more valid statistical tools that could help us estimate more accurately the effect of competence and other process variables in therapy, such as the Complier-Average Causal Effect estimate (Reference Angrist, Imbens and RubinAngrist et al, 1996). This has been used to estimate the ‘dose’ response effect of psychological intervention in randomised controlled trials where the number of sessions patients attended and therapeutic alliance were variable (Reference Dunn and BentallDunn & Bentall, 2007). This type of analysis should be used in the future to provide less biased estimates of therapist effects on outcome.

Clinical practice

This complexity of quality assurance of therapies delivered in the National Health Service (NHS) presents a number of dilemmas. For example, if competence is defined too simply or in generic terms, everyone will meet the specification. If defined too exhaustively, it may be difficult to monitor and/or very few practitioners will achieve the standard (Reference Roth and PillingRoth & Pilling, 2008). However, this does not absolve us from trying and given the proposed expansions in the availability of therapies, the introduction and routine use of standardised measures of process and outcome are required. The governance of psychological therapies within the NHS is patchy and so the introduction of psychological therapy monitoring committees (as is established practice for medications with drugs and therapeutics committee) are also needed. Lastly, supervision structures need to be reviewed – too many professionals receive generic caseload management support and too few are offered regular ongoing systematic supervision such as they received when they trained in a specific therapy approach. These measures would seem to be a bare minimum if we are committed to not just expanding access to therapies and therapists, but ensuring the quality of what is offered.

Declaration of interest

None.