Introduction
This research focuses on the identification of cognitive behavioural therapy (CBT) higher-order skills, also called meta-competencies. It aims to explore meta-competencies through comparing therapists of different levels of experience with respect to clinical scenarios likely to pose challenges in therapeutic practice. This includes reflections on how they would respond in those situations, and judgements about the therapy skills most likely to be needed. It is proposed that meta-competencies are desirable working with all cases and potentially more important when clients’ needs are complex and/or when atypical situations arise in therapy.
What is competence in CBT?
Competence can be described as ‘the extent to which a therapist has the knowledge and skill required to deliver a treatment, to the standard needed for it to achieve its expected effects’ (Fairburn and Cooper, Reference Fairburn and Cooper2011). As different skills are required for different therapies, it is important to assess the ‘limited-domain intervention competence’ of therapists (Barber et al., Reference Barber, Sharpless, Klostermann and McCarthy2007), referring to the ability to implement a specific type of treatment to an acceptable standard. When assessing therapist competence in CBT, the capacity of the therapist to deliver the treatment to an acceptable standard is evaluated, based on an appraisal of a specific set of therapist behaviours, sampled from therapy sessions. The appraisal is based on the presence of (adherence) and quality of (competence) therapist behaviours within those sessions. Measures such as the Cognitive Therapy Scale (CTS; Young & Beck, Reference Young and Beck1980), the Revised Cognitive Therapy Scale (CTS-R; Blackburn et al., Reference Blackburn, James, Milne and Reichelt2001), the Assessment of Core CBT Skills (ACCS; Muse et al., Reference Muse, McManus, Rakovshik and Thwaites2017), and the Cognitive Therapy Adherence and Competence Scale (CTACS; Barber et al., Reference Barber, Liese and Abrams2003) have all been widely used in research and training contexts, with trainee therapists demonstrating enhanced competence across the training process (McManus et al. (Reference McManus, Westbrook, Vazquez-Montes, Fennell and Kennerley2010).
Nevertheless, there are limitations in the evidence base for the use of competence measures (Barber et al., Reference Barber, Liese and Abrams2003; Muse and McManus, Reference Muse and McManus2013; Wampold and Imel, Reference Wampold and Imel2015). Empirical studies indicate that it is difficult to establish adequate inter-rater reliability (Fairburn and Cooper, Reference Fairburn and Cooper2011), and it has been problematic defining and operationalising some aspects of CBT (Jacobson and Gortner, Reference Jacobson and Gortner2000). The structure of competence scales emphasises the presence or absence of a particular feature, potentially at the expense of adaptations to individual clients. Muse et al. (Reference Muse, Kennerley and McManus2022) explored the potential benefits and challenges of competence assessment and highlighted the difficulty in researchers agreeing on a clear working definition of competence. Although a broad range of CBT treatment approaches is helpful when treating patient problems, the ever-evolving nature of CBT means there is a recurrent challenge establishing a consensus about what constitutes effective and competent CBT practice. A greater consensus, and more sensitive and reliable scales, may be possible if measures are extended to include meta-competencies, not just competencies.
What is meta-competence in CBT?
Meta-competencies are defined as ‘competencies that are used by therapists to work across all levels, adapting CBT to the needs of each individual patient’ (Roth and Pilling, Reference Roth and Pilling2007). Meta-competencies govern the application of other competencies, and allow therapists to know why, and when, particular skills are necessary to apply interventions in ways that address individual client needs. The work of the project team involving Roth and Pilling (Reference Roth and Pilling2007) was overseen by an Expert Reference Group (ERG) whose advice contributed significantly to the work. The ERG agreed on the relevance of meta-competencies in clinical practice, and divided them into two categories: generic and CBT-specific. Generic meta-competencies (e.g. the capacity to use clinical judgement when implementing treatment) reflect the therapist’s ability to introduce a flexible and responsive intervention. CBT-specific meta-competencies (e.g. capacity to formulate/apply CBT models to the individual clients) refer to CBT being implemented in a way that represents its underlying principles in diverse situations (Whittington and Grey, Reference Whittington, Grey, Whittington and Grey2014).
CBT-specific meta-competencies are more likely to be needed when working with atypical or complex cases. As cases become more complex, client, therapist and/or healthcare factors interact in ways that create barriers to the working relationships needed to provide effective therapy (Barton et al., Reference Barton, Armstrong, Wicks, Freeman and Meyer2017). As therapists become more experienced, it is highly likely that they encounter more challenging and complex presentations, where CBT protocols need to be applied in principled and creative ways so that the active components of a treatment are delivered, in spite of challenges or complications. Arguably, the impact of clinical complexity has been under-emphasised in extant competence measures, due to these scales being developed for core competencies with standard cases. It is questionable whether these scales allow for sufficient personalisation with respect to atypical or complex cases. In the future, it may be possible to develop a scale that measures meta-competencies, but to accomplish this, the field first needs to explore and discover what meta-competencies are.
It has been proposed that there are ‘super-shrinks’ (also called ‘meta-competent’ therapists; Okiishi et al., Reference Okiishi, Lambert, Nielsen and Olgles2003), whose clients demonstrate quicker clinical improvements and reach recovery more often. Both Heinonen et al. (Reference Heinonen, Lindfors, Laaksonen and Knekt2012) and Okiishi et al. (Reference Okiishi, Lambert, Eggett, Nielsen and Dayton2006) suggest that the identification of characteristics of these therapists would be important for patient treatment and for training programmes. Green et al. (Reference Green, Barkham, Kellett and Saxon2014) found that Psychological Wellbeing Practitioners (PWPs) had differential outcomes predicted by self-rated resilience, suggesting that less effective PWPs may have ‘resilience deficits’ (i.e. reduced ability to cope with challenges). Less effective PWPs also had more limited reflective capacity; for example, during supervision they were more likely to focus on technique and seeking reassurance. In contrast, more effective PWPs were open to developing enhanced self-awareness via reflection. If resilience and reflective capacity are higher-order skills, they could have significant impacts on patient outcomes.
The relationship between competencies and meta-competencies
The framework presented in Fig. 1 is a proposed synthesis of how competencies and meta-competencies relate. Level 1 skills optimise the delivery of treatment based on a clients’ particular problems (e.g. recognising reassurance seeking in OCD). Level 2 skills deliver core aspects of CBT across disorders (e.g. reviewing homework). Level 3 skills help therapists to decide which level 1 and 2 skills should be prioritised at any point, with regard to the specific client and stage of treatment. Level 3 skills have a co-ordinating effect that enables therapy to be delivered with good fidelity and in a personalised way: they have a ‘meta effect’ on other skills due to their higher-order organising function. Extant competency measures are calibrated at level 1 and/or level 2, which reflects the zone of proximal development of trainee and recently qualified therapists. The development of level 3 skills may or may not be encouraged during training, but clinical meta-competence is not assessed. They are assumed to be the consequence of greater experience and continuing professional development.
Figure 1. CBT competency pyramid.
According to Bennett-Levy’s DPR model (Bennett-Levy, Reference Bennett-Levy2006), therapists have declarative knowledge (e.g. of theory and evidence), procedural knowledge (of action and how it is performed) and reflective capacity to critically analyse the ‘personal and therapist self’, in relation to declarative and procedural knowledge. In other words, the reflective system mediates the declarative and procedural systems in order to respond appropriately to patient need. When declarative and procedural knowledge is unregulated by reflection, therapists may implement therapy with insufficient flexibility or personalisation. If reflective capacity is a meta-competence (Bennett-Levy et al., Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001), this would be consistent with the competency framework in Fig. 1: declarative and procedural knowledge would be emphasised at levels 1 and 2, with reflective capacity emphasised at level 3. This is the expected zone of proximal development of more experienced therapists, especially those who work with complex or atypical cases (Dreyfus, Reference Dreyfus2011). With this framework in mind, the current study sought to explore level 3 meta-competencies by comparing therapists with different levels of experience, the expectation being that meta-competencies would be more developed in experienced CBT therapists, and more evident in their responses to complex clinical scenarios. The hypotheses were exploratory, to find out whether the experienced group differed from the other groups when reflecting on, and making judgements about, complex cases.
Method
Pilot study
Five hypothetical complex clinical scenarios were developed and tested in a pilot study. Figure 2 presents an example scenario. All items from the competency measures listed in Table 1 were collated to create a single list of 30 CBT competencies, removing all duplications (Table 2). Doctorate in Clinical Psychology students (n=8) from Newcastle and Plymouth Universities were interviewed and presented with the five scenarios, either virtually (via Microsoft Teams) or in person. They rated the likely importance of deploying the 30 CBT competencies in each scenario. The CBT skills that were rated most important (*) were then included in the main study (Table 2). Participants were also invited to reflect on other skills they might deploy in each of the scenarios. From these responses, four putative meta-competencies were generated, all reflective in nature and consistent with the DPR model (Bennett-Levy, Reference Bennett-Levy2006): (1) monitor their personal reactions; (2) reflect before responding; (3) consider their interpersonal style; and (4) conceptualise within the case formulation (i.e. live conceptualisation of what is happening in the room, rather than an overall formulation).
Figure 2. Example scenario.
Table 1. Competency/disorder-specific competency scales
Table 2. List of competencies
* Final competencies selected for the main study.
Main study
Participants
Recruitment was conducted between August and November 2022, via social media (LinkedIn, Twitter, Instagram), clinical services offering CBT, and all universities across the UK offering a postgraduate CBT Diploma. Convenience and snowballing sampling were used. The initial plan was to recruit only BABCP-accredited therapists in the two qualified groups, and to measure years of accredited practice to identify ‘expert’ therapists. However, this excluded experienced CBT therapists who were not accredited, and also limited recruitment. The decision was made to include therapists who had received a PG Diploma in CBT, whether or not they had subsequently sought BABCP accreditation – identified in the study as ‘experienced’ therapists. Participants (aged 18+) formed three groups of CBT therapists: trainees (n=20; 1–4 years of CBT experience), recently qualified (n=17; 5–10 years of CBT experience) and experienced (n=21; 10+ years of CBT experience). UK-based therapists were recruited, reflecting a variety of cultural backgrounds. Table 3 summarises the participant data. To manage missing data, participants who failed to respond to all scenarios were excluded. Any missing data relating to demographic information was followed up via email.
Table 3. Participant demographics
Procedure
Trainee, recently qualified and experienced therapists were asked to consider four of the five clinical scenarios tested in the pilot phase (see Appendix 1 in the Supplementary material). For each scenario, they were asked to make judgements about 14 CBT skills: the 10 competencies identified as most important during the pilot study, and the four putative meta-competencies also identified. They were asked to rate the likelihood that each skill would be important to use in each scenario, and ratings were made on a 5-point scale: 0 (‘definitely not important’) to 4 (‘definitely important’). They were also asked to rate how much they would us the skill in each situation, also on a 5-point scale: 0 (‘not at all’) to 4 (‘a great deal’).
Participants were then given the opportunity to reflect and speak about how they would respond in each scenario, including using other therapy skills that were not previously listed. Interviews were conducted remotely using Microsoft Teams and participants consented to being video recorded so that their verbal responses could be analysed later. Statements were classified into action-based (e.g. ‘I would write him a letter’) and reflective types (e.g. ‘I suppose what I’m thinking is, she is a 14-year-old very frightened girl, who has a lot of control taken away from her’), with the totals of each recorded for each scenario (see Appendix 2 in the Supplementary material).
Variables
The variables measured are: (1) skill importance: participant ratings of the likely importance of given CBT skills in the scenarios presented to them, and (2) reflective statements in free speech: the comparative number of reflective statements and those indicating actions that participants proposed to take in response to the scenarios presented to them. This was measured by counting the number of reflective and action statements made by the participants (see Appendix 2 in the Supplementary material).
Results
The design was mixed between-group (three groups) and within-group (competencies and meta-competencies; reflective and action-based statements). Two-way ANOVAs were conducted using statistical software SPSS, and post-hoc analysis was completed using one-way ANOVAs when significant main effects and/or interactions were observed. The minimum assumptions for the ANOVAs were all met.
Skill ratings
Hypothesis 1 : There will be group differences in ratings of skill importance of which competencies and meta-competencies are likely to be needed in complex scenarios
The two-way ANOVA revealed no main effect of group, F 2,55 = 0.563, p = .573, and a significant main effect of competency type (competencies/meta-competencies), F 1,55 = 29.784, p = .001, with the ratings for competencies higher than meta-competencies. There was no group × competency-type interaction, F 2,55 = 0.226, p = .798 (Fig. 3; Table 4).
Figure 3. Skill importance of meta-competencies (MC) and competencies (C).
Table 4. Means and SD for the skill importance of meta-competencies and competencies
Participant statements
Hypothesis 2: Experienced therapists will make more reflective and action-based statements than recently qualified, and recently qualified will make more than trainees
The two-way ANOVA revealed no main effect of group F 2,55 = 1.951, p = .152, and a significant effect of statement-type, F 1,55 = 37.336, p < .001 (i.e. whether the statements were action-based or reflective). There was also a highly significant group × statement-type interaction F 2,55 = 10.210, p < .001, with a large effect size (ŋ2 = 0.152) comparable to Cohen’s d of 0.9 (http://www.psychometrica.de/effect_size.html). This was further explored with one-way ANOVAs. There were no group differences on action-based statements F 2,55 = 0.171, p = .843 (Fig. 4; Table 5). The one-way ANOVA exploring reflective statements found a significant group difference F 2,55 = 4.913, p = .011. A Bonferroni test was conducted post-hoc to test group differences with respect to the reflective statements, and a significant difference was found between trainees and experienced therapists (standard error = 1.83; mean difference = –5.6; p = .01). No significant differences were observed between the trainee and recently qualified groups, and the recently qualified and experienced groups. There was a potential confound, that more reflective statements were made by the experienced therapists because they used more words during their reflections on each scenario. This hypothesis was tested using a one-way ANOVA of the mean number of words used, and there was no significant difference between the groups, F 2,55 = 1.376, p = .261.
Figure 4. Means of reflections and actions.
Table 5. Means and SD for reflections and actions
Inter-rater reliability
Cohen’s κ was run to determine if there was agreement between the primary rater and a second rater, on the number of reflective and action-based statements, taken from a random selection of transcripts. The rater was sent one transcript from each participant group, using a random number generator. There was moderate agreement between the two raters (κ = 0.659, 95% CI [0.444, 0.874], p < .001. Agreement interpretations are: 0.4, weak; 0.6, moderate; 0.8, strong (McHugh, Reference McHugh2012).
Discussion
This study explored CBT meta-competencies by comparing trainee, recently qualified and experienced therapists in their judgements about, and reflections on, clinically complex scenarios. The research strategy was to explore differences between groups, specifically whether the experienced group differed from the other groups, as a potential indicator of meta-competencies.
There were no group differences in judgements about the types of CBT skills that could or should be deployed in these scenarios. This suggests that trainees and recently qualified therapists share similar knowledge to experienced therapists concerning the types of skills that need to be deployed under complex conditions. In other words, the groups have comparable declarative knowledge about the CBT procedures that are likely to be needed. Alternatively, it may be that the judgements made in this study were not sufficiently sensitive to detect group differences, because there was a relatively small amount of information given about each case. More detailed information about the cases could potentially have identified differences in judgements about the skills that need to be prioritised. This is something that should be considered in future replications.
The study observed a significant difference between trainees and experienced therapists with respect to reflective statements about each scenario, consistent with the view that reflective capacity is enhanced as a consequence of greater therapeutic experience, and could be a higher-order skill or meta-competency. This difference in reflective statements was not explained by experienced therapists saying more words during the task. As therapists are likely to be exposed to more complex cases over their careers (Davis et al., Reference Davis, Thwaites, Freeston and Bennett-Levy2015), this finding suggests that reflective capacity increases over time as therapists are confronted with a broader range of non-standard clinical presentations. The is consistent with Bennett-Levy et al.’s (Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001) proposal that reflective capacity is used to critically analyse the ‘personal and therapist self’, built on the foundation of declarative and procedural knowledge. This could be refined further: is reflection essentially a verbal act? Bennett-Levy’s idea that the reflective system is ‘content-free’ might suggest a non-verbal component or stage. Or are there varying levels or stages of reflection? (e.g. the act of reflection itself, and the ability to verbalise the product of reflection). This could be an area for future research.
No significant differences were found in reflective statements between trainee and recently qualified, and recently qualified and experienced, which may be attributable to the slow and progressive development of reflective skills, supporting the idea that reflective capacity is developed over several years of experience. The difference in reflective statements was not reflected in differences in action- or potential action-statements between groups. This could suggest that all participants shared a sense of the importance of retaining core competencies in complex situations.
There may be other explanations for the group difference in reflective statements; for example, experienced therapists may be more confident in sharing hypotheses about complex cases. Aarons et al. (Reference Aarons, Cafri, Lugo and Sawitzky2012) discovered that therapists with more clinical experience were more confident in their skills, and more likely to perceive psychological practice as a creative skill, rather than a science. Additionally, therapists with more clinical experience were associated with less frequent use of treatment manuals (Simmons et al., Reference Simmons, Milnes and Anderson2008), and a less positive attitude towards their use (Barry et al., Reference Barry, Fulgieri, Lavery, Chawarski, Najavits, Schottenfeld and Pantalon2008). Arguably, this may further indicate that experienced therapists are more likely to be less prescriptive than those with less experience, and therefore experienced therapists could be viewed as clinicians who engage in ‘therapist drift’ (Waller and Turner, Reference Waller and Turner2016), rather than meta-competent.
Limitations
The clinical scenarios represented a spectrum of specialisms (e.g. child, learning disability) and potentially could have had more complex elements included. Complexity develops when interactions between psychopathology and biopsychosocial factors (client, therapist, and system factors), create barriers to therapeutic and other working relationships. Two of the scenarios made reference to other healthcare systems (e.g. community treatment team, palliative care); however, it would have been helpful for all scenarios to have more focus on therapist and systemic factors, to better reflect clinical complexity. For future research, it would be helpful to have experienced therapists offer feedback and guidance on the scenarios and their representation of complexity. Considering the methodology, unstructured speech yielded group differences and structured skill ratings did not. It may have been helpful to have a short description of each competency to ensure all participants interpreted them in the same way, and/or to have more information provided in the clinical scenarios. Moderate inter-rater agreement was observed on the ratings of reflective statements (κ = 0.659), and although moderate agreement is acceptable, it suggests some disagreement between raters.
Additionally, a significant weakness relates to using trainees to guide which competencies were the most important. Trainees were presented with the list of 30 competencies which included the ERGs development ideas of meta-competencies (Roth and Pilling, Reference Roth and Pilling2007). However, they did not rate any of these skills to be the most important, and therefore none of those skills was included in the final list. It could be argued that trainees did not identify any of these meta-competencies as ‘definitely likely to be important’, because their skills are still developing. Therefore, for future replication, it may be advisable to use more experienced therapists to rate skill importance, or present all therapists with the list of meta-competencies from Roth and Pilling’s framework asking them to make judgements of skill importance. Nevertheless, the lack of a difference found between trainee and experienced therapists regarding importance ratings of the final list of meta-competencies and competencies may suggest otherwise.
Conclusions and implications
There are potential implications for CBT training, giving reflection sufficient priority within the training process to enable its subsequent development in future clinical practice. The Self-Practice/Self-Reflective (SP/SR) programme (Bennett-Levy et al., Reference Bennett-Levy, Turner, Beaty, Smith, Paterson and Farmer2001) invites therapists to practise CBT techniques on themselves and then reflect on their experience. This programme would highlight the opportunity for trainees to continue the advancement of their reflective skills, especially after formal CBT training. Additionally, the prioritisation of engaging in specific tasks to enhance reflective ability is another factor that can support the long-term development of this skill.
Overall, the main findings support the idea that more experienced therapists have a greater capacity for reflection under conditions of clinical complexity. This research has potentially discovered the presence of a specific meta-competency, which could be the first step in developing a CBT meta-competency scale. Further studies are needed to investigate the relationship between reflective capacity and other putative meta-competencies, such as clinical formulation, therapist responsiveness (Eells et al., Reference Eells, Lombart, Kendjelic, Turner and Lucas2005) and interpersonal responsiveness.
Key practice points
-
(1) Based on the findings, it may be important for both trainee and recently trained therapists to continue developing their reflective skills through SP/SR practice, and/or via incorporating reflective approaches in clinical supervision and self-supervisory work (Haarhoff and Thwaites, Reference Haarhoff and Thwaites2016).
-
(2) Current CBT training programmes and accreditation bodies for training programmes, may wish to consider integrating the development of reflective skills into CBT curriculum, either within current modules or as a separate module and assessments methods.
-
(3) When working with complex clients, it is important for therapists to allow extra time for reflection, and to be given the space to consider the needs of the case alongside core competencies. This may result in some competencies being prioritised over others, or some being left out of particular sessions, or even courses of therapy, so long as these decisions can be justified via rigorous supervision and documented accordingly.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1754470X24000047
Data availability statement
The data that support the findings of this study are openly available in Newcastle University’s repository at http://doi.org/10.25405/data.ncl.23541702 (participant data), http://doi.org/10.25405/data.ncl.23541879 (rater information), http://doi.org/10.25405/data.ncl.23541663 (trainee therapist transcripts), http://doi.org/10.25405/data.ncl.23541654 (recently qualified therapist transcripts), and http://doi.org/10.25405/data.ncl.23541588 (experienced therapist transcripts).
Acknowledgements
Thank you to all therapists and trainees who took part in the research, for meeting with the primary investigatory, and taking the time to reflect on their own therapeutic skills in order to answer the study questions. Thank you for your positive feedback and encouraging others to take part.
Author contributions
Denika Campbell-Lee: Conceptualization (equal), Data curation (lead), Formal analysis (lead), Investigation (lead), Methodology (lead), Project administration (lead), Resources (equal), Visualization (lead), Writing – original draft (lead), Writing – review & editing (lead); Stephen Barton: Conceptualization (equal), Formal analysis (supporting), Methodology (supporting), Resources (equal), Supervision (lead), Validation (equal), Visualization (supporting); Peter Armstrong: Conceptualization (equal), Methodology (supporting), Resources (equal), Supervision (lead).
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
The authors declare none.
Ethical standards
Ethical approval was obtained by Newcastle University Ethics Committee (protocol number: 14644/2020). Informed consent was sought, and the right to withdraw at any time was given. The information pages provided therapists with the specifics of the study. Authors have abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the BABCP and BPS.
Open access
Comments
No Comments have been published for this article.