Evidence-based practice (EBP) in psychotherapy has been defined as ‘the integration of the best available research with clinical expertise in the context of patient characteristics, culture and preferences’ (American Psychological Association Presidential Task Force on Evidence-Based Practice, 2005, p. 5). It has been suggested that using EBP in psychological treatment enables clinicians to ensure that they make recommendations about treatment that offer the maximum chance of treatment benefit with the minimum risk of harm to clients; more broadly, EBP can ensure that costs of delivered treatments are acceptable, given their likely benefit (Australian Psychological Society, 2010).
In the implementation of psychological therapies, it has been proposed that EBP has three core components (Lilienfeld, Ritschel, Lynn, Cautin, & Latzman, Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013). The first component is the clinician's ability to identify and use the best available evidence regarding treatment, enabling selection of the treatment demonstrated to be most effective for a client presentation, and an understanding of why a treatment is effective. Such evidence includes treatment efficacy studies in research settings, effectiveness studies in ‘real-world’ settings, and research into psychological processes relevant to therapies (Lilienfeld et al., Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013). The second component of EBP entails the clinicians’ expertise, comprised of clinical skills and experience. Lilienfeld et al. (Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013) argue that clinical expertise is a necessary part of decision making in psychological therapies, as ‘data simply are not available to dictate every decision within a psychotherapy session’ (p. 886). The third component of EBP comprises clinicians’ knowledge of client values and preferences, as these may have an impact on choice of therapy; for instance, a treatment found to be effective for a client's problem may not be acceptable to a client (Lilienfeld et al., Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013).
Despite the suggested benefits of EBP in delivering psychological therapy, research suggests that a substantial proportion of clinicians offering psychological therapies do not adopt all components of EBP, relying predominantly on their clinical expertise, with resistance to EBPs most consistently related to use of external evidence (e.g., Cowdrey & Walller, Reference Cowdrey and Waller2015; Gyani, Shafran, Myles, & Rose, Reference Gyani, Shafran, Myles and Rose2014; Lilienfeld et al., Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013; Pilecki & McKay, 2014; Safran, Abreu, Ogilvie, & DeMaria, Reference Safran, Abreu, Ogilvie and DeMaria2011). A number of reasons for this finding have been proposed, including clinicians: (1) not finding research relevant or applicable to their practice (Kazdin, Reference Kazdin2008; Safran et al., Reference Safran, Abreu, Ogilvie and DeMaria2011); (2) not updating their knowledge of research (Gyani et al., Reference Gyani, Shafran, Myles and Rose2014); or (3) lacking familiarity with the theory underpinning treatments (Pilecki & McKay, Reference Pilecki and McKay2013). Lilienfeld et al. (Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013) proposed that while many clinicians agree that external, research-based evidence is useful in therapy practice, other sources of information (most consistently their own clinical experience) are endorsed as most useful and used more frequently.
Allegiance to a therapeutic approach may play a role in influencing clinician attitudes toward using research. Studies have indicated that clinicians using cognitive behavioural therapy (CBT) regularly are more likely to be positive about EBPs, compared to clinicians who apply other theoretical orientations (Gyani et al., Reference Gyani, Shafran, Myles and Rose2014). CBT approaches also have a comprehensive evidence base in terms of theoretical concepts, treatment methodologies, and validation (Tarrier & Calam, Reference Tarrier and Calam2002; Nezu, Martell, & Nezu, Reference Nezu, Martell and Nezu2014). Therefore, clinicians using CBT may be more aware of EBPs, or more likely to implement EBPs in their practice.
Within the practice of CBT, a specific therapy process suggested to play an integral role in incorporating evidence into individual treatments is case formulation (CF; Dudley, Kuyken, & Padesky, Reference Dudley, Kuyken and Padesky2011; Nezu, Nezu, & Lombardo, Reference Nezu, Nezu and Lombardo2004; Persons, Reference Persons2006, Reference Persons2008). It is during the CBT CF process that the clinician can integrate the ‘individual particularities of a given case, relevant theory and research’ (Kuyken, Reference Kuyken and Tarrier2006, p. 12). By incorporating empirical findings into the CF, the clinician is prompted to identify aspects of the individual's presentation that correspond with theoretical models and that could be targeted in treatment (Persons, Roberts, Zalecki, & Brechwald, Reference Persons, Roberts, Zalecki and Brechwald2006).
Despite the importance of CF to the practice of CBT, surprisingly, there is a paucity of research regarding clinician practice of CF. In one of the few studies to focus on clinician perspectives regarding CBT CF, Flitcroft, James, Freeston, and Wood-Mitchell (Reference Flitcroft, James, Freeston and Wood-Mitchell2007) aimed to identify the essential features of CBT CF for depression using a Q-sort methodology. Twenty-eight CBT clinicians were asked to order, in terms of importance, expert-developed statements describing CBT CF processes, and then to select three statements that they perceived represented what was most essential to CF, and three statements representing what was least essential. Flitcroft et al. (Reference Flitcroft, James, Freeston and Wood-Mitchell2007) identified that the descriptive statements rated as most important referred to the formulation as: (1) explaining how problems are maintained, (2) being acceptable to the client and others, (3) helping to make sense of what the client was experiencing, and (4) guiding possible interventions. Their analysis of results identified a three-factor model, which Flitcroft et al. (Reference Flitcroft, James, Freeston and Wood-Mitchell2007) argued represented three different viewpoints of what is most important in CF. However, this outcome should be interpreted with caution, as many of the statements appear to cross-load on the three factors, making interpretation of factor structure difficult. Moreover, the factor extraction method used in this study, the Kaiser criterion, has been criticised as one of the least accurate methods for selecting the number of factors to retain (Costello & Osbourne, Reference Costello and Osbourne2005), as it may lead to identification of too many factors (Velicer & Fava, Reference Velicer and Fava1998). The findings from Flitcroft et al.’s (Reference Flitcroft, James, Freeston and Wood-Mitchell2007) study were also likely affected by the very small sample of participants (N = 28), well below the sample suggested as the lowest number acceptable for factor analysis (Velicer & Fava, Reference Velicer and Fava1998).
More recent research by Nattrass, Kellett, Hardy, and Ricketts (Reference Nattrass, Kellett, Hardy and Ricketts2014) evaluated the content of CFs presented by eight clinicians to 29 clients who received CBT-oriented treatment for obsessive compulsive disorder. In the 70 sessions analysed, Nattrass et al. identified that clinicians appeared to focus on information about symptoms and maintaining factors, with most frequently mentioned content being descriptions of the behavioural and cognitive components of the client's psychological difficulties. However, the authors concluded that their focus on analysing the CF content as presented to the client did not enable them to examine the complexity of CF over time.
Neither of these aforementioned studies provide specific information about what practices CBT clinicians consider important to CF, what practices they report implementing, and whether this incorporates use of EBP. Accordingly, the overarching objective of the current study was to address this gap in the literature; notably, to examine which practices CBT clinicians report are important to CF, and which practices they report implementing.
The CF process itself has been proposed to comprise three stages (Eells, Reference Eells2007). First, the clinician gathers information about the client's current thoughts, emotions, behaviours, and contextual and historical factors in order to identify what client difficulties are to be the focus of treatment (Eells, Reference Eells2007). In the second stage, using information from the client and knowledge of the theoretical basis of treatment, the clinician develops hypotheses about how the client's current psychological difficulties and problems are maintained (Eells, Reference Eells2007). These hypothesised relationships can then be used to develop an initial treatment plan (Eells, Reference Eells2007). In the third stage, both the CF and the treatment plan based upon it are evaluated and revised by the clinician as treatment progresses (Eells, Reference Eells2007). A possible integration of EBPs as outlined by Lillenfeld et al. (Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013) and as applied to the practice of CF as described by Eells (Reference Eells2007) is presented in Table 1.
TABLE 1 Synthesis of the Three Steps of Case Formulation and Evidence-Based Practices
Note: *Based on Eells, (Reference Eells2007). **Based on Lilienfeld et al., (Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013).
Study Aims and Hypothesis
Considering the notable paucity of research that has focused on clinician perceptions and applications of CBT CF, the aim of the current study was to investigate whether clinician responses regarding what they believed was important in CBT CF, and what practices they implemented, reflected Eells’ (Reference Eells2007) grouping of CF activities into three stages. For the purposes of the current study, the Nezu et al. (Reference Nezu, Nezu and Lombardo2004, Reference Nezu, Nezu, Cos and Eells2007) and Persons (Reference Persons2006, Reference Persons2008) CF models were selected as the basis for investigating CF beliefs and practice among clinicians, as these models provide guidance on using information from the three sources defined by Lilienfeld et al. (Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013) as comprising EBP. In particular, the approaches by Nezu et al. (Reference Nezu, Nezu, Cos and Eells2007) and Persons (Reference Persons2006, Reference Persons2008) provide specific guidance regarding how to incorporate empirical findings and relevant external evidence into the CBT CF process.
Four specific hypotheses were tested. First, on the basis that: (a) research has shown that clinicians tend to rely on their experience, existing skills, or consultation with colleagues, when engaged in activities such as selecting treatments, and that there is resistance or reluctance to incorporate external evidence in clinical practice (e.g. Stewart, Stirman, & Chambless, Reference Stewart, Stirman and Chambless2012), and (b) studies have further shown that less experienced CBT clinicians may not identify all problems relevant to treatment (Haarhoff, Flett, & Gibson, Reference Haarhoff, Flett and Gibson2011) and struggle to identify relevant theory-driven components of CF (Dudley, Park, James, & Dodgson, Reference Dudley, Park, James and Dodgson2010), it was first hypothesised that clinicians would report that practices related to incorporating external evidence were less important, and that these practices were less likely to be implemented. Second, it was predicted that more experienced clinicians would be more likely to endorse practices related to using evidence in CF, as these practices may reflect higher levels of skill using theory to inform practice. Furthermore, research has documented that training also seems to improve CBT-based CF skills (e.g., Dudley, Ingham, Sowerby, & Freeston, Reference Dudley, Ingham, Sowerby and Freeston2015; Haarhoff, Gibson, & Flett, Reference Haarhoff, Gibson and Flett2011; Zivor, Salkovis, Oldfield, & Kushir, Reference Zivor, Salkovskis and Oldfield2013). Accordingly, the third hypothesis tested was that clinicians with higher levels of training would endorse more practices related to use of external evidence, on the basis that use of external evidence may be considered a more advanced CF skill. Finally, it was also hypothesised that participants with higher levels of professional accreditation would also endorse more practices related to use of external evidence.
Method
Participant Characteristics
A sample of 79 psychologists registered in Australia with the Psychology Board of Australia at the time of recruitment took part in an online survey. The participants were predominantly female (N = 68; 86%), while the average age of the sample was 40 years (SD = 11.23, range 26–69 years). Participants were required to meet the following inclusion criteria: (1) had completed their professional registration as a psychologist in Australia, (2) applied CBT at least monthly in their current work, and (3) had more than 6 months’ experience using CBT with clients. Of the 79 participants who started the survey, 9% did not complete all sections.
Measures
The online survey completed by participants consisted of items regarding participants’ demographic characteristics, including professional title, highest level of professional qualification, current work setting characteristics (i.e., own practice, outpatient or inpatient settings), current client caseloads (e.g., anxiety, PTSD), years of experience using CBT, training in CF and encouragement to use CF in the workplace.
Following a review of the published literature for CBT-based CF, two CBT CF methodologies were identified that contained specific guidance relating to use of external evidence during the CF process (Nezu et al., Reference Nezu, Nezu and Lombardo2004; Nezu, Nezu, & Cos Reference Nezu, Nezu, Cos and Eells2007; Persons, Reference Persons2006, Reference Persons2008). These published methodologies were used to develop statements related to general practice in CBT CF, and statements related to EBP in CBT CF. Statements were developed based on this literature, given no validated questionnaire assessing beliefs and practices related to CF in CBT could be identified. CF survey items derived from the aforementioned CBT CF methodologies described different activities associated with the three steps of CF outlined by Eells (Reference Eells2007). In summary, these steps related to: (1) the clinician assessing and describing the client's presenting problems, (2) generating hypotheses about how the presenting problems are being caused or maintained, and (3) evaluating these hypotheses. Within each of these three stages, activities relating to use of externally derived evidence were identified and described.
These statements were incorporated into the survey. For clinician beliefs about CBT CF activities, 13 statements focused on participants’ belief in the importance of different CF activities (see Table 4 for items). Participants were asked to rate the extent to which they agreed an activity was important in CF on a 5 point scale, with a score of 1 representing not important to case formulation, a score of 2 equivalent to of little importance to case formulation, 3 representing moderately important to case formulation, a score of 4 denoting important to case formulation and scores of 5 representing very important to case formulation.
A further 13 statements (see Table 5) assessed how frequently participants reported implementing these activities in their current CF practice (where current practice was defined as ‘the last 8 weeks of practice’). Frequency was indicated using a 5-point scale, where a score of 1 represented I never do this as part of case formulation, 2 equivalent to I rarely do this as part of case formulation, 3 represented I sometimes do this as part of case formulation, 4 represented I usually do this as part of case formulation, and a score of 5 represented I always do this as part of case formulation. Three final statements assessed how participants used CF in their current practice. This was scored using a 5 point scale, identical to the activity item score (ranging from a score of 1, representing I never do this to 5, equivalent to I always do this).
Following institutional ethics approval, the survey was piloted with two psychologists (a generalist psychologist and a clinical psychologist). Following feedback, one question relating to psychology registration was rephrased to ensure that only provisional psychologists were excluded from completing the survey.
Procedure
An email invitation was circulated to a variety of relevant professional networks and organisations throughout Australia. These included the Australian Psychological Society (APS), Australian Clinical Psychology Association (ACPA), Australian Association for CBT (AACBT), Headspace offices, Primary Health Networks, and various Facebook groups used by psychologists in Australia to discuss professional concerns. A snowball approach was also utilised, requesting that psychologists who completed the survey to also consider inviting colleagues to participate.
Data Analysis
Data were analysed using SPSS Version 22 to derive descriptive statistics, chi-square comparisons, repeated measures t tests to evaluate whether differences emerged between responses to survey items related to using external evidence and other survey items, and independent sample t tests to compare participants’ responses by professional title, and years of experience with CBT. As the survey on CBT CF beliefs and practice was developed for this study, exploratory factor analysis revealed adequate internal consistency, with Cronbach's alpha coefficient reported at 0.80 for the 13 items related to CF beliefs, and 0.79 for the 13 items related to CF practices. To assess whether a three-factor model could be derived from belief and activity items, confirmatory factor analysis was conducted. Based on recommendations in the literature on factor analysis (Costello & Osborne, Reference Costello and Osbourne2005; Fabrigar, Wegener, MacCallum, & Strahan, Reference Fabrigar, Wegener, MacCallum and Strahan1999), data reduction was conducted using principal axis factor analysis and oblique rotation. Results indicated that items relating to beliefs about importance of activities could be explained by a three-factor model, but reduction of items relating to implementation of activities did not produce coherent latent variables. Given this outcome, individual statements were used to explore the aims of the study.
Results
Analyses relating to participant characteristics were conducted prior to hypothesis testing. Sixty-one percent of the participants categorised themselves as clinical psychologists, and 39% as general psychologists. About half of participants (53%) had a master's degree as their highest qualification, with 32% reporting they had a doctorate-level qualification. Due to the small number of participants with certificate, diploma or bachelor studies as their highest level of qualification, these categories were combined, representing 14% of the total sample. As most participants with doctoral qualifications also held masters-level qualifications, these groups were not considered sufficiently different to compare, a finding confirmed by preliminary analysis. The number of participants with certificate/diploma/bachelor degrees as their highest level of qualification was small. As such, planned comparisons using qualifications could not be conducted.
Participants reported clinical experience using CBT ranging from less than 2 years to more than 5 years. Due to the small number of participants with less than 2 years’ experience, the categories of ‘less than 2 years’ experience’ and ‘2–5 years’ experience’ were combined. Almost two-thirds (63%) of participants reported having more than 5 years’ experience using CBT in their work, and 38% reported having 5 years’ experience or less. The professional characteristics of the total sample are summarised in Table 2. Participants with more than 5 years’ experience were significantly older than participants who had less than 5 years’ experience, t(77) = -5.61, p = .003; they were also more likely to have a doctoral qualification, χ2(2) = 6.99, p = .032. There were no significant differences in work settings or client characteristics when experience formed the basis for comparison (all ps > 0.05).
TABLE 2 Sample Demographics and Professional Details
Note: Some participants endorsed more than one work setting; Superscript a and b indicate p < .05.
In comparing accreditation status, clinical psychologists were found to be significantly older, t(77) = -2.85, p = .006, more likely to have completed doctoral level studies χ2(2) = 24.07, p < .001, and had more years of experience using CBT than psychologists, χ2(2) = 10.02, p = .002. Comparisons between psychologists and clinical psychologists revealed no significant differences in relation to current work setting, age of clients, or client presentations (all ps > .05).
Participants’ responses regarding specific training in CF indicated that there were no significant differences when responses were compared for psychologists and clinical psychologists, or when comparing responses for participants with more than 5 years’ or less than 5 years’ experience (all ps > 0.05; see Supplementary material online, Table 1). Overall, the most common source of CF training was self-directed (e.g., reading books and articles), with 89% of participants endorsing this option. Eighty-six percent of participants reported they had received training in CF during their studies, and 75% stated they had received training in CF at work (e.g., from a supervisor or from work-sponsored training). The majority of participants endorsed receiving training from all three sources. Participants were also asked to report whether they currently received encouragement to use CF in their workplace. Again, no significant differences emerged between groups when compared by professional title or years of experience. Overall, 78% of participants reported they were encouraged to use CF skills by their supervisor, and 68% agreed they were encouraged to use CF skills by work colleagues.
Factor Analysis of Beliefs and Practices Data
Responses to items about beliefs in importance of CF activities (N = 79) and CF activities implemented (N = 72) were reduced, to identify whether a three-factor structure emerged that reflected the grouping of CF activities as described by Eells (Reference Eells2007). Measures of sampling adequacy indicated that both sets of data were suitable for data reduction (KMO Measure of sampling adequacy = 0.74–0.77; Bartlett's test of sphericity significance = 0.000; all anti-image correlations > 0.50). Based on recommendation of Costello and Osbourne (Reference Costello and Osbourne2005), only items with correlations above .32 with a factor were retained.
Data reduction of item responses regarding beliefs in importance of CF activities indicated that a three-factor structure could be derived (see Table 3). Each factor contained three or more items loading above .5 on the factor, which Costello and Osbourne (Reference Costello and Osbourne2005) suggest is the minimum factor loading that can be considered stable. Factor 1 was moderately correlated with both factor 2 (r = -.34) and factor 3 (r = +.34); factors 2 and 3 were weakly correlated with each other (r = -.14). Data reduction of items related to activities implemented resulted in less coherent groupings of items, with lower item loadings on factors, and cross-loading of items on multiple factors. Factors 1 and 3 were moderately correlated (r = =.37), and factor 2 was weakly correlated to Factor 1 (r = +.11), and Factor 3 (r = -.07).
TABLE 3 Factor Analysis of Items Related to Importance of Activities to CBT CF
Beliefs and Practices in CBT CF
To test the first hypothesis, for the full sample (N = 79), means were calculated for each item that assessed the importance given to different CF activities, and the frequency with which participants reported engaging in these activities (see Tables 4 and 5 respectively). Mean scores indicated that the activities described were seen as relevant to CBT CF, with all activities achieving mean scores above 3. Participants’ scores for frequency of implementation of activities for the full sample (N = 72) indicated slightly more variation in responses compared with belief ratings, but overall, activities all achieved mean scores above 3.
TABLE 4 Beliefs about Importance of CF Methods
Note: ∧activities described as incorporating or using external evidence according to Nezu et al. (Reference Nezu, Nezu and Lombardo2004, Reference Nezu, Nezu, Cos and Eells2007) and Persons (Reference Persons2006, Reference Persons2008).
*p < .025; **Population variances unequal, t statistic, df and p results reported with equal variances not assumed. Clin psych = Clinical psychologist.
TABLE 5 CF Activities Implemented
Note: ∧activities described as incorporating or using external evidence according to Nezu et al. (Reference Nezu, Nezu and Lombardo2004, Reference Nezu, Nezu, Cos and Eells2007) and Persons (Reference Persons2006, Reference Persons2008).
*p < .025; **Population variances unequal, t statistic, df and p results reported with equal variances not assumed.
To compare responses on items describing activities related to use of external evidence, paired-sample t tests were conducted. The mean of five items in the survey on beliefs (these items are summarised in Table 4) was compared to the mean of the eight items describing other activities. Cronbach's alpha for the scales indicated acceptable internal consistency of scale item (see Supplementary Table 2). Results of scale comparisons indicated that participants rated items related to use of external evidence as significantly less important, compared to their ratings of importance of other CF activities t(78) = 10.901, p < .001. The mean of the five items describing implementation of practices related to use of external evidence was compared to the mean of the eight survey items describing other practices; the five items relating to use of external evidence are presented in Table 5. Cronbach's alpha for the scale related to use of external evidence was adequate, but questionable for the eight-item scale. Participants reported significantly less frequent implementation of activities related to use of external evidence compared to their implementation of other activities, t(71) = 10.070, p < .001.
Beliefs and Practices in CBT CF: Comparisons Between Groups
Participants’ responses related to beliefs and practices in CF were compared using differences in years of experience with CBT and differences in professional accreditation (general psychologist vs. clinical psychologist). A Bonferroni correction was used to adjust p, set at p < .025 for this set of analyses.
For the full sample, comparisons of responses about beliefs relating to CF indicated few differences. Participants with less than 5 years’ experience using CBT rated some activities as significantly less important to CBT CF; including evaluating explanations during therapy, t(77) = -2.72, p = .01, consulting theory or evidence relevant to a client's presenting problems, t(77) = -3.00, p < .001, and using a structured case formulation, t(77) = -2.32, p = .02. Few significant differences emerged when clinical psychologists and generalist psychologists were compared. Generalist psychologists’ ratings of the importance of consulting theory or evidence were significantly lower when compared to ratings given by clinical psychologist, t(77) = -2.34, p = .02, as were generalist psychologists’ ratings of the importance of using a structure case formulation, t(77) = -2.41, p = .018.
Comparisons of responses relating to frequency of implementation of CF practices also indicated very few overall differences, with no significant differences identified when participants with differing levels of experience were compared. Psychologists reported less frequent implementation of evaluation of their hypotheses about causal and maintaining factors, t(70) = -2.68, p = .01, and less frequently consulting theory or evidence relevant to the client's presenting problems, t(70) = -2.35, p = .02, compared to clinical psychologists.
Discussion
This study was the first survey conducted with psychologists (generalist and clinical psychologists) pertaining to CBT CF. Before the main aim and hypotheses could be tested, an initial goal was to identify whether participants’ responses reflected the three-step model of CF suggested by Eells (Reference Eells2007), which grouped CF activities into those: (1) that describe presenting problems, (2) activities aimed at generating hypotheses about presenting problems, and (3) activities aimed at evaluating hypotheses. Factor analysis of item responses related to beliefs about importance of activities in CBT CF supported a three-factor model, but the factors identified did not appear to support Eells’ (Reference Eells2007) model in its entirety. Factor 1, consisting of four items, appeared to comprise activities in which the clinician seeks contextual information relevant to planning treatment, such as finding out about client factors that may help or hinder treatment, consulting theory or evidence relevant to the presenting problems, and identifying the client's goals for treatment. Factor 2, comprising three items, related to activities used by clinicians to structure or check CF. Factor 3, which included five items, comprised activities related to the clinician describing and hypothesising about the client's presenting problems, including maintaining factors.
The three factors indicated that clinician responses grouped activities related to understanding client presenting problems (including hypothesising about those problems) separately from activities related to understanding psychosocial factors that may have an impact on treatment, and that were distinct from activities related to structuring and checking hypotheses. These results imply that clinicians view structuring and evaluation of hypotheses as a discrete activity in CBT CF, as suggested by Eells’ (Reference Eells2007) model. However, these findings also indicate that clinicians may perceive the distinction between activities related to description and hypothesis generation as less salient. These results should, however, be interpreted with caution. While loading of the items on factors can be described as adequate, the sample size can only be considered modest (Costello & Osbourne, Reference Costello and Osbourne2005). Furthermore, this pattern of item loadings was not reflected in data reduction of items describing reported practice of activities. Factor analysis of practice-related items provided support for grouping items related to use of external evidence, as items related to consulting theory, evaluating hypotheses and using assessments were related. Overall, however, the factor structure for these items did not appear to reflect Eells’ (Reference Eells2007) description, or suggest an alternative coherent pattern.
The specific aim of this study was to investigate clinician beliefs and practices related to EBP in CBT CF. On the basis of prior studies (Lilienfeld et al., Reference Lilienfeld, Ritschel, Lynn, Cautin and Latzman2013), it was hypothesised that CF activities, derived from published CBT CF methodologies that explicitly involved reference to external evidence would be seen as less important, and would be less frequently implemented by clinicians, and the findings supported this prediction. Participants rated consulting theory or evidence related to the presenting problem using third-party assessments (such as self-report questionnaires) to identify relevant constructs (thoughts or behaviours or feelings), using a structured case formulation, identifying how to evaluate hypotheses, and then evaluating hypotheses during therapy, as less important. These activities were also less frequently implemented compared to activities related to describing and explaining the presenting problem, and activities related to understanding factors that could affect treatment. However, it should be noted that the absolute differences were not large. That is, almost all activities were reported at least moderately important and were implemented at least sometimes during CF.
Furthermore, in line with our predictions, more experienced clinicians reported evaluating hypotheses during therapy and consulting theory related to the presenting problems, and using a structured CF to guide their CF, as significantly more important than less experienced clinicians. However, no significant differences were identified in reported frequency of implementation of any CF activities between clinicians according to experience. This outcome indicates that while experience was related to clinicians endorsing practices related to use of external evidence as more important to CF, this did not translate into observed differences in their reported practice.
Not surprisingly, in the current study, registered clinical psychologists endorsed higher levels of training (including having participants with a greater proportion of doctoral qualifications) and experience compared to general psychologists. However, few differences in ratings of importance of CF activities emerged between generalist psychologists and clinical psychologists. Clinical psychologists rated consulting theory or evidence relevant to presenting problems, and using a structured CF to guide CF as significantly more important compared to psychologist’ ratings of the importance of these activities, and they reported consulting theory or evidence related to the client's presenting problem with significantly higher frequency than registred generalist psychologists. These findings provide only partial support for the hypothesis that training level and experience influence use of external evidence in CBT CF.
Interestingly, however, when comparing participants with more or less experience, results indicated no significant differences in reported implementation of CF activities. Thus, it is possible that observed differences in reported practice between clinical psychologists and generalist psychologists may be due to differences in training background, although this explanation can only be considered tentative. While clinical psychologists had higher levels of training overall, a substantial proportion (67%) of general psychologists reported having a masters’ qualification, and it is likely that a number of those participants who marked the general psychologist category were in the process of working towards endorsement as clinical psychologists, under the Australian regulatory frameworks. Moreover, given the structure of many doctorate degree programs in Australia, it is likely many of the clinical psychologists who completed doctoral-level qualifications had also completed a masters’ degree within that qualification. As such, it is difficult to draw distinctions between the two groups in relation to differences in university-based training.
Given that clinical psychologists were identified to be significantly more experienced than psychologists, this may in part have contributed to difference in their CF practice. However, it is also possible that clinical psychologists are more likely to work in organisational contexts where such CF practices are encouraged, and/or have supervisory roles that encourage modelling of certain behaviours, or that client characteristics also play a role (e.g., greater client complexity or comorbidity, neither of which were assessed in this study, may be related to greater reliance on theory or external evidence when planning treatment). Therefore, further exploration of the role of professional accreditation in contributing to observed differences in CF practice is warranted, to determine the possible contributions of factors associated with accreditation including training and work setting, as well as considering client characteristics.
The results from this study indicate that CF practices related to using external evidence are more likely to be endorsed as important and more frequently put into practice by clinical psychologists and clinicians who have more experience. This finding has several implications. First, it supports the view that using external evidence effectively in CF is a clinical skill that can be developed. As such, it is encouraging that most participants in this survey indicated that they received training in CF through multiple formats, including university training, workplace training, and through self-directed learning. While self-directed training is to be encouraged, evidence indicates that it needs to be carefully directed to be an effective form of training in CF (Haarhoff et al., Reference Haarhoff, Flett and Gibson2011), and that more extensive training with supervision is more likely to result in improved skills (Zivor et al., Reference Zivor, Salkovskis and Oldfield2013). Moreover, the significantly lower rate of endorsement of activities related to use of external evidence, all of which were derived from published CBT CF methods (and are therefore accessible to clinicians), indicates that current training approaches used may not be effective in helping clinicians develop specific skills related to use of external evidence to supplement their understanding of the client's presentation, generate CF hypotheses, and identify and implement evaluation of those hypotheses.
A further implication of participants’ responses is the finding that clinicians with less experience and lower levels of accreditation who, it could be argued, are most in need of the guidance offered by empirical research, are less likely to believe it is important and may be less likely to use it. These clinicians are also less likely to have developed significant clinical expertise, and yet these results indicate that they may be likely to rely on their limited clinical experience in relation to CBT CF with clients. Research indicates that clinicians with less skill in CF are more likely to rate themselves as competent, and are less able to identify low quality formulations (Zivor et al. Reference Zivor, Salkovskis and Oldfield2013); therefore, it is possible that clinicians with less experience and training may not be aware of the gap between their perceived and actual competence in relation to CF. As such, they may be less likely or less able to use external evidence as a way to compensate for reduced competence. This has implications for training of psychologists in terms of their ability to develop and assess their CBT CF competency, as well as to learn strategies that can mitigate low competency levels.
This study has several limitations that should be taken into account when interpreting the findings. First, while the psychometric properties of the survey items on CF beliefs and practices were adequate (both with an alpha above 0.7), the measure still needs further validation in future research. The relatively small differences in responses to both belief and practice items may reflect that the scales used to capture differences did not do so adequately, or may reflect that, in fact, there is relatively little variation in perceived importance and implementation of CF activities. Capturing responses from a larger and more varied sample is therefore warranted in future research. A further limitation of this study was the difficulty in assessing differences in training level of participants, given the various current pathways to registration as a psychologist in Australia. Researchers Zivor et al. (Reference Zivor, Salkovskis and Oldfield2013) and Kuyken, Fothergill, Musa, and Chadwick (Reference Kuyken, Fothergill, Musa and Chadwick2005) sought to resolve this difficulty by testing participants who took part in a workshop on CBT CF either before or after they completed the workshop, and comparing responses, but acknowledged this approach also had limitations. It appears that future research in CBT CF would benefit from more robust methods of delineating training level, to assist in identifying what training, and how much training, results in improvements in CF skills.
Notwithstanding these limitations, the current study was the first to investigate clinicians’ beliefs and practices related to CBT CF, and use of external evidence within the CF, using activities derived from literature on CBT CF methodologies. Results indicate that clinicians focus on describing the client presentation, and developing explanations relating to that presentation. Beliefs and practices related to use of external evidence, and evaluation of the CF, appear to be seen as less important, and are less likely to be implemented. These findings indicate that further research is warranted on barriers to use of external evidence and evaluation in the CBT CF process. Clinician skills related to use of external evidence to develop their understanding of the client's presenting difficulties and evaluate that understanding are essential if psychologists are to confidently claim that their practice is fully evidence-based.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/bec.2018.5
