Introduction
Manual-based treatments are widely available in mental health care, including addiction care (Wilson, Reference Wilson1996). Treatment manuals indicate how professionals should provide treatment, in a step-by-step fashion in daily practice. As such, they assist professionals in their work and increase the quality of health care (Woolf and Grimshaw, Reference Woolf, Eccles and Grimshaw1999).
A commonly used manual-based therapy in addiction care is the Community Reinforcement Approach (CRA) (Meyers and Smith, Reference Meyers and Smith1995). CRA is a behavioural therapy based on operant learning theory and focusing on positive reinforcement. CRA is efficacious for treating different addictions (De Crescenzo et al., Reference De Crescenzo, Ciabattini, D’Alò, De Giorgi, Del Giovane, Cassar, Janiri, Clark, Ostacher and Cipriani2018; Meyers et al., Reference Meyers, Villanueva and Smith2005; Roozen et al., Reference Roozen, Kerkhof and Van den Brink2003; Schottenfeld et al., Reference Schottenfeld, Pantalon, Chawarski and Pakes2000). CRA includes multiple interventions, the so-called CRA procedures, creating flexibility to match the approach with the individual patient’s needs. Examples of these procedures are the Functional analysis of substance use, Communication skills, and Refusing substances skills.
Adherence to the CRA manual by therapists is thought to be an important aspect of its effectiveness (Campos-Melady et al., Reference Campos-Melady, Smith, Meyers, Godley and Godley2017; Garner et al., Reference Garner, Godley, Funk, Dennis, Smith and Godley2009; Garner et al., Reference Garner, Hunter, Funk, Griffin and Godley2016). In accordance, two randomized trials on the Adolescent version of CRA (A-CRA; N=399) showed that adolescents receiving 12 or more A-CRA procedures were more likely to be in recovery at follow-up (55 vs 35%) than those receiving less procedures (Garner et al., Reference Garner, Godley, Funk, Dennis, Smith and Godley2009; Garner et al., Reference Garner, Hunter, Funk, Griffin and Godley2016). Similarly, in a study among adolescent patients with substance use disorders (N=384), therapists’ adherence to A-CRA was predictive of patient substance use over 1-year follow-up (Campos-Melady et al., Reference Campos-Melady, Smith, Meyers, Godley and Godley2017).
Nevertheless, it is well known that manual-based treatments are often not delivered according to protocol (Shafran et al., Reference Shafran, Clark, Fairburn, Arntz, Barlow, Ehlers, Freeston, Garety, Hollon, Ost, Salkovskis, Williams and Wilson2009, Waller, Reference Waller2009). For instance, a study on Cognitive Behavioral Therapy (CBT) for eating disorders found that half of the therapists did not routinely use any of the advised techniques (Waller et al., Reference Waller, Stringer and Meyer2012). Similarly, a study in patients with mood disorders showed that only 21% of patients with a major depressive disorder received psychological treatment according to the protocol (Kessler et al., Reference Kessler, Merikangas and Wang2007).
Studies exploring causes and consequences of therapists’ non-adherence to treatment manuals suggest that treatment-, therapist- and organizational-related determinants may either facilitate or impede protocol adherence (Perepletchikova and Kazdin, Reference Perepletchikova and Kazdin2005). Regarding treatment-related determinants, a qualitative study among clinicians of 68 organizations suggests that A-CRA’s complexity might be related to A-CRA adherence (Hunter et al., Reference Hunter, Han, Slaughter, Godley and Garner2015). Similarly, the requirement of multiple materials was associated with poor therapist adherence in a narrative review on therapists’ protocol adherence in general (Perepletchikova and Kazdin, Reference Perepletchikova and Kazdin2005).
Therapist-related determinants have also been related to therapists’ adherence (Waller, Reference Waller2009). In CRA, only one observational study explored therapist-related determinants of adherence. Among therapists in addiction care (N=69), the self-reported delivery of CRA procedures correlated with the level of received CRA training (Kraan et al., Reference Kraan, Dijkstra and Markus2018). In other fields of mental health care, therapist-related determinants of protocol adherence have also been identified. For instance, in a study on CBT for eating disorders, carried out among 139 clinicians, therapist anxiety was negatively associated with self-reported application of exposure and behavioural experiments (Mulkens et al., Reference Mulkens, de Vos, de Graaff and Waller2018).
Finally, with regard to organization-related determinants of therapists’ adherence, the authors of the above-mentioned qualitative study on A-CRA in addiction care conclude that the consistency of the intervention with the overall mission of the organization might facilitate implementation of A-CRA in clinical practice (Hunter et al., Reference Hunter, Han, Slaughter, Godley and Garner2015).
In sum, several studies have demonstrated that therapists’ adherence to treatment manuals is relevant for the outcome of manual-based treatments. It is unclear, however, which specific determinants contribute to adherence of therapists to the CRA manual in addiction care. The current study assessed the association of a broad range of treatment-, therapist- and organization-related determinants with therapists’ self-reported CRA adherence. First, we assessed which determinants were associated with therapists’ CRA adherence. Next, we explored which independent determinants contributed most to this self-reported adherence to CRA procedures.
Method
Study design
A cross-sectional design was used to assess therapists’ self-reported adherence to the CRA manual. This study was a part of a larger study of which part of the data have already been published (Kraan et al., Reference Kraan, Dijkstra and Markus2018). The current study focused on the same therapists and on a broad range of determinants associated with CRA adherence. Data collection took place between April and October 2015. The local institutional ethics board of both participating treatment centers judged that external medical-ethical approval was not needed for the study.
Study context
The study was carried out within two addiction care centres in the Netherlands, henceforth identified as centres A (IrisZorg) and B (Novadic-Kentron). Both use CRA as their main treatment approach, although this can be supplemented with additional interventions such as pharmacotherapy, social support, etc. Both centres started with the implementation of CRA in 2011. Therapists worked in out-patient settings in both urban and rural areas.
Participants
Participants were 69 therapists (23.2% male) working at two addiction centers (A: N=31; and B: N=38). Therapists worked at 13 different treatment sites. Their mean age was 42.2 years (SD=11.7) and they had been employed at the treatment centres for on average 12.2 years (SD=9.0). Therapists had diverse professional backgrounds, including social work, nursing and psychology. On average, they had worked with CRA for 3.3 years (SD=1.8). All therapists had previously attended a two-day basic CRA training provided by a licensed CRA trainer, and received regular supervision provided by certified CRA supervisors employed at the treatment centres. Supervision included demonstrations by supervisors, case reviews and practising CRA procedures by role-playing. One third of the therapists were certified CRA therapists, while 21.7% were still in the process of attaining certification. For certification purposes, audio recordings of sessions were evaluated by trained reviewers, based on a coding manual for evaluating CRA fidelity (Smith et al., Reference Smith, Lundy and Gianini2007). Our previous work showed that there were no differences between the characteristics of the therapists of the two treatment centres with the exception of years working at that centreFootnote 1 (Kraan et al., Reference Kraan, Dijkstra and Markus2018).
Treatment programme
CRA consists of 12 CRA procedures; see Table 1. Therapists choose which CRA procedure they want to use in each session, depending on the goals of the patient. Individual patients may have received some procedures multiple times and may have never received others (Campos-Melady et al., Reference Campos-Melady, Smith, Meyers, Godley and Godley2017).
Table 1. Description of the CRA procedures (based on Meyers et al., Reference Meyers, Roozen and Smith2011)
Measures
The CRA survey used here (Kraan et al., Reference Kraan, Dijkstra and Markus2018) was based on the Measurement Instrument for Determinants of Innovations (MIDI), an implementation framework developed to gain better understanding of professional adherence to innovations in health care (Fleuren et al., Reference Fleuren, Paulussen, Van Dommelen and Van Buuren2014). The MIDI was developed based on a systematic review of literature, Delphi procedures and expert opinion; see Fleuren and collegues (Reference Fleuren, Paulussen, Van Dommelen and Van Buuren2014) for further details. The MIDI consists of 29 determinants that facilitate or impede implementation of an innovation. The determinants are grouped into innovation-, professional-, organizational- and socio-political-related determinants. Most determinants consist of single questions but some of multiple questions. Depending on the implementation or innovation, the user is encouraged to adapt the items, the number of items and the selection of determinants which are deemed relevant. The MIDI has been used successfully to evaluate the adherence to guidelines on preventing child abuse and to study barriers and facilitators to the implementation of a pediatric palliative care team (Konijnendijk et al., Reference Konijnendijk, Boere-Boonekamp, Fleuren, Haasnoot and Need2016; Verberne et al., Reference Verberne, Kars, Schepers, Schouten-van Meeteren, Grootenhuis and van Delden2018).
The CRA survey used in this study consisted of two parts, divided into: self-reported adherence (part 1) and determinants (part 2). The first part of the CRA survey regarding adherence to CRA consisted of 12 items scored on a 100 mm visual analogue scale (VAS). Therapists rated adherence to each CRA procedure (‘In the past six months, what % of your patients did you offer CRA procedure [name procedure]?’). Overall CRA adherence was derived by calculating the average of the adherence scores of all CRA procedures. A higher score indicated a more frequent delivery of CRA procedures per therapist. Although the CRA survey involves a self-reported meausure of adherence, our initial study found that the survey findings were largely consistent with a repetitive registration list (Kraan et al., Reference Kraan, Dijkstra and Markus2018).
In the second part of the CRA survey, therapists scored 53 items (or theses) about CRA procedures which best fitted their judgement. The items had a 5-point Likert-type scale (1: ‘totally disagree’; 2: ‘disagree’; 3: ‘neither agree/nor disagree’; 4: ‘agree’; and 5: ‘totally agree’. Only ‘Descriptive norm’ had a 6-point Likert-type scale and scores ranged from ‘none of my colleagues’ to ‘almost all colleagues’. The items covered 16 determinants, grouped in three categories: treatment-, therapist- and organization-related determinants. See Table 2 for an overview of the determinants. Fifteen determinants were based on the determinants of the MIDI. We added one non-MIDI determinant ‘Therapist insecurity’, because research suggests that this might be relevant for therapists’ protocol adherence (Mulkens et al., Reference Mulkens, de Vos, de Graaff and Waller2018; Waller, Reference Waller2016). Because of the scope of the study, we did not include MIDI determinants associated with the socio-political context (e.g. laws and regulations), and included only two organization-related determinants applicable in the current context. Other organizational determinants, e.g. financial resources, were not included.
Table 2. Description of the determinants of the CRA survey
Procedure
Therapists were asked to fill in the survey during team meetings to assess the self-reported adherence to the 12 CRA procedures and their judgement of the different determinants.
Data analysis
Survey data were entered in Statistical Program for the Social Sciences (IBM SPSS Statistic 22 for Windows; IBM Corporation, 2013). Scores on negatively formulated items were reversed, so that higher scores indicated a more positive answers for all items and determinants.
Reliability analyses were performed for determinants that were measured by multiple items. Analyses revealed Cronbach’s alpha values between .59 and .97. Most determinants had respectable Cronbach’s alpha values (see Table 2). Two determinants (‘Self-efficacy’ and ‘Practicality’) were excluded from further analysis because of unacceptable Cronbach’s alphas (<.70). Next, three items were shifted to the determinant ‘Outcome expectations’ because the content of the items was more in line with the original formulation of this determinant of the MIDI. It concerned the following three items: (1) ‘I do not perform some procedures because they are of no use to patients’, (2) ‘I expect that with CRA procedures the following objectives will actually be achieved in my patients…’, and (3) ‘Procedures fit well within the framework of scientific research, but not in clinical practice’.
A correlation analysis was used to test which determinants were associated with CRA adherence. The dependent variable (therapists’ self-reported adherence to CRA procedures) was normally distributed (Kolmogorov-Smirnov: .08), but the independent variables (determinants) had an ordinal level, so Spearman’s rho correlation (r) was used. Correlations in the range of .10–.29 were considered ‘small’, .30–.49 as ‘medium’ and >.50 as ‘large’ (Field, Reference Field2013).
In order to test which determinants contributed most to CRA adherence, and to estimate effect sizes, all determinants significantly correlating with adherence were included in a regression analysis with CRA adherence as a dependent variable. We started with the complete model (backward method) in which the effects of all variables simultaneously is considered. Because the survey contained both 5- and 6-point Likert scale items and are therefore not comparable, standardized coefficients Beta are reported.
Results
Descriptives
Table 3 summarizes scores for all determinants and the correlations with therapists’ self- reported CRA adherence. Scores on the determinants with a 5-point Likert-type scale ranged from 2.9 (therapist-related determinant ‘Insecurity’) to 3.8 (treatment-related determinant ‘Complexity’), indicating that therapists rated determinants as neutral to positive. Therapists perceived CRA procedures as not complex (82.3% scored >2) and complete (81.9% scored >2). Furthermore, therapists perceived patients’ cooperation to CRA procedures as neutral to positive (79.4% scored >2). The only determinant with a 6-point Likert-type scale, ‘Descriptive norm’, showed that therapists had the impression that at least half of their colleagues delivered CRA procedures (72.1% scored >3).
Table 3. Descriptives of the determinants of the CRA survey and the correlation with CRA adherence
* 6-point Likert-type scale (instead of 5-point Likert-type scale).
Associations between determinants and CRA adherence
Significant associations with therapists’ self-reported CRA adherence were found for nine out of 14 determinants, although most values represented medium effects (see Table 3). Determinants with correlation coefficients higher than .50 included ‘Compatibility’ (r=.51, p<.001), ‘Outcome expectations’ (r=.59, p<.001) and ‘Task perception’ (r=.53, p<.001).
Regression analysis
All nine determinants significantly correlating with therapists’ self-reported adherence were included in a multi-level regression analysis. Regression analysis showed a strong association with CRA adherence on treatment-related determinant ‘Compatibility’, and two therapist-related determinants, ‘Outcome expectations’ and ‘Descriptive norm’ (see Table 4). The higher therapists scored on these determinants, the higher their adherence to CRA procedures. These three determinants together explained 43% of variance in therapists’ CRA adherence.
Table 4. Regression analysis
R 2=.458 and adjusted R 2=.428 for this model. Determinants included in the multi-level regression analysis: Completeness, Observability, Compatibility, Personal benefits/drawbacks, Outcome expectations, Task perceptions, Patient cooperation, Descriptive norm and CRA supervision.
Discussion
The aim of this explorative study was to identify which treatment-, therapist- and organization-related determinants influence therapists’ self-reported adherence to the CRA manual. While most of the studied determinants correlated positively with therapists’ CRA adherence, correlations were small to medium. Regression analyses identified three determinants with the largest contribution to therapists’ adherence to CRA, namely ‘Compatibility with previous working methods’, ‘Outcome expectations of the therapist’ and ‘Descriptive norm’. These three determinants combined explained 43% of the variance in adherence to the CRA manual. We will discuss the findings with regard to these three determinants below.
First, the observation that the treatment-related determinant ‘Compatibility with previous working methods’ was associated with therapists’ CRA adherence is in line with previous work suggesting that therapists have more positive attitudes towards treatment protocols that are more familiar and similar to the approaches they have used in the past (McGovern et al., Reference McGovern, Fox, Xie and Drake2004). It helps therapists to master a new treatment protocol when that matches with what therapists are familiar with and are already doing. New practices are learned more easily when there are no previously learned procedures that must be disregarded in order to adopt a new treatment protocol (Garner et al., Reference Garner, Hunter, Godley and Godley2012).
Second, the therapist-related determinant ‘Outcome expectations of the therapist’ was associated with therapists’ CRA adherence, which shows that therapies that are perceived as more effective by therapists are implemented with higher protocol adherence than those viewed as less effective (Perepletchikova and Kazdin, Reference Perepletchikova and Kazdin2005). In accordance, knowledge about CRA’s effectiveness contributes positively to therapists’ outcome expectations, and increased therapists’ adherence in previous studies (Campos-Melady et al., Reference Campos-Melady, Smith, Meyers, Godley and Godley2017). It is also known that therapists who are convinced of the effectiveness of the treatment convey this belief to patients which in turn contributes to patients’ own outcome expectations (Glass et al., Reference Glass, Arnkoff and Shapiro2001; Greenberg et al., Reference Greenberg, Constantino and Bruce2006). In accordance, it has been suggested that, for an effective implementation of a treatment method, trainers should emphasize the relevance and effectiveness of that method to therapists that start working with it (McGovern et al., Reference McGovern, Fox, Xie and Drake2004).
Finally, the therapist-related determinant ‘Descriptive norm’ (perceived level of adherence to CRA by their colleagues) was associated with adherence to the CRA manual. The impact of descriptive norms on behaviour has widely been studied in the light of the theory of planned behaviour and the reasoned action approach (Ajzen, Reference Ajzen1991; Fishbein and Ajzen, Reference Fishbein and Ajzen2010; McEachan et al., Reference McEachan, Taylor, Harrison, Lawton, Gardner and Conner2016; Rivis and Sheeran, Reference Rivis and Sheeran2003). Descriptive norms, which involve perceptions of what others actually do, influence behaviour indirectly through intentions and also directly reflecting modelling or other processes (McEachan et al., Reference McEachan, Taylor, Harrison, Lawton, Gardner and Conner2016). In a meta-analysis, McEachan et al. (Reference McEachan, Taylor, Harrison, Lawton, Gardner and Conner2016) found a modest but significant correlation between descriptive norms and behaviour. They also concluded that the descriptive norm–behaviour relationship was significantly stronger in young people. It may therefore be useful to use the power of such descriptive norms in the implementation of treatments with strategies such as modelling and group performance, specific in case of younger therapists.
Although the aforementioned three determinants explained 43% of the variance regarding CRA adherence, it is still unclear which determinants might explain the remaining variance. Hunter et al. (Reference Hunter, Han, Slaughter, Godley and Garner2015) stressed the importance of several organizational determinants with regard to adherence, such as financial and political stability, and implementation quality, as well as the important role of mid-level managers who can either facilitate or impede implementation (Urquhart et al., Reference Urquhart, Porter, Sargeant, Jackson and Grunfeld2014). In addition, patient characteristics like co–morbidity and intellectual or educational level have been shown to influence therapists’ adherence (Addis et al., Reference Addis, Wade and Hatgis1999; Boswell et al., Reference Boswell, Gallagher, Sauer-Zavala, Bullis, Gorman, Shear, Woods and Barlow2013). These determinants were not included in the current study.
Clinical implications
The current findings suggest that new therapists should receive a tailored training in the treatment method used, based on their previous treatment experience. In the case of CRA, those who are familiar with CBT, especially in a similar setting, may need less instruction than those with other backgrounds. Specific attention should be given to commonalities and important differences on critical dimensions with previous therapeutic models with which they are familiar (Perepletchikova and Kazdin, Reference Perepletchikova and Kazdin2005).
Second, it might be important to pay attention to create positive outcome expectations so that therapists can confidently propagate the effectiveness of the treatment to patients. Investing in a realistic but positive learning environment by sharing success stories, using routine outcome monitoring to make patient outcomes visible, keeping therapists informed on new scientific findings and providing supportive feedback in supervision, may further augment outcome expectations. In addition, the relevance of continued coaching after completing basic training with regard to protocol adherence has been demonstrated for many treatment modalities (Lochman et al., Reference Lochman, Boxmeyer, Powell, Qu, Wells and Windle2009; Miller et al., Reference Miller, Yahne, Moyers, Martinez and Pirritano2004; Webster-Stratton et al., Reference Webster-Stratton, Reid and Marsenich2014). Coaching and supervision by an expert can help increasing therapists’ proficiency in providing a treatment as prescribed in the manual (Webster-Stratton et al., Reference Webster-Stratton, Reid and Marsenich2014).
Finally, next to training and education, the influence of descriptive norms on behaviour can be explicitly used within teams to optimize adherence, for example by providing benchmark information. For instance, experiences of colleagues and teams that excel can be used. In line with Garner et al., it may be helpful to provide a target norm of treatment sessions (session exposure) and CRA procedures (procedure exposure) for each patient (Garner et al., Reference Garner, Godley, Funk, Dennis, Smith and Godley2009; Garner et al., Reference Garner, Hunter, Funk, Griffin and Godley2016; Garner et al., Reference Garner, Hunter, Slaughter, Han and Godley2017) because of the positive association of CRA procedure exposure with decreased substance use (Garner et al., Reference Garner, Hunter, Funk, Griffin and Godley2016).
Limitations and future research
The findings should be interpreted in the context of certain limitations. First, the observational nature of the study prohibits causal inferences. The use of retrospective self-report may have biased the findings regarding CRA adherence. Previous research showed that therapists tend to over-estimate their intervention delivery (Hogue et al., Reference Hogue, Dauber, Lichvar, Bobek and Henderson2015). However, an additional repetitive registration list used in our first study yielded corresponding results indicating realistic adherence rates (Kraan et al., Reference Kraan, Dijkstra and Markus2018).
Furthermore, this study focused on therapists’ CRA adherence but this did not include the quality of the treatment, including the extent to which CRA procedures matched the patients’ goals and capabilities. Therapists’ competence is also associated with better treatment outcomes (Campos-Melady et al., Reference Campos-Melady, Smith, Meyers, Godley and Godley2017) and therefore research should not only focus on adherence (the what) but also on how therapists deliver CRA. In addition, the survey was mainly based on a promising but not extensively validated instrument which was supplemented with some self-formulated items. However, multi-item determinants based on the MIDI items had high Cronbach’s alphas, which suggests that they were reliable. Finally, the MIDI, which is often used in research on the implementation of innovations and as such can be used before and after the introduction of an innovation, may be less suitable for a process evaluation four years after CRA was introduced within the two participating treatment centres. However, the MIDI itself does not prescribe ‘appropriate timeliness’.
Future research should take the aforementioned methodological issues into account and should also consider other measures for therapists’ adherence and other determinants that underlie adherence to treatment manuals, such as organizational determinants and focus on clarifying causality between determinants and adherence. In addition, more research is needed on the relevance of CRA adherence regarding treatment outcome, as has been done for A-CRA (Garner et al., Reference Garner, Godley, Funk, Dennis, Smith and Godley2009). It might be relevant for clinical practice to identify whether a target level of CRA procedures is helpful in improving a patient’s recovery. Finally, it should be explored whether the current findings are also relevant for other treatment approaches and in patients with other conditions than addictive disorders.
Conclusions
This study identified three main determinants of therapists’ self-reported adherence to CRA: (1) ‘Compatibility’ with the way therapists are used to work, (2) therapists’ ‘Outcome expectations’, and (3) therapists’ perceived number of colleagues also working with the treatments’ interventions (‘Descriptive norm’). These determinants should be taken into account when implementing a behavioural treatment like CRA in mental health care or addiction care. In addition, more is to be learned about other determinants associated with CRA adherence, as the determinants identified in this study explained 43% of the variance in therapists’ adherence. Future studies should explore which other determinants influence therapists’ protocol adherence, and how these factors could be targeted to improve adherence to and implementation of (complex) interventions.
Key practice points
-
(1) Therapists are frequently non-adherent to the protocol of (complex) behavioural interventions, which negatively impacts treatment outcome.
-
(2) We identified three determinants of CRA adherence in addiction care, namely Compatibility, Outcome expectations and Descriptive norm, together explaining 43% of the variance in therapists’ CRA adherence.
-
(3) When implementing a (complex) behavioural interveniton like CRA, attention must be paid to tailored training depending on the previous treatment experience of therapists.
-
(4) Training and education should create positive outcome expectations so that therapists can confidently propagate the effectiveness of the treatment to patients.
-
(5) The influence of descriptive norms on behaviour can be explicitly used within teams to optimize adherence, for example by making experiences explicit of therapists and teams that excel.
Data availability statement
The data collection protocols and datasets used during this study are available from the corresponding author.
Acknowledgments
We thank addiction centres IrisZorg and Novadic-Kentron, as well as all the individuals who assisted in this study, especially Reinier van den Haak for data collection and management. In particular, we thank the participating therapists. We also thank Hendrik Roozen as CRA trainer in The Netherlands for his valuable input regarding the study.
Author contributions
Anneleen Kraan: Conceptualization (lead), Data curation (lead), Formal analysis (lead), Investigation (lead), Methodology (lead), Project administration (lead), Software (lead), Visualization (lead), Writing – original draft (lead); Wiebren Markus: Conceptualization (supporting), Methodology (supporting), Supervision (supporting), Writing – review & editing (supporting); Arnt Schellekens: Supervision (supporting), Writing – review & editing (supporting); Boukje Dijkstra: Conceptualization (supporting), Formal analysis (supporting), Methodology (supporting), Supervision (lead), Writing – review & editing (supporting).
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
Competing interests
The authors declare none.
Ethical standards
This research conforms to the Declaration of Helsinki. The local ethical committees judged that there was no need for ethics approval for this study. Informed consent was received from all participants prior to data collection.
Open access
Comments
No Comments have been published for this article.