Progress on NICE guideline implementation in mental health trusts: meta-analyses

AIMS AND METHOD 

To investigate implementation of National Institute for Health and Clinical Excellence (NICE) guidelines in mental health, focusing on the schizophrenia guideline. Data analyses centred on implementation of the guideline, as well as looking at a set of markers mapped to the NICE principles of implementation and other identified clinical prerequisites. A self-report questionnaire tool was sent to senior executives at mental health trusts containing questions linked to the markers of implementation and clinical prerequisites; responses were analysed with data from the Healthcare Commission audit of implementation of the guideline to show key relationships.

RESULTS 

Information from both data-sets (senior executive data collection and the audit) showed that implementation is patchy, with pockets of good implementation. Findings indicate that higher levels of implementation are linked to corporate commitment and leadership, as well as support from commissioners.

CLINICAL IMPLICATIONS 

Implementation might be improved by corporate commitment and leadership and better support from commissioners.


AIMS AND METHOD
To investigate implementation of National Institute for Health and Clinical Excellence (NICE) guidelines in mental health, focusing on the schizophrenia guideline. Data analyses centred on implementation of the guideline, as well as looking at a set of markers mapped to the NICE principles of implementation and other identified clinical prerequisites. A self-report questionnaire tool was sent to senior executives at mental health trusts containing questions linked to the markers of implementation and clinical prerequisites; responses were analysed with data from the Healthcare Commission audit of implementation of the guideline to show key relationships.

RESULTS
Information from both data-sets (senior executive data collection and the audit) showed that implementation is patchy, with pockets of good implementation. Findings indicate that higher levels of implementation are linked to corporate commitment and leadership, as well as support from commissioners.

CLINICAL IMPLICATIONS
Implementation might be improved by corporate commitment and leadership and better support from commissioners.
To date there is little research about how successfully trusts are implementing guidance from the National Institute for Health and Clinical Excellence (NICE), but what there is suggests that implementation is inconsistent (Sheldon, 2004). A large amount of information on evidence-based implementation methods is published by NICE to support trusts with this task (National Institute for Health and Clinical Excellence, 2006). In order to determine the degree of progress and the key implementation processes used, the National Collaborating Centre for Mental Health (NCCMH) carried out a survey of mental health trusts, analysing data alongside findings from the Healthcare Commission audit of the implementation of the schizophrenia guideline.

Method
We used data from two sources to unpack progress with implementation: data from a bespoke questionnaire tool, and a meta-analysis using this alongside data from the Healthcare Commission audit of the implementation of the NICE schizophrenia guideline. The questionnaire items were derived from two sources, to answer two key questions: first, how far have trusts followed NICE's principles of implementation; and second, what evidence is there that trusts are implementing guidelines? Question one is answered by assessing the extent to which trusts gave positive responses to specific questionnaire items derived from the NICE principles. To Commander & Rooprai Long-stay patients on acute psychiatric wards original papers BOARDMAN, J. & PARSONAGE, M. (2007)  answer the second question, expert advisors to the project have identified key implementation markers, which show that a trust is fully engaged with implementation. Questionnaire items were mapped to the principles summarised in Box 1. The forms were sent to all chief executives and medical, clinical and clinical governance directors at all National Health Service (NHS) mental health trusts (including primary care trusts providing mental health services) in England as well as combined trusts in Wales (a total of 488 questionnaires sent to 79 trusts).
Data from the recent audit of implementation of the schizophrenia guideline (Healthcare Commission, 2006), carried out as part of the Healthcare Commission joint review of community mental health services, were reanalysed alongside data from the present study. The audit identified a number of key implementation markers, and gathered data to assess adherence with these: availability and use of family interventions and cognitive^behavioural therapy (CBT), provision of information and occupational needs, physical health checks, prescribing (atypical antipsychotics, depot formulations, treatment-resistant schizophrenia), monitoring, medication and advance directives. A proxy measure of guideline implementation was created within the data-set by summing the trustlevel scores for each of the indicators in the audit.
Data were analysed using SPSS version 14 for Windows. Correlations and one-way analyses of variance (ANOVAs) were used to identify significant univariate relationships; linear regression was used for multivariate effects. In order to examine fully the effect of corporate commitment and leadership, a new variable was constructed comprising identified executive lead, identified clinical director lead, clinical champion, team to support the clinical champion, and reports to trust board. The new variable showed an acceptable Cronbach's a value of 0.68.

Results
A total of 209 usable forms were returned, an adjusted response rate of 47% (209/(488744)). Data are available for the various professional groups sampled: 59% of forms were returned from chief executives, 42% from clinical directors, 47% from medical directors and 59% from directors of clinical governance. Data were returned from 69 of the 79 trusts sampled (87%), ranging from 0% to 100% of forms returned (mean 48%).

Descriptive results
Corporate and local leadership (mapped to NICE principle 1) . 56% indicated that the trust had identified a clinical director-level lead for guideline implementation . 46% indicated that the trust had identified a clinical guideline implementation champion; of the clinical champions, 57% had a team to support them . 85% indicated that the trust had a committee to oversee implementation of guidelines . 57% indicated that trust boards routinely received reports on guideline implementation.
Staffing and resources (mapped to NICE principles 2 and 3) . 32% indicated that the trust had appointed staff specifically to implement NICE guidelines . 42% indicated the trust had calculated likely resource implications for guidelines . 38% indicated the trust had committed funds to implementation . funds committed ranged from »10 000 to »1.2 million (mean »34 000).
Commissioner resourcing (mapped to NICE principle 4) . 84% indicated that recent NHS funding problems had prevented (or would be likely to prevent) implementation of guidelines . 71% rated the trust's support from their commissioning primary care trust in implementing health technology appraisals as poor, very poor or non-existent . 82% rated the trust's support from their commissioning primary care trust in implementing guidelines as poor, very poor or non-existent.
Accessibility of guidelines (mapped to NICE principle 5) . 70% indicated that NICE guidelines were easily available on the trust intranet . 67% indicated that the trust also made guidelines accessible in the course of clinical practice in another way . 86% had looked for implementation information on the NICE website.
Operational implementation (mapped to NICE principle 5) The use of outcome measures is a proxy for implementation, because it is a precursor for appropriate recording of patient information and an indicator that diagnosis is being recorded. Respondents were asked if clinical teams routinely used outcome measures. Responses are shown in Fig. 1. Seventy per cent of respondents indicated that their trust used a computerised clinical information system for patient details.
Monitoring (mapped to NICE principle 6) . 73% indicated that the trust had audited health technology appraisal implementation . 84% indicated that the trust had audited guideline implementation.

Inferential results
Commissioner support of health technology appraisal implementation One-way analysis of variance showed that a lower commissioning score is associated with: . Correlation showed that a lower score for commissioning support is linked to a lower level of commitment and leadership (see below; R=0.180, P=0.048).

Commissioner support of guideline implementation
Our hypothesis was that, as with support for health technology appraisals, commissioning support of guideline implementation might exert an effect on other variables. One-way ANOVA showed that poorer commissioning support of guidelines was associated with: . higher level of health technology appraisal audit (F=2.471, d.f.=4, P=0.046) . poorer access to guidelines other than on the intranet (F=3.184, d.f.=4, P=0.005).

NHS funding problems
Our hypothesis was that funding problems might curtail implementation. This proved not to be the case for audit of health technology appraisals, where reported funding problems were associated with: . higher likelihood of audit taking place (F=14.667, d.f.=1, P50.001); . poorer level of access to guidelines other than on the trust intranet (F=7.375, d.f.=1, P=0.007).

Commitment and leadership
One-way ANOVA showed that better levels of commitment and leadership were associated with: . staff appointed to support implementation ( Correlation showed that better commitment and leadership is associated with more prevalent use of outcome measures (R=0.194, P=0.008).

Commissioner support, and corporate commitment and leadership
Correlation shows a positive relationship between commissioner support for health technology appraisals and commitment and leadership (R=0.180, n=181, P=0.016).

Analyses of the Healthcare Commission audit
A series of analyses were carried out to look at the relationship between previously identified 'causal' factors and implementation as measured by the Healthcare Commission audit of implementation of the schizophrenia guideline.

Descriptive results
Items within the audit were collapsed into a number of key markers showing adherence levels (

Inferential results
In order to look at statistical relationships, the individual scores for the audit were combined to give an overall proxy measure for implementation. Correlations showed that better levels of implementation were associated with the following factors (data at trust level): . better corporate commitment and leadership (R=0.342, n=57, P=0.009) . existence of a committee to oversee implementation (R=0.429, n=57, P=0.001) . commissioner support for health technology appraisal (R=0.459, n=57, P=0.000) . less likelihood of guideline audit having been carried out (R=70.270, n=57, P=0.042).
The data were submitted to a step-wise linear regression. All variables from the NCCMH survey were input as independent variables, the schizophrenia proxy as the dependent. The model yielded a reasonable R 2 adj of 0.284 using three variables, model ANOVA significant (F=8.410, d.f.=3, P50.001), and collinearity statistics were within acceptable range (condition index=7.72). Variable (b) weights showing direction and strength of effect in the model were: . commissioner support for TA, b=0.39 . corporate support, b=0.26 . identified clinical director lead, b=0.23.
This interaction between commissioner support and corporate support has a powerful predictive effect on implementation.

Limitations
The response rate was below 50%, and thus the findings must be seen as illustrative rather than definitive. To present the results from the survey we used all individual responses, as opposed to overall trust responses. There is, in some cases, variation between responses from different individuals from the same trust (although analysis of variance showed no significant results). Since the study focused only on mental health trusts, findings are not generalisable to other healthcare settings; the survey also provides a snapshot only of the current situation. The use of a self-report method (for both the questionnaire and the schizophrenia audit) will inevitably bring in elements of bias.

NICE guidance implementation and key implementation indicators
The NICE principles of implementation (Box 1) are not being closely followed by many trusts. It is encouraging, however, that a majority of trusts seem to have some identified prerequisites for implementation in place, showing engagement with guideline implementation, although it is arguable that these might be seen as prerequisites for an effective clinical service, and that their existence owes more to this than any link to guideline implementation.

Markers from the schizophrenia audit
Some of the markers show high levels of adherence (use of depot medication, documentation of response and side-effects in care plan, avoidance of polypharmacy), whereas others are poorly adhered to (notably provision of CBT and family interventions). The high levels of adherence on the top five markers may be due to coincidence with established best practice rather than to change driven by guidance. The low levels of CBT and family interventions are possibly due to lack of availability and funding rather than commitment from clinical teams to implementation.

Key findings
Hypothesis testing shows a clear link between corporate commitment and leadership and implementation of guidelines. Corporate commitment is linked to many key implementation markers -where it is lacking, implementation will not be very far advanced. This hypothesis is clearly supported by the secondary analyses of the Healthcare Commission data on NICE schizophrenia guideline implementation. There is a clear link between corporate commitment and leadership and implementation of this specific guideline, from both the correlation and the regression analyses, the latter showing the interactive effect of commissioner support and commitment and leadership (lack of commissioner support is linked to poorer corporate commitment and leadership).
It is interesting to note that a higher level of audit of health technology appraisals is linked to poor commissioning support and funding problems. This may be a function of two possibly unrelated effects: it may be that commissioners are insisting that an audit of an appraisal is undertaken as a prerequisite of funding being allocated; it is also possible that this audit is being conducted as a lobbying activity to persuade commissioners to allocate funding (via a demonstration of need).

Implications
This study shows empirically that in order for implementation to succeed, the trust's board and senior managers must be engaged. Further, it shows that commissioners must demonstrate support. These findings give ammunition to trusts and to NICE to enable pressure to be put on commissioners in order to improve support of NICE guidance and of its implementation. Further, these data show that some trusts have advanced well in implementing guidelines, providing identifiable best-practice models for others to follow.

AIMS AND METHOD
To compare local practice with national guidelines, examine the areas of divergence, and establish complication rates for methadone and diamorphine. Fifty-one patients from Cornwall treated with injectable methadone or diamorphine were interviewed using a standard questionnaire.

RESULTS
Fewer problems were reported by individuals using injectable diamorphine, though for both drugs intramuscular injection was more problematic than intravenous injection. Injections into the groin were common, as was problem drinking.

CLINICAL IMPLICATIONS
Intramuscular administration of medications may be more likely to cause abcesses or cellulitis. Ongoing groin injecting and alcohol misuse is common, but should probably be tolerated if other harm reduction benefits accrue. It may be prohibitively expensive to set up injecting rooms in rural parts of the UK and future policy should reflect this.
In the UK, injectable opiates have been used longer than in any other country as a treatment for heroin misuse (Strang & Gossop, 1994). Also, while others have started using injectable diamorphine with promising results (Perneger et al, 1998;Van Den Brink et al, 2002), the UK remains almost unique in continuing to provide injectable methadone. In 1995 this was the chief injectable used, comprising 90% of injectable prescriptions in England and Wales (Strang et al, 1996).
In the studies above carried out in mainland Europe, injectable medications have been administered under high levels of supervision in dedicated injection rooms. The Randomised Injecting Opioid Treatment Trial (RIOTT) study under way in the UK (Lintzeris et al, 2006) has recreated these conditions to a large extent and it will add invaluable information to the burgeoning scientific literature on injectable medications. There is, however, still a need to do more to examine the current practice in the UK as it has evolved over 30 years within the context of less close monitoring arrangements.
Reflecting on these arrangements, in 2003 the National Treatment Agency published a report with guidelines on the use of injectable medications (National Treatment Agency, 2003). It stated that 'injectable maintenance treatment is most appropriate for addicts who have not responded to oral maintenance treatment'. It also listed eligibility criteria and a number of cautions and precautions. However, it contained little information on a number of issues which are of practical concern to clinicians in the field, for example, intramuscular or subcutaneous administration and appropriate responses to individuals who persist in using their medications in a potentially harmful manner.
Thus, although it provides a template for good practice, some aspects of the report appear more aspirational than achievable. A survey of 104 people receiving injectable opiates in the north-east of England was carried out in 2000 (Sell & Zador, 2004). Among this sample, 93% used their medication intravenously and more than half injected it into the femoral vein (groin); injectable diamorphine was preferable to injectable methadone. Though the reasons for this preference were not explored in detail, there was a clearly stated desire among those on methadone to be switched to diamorphine. The survey did not study the previous experience of individuals who had had both medications, but it highlighted the fact that very few of them appeared motivated to stop injecting or to detoxify, and for many the immediate goal in treatment was increasing the dose.
We felt it was important to audit the safety and effectiveness of local practices to measure them against the national standards as outlined in the National Treatment Agency report (2003), to understand how clients use injectable medications when unsupervised (including whether they prefer to apply them intravenously or intramuscularly) and to establish a baseline measure of complication rates with both medications. We original papers Mears et al NICE guideline implementation the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients' notes, and interviews. BMJ, 329, 999.