Need to translate mental health research to practice

Guidelines are syntheses of best available evidence that support decision-making by clinicians, managers and policy-makers about the organization and delivery of health care, but population-based studies show that guidelines produced by prominent agencies for chronic and acute conditions continue to be underused (Browman et al. Reference Browman, Levine, Mohide, Hayward, Pritchard, Gafni and Laupacis1995; McGlynn et al. Reference McGlynn, Asch, Adams, Keesey, Hicks, DeCristofaro and Kerr2003; Brown et al. Reference Brown, Johnson, Majumdar, Tsuyuki and McAlister2004; FitzGerald et al. Reference FitzGerald, Boulet, McIvor, Zimmerman and Chapman2006; Grimshaw et al. Reference Grimshaw, Eccles, Thomas, MacLennan, Ramsay, Fraser and Vale2006; Latosinsky et al. Reference Latosinsky, Fradette, Lix, Hildebrand and Turner2007; Francke et al. Reference Francke, Smit, de Veer and Mistiaen2008). Modelling by the World Health Organization found that for cancer, a third of the cases could be prevented, another third cured and the rest effectively managed if care consistently complied with the existing guidelines (World Health Organization: Cancer, 2012). The same may be true of other conditions, including mental health, for which studies show that many patients with mental health disorders do not receive guideline-recommended care for those or other medical conditions (Becker et al. Reference Becker, Martinez-Tyson, DiGennaro and Ochshorn2011; Kilbourne et al. Reference Kilbourne, Pirraglia, Zongshan, Bauer, Charns, Greenwald, Welsh, McCarthy and Yano2011). Thus it is imperative that we improve or seek new ways of promoting guideline use.

Many countries have launched policies and programmes to monitor the quality of mental health care (Spaeth-Rublee et al. Reference Spaeth-Rublee, Pincus and Huynh2010). This alone may not be sufficient to bring about improvements in the organization, delivery and outcome of mental health services. Instead, improvements may be needed along a spectrum of activities, including guideline development and implementation. Others have demonstrated the complexity of developing guidelines for mental health, and have adapted the existing methods for doing so (Barbui et al. Reference Barbui, Dua, van Ommeren, Yasamy, Fleishcmann, Clark, Thornicroft, Hill and Saxena2010). A recent systematic review of 187 eligible studies that focused on implementation of mental health research into practice found that most publications were theoretical or policy discussions (Goldner et al. Reference Goldner, Jeffries, Bilsker, Jenkins, Menear and Petermann2011). Empirical research largely consisted of quantitative, qualitative or mixed method evaluations of interventions to promote changes in provider knowledge or behaviour, shared decision-making between patients and providers or public campaigns to enhance mental health literacy. Of particular note, the review identified that all the studies were published quite recently, between 2007 and 2010, highlighting growing interest in optimizing use of mental health care guidelines and associated beneficial outcomes, and the need for more research on how to effectively do so.

The study of guideline implementation and use is not new, so a review of the key concepts drawn from that body of knowledge may guide ongoing practice and research in mental health. In this editorial, those key concepts are organized according to the steps in the iterative cycle of guideline implementation, including tailoring guidelines, assessing barriers of guideline use, selecting and tailoring implementation interventions and monitoring guideline use and outcomes (Graham et al. Reference Graham, Logan, Harrison, Straus, Tetroe, Caswell and Robinson2006).

Tailoring guidelines

Evidence suggests that we could modify the content and format of guidelines to facilitate their use (Grilli & Lomas, Reference Grilli and Lomas1994; Shekelle et al. Reference Shekelle, Kravitz, Beart, Marger, Wang and Lee2000; Michie & Johnston, Reference Michie and Johnston2004; Michie & Lester, Reference Michie and Lester2005; Cochrane et al. Reference Cochrane, Olson, Murray, Dupuis, Tooman and Hayes2007; Dobbins et al. Reference Dobbins, Hanna, Ciliska, Manske, Cameron, Mercer, O'Mara, Decorby and Robeson2009). Including information in guidelines beyond the clinical recommendations may help users to themselves implement the guidelines by promoting greater understanding of how they are to overcome barriers, stimulating confidence in the ability to practice the recommended behaviour, leading to greater intent to use guidelines, and possibly actual use. The concept of implementability was first defined by Shiffman as characteristics of guidelines that may enhance their implementation by users, who issued criteria for generating guideline recommendations with actionable wording (Shiffman et al. Reference Shiffman, Dixon, Brandt, Essaihi, Hsiao, Michel and O'Connell2005). To further investigate the concept of implementability, we reviewed the medical literature to identify features desired by different users or associated with guideline use (Gagliardi et al. Reference Gagliardi, Brouwers, Palda, Lemieux-Charles and Grimshaw2009). The guideline implementability framework included 22 elements organized within eight domains: adaptability, usability, relevance, validity, applicability, communicability, resource implications, implementation and evaluation. Subsequent analysis of 20 high-quality guidelines on various clinical indications found that most did not contain implementability elements, highlighting numerous opportunities to potentially improve guideline development and use by integrating one or more of these elements (Gagliardi et al. Reference Gagliardi, Brouwers, Palda, Lemieux-Charles and Grimshaw2011). In collaboration with the Guidelines International Network (G-I-N) and several member agencies, we are identifying and developing tools that guideline developers, implementers or users can apply to address Resource Implications (equipment or technology needed; industrial standards; policies governing their use; type and number of health professionals needed to deliver services; education, training or competencies needed by staff to deliver services; anticipated changes in workflow or processes during or after adoption), Implementation (identifying barriers associated with adoption; selecting and tailoring implementation strategies that address barriers) and Evaluation (tools based on performance measures to assess baseline and post intervention compliance with guidelines). These tools will be pilot tested and then ultimately packaged with guidelines for various clinical indications to assess their impact, alone and combined with other strategies. As part of this effort we would welcome the opportunity to collaboratively examine the implementability of mental health guidelines, and develop and evaluate the impact of implementability tools that support guideline-recommended mental health care delivery.

Identifying barriers of guideline use

Many factors intrinsic and extrinsic to guidelines challenge their use, including guideline characteristics (quality of format and content), and individual (provider characteristics), institutional (capacity to collect, adapt, share and apply evidence) and system level (policies and resources) issues (Carter et al. Reference Carter, Battista, Hodge, Lewis, Basinski and Davis1995; Fleuren et al. Reference Fleuren, Wiefferink and Paulussen2004; Davis et al. Reference Davis, Mazmanian, Fordis, Van Harrison, Thorpe and Perrier2006, Reference Davis, Powell and Rushmer2007). Furthermore, many guidelines are passively distributed. A repeat survey of Canadian guideline developers in 1994, 2002 and 2008 found that a few implemented their guidelines, citing limited resources (Kryworuchko et al. Reference Kryworuchko, Stacey, Bai and Graham2009). A survey of the guideline developers in Africa, Asia, Europe, Latin America, the United States and United Kingdom found that most believed that the target users should be responsible for implementation (Lavis et al. Reference Lavis, Oxman, Moynihan and Paulsen2008). Through interviews, we found that the professionals who fund, manage and deliver health services lack knowledge about how to implement the guidelines, and may not have the mandate or resources to do so (not yet published). In the mental health arena, education, tools and resources are needed to guide and support all stages of guideline implementation, including barrier assessment. Recommendations have been issued to help mental health care professionals address the barrier of patient adherence (Velligan et al. Reference Velligan, Weiden, Sajatovic, Scott, Carpenter, Ross and Docherty2010). Additional research is needed to identify and describe other intrinsic and extrinsic barriers specific to the implementation and use of mental health care guidelines.

Selecting and tailoring interventions to promote guideline use

Considerable research has examined the effectiveness of strategies to implement guidelines, including educational (materials and meetings), social engagement (opinion leaders and educational outreach), embedding (clinical support systems and reminders) and incentive (audit and feedback and pay-for-performance) approaches. These strategies can have a small to moderate impact either alone or in combination, but not consistently (O'Brien et al. Reference O'Brien, Oxman, Davis, Haynes, Freemantle and Harvey1997, Reference O'Brien, Freemantle, Oxman, Wolf, Davis and Herrin2001; Davis et al. Reference Davis, O'Brien, Freemantle, Wolf, Mazmanian and Taylor-Vaisey1999; Wolff et al. Reference Wolff, Taylor and McCabe2004; Garg et al. Reference Garg, Adhikari, McDonald, Rosas-Arellano, Devereaux, Beyene, Sam and Haynes2005; Grimshaw et al. Reference Grimshaw, Eccles, Thomas, MacLennan, Ramsay, Fraser and Vale2006; Jamtvedt et al. Reference Jamtvedt, Young, Kristoffersen, O'Brien and Oxman2006; Doumit et al. Reference Doumit, Gattellari, Grimshaw and O'Brien2007; Mehrotra et al. Reference Mehrotra, Damberg, Sorbero and Teleki2009). Despite the variable impact, there is little evidence to guide the selection of implementation interventions for any given clinical indication or context. Practically speaking, a number of issues should be considered. The interventions must address specific barriers identified through previous research or a needs assessment. In addition to the barriers, the needs assessment should identify the current state of practice with respect to the guidelines, including awareness, acceptance, adoption and adherence, which comprise the Phases of Change (Pathman et al. Reference Pathman, Konrad, Freed, Freeman and Koch1996). A different implementation strategy may be needed if stakeholders are aware and accepting of a guideline, but are impeded in adopting it by organizational or system issues. Although single interventions may have comparable impact compared to multipronged interventions (Grimshaw et al. Reference Grimshaw, Thomas, MacLennan, Fraser, Ramsay, Vale, Whitty, Eccles, Matowe, Shirran, Wensing, Dijkstra and Donaldson2004), if multiple interventions are applied simultaneously, it may not be clear which component was influential or which component to modify if there is little change. The implementation process is ideally cyclical so it may be best to first implement and evaluate the impact of one intervention before modifying it or packaging it with additional interventions. Research shows that in-person contact is an important predictor of knowledge use, so involvement of a facilitative intermediary is likely to prove beneficial (Innvaer et al. Reference Innvaer, Vist, Trommald and Oxman2002; Ouimet et al. Reference Ouimet, Landry, Amara and Belkhodja2006). Feasibility, cost, probable impact and unintended consequences associated with interventions or their impact must also be considered. Given these numerous issues assistance is clearly needed to more systematically guide the selection of implementation strategies appropriate to a given context, including mental health care.

There is some evidence to support the need for tailoring of interventions. A recent Cochrane systematic review of 26 randomized controlled trials found that interventions that had been tailored to address identified barriers were more likely to improve professional practice compared with either no intervention or dissemination of guidelines (Baker et al. Reference Baker, Camosso-Stefinovic, Gillies, Shaw, Cheater, Flottorp and Robertson2010). However, most studies provided little information about how interventions were tailored so the review concluded that there is insufficient evidence on the most effective approaches for tailoring. Therefore, we examined select literature to identify the factors that appear to positively influence the impact of interventions, and which warrant further research to establish their effectiveness as tailoring strategies (Gagliardi, Reference Gagliardi2011). For example, the impact of educational strategies could be improved by focusing on topics involving less complex behaviour, offering a series of events, including interactive components and opportunities for reflection, and requesting commitments for behavioural change. The practice and impact of self-assessment could be improved by better supporting health professionals with training, tools and guidance. The impact of public reporting of performance data could be improved by optimizing the content, format and delivery of this information, and by periodic and ongoing rather than one-time reporting, as could audit and feedback data. Opinion leader interventions could be improved if multiple methods were used to identify and train the opinion leaders, the opinion leaders served in a variety of roles (educate or persuade, contextualize and assist with implementation) the participants were given multiple opportunities to interact with the opinion leaders, and non-physicians were considered for the role of opinion leaders. Broader systematic reviews of tailoring strategies that consider research from different disciplines and featuring a variety of approaches are recommended. Non-traditional systematic reviews that explain ‘what works for whom, in what circumstances, and in what respect’ are becoming more common, and include realist, narrative and ethnographic approaches (Britten et al. Reference Britten, Campbell, Pope, Donovan, Morgan and Pill2002; Greenhalgh et al. Reference Greenhalgh, Roberts, Macfarlane, Bate, Kyriakidou and Peacock2005; Pawson et al. Reference Pawson, Greenhalgh, Harvey and Walshe2005). Primary research is also needed to develop, implement and test the effectiveness of tailored interventions. Such investigations are needed in the area of mental health care, and would contribute to the larger body of knowledge on how to optimize interventions for implementing guidelines.

Evaluating guideline implementation, use and outcomes

As mentioned, guideline implementation is meant to be iterative. Following implementation, guideline use and outcomes should be monitored, and the findings used to inform ongoing quality improvement efforts. Although mental health care quality indicators have been rigorously developed and form the basis of national and subnational monitoring initiatives, the same issues that challenge performance measurement in other health care sectors influence the effectiveness of these programmes, including limited evidence regarding appropriate care upon which to base indicators, lack of coordination and common indicators across programmes and lack of information infrastructure to capture descriptive clinical data (Kilbourne et al. Reference Kilbourne, Keyser and Pincus2010; Waraich et al. Reference Waraich, Saklikar, Arube, Jones, Haslam and Hamill2010; Pincus et al. Reference Pincus, Spaeth-Rublee and Watkins2011). Time and resources will lead to improvements in data collection. In the meantime, there are several factors that must be considered when planning how to evaluate the impact of guideline implementation.

First, it is important to evaluate the completeness and success of the chosen implementation strategy. Adoption of an innovation is not a discrete event that can be assessed as a binary (yes/no) measure (Dopson, Reference Dopson2007). There are different degrees of adoption (none, partial, complete and customized) and this can change over time, therefore longitudinal evaluation is needed. Implementation fidelity is defined as the degree to which an intervention is delivered and/or used as intended and, unless measured, it cannot be determined whether an outcome is associated with the intervention, and therefore whether and how to refine the intervention (Carroll et al. Reference Carroll, Patterson, Wood, Booth, Rick and Balain2007). Decisions must also be made about what to measure, and how. The measures may include immediate, intermediate and longer-term outcomes. For example, one may wish to assess reaction to the implementation strategy (satisfaction, participation, perceived effort, applicability, attitude and confidence), learning (knowledge, skill and intent to apply learning), behaviour (application of learning in practice, self-evaluation, nature of changes and if sustained) and outcomes (volume, percentages, timescales or other quantifiable performance measures relevant to clinical care or qualitative views about the nature of outcomes) (Kirkpatrick & Kirkpatrick, Reference Kirkpatrick and Kirkpatrick2006). How to undertake measurement includes consideration of the level of impact, and the models. The level may be influenced by the specific measure, availability of data and what is appropriate to report, and includes individual (provider and patient), organizational, system and population level impact. The models refer to theories or frameworks that suggest or explain factors that influence the potential impact of an intervention or their interrelationship, which informs data collection and analysis. Numerous models are available by which to plan and evaluate the impact of guideline implementation (Estabrooks et al. Reference Estabrooks, Thompson, Lovely and Hofmeyer2006; Sarjeant et al. Reference Sarjeant, Borduas, Sales, Klein, Lynn and Stenerson2011). The methods of evaluation are dependent on the measures chosen and sources of data by which to evaluate those measures, but may include quantitative, qualitative or mixed methods.