QI and audit in times of financial constraint

Clinical audit has long been seen as an essential cornerstone of clinical governance. Audit was introduced in to the National Health Service (NHS) by the government's white paper Working for Patients (Department of Health 1989). Audit is the process of setting explicit standards, then measuring practice against these standards and implementing the necessary change to improve patient care, thereby addressing any deviation from the standards (Reference PalmerPalmer 2002). It has been criticised as being overly focused on the process of data collection, with inadequate emphasis on bringing about the actual improvements (Reference Hillman and RouechéHillman 2011). When used to assess practice against minimum standards, audit becomes an exercise in quality assurance, focused on improving outcomes to meet a ‘good enough’ target, in contrast to continuous quality improvement, which seeks to achieve the ‘best possible’ outcomes through improved healthcare processes (Reference GoldstoneGoldstone 1998).

Clinical audit is not completely distinct from quality improvement; they share the repeated collection of data to inform change efforts, with a focus on improving patient outcomes and experience in a climate of financial restraint. Emphasising that healthcare is delivered within complex systems, Reference Langley, Moen and NolanLangley et al (2009) define systems as ‘interdependent group[s] of items, people, or processes working together toward a common purpose’ (p. 77). Audit generally involves staff in a discrete part of a system regularly accessing data within their local context to evaluate ‘what is happening in clinical practice against explicit criteria’ (Reference Hill and SmallHill 2006).

QI recognises that improved performance of individual parts cannot, on its own, maximise the effectiveness of the system as a whole (Reference Langley, Moen and NolanLangley 2009). It moves beyond clinical audit to multidisciplinary scrutiny of a system, emphasises integration, uses open-ended questions and seeks qualitative as well as quantitative data. QI seeks to bring about better outcomes while either reducing expenditure or, if this is not possible, ensuring that the best possible value is achieved for the investment made (Reference PorterPorter 2010). The process of improvement itself can bring about substantial savings by eliminating unnecessary activity, reducing costly harms and increasing efficiency. An intervention in a psychiatric inpatient unit designed to reduce violence has the potential to save money in a number of ways, as observed by Reference Brown, Fawzi and McCarthyBrown et al (2015), who conducted a financial analysis of their work to reduce physical violence on three older adults’ wards and observed a reduction in direct costs of £58 612, balanced against the £2000 required to fund environmental improvements.

The Model for Improvement

Developing knowledge of QI principles and methodology requires dedicated study and the opportunity to apply learning in an experiential environment. There are several different approaches, with their own perspectives for understanding and influencing positive change. By way of an example, the Model for Improvement (Fig. 1) provides an accessible framework to accelerate improvement efforts, posing three fundamental questions that are combined with Deming's structure for iteratively testing changes, the plan–do–study–act (PDSA) cycle (Reference DemingDeming 2010).

FIG 1

The Model for Improvement, developed by Associates in Process Improvement. Redrawn with permission from Reference Langley, Moen and NolanLangley et al (2009).

This model encourages those who would like to bring about change to engage with other stakeholders across the system and systematically plan changes they believe will be beneficial. Couched in the plural, the approach emphasises that sustainable change in a complex system necessitates joint working by people with multiple perspectives, across disciplines, professions and even organisations.

Having identified an area for improvement and outlined the scope and time frame of the work, the team need to determine what they must measure in order to find out whether a change being tested does indeed lead to improvement. The group agree in advance what evidence will be required to justify either further scrutiny or continued investment of effort with regard to a particular intervention. They then generate ideas about what could be tested and decide by consensus which change ideas will be prioritised for testing.

During testing, the PDSA cycle is used to refine and improve the change being attempted until the group of stakeholders is confident about the effectiveness of the change. Having proposed what will be done (plan), a change is carried out (do) and then reflected upon using relevant qualitative and quantitative information as appropriate, which is compared against the team's predictions (study). The learning from this informs the decision made by the team about how to proceed (act), and the next test of change is planned. This cyclical process helps the team to continuously build knowledge using an evidence base that is accumulated through their own practice. It is used to test changes on a small scale at first – even as small as a single patient interaction, for example – thereby reducing the potential risk involved in doing something differently. Reference Langley, Moen and NolanLangley et al (2009) assert that this sequential ‘trial-and-learning’ approach, although it appears counterintuitive, is a more effective way of making improvements in large or complex systems than ‘spending time in a conference room trying to perfect a change and then trying to “install” it’ (Reference Langley, Moen and NolanLangley 2009: p. 102).

The collected data can be analysed using statistical methods designed to understand variation over time. These are routinely used in manufacturing industries but have now been adapted and transposed to healthcare settings, allowing clinicians to consider information in new ways. Unlike traditional methods that aggregate data into quarterly or annual summaries to understand past performance against a ‘quality’ standard, statistical process charts track variation in a way that corresponds with clinically meaningful segments of time. This is determined by the needs of the system, and might be by the hour, week or month, or it might track time between rare events. Maintaining real-time relationships between data and events that accompany them preserves the richness of information being generated, permitting associations between changes and outcomes to be postulated and allowing future system performance to be predicted. This permits improved planning of resources, helps teams to guide patients’ expectations of the service – for example, waiting times – and supports managers to identify which improvements to quality outcomes will require system redesign.

Adaptive v. technical challenges

Identifying aspects of work where applying QI methodology can add value is an important consideration for anybody planning an initiative to improve quality. Reference Heifetz and LinskyHeifetz & Linsky (2002) draw a distinction between two different types of challenge: technical and adaptive.

Technical challenges are those where the problem and its likely solution are understood, requiring internal expertise to reach resolution. An example might be the challenge to improve compliance with mandatory staff training using line management processes. Adaptive problems are different; they are not linear and indeed it may not even be possible to resolve them, for example avoiding pressure ulcers in vulnerable patients. Typically, attempts to tackle adaptive problems will increase understanding of the problem and require the system to change in a sustainable manner if change is to be accomplished. Development of new attitudes, values and behaviours is essential to this process, and success may be understood in terms of the system moving in the right direction.

Traditional management strategies have developed to deal with technical problems, and they can exacerbate problems if implemented with adaptive challenges (Reference HeifetzHeifetz 1994). QI offers an alternative way of dealing with the complex problems we so often face in healthcare systems, and the potential to make changes that lead to ‘improvement beyond what has been seen before’ (Reference Langley, Moen and NolanLangley 2009: p. 16). Any theory of change or logic model to tackle a complex adaptive problem will inevitably involve some technical aspects. QI often includes testing and learning about areas where existing knowledge and experience are weak, and facilitates shorter testing cycles with accelerated implementation in technical areas.

Factors that influence success in QI

The context in which QI is used within healthcare to attempt improvements has been recognised as an important determinant of success, and work is ongoing to establish which contextual factors are sufficiently modifiable to warrant investment at different stages of QI (Reference Fulop and RobertFulop 2015). Reference Kaplan, Provost and FroehleKaplan et al (2011) have explored contextual factors as they relate to QI in healthcare using a systematic review of the literature and structured input from a diverse group of QI experts. Having defined success for a QI project as ‘the implementation of system and process changes and associated outcome improvements’, they identified 25 factors they believed were likely to influence success and created a conceptual model applicable to QI in healthcare. The resulting Model for Understanding Success in Quality (MUSIQ) organises these into several themes: the environment external to the organisation in which QI work is being done; the organisation itself; local QI support and capacity; the team working on the project; and the microsystem in which the team functions (Fig. 2). A related MUSIQ tool allows projects to be scored in each domain, highlighting areas where contextual factors may be supporting or failing to support improvement efforts.

FIG 2

Contextual factors and the level of the healthcare system in which they operate, according to the Model for Understanding Success in Quality (MUSIQ) developed by Reference Kaplan, Provost and FroehleKaplan et al (2011). QI, quality improvement.

Case study: East London NHS Foundation Trust

East London NHS Foundation Trust (ELFT) is one of the UK's largest specialist healthcare providers, delivering mental health services to a population of 750 000 people in east London and, since April 2015, a further 630 000 people in Bedfordshire and Luton. Other provisions in London include community health services in Newham, psychological therapy services in Richmond, and children and young people's speech and language therapy services in Barnet. Services operate from over 100 community and inpatient sites, with over 735 general and specialist in-patient beds, 5000 permanent employees and an annual income exceeding £350 million.

ELFT's organisational mission is to provide the highest quality mental health and community care in England. A large-scale QI programme was launched in February 2014 to help the organisation achieve this. The QI programme is designed to support teams to continuously improve care and to change organisational culture so as to shift power and decision-making to the front line. It encourages teams and patients to drive change, collaborating to creatively tackle some of the trust's most complex quality issues. Two initial ‘stretch’ goals, clearly communicating that maintaining the status quo is not an option, were set: to reduce harm by 30% every year; and to ensure that every patient receives the right care, in the right place at the right time (Fig. 3). Two years after the QI programme was introduced at ELFT, improvement efforts have grown in scale and breadth to over 150 active projects across all areas of the organisation, including corporate services, all clinical services and the Council of Governors (ELFT 2015).

FIG 3

Number of active quality improvement projects and teams testing changes in East London NHS Foundation Trust. Two ‘stretch’ goals are divided into four key priority areas.

Reducing violence

The most frequently reported safety incident in the trust is physical violence, which has therefore been a priority area for improvement and provides an illustration of a learning system being developed across multiple units.

QI methodology was used on the acute adult mental health in-patient ward with the highest frequency of violence to test a number of change ideas, resulting in an increased average time between incidents of physical violence from 5 days to 11 days. The learning from this ward was scaled up to all six in-patient wards in the Tower Hamlets directorate using a collaborative learning approach. The teams meet every 6 weeks to discuss their tests of change and review data. Since the beginning of the project, Tower Hamlets has seen a 40% decrease in incidents of physical violence in its six in-patient mental health wards (Fig. 4), with the four acute adult wards seeing a 57% reduction (Fig. 5).

FIG 4

Rates of incidents of physical violence across all six in-patient mental health wards in Tower Hamlets. LCL, lower control limit; OBD, occupied bed-day; PDSA, plan–do–study–act; UCL, upper control limit. Adapted with permission from Reference Taylor-Watt, Cruickshank and InnesTaylor-Watt et al (2017).

FIG 5

Rates of incidents of physical violence across in-patient mental health wards in Tower Hamlets: the top chart relates to acute wards and the bottom chart relates to intensive care wards. LCL, lower control limit; OBD, occupied bed-day; PDSA, plan–do–study–act; UCL, upper control limit. Adapted with permission from Reference Taylor-Watt, Cruickshank and InnesTaylor-Watt et al (2017).

Across ELFT, a range of ideas have been tested in different settings, including older adult wards, adult mental health wards, intensive care wards and secure forensic wards. The ideas being tested come from both the limited evidence base in this field and the knowledge and experience of our staff and patients. The changes being tested and implemented are diverse and include the use of safety huddles, providing more meaningful activities throughout the week, and sensory stimulation.

The future medical workforce

Improvement requires a progressive accrual of knowledge, skills, values and behaviours; any QI curriculum needs to reflect this. QI learning in the healthcare workforce needs start as early as possible in practitioners’ careers if it is to be embedded in practice. Psychiatrists in training should have the opportunity to develop a basic understanding and engage in practical experience of QI during medical school and foundation training. Examples of effective partnerships between education and health organisations are emerging, for example, collaboration between City University and ELFT, and Stirling University's adapted IHI practicum. These education initiatives are supported by mentors, managers and other clinical staff and provide undergraduate nursing students with structured teaching while aligning them with practice of QI.

The Academy of Medical Royal Colleges’ (AoMRC's) Joint Academy Training Forum commissioned a ‘task and finish’ group to consider this issue as it relates to the medical profession. The group involved key stakeholders from all four UK countries, the aim being to create an increased capacity across the workforce in order to allow healthcare teams to have a positive impact on the delivery of safe and effective patient care.

The final report, Quality Improvement – Training for Better Outcomes, recognises that trainees have generally been expected to focus on audit, which can be a token exercise in data collection, and makes recommendations designed to align efforts to implement QI training as a core component of every doctor's development (AoMRC 2016a). The group's recommendations (Box 1) are designed to provide a robust framework for embedding improvement methodology as a core competence for all doctors.

This learning will be augmented throughout core and higher training in preparation for senior leadership of QI work as consultants. Current consultants who are trainers should familiarise themselves with the expectations for students and trainees at different stages in order to provide appropriate support. This may also involve trainers undertaking additional training in order to feel confident in providing such support.

The report also suggests that each organisation in healthcare should have a local leader responsible for quality who would lead efforts to build capability and capacity. This would need to take account of the training environment and adapt existing learning opportunities to include an element of QI training. There is also a need to incorporate improvement education into the research agenda, building on the work of the Institute of Medicine (2001). It will be essential to involve patients and carers at every stage of the process, as they have a significant and diverse contribution to make.

One aim of the Academy's work has been to share best practice and map the resources that are available to clinicians to develop QI competencies. These have been gathered together on an informative, although not exhaustive, interactive map (AoMRC 2016b). They also developed a series of illustrative case studies (AoMRC 2016c).

For all of this work to happen there needs to be organisational support at every level, including strategic direction from a slew of national bodies. Psychiatrists will have a pivotal role in ensuring that this is translated from policy into practice within mental health services, so that the next generation of psychiatrists has the necessary tools to be active improvers.

In modern healthcare, it is essential for psychiatrists to supplement specialist medical expertise, applicable to one-to-one interactions between clinicians and patients, with knowledge and skills to improve the quality of care and to work safely and effectively both within a team and across the healthcare system (Reference Jones and WoodheadJones 2015). Learning resources about QI are increasingly being made available through the workplace, with numerous additional offerings available free or at low cost (Box 2).

MCQs

Select the single best option for each question stem

1. 1 In the 1920s, methods of understanding variation to drive improvements in industry were developed by:

   1. a Jerzy Neyman

   2. b William Deming

   3. c Walter Shewhart

   4. d Ronald Fisher

   5. e Donald Berwick.

2. 2 Regarding clinical audit and quality improvement (QI):

   1. a QI and audit rely on repeated collection of system performance data

   2. b clinical audit involves making changes to deliver improvements

   3. c QI involves making changes to deliver improvements

   4. d QI involves staff in a discrete part of the system

   5. e clinical audit involves staff in a discrete part of the system.

3. 3 In a typical technical challenge:

   1. a skills to tackle the challenge exist within the system

   2. b resolution is difficult to define

   3. c further challenges are revealed when solutions are attempted

   4. d attitudes, values and behaviours need to adapt

   5. e those involved in the work develop solutions.

4. 4 The Model for Improvement does not involve:

   1. a determining what should be accomplished

   2. b deciding how improvements will be recognised

   3. c generating change ideas that can be tested

   4. d establishing effectiveness using aggregated data retrospectively

   5. e making predictions based on real-time data.

5. 5 The MUSIQ tool is likely to be least effective when used by:

   1. a a QI mentor, working with a project team

   2. b project team, identifying their own needs

   3. c clinical leader, deciding what QI support is needed for an initiative

   4. d non-clinical manager, deciding what support needed for a QI initiative

   5. e an organisation, determining their QI capability.

MCQ answers

1

c

2

e

3

a

4

d

5

e