2.1 Definition and Description of Healthcare Simulation

Simulation as an imitation of a situation or process has a long history within fields such as aviation and construction. Since the turn of the century, simulation has been adopted in healthcare as ‘a technique that creates a situation or environment to allow persons to experience a representation of a real event for the purpose of practice, learning, evaluation, testing, or to gain understanding of systems or human actions’.Reference Lioce and Lopreiator¹

In his seminal work, The future vision of simulation in health care,Reference Gaba² Gaba outlines 11 dimensions that highlight the various applications of simulation. Positing that ‘simulation is a technique, not a technology’, Gaba underscores the diversity of simulation techniques. Simulation can look like many different things, in different places, with different people. In a medical school, for example, students use simulation when they practise suturing on task trainers – plastic models with fake skin. Within a hospital context, a simulation may be conducted ‘in situ’ (within a real clinical space) with a manikin acting as the patient. Equipped with technology to emulate a heartbeat, vital signs, realistic lungs, and electronic haptic (touch) feedback, this could allow an interventional cardiology team to catheterize the heart while the intensive care team resuscitates the patient. In a resuscitation bay, an emergency department team standing around an empty stretcher could be engaging in a brief mental simulation exercise to start their shift.

In short, there is no single recipe for a simulation programme or simulation exercise. Box 1 describes a hypothetical case vignette of applying simulation to a specific healthcare improvement goal – improving performance in emergencies on a cardiac surgery ward. The vignette illustrates the complexity of the clinical performance being explored and the variety of simulation techniques that might be employed to achieve the improvement goal. In Table 1, we then explore that example through the lens of Gaba’s 11 dimensions of simulation.Reference Gaba²

2.2 How Simulation Became Integrated into Approaches to Improve Quality and Safety

The benefits of healthcare simulation for education and training in a variety of contexts are well described.Reference McGaghie, Issenberg, Cohen, Barsuk and Wayne⁴ Historically, simulation was assumed to improve patient safety and care quality through the education of individual healthcare professionals and teams.

Early use of simulation focused on practising procedural skills using part task trainers – for example, using oranges to practise intramuscular injection, plastic arms to practise intravenous cannulation, and plastic head and neck simulators to practise airway management techniques. As technology has improved, educational applications for procedural skills now extend to virtual reality and software-based simulation of complex procedural tasks, such as laparoscopic surgery.

Improving a wider range of clinical skills such as communication is also a common use of simulation. Simulated patients are trained educators acting as patients, recreating everyday and challenging conversations, such as history taking, discussing bad news, or end-of-life conversations, and offering thoughtful feedback to learners in real time.Reference Nestel and Bearman⁵

Computer-based simulation of patient care scenarios, which require learners to synthesise information and make decisions about investigations and treatments, can improve decision-making and support cognitive aspects of healthcare delivery.Reference Zary, Johnson, Boberg and Fors⁶

And, in recognition of the critical role of teamwork in healthcare, simulations can be focused on teamwork behaviours. These involve teams of healthcare practitioners caring for a patient to support learning about both common and rare presentations, while providing opportunities to practise role allocation, leadership, and communication within the team.

If appropriately embedded within an educational framework,Reference McGaghie, Draycott, Dunn, Lopez and Stefanidis⁷ these examples of simulation-based education can lead to faster and more effective learning without the attendant risks of subjecting patients to practitioners’ learning curves. Best practice for educationally focused simulation includes integrating simulation into curricula, capturing clinical variation, allowing repetitive practice, and incorporating useful feedback or time for reflection.Reference Issenberg, McGaghie, Petrusa, Lee Gordon and Scalese^8,Reference Motola, Devine, Chung, Sullivan and Issenberg⁹ An exponential growth in educational simulation research since 1980 has also led to an increased emphasis on sound educational principles – for example, maintaining psychological safety for participants and increasing emphasis on debriefing and reflective practice.

Box 2 describes a hypothetical case vignette illustrating the need for simulation activities to be supported by educational frameworks (including assessment) and cultural change to be successful.

Box 2Attempt to improve operating theatre efficiency through simulation

A hospital wants to improve its operating theatre efficiency. One factor affecting current performance is the time taken for trainees to perform operations, including laparoscopic appendicectomies. Relatively junior trainee surgeons are allowed to perform these procedures, taking on the role of the primary operator, but they may be slow since they are going through a learning curve.

A training programme is developed to improve the skills of trainees through laparoscopic simulation, using both simple task trainers (which trainees can even take home) and highly complex, virtual reality simulators (Figure 1). Attendance at training is variable, due to competing clinical demands on trainee surgeons’ time. Trainees who do attend demonstrate rapid improvement in skills. The supervising consultants are aware of the training, but they still consider the laparoscopic appendicectomies on real patients as excellent opportunities for the trainees to practise, and they allow the slow operations to continue. There is no accepted credentialing process to become a primary operator at the institution. Nurses and technical specialist staff in the operating theatre are not involved in any of the training exercises, and administrative staff are not engaged to review the scheduling of operations. After 12 months, no change is demonstrated in operating theatre efficiency.

Figure 1 A simple task trainer (A) and a virtual reality simulator (B) for laparoscopic simulation

© Victoria Brazil

Those with oversight of the training programme reflect that the lack of robust assessment and credentialing process, and their inability to change supervision practice and operating theatre culture, has meant that the simulation training has had minimal impact on operating theatre efficiency.

Recent years have seen widespread adoption of simulation in healthcare professions’ curricula for education, continuing professional development, and team improvement.Reference Motola, Devine, Chung, Sullivan and Issenberg⁹ Here, simulation is seen as an educational adjunct, manifested in a desire for standardised educational opportunities, the need to practise skills before applying them in a clinical environment, and to supplement scarce clinical training sites. The educational focus mostly reflects a person-centred approach to safety, in which the purpose of training is to decrease the number of errors by individuals and teams.

Coinciding with the integration of simulation into education has been a growing understanding of the contribution of behaviour and other non-technical skills to team performance in complex health systems.^10–Reference King, Battles, Baker, Henriksen, Battles, Keyes and Grady¹³ Team-based, crew-resource management training that includes simulation has been associated with improved teamwork and confidence among a variety of healthcare teams.Reference Figueroa, Sepanski, Goldberg and Shah^14,Reference Meurling, Hedman, Sandahl, Felländer-Tsai and Wallin¹⁵ Such an approach has undoubted value: training for teamwork, communication, and procedural skills is necessary for improved patient care. But there remains limited understanding of how long these improvements persist and what impact they have on clinical outcomes. Further, on their own, educational uses of simulation are unlikely to be sufficient to address the need for more systems-based approaches that are now recognised as fundamental to securing quality and safety.Reference Reason¹⁶

There is a growing understanding of the positive roles that simulation can play, beyond education and training, in improving quality and safety in organisations. Consistent with Gaba’s proposal that ‘using simulation to improve safety will require full integration of its applications into the routine structures and practices of health care’,Reference Gaba² there is evidence of increased use of simulation for the express purpose of healthcare improvement, such as identifying latent safety threats or improving processes. The term translational simulation was coined in 2017 to describe those simulation activities ‘connected directly with health service priorities and patient outcomes, through interventional, testing and diagnostic functions’.Reference Brazil¹⁷ A brief PubMed search shows that in the year 2000, there were 21 publications related to ‘simulation and “patient safety”’; in 2021, that same search yielded 741 results. Many institutions, teams, and researchers are realising, honing, and advancing Gaba’s original vision.

These shifts are occurring alongside – and to some extent inspired by– the evolution of paradigms for safety thinking. In 2013, a white paper by Hollnagel et al.Reference Hollnagel, Wears and Braithwaite¹⁸ prompted a shift towards what the authors call a ‘Safety II’ perspective. They argue that we should cease to focus exclusively on how to stop things going wrong, and emphasise instead why things go right. A Safety I approach to healthcare presumes that things go wrong because of identifiable failures of specific components of a system, but such an approach does not address the contribution of the system as a whole, including its culture and variability.^10–Reference King, Battles, Baker, Henriksen, Battles, Keyes and Grady¹³ By contrast, Safety II focuses on proactively fostering a system that allows as many things as possible to go right, with an effort to continuously anticipate issues and embrace humans as contributors to flexibility and resilience.^10–Reference King, Battles, Baker, Henriksen, Battles, Keyes and Grady¹³ Simulation has emerged over the past decade as having potential to purposefully uphold and complement a Safety II approach.Reference Wheeler, Geis, Mack, LeMaster and Patterson^19–Reference Dieckmann, Patterson and Lahlou²² This is recognised in the Society for Simulation Healthcare’s accreditation of programmes that undertake ‘systems integration’ simulation:

Many simulation researchers and practitioners embraced this change in the safety paradigm, and began to reconceptualise the role of simulation as going beyond an educational adjunct. Simulation techniques emerged that were system focusedReference Dubé, Posner and Stone²³ and more integrated into improvement approaches.

3.1 Exploring Working Environments and the Practices and Behaviours of Those in Them

Simulation offers a broad range of opportunities and methods for examining current healthcare practice, including the various factors that shape performance at an individual, team, technology, working environment, or system level.Reference Lamé and Dixon-Woods^31,Reference LeBlanc, Manser and Weinger³²

Simulation is an attractive option when study in an actual clinical setting would be difficult due to practical constraints or ethical concerns – for example, interrupting nurses during medication rounds to determine error rates. Increasingly, simulated explorations are now frequently undertaken as part of an initial or ongoing improvement strategy too. Diverse techniques can be employed: for example, task trainers to study procedural skill performance (e.g. a plastic arm that allows for intravenous cannula insertion); scenario-based immersive simulations to study team performance; role play with simulated patients to review communication; and computer modelling simulation to examine patient flow through an emergency department.

Simulations conducted within the actual care setting (in situ simulationReference Patterson, Geis, Falcone, LeMaster and Wears³³) may be particularly valuable in evaluating system performance and identifying latent conditions that pose threats to patient safety. This more naturalistic approach enables participants to practise within the same physical environment, healthcare team, and care processes that are used in real clinical practice. It recognises that clinical practice happens under conditions of ‘considerable complexity, change and surprise’,Reference Macrae and Draycott²¹ which are difficult to capture in dedicated simulation laboratories.

In situ simulation programmes claim to ‘ … accomplish … the dual goals of identifying and remedying [latent safety threats] as well as providing continuous opportunities to deliberately practice technical and non-technical skills’.Reference Patterson, Geis, Falcone, LeMaster and Wears³³ Parallel exploration may also occur within team relationships, roles, and cultureReference Brazil, Purdy, Alexander and Matulich^20,Reference Macrae and Draycott^21,Reference Purdy, McLean and Alexander³⁴ – which are equally likely sources of latent safety threats to health service performance or safety. A typical approach might involve a hospital department simulating scenarios that are representative of their patient profile and require a team-based approach to care, located within an actual patient care space. Exercises are generally accompanied by a debriefing session in which the professionals involved reflect on their performance and the opportunities and constraints afforded by the physical space and other resources available. Programmes have been conducted in emergency departments, operating theatres, maternity units, general wards, pre-hospital environments, and primary care contexts.

Importantly, simulations that explore working environments are also an opportunity for learning from success,Reference Dieckmann, Patterson and Lahlou²² as they enable tacit expertise and examples of positive deviance can be identified and elaboratedReference Lawton, Taylor, Clay-Williams and Braithwaite³⁵ (the positive deviance approach is explored in another Element in this seriesReference Baxter, Lawton, Dixon-Woods, Brown and Marjanovic³⁶). In situ exercises become an opportunity to ‘investigate and optimize human activity based on the connected layers of any setting: the embodied competences of the healthcare professionals, the social and organizational rules that guide their actions, and the material aspects of the setting’.Reference Dieckmann, Patterson and Lahlou²² As such, we see alignment with trends in patient safety towards Safety II approaches – reinforcing the role of efficient adaptation and organisational resilience in the face of errors and obstacles arising.Reference Braithwaite, Wears and Hollnagel³⁷

Box 3 outlines a hypothetical case vignette in which in situ simulation is used to explore the working environment, latent safety threats, team function, and positive deviance in caring for paediatric patients with anaphylaxis. It illustrates some of the challenges in translating to change in practice.

When using simulation to explore working environments, the delivery methods vary greatly. Scenarios may be conducted in actual patient care areas or nearby to facilitate team attendance. Sessions may be unannounced and unexpected, simulating real response processes, or they may be planned and scheduled in advance.Reference Posner, Clark and Grant²⁹ Each design decision is likely to require some trade-offs between the feasibility of conducting simulation exercises in a clinical environment and the veracity of the system-probing function.Reference Sørensen, Østergaard and LeBlanc³⁸ At present, however, there are no design standards nor even a consensus on terminology.Reference Posner, Clark and Grant²⁹ Frequently identified practical challenges include those relating to equipment, medication, use of physical space, and call systems.

Using in situ simulation to explore working environments is a potentially attractive approach in healthcare improvement, but is immature in its methods, consistency, and integration with other improvement strategies. Auerbach et al. report, for example, that most paediatric simulation programmes they surveyed used in situ simulation, but also found inadequacies in how latent safety threats were identified, reported, and acted on.Reference Auerbach, Kessler and Patterson²⁴ A systematic review of studies reporting in situ simulation activities found that ‘approaches to design, delivery, and evaluation of the simulations were highly variable across studies’, and that performance measurement practices were suboptimal.Reference Rosen, Hunt, Pronovost, Federowicz and Weaver³⁹

Colman et al. have developed a more standardised approach to simulation-based testing of clinical systems,Reference Colman, Doughty and Arnold⁴⁰ providing documentation and evaluation tools to help in identifying inefficiencies and risks to safety. But as yet there is no consensus on the best approach. There is also conflicting evidence about whether improvements to working environments are sustained, with some arguing that it is most likely to be effective if seen as a long-term commitment requiring regular participation that is intrinsic to an ongoing patient safety strategy.Reference Fent, Blythe, Farooq and Purva²⁷

3.2 Improving Clinical Performance and Outcomes

Simulation can be applied to a broad range of healthcare targets: anything from a single patient journey at one institution to improvement of system outcomes. The clearest examples include simulation projects designed to improve time-based targets or other easily measurable indicators related to individual patient journeys, such as time to thrombolysis in stroke care,Reference Ajmi, Advani and Fjetland⁴¹ time for trauma patients to go to CT scan,Reference Knobel, Overheu, Gruessing, Juergensen and Struewer⁴² resuscitation outcomes,Reference Andreatta, Saxton, Thompson and Annich⁴³ teamwork in trauma,Reference Steinemann, Berg and Skinner⁴⁴ or success rates in paediatric intubation.Reference Long, Cincotta and Grindlay⁴⁵ Simulations designed for such a purpose may include dedicated educational programmes to improve individual and team performance – for example, deploying part-task training for procedural skills and immersive simulations for team-based tasks, combined with appropriate didactic or other training methods. The hypothetical case vignette in Box 4 is a typical example.

In a review of the clinical outcomes of simulation-based ‘mastery learning’ (learning that helps students to master or reach a high level of achievement), Griswold-Theodorson et al. identified studies reporting improvements following training interventions, including better performance level, better procedural success rate, reduced patient discomfort, shorter procedure times, reduced error rate, and lower healthcare costs.Reference Griswold-Theodorson, Ponnuru and Dong⁴⁶ Reviews of in situ simulation practice have also demonstrated improved patient morbidity and mortality.Reference Goldshtein, Krensky, Doshi and Perelman⁴⁷ In situ simulation is likely to be especially important given that outcomes are dependent on how individuals and teams perform within the constraints and opportunities of hospital systems and complex departmental interfaces.

Reported examples of successful improvement programmes often relate to interventions conducted in a single institution, but simulation has the potential to impact healthcare beyond these examples. Simulation may be used to improve healthcare management and policy-making at a state or national level,Reference Lamé and Simmons⁴⁸ where the impact is more distributed. Nataraja et al. report a significant national improvement in the management of paediatric intussusception (an acute bowel emergency) in Myanmar following the introduction of a focused, simulation-based intervention.Reference Nataraja and Kyaw⁴⁹ Preparing a disaster plan or evaluating strategies to minimise infections during a pandemic such as COVID-19 might involve computer modelling techniques, combined with live simulations to test protocols for safe patient care,Reference Brazil, Lowe and Ryan²⁵ and training simulations to determine if personal protective equipment is adequate.Reference Begley, Lavery, Nickson and Brewster⁵⁰

Relationships and culture within and between healthcare teams are less frequent targets for simulation-based interventions, despite the recognised role they both play in health system performance. In one study of a relationship-based approach to improve trauma care at an institution, Purdy et al.Reference Purdy, McLean and Alexander³⁴ illustrate the considerable impact of regular interprofessional, multidisciplinary, in situ simulation on the relational aspects of care and the development of a collaborative culture.Reference Brazil, Purdy, Alexander and Matulich²⁰ Another study of a programme involving regular emergency department simulation showed that simulation is a place to foster familiarity and psychological safety, which can have a direct impact on clinicians’ work in real clinical settings.Reference Purdy, Borchert and El-Bitar⁵¹ For further discussion of some of the issues relating to culture in healthcare, see the Element on making culture change happen.Reference Mannion, Dixon-Woods, Brown and Marjanovic⁵²

Overall, the simulation techniques used for targeting system improvements are variable – combinations of in situ simulation, educationally focused simulation in dedicated facilities, procedural skills practice, and scenario-based team training. The design requires clarity on the hopefully meaningful target(s) and appreciation of the relative benefits of various simulation methods to achieve improvement, while being feasible and cost-effective to implement.Reference Brazil^17,Reference Petrosoniak, Brydges, Nemoy and Campbell²⁸ Simulation itself is agnostic towards healthcare improvement frameworks and is frequently one part of a more comprehensive improvement strategy,Reference Long, Cincotta and Grindlay⁴⁵ which is pragmatic and appropriate, but which makes it difficult to ascertain the specific impact of the simulation elements on the overall effectiveness of the approach.Reference Brazil, Purdy and Bajaj²⁶ Liberati et al. highlight this interrelationship by outlining how a host of quality and safety mechanisms within a maternity unit were ‘nurtured and sustained’ through the simulation-based Practical Obstetric Multi-Professional Training (PROMPT) programme.Reference Liberati, Tarrant and Willars⁵³ The PROMPT course is focused on multi-professional teams learning how to manage obstetric emergencies, working on their own labour ward, using their own emergency equipment, local procedures, and systems.Reference Abdelrahman and Murnaghan⁵⁴

3.3 Testing Planned Interventions and Infrastructural Changes

Simulation can enable evaluation of the feasibility, safety, acceptability, or effectiveness of planned interventions, new healthcare facilities, and changes to infrastructure. This provides opportunities to develop and test ergonomics and workflows, and to identity human factors flaws and latent safety threats before going live or being introduced into the real clinical environment.Reference Lamé and Dixon-Woods^31,Reference Colman, Doughty and Arnold^40,Reference Barlow, Dickie, Morse, Bonney and Simon^55–Reference Petrosoniak, Hicks and Barratt⁵⁷ Effectively designed simulations can also be used to test new processes. For example, they have been used to test cognitive aids for emergencies,Reference Marshall, Sanderson, McIntosh and Kolawole⁵⁸ guidelines for massive transfusion and other care pathways, and the introduction of new equipment or bundles, such as boxes for the management of postpartum haemorrhage.Reference Spiegelman, Sheen and Goffman⁵⁹ Strategies can encompass a range of techniques and targets, with success dependent on the authenticity of the simulation and the adequacy of data collection.Reference Barlow, Dickie, Morse, Bonney and Simon⁵⁵ Simulation can be used as one part of a mixed-methods design, where data collected through simulation can be triangulated with other sources of information and intelligence.Reference Lamé and Dixon-Woods³¹

Testing may include tabletop mock-ups, full-scale recreations of facilities, and individuals or teams working within test environments to varying degrees of realism. This requires more than a single event – it requires a strategy for testing and data collection. Petrosoniak et al.Reference Petrosoniak, Hicks and Barratt⁵⁷ propose a ‘design thinking’ approach, with multimodal simulation techniques and an emphasis on end user engagement, to iteratively test and improve planned changes to a trauma resuscitation bay in an emergency department. Although frameworks have been described in the literature,Reference Colman, Doughty and Arnold⁴⁰ there are no endorsed standards and no accepted consensus approach for using simulation to test new healthcare facilities.

However, some important examples are appearing. Prior to the opening of a newly constructed paediatric outpatient clinic, Colman et al.Reference Colman, Stone and Arnold⁶⁰ conducted 31 simulated scenarios over 3 months to identify system flaws (latent safety threats) that posed a potential risk to patients. Failure mode and effects analysis was used to prioritise threats. In all, the authors identified 334 latent safety threats, including 36 ‘very high priority’ threats. High-priority examples included emergency preparedness and the emergency notification system, the proximity of antibacterial hand sanitiser to clinic rooms, the location of the sharps disposal container, infection control regarding the movement of cystic fibrosis patients throughout the building, the accessibility of resuscitation bags, and the impact of the building’s climate on testing reagents.Reference Colman, Stone and Arnold⁶⁰ In a subsequent paper, Colman et al. offer guidelines for simulation-based testing of clinical systems to develop, implement, and evaluate newly built clinical environments using principles and tools derived from the Agency for Healthcare Research and Quality.Reference Colman, Doughty and Arnold⁴⁰

An iterative approach to testing and embedding was evident in the response to the COVID-19 pandemic, when many healthcare workflows and practices had to be rapidly adjusted to minimise infection risks.Reference Brazil, Lowe and Ryan^25,Reference Brydges, Campbell and Beavers⁶¹ Simulation strategies were used to explore the risks of COVID-19 transmission within current practices, and to assess changes designed to reduce risks at the individual, team, and system level. Some initially promising interventions, such as Perspex boxes to protect airway teams from exposure to COVID-19 during intubation, turned out not to be effective or feasible when tested in simulated practice.Reference Begley, Lavery, Nickson and Brewster⁵⁰

Box 5 describes a hypothetical case vignette in which a design thinking approach is used to rapidly design a fever clinic for COVID-19 testing, drawing on anecdotal experience of colleagues during the COVID-19 pandemic.

Simulation can be used proactively to test and refine planned changes in specific contexts, but the generalisability of the findings is not always straightforward. For example, the findings of simulations of the effectiveness of specific cognitive aids in helping to select paediatric anaesthesia equipment in one hospital may, if the human factors principles are the same, be useful across multiple contexts. But the success of modifications to a massive transfusion protocol developed through simulation may depend on the interrelationships of that care pathway with local hospital systems, teams, and capabilities, and hence require development and testing at a local level.

3.4 Helping Healthcare Professionals to Learn about and Embed a Culture of Improvement

Learning about the theory and practice of healthcare improvement is now a requirement in many undergraduate and postgraduate training programmes in medicine and other health professions.Reference Wong, Ackroyd-Stolarz and Bukowskyj⁶² Simulation techniques can support experiential methods of education on reporting and investigating patient safety incidents, process mapping, plan-do-study-act cycles, intervention design, and culture change.Reference Worsham, Swamy, Gilad and Abbott⁶³ Simulation-based activities can invite practitioners to reflect on their practice through a quality and safety lens and the actions and behaviours that might be needed to improve it. In one study involving a collaborative ethnography of a trauma service, on-the-ground care providers with no formal role in healthcare improvement reflected that a programme of regular in situ simulation allowed them to feel engaged in ‘process review and improvement’ and empowered teams to form a habit of ‘team reflection’.Reference Brazil, Purdy, Alexander and Matulich²⁰ Other examples involve inviting professionals to explore problems such as emergency room crowding or hospital-acquired infections using tabletop or computer simulations. In these exercises, professionals are asked to develop and undertake healthcare improvement efforts that have consequences as the simulation unfolds. After the simulation exercise, they take part in facilitated reflection on the impacts of their improvement efforts in the example and to draw out wider learning. Another useful technique is debriefing to marginal gains – that is, exploring what went well and what could go just 1% better at an individual, team, or systems level. This is a simple way to inspire an improvement mindset in individuals and teams and further support the cultural foundation of a Safety II approach. The hypothetical case vignette described in Box 6 illustrates how a mindset can be engendered in simulation and then translated to learn from real patient care.

When simulation is used to explore and enhance healthcare performance in this way, it has the added benefit of signalling improvement as a priority and may also contribute to changing the safety culture of the system.Reference Patterson, Geis, Falcone, LeMaster and Wears³³

4.1 Is Simulation an Effective Technique for Improvement?

The challenges of evaluating simulation as an educational technique have been extensively discussed,Reference McGaghie, Issenberg, Cohen, Barsuk and Wayne⁶⁴ and similar challenges surface when considering evaluation of simulation-based interventions for healthcare improvement. Given the variety of techniques encompassed by the term simulation and the diverse contexts in which the method may be applied, no single study is likely to provide the answer to what works and why in simulation, notwithstanding some interesting examples.Reference Brazil, Purdy and Bajaj^26,Reference Ajmi, Advani and Fjetland^41,Reference Long, Cincotta and Grindlay⁴⁵

Potential unintended negative consequences of simulation in the setting of healthcare improvement remain underreported and underexplored. Simulation can be intuitively appealing as a safe approach to improvement – practising skills and teamwork seem likely to improve performance, and practising on plastic manikins or with actors who simulate patients seems inherently safer than with real patients. But there are also well-described safety risks of conducting simulation activities.Reference Raemer, Hannenberg and Mullen^65,Reference Bajaj, Minors, Walker, Meguerdichian and Patterson⁶⁶ Ironically, for example, in situ simulation exercises can themselves pose a potential threat to safe and efficient service delivery to real patients in clinical areas: by deploying staff from clinical care into a simulation, by preventing a real patient from using a physical space, and by mixing simulated and real equipment and medications.Reference Raemer, Hannenberg and Mullen⁶⁵ Simulation programmes have adopted systems and processes to mitigate these risks, including the development of formal no-go criteria for cancelling in situ simulation exercises,Reference Bajaj, Minors, Walker, Meguerdichian and Patterson⁶⁶ and guidance on developing simulation safety policies.Reference Brazil, Scott, Matulich and Shanahan⁶⁷ If simulation sessions are poorly facilitated, there are also potential threats to the psychological safety of teams, which could have downstream effects on patient safety.

Simulation will always be an imperfect recreation of the complex healthcare environment, and there are risks of embedding bad habits (e.g. medical students not wearing gloves in simulation) or even perpetuating culturally embedded bias or prejudices (e.g. using predominantly white-skinned and male manikins).Reference Conigliaro, Peterson and Stratton⁶⁸ Growing interest in equity, diversity, and inclusion within healthcare has prompted important reflection among simulation facilitators,Reference Purdy, Brazil and Symon⁶⁹ and there is increasing interest in simulation as a technique for addressing equity, diversity, and inclusion issues.Reference Buchanan and O’Connor⁷⁰

One challenge for evaluation is that simulation is often used as one part of a more comprehensive improvement strategy.Reference Long, Cincotta and Grindlay⁴⁵ While often pragmatic and appropriate, it can complicate efforts to evaluate the specific contribution of the simulation elements to the outcomes.Reference Brazil, Purdy and Bajaj²⁶ As such, it can be difficult to determine the interrelationships between findings from exploratory simulation informing other improvement strategies, versus simulation design being informed by data collected as part of the broader improvement effort.Reference Brazil¹⁷ Even though academic reports commonly separate or seek to separate the two cleanly, the reality is that simulation may have roles in both informing and being informed by other healthcare improvement efforts that cannot easily be distinguished.Reference Liberati, Tarrant and Willars^53,Reference Abdelrahman and Murnaghan⁵⁴

4.2 How Should We Integrate Simulation into Healthcare Improvement?

The integration of simulation into the healthcare improvement strategies of organisations offers considerable potential but is often not fully realised. Aligning simulation with contemporary approaches to healthcare improvement is important, and should prevent conflicting or competing agendas, philosophies, or claims on resources. For example, testing whether planned interventions are feasible, acceptable, or effective in simulated environments and teams aligns well with the Safety II approach to focusing on ‘work as done’ rather than ‘work as imagined’.Reference Hollnagel, Wears and Braithwaite¹⁸ Barlow et al.Reference Barlow, Dickie, Morse, Bonney and Simon⁵⁵ use language and tools drawn from healthcare improvement when outlining a framework for documenting and reporting latent system threats unearthed during simulation scenarios. Drawing on human factors and plan-do-study-act constructs, the framework supports the capture and reporting of findings on system deficits to key decision-makers. Connecting simulation-based approaches to other improvement initiatives within an organisation can help in the same way. For example, simulation experts with skills in managing reflective conversations might take a lead in developing clinical event debriefing programmes for healthcare teams, which can be used to discuss opportunities for improvement after real patient care encounters.

However, barriers in integrating simulation-based strategies in overarching healthcare improvement approaches are posed by different traditions of scholarship and practice.Reference Brazil, Purdy and Bajaj²⁶ This may be manifest in disconnected terminology and in organisational structures and professional groups. With some notable exceptions,Reference Dench, Barwick and Barlow⁷¹ healthcare management journals tend not to cover simulation-based healthcare improvement. Within organisations, simulation programmes may be situated in educational structures and staffed by educational experts, while healthcare improvement teams may use tools and language unfamiliar to simulation delivery teams or clinicians. Where a simulation programme sits within a healthcare organisation, it will tend to drive both the organisation’s focus and sphere of influence.Reference Trawber, Sweetman and Proctor⁷² Simulation programmes that have weaker links to the quality and operational structures within their organisation are less likely to help inform the organisation’s strategic direction.

The consumer voice has not been well established in design or delivery of simulation activities, either for education or for improving quality and safety. Drawing on established frameworks for healthcare consumer engagement,Reference Carman, Dardess and Maurer⁷³ there clearly is a role for consumers in simulation design, delivery, and strategy development,Reference Barwick and Brazil⁷⁴ but it so far appears to be a missed opportunity.

While the field continues to develop, the questions posed in Box 7 may be helpful. Also important is recognition that skills in running simulations – including design and execution of scenarios, skills in managing debriefing or reflective practice conversations, and systems-focused debriefingReference Dubé, Reid and Kaba³ – are highly specialised and require specific training. Although simulation techniques are varied and evidence of the relative benefits of various simulation methods to achieve improvement is still emerging, we do know that it is essential for the goals to be clearly defined and the design to be feasible and cost-effective to implement.Reference Brazil^17,Reference Petrosoniak, Brydges, Nemoy and Campbell²⁸

4.3 Can We Build a Business Case for Simulation?

While there is increasing awareness of simulation as a quality and safety technique, building a business case for simulation at scale remains challenging. Simulation activity is often resource-intensive, and, for all its potential to improve quality, there are significant downsides to simulation. Activities can be expensive – for example, in relation to equipment, staff time, and use of clinical areas for simulation. There is also limited information about what dose and frequency of simulation is effective. Recruiting and training simulation facilitators to the required level of expertise in both simulation techniques and the skills needed for clinical redesign and healthcare improvement is difficult.

Evidence of return on investment may therefore be much in demand, but little published data are yet available.Reference Nestel, Brazil and Hay⁷⁵ The lack of evidence arises partly because of the emerging nature of the field and because some important impacts of simulation and debriefing are in areas such as team trust and psychological safety, which are difficult to place on a balance sheet. Lin et al.Reference Lin, Cheng, Hecker, Grant and Currie⁷⁶ offer steps to gather the necessary information to conduct an economic evaluation of simulation-based education programmes and curricula, and describe the main approaches to conducting an economic evaluation.

A useful framework is offered by Shah and Course to ‘help identify, understand, and evaluate return on investment from large-scale application of [quality improvement]’.Reference Shah and Course⁷⁷ It describes six domains:

• patient, carer, and family experience outcomes

• staff experience

• productivity and efficiency

• cost avoidance

• cost reduction

• revenue.

Although the framework has limitations, including the absence of links to patient outcomes and provider effectiveness, it encourages a focus on domains that the clinicians undertaking improvement-focused simulation work may not instinctively think about. We encourage those involved in healthcare improvement simulations to begin framing design and measurement of impact around these domains to support a business case for simulation in their organisation.

In Box 8 we apply the Shah and Course framework to two of our prior hypothetical examples.

The application of return-on-investment frameworks to simulation activity is unfamiliar territory for many of those who plan and facilitate simulation. This highlights why early collaboration between simulation facilitators, those with improvement expertise, and those with skills in economic evaluation should be the standard for simulation programmes seeking to demonstrate return on investment.