Introduction
World-wide attention has focused on coronaviruses, owing to the discovery of a highly pathogenic human coronavirus (severe acute respiratory syndrome [SARS]-CoV-2), which was discovered in December 2019 in China and later named coronavirus disease 2019 (COVID-19).Reference Opriessnig and Huang1 Since its detection, it has rapidly spread beyond China to the rest of the world.Reference Bedford, Enria and Giesecke2 On March 11, 2020, the World Health Organization (WHO; Geneva, Switzerland) defined the COVID-19 situation as a pandemic and declared an international emergency.Reference Bedford, Enria and Giesecke2,Reference Neher, Dyrdak and Druelle3 Though human coronavirus infections are typically mild, COVID-19 can cause fatal illness.Reference Fregene, Nadarajah and Buckley4,Reference Huang, Wan and Li5
The rise of COVID-19 has shown that protocols and policies for initial management need to adapt, using science-based evidence.Reference Schuchat, Bell and Redd6 Job-related stress, segregation, suspicion and fear of other people, and concerns about infection, family health, and sickly peers can create additional challenges in a pandemic.Reference Aiello, Khayeri and Raja7 A heavy sense of responsibility, lack of proper personal protection equipment (PPE), shortage of certain medications, extensive media reporting, and feelings of being inadequately supported can add significantly to the psychological burden borne by health care workers.Reference Lai, Ma and Wang8,Reference Gaeta and Brennessel9 Studies have found that one-third to one-half of health care workers showed significant stress during the SARS outbreak, with those in quarantine or treating colleagues showing higher levels of distress.Reference Maunder, Leszcz and Savage10,Reference Daugherty, Carlson and Perl11 Medical and psychiatric management can help prevent the fear, uncertainty, and stigmatization that are commonly caused by epidemics.Reference Park and Park12
The COVID-19 pandemic has shown that continued commitment to global preparedness is essential. Previous epidemics have led to an initial increased awareness of global health, but once the initial anxiety has passed, preparedness is given a lower priority—a panic-then-forget cycle, that has yet to be broken. To break this cycle, it is essential to prioritize preparedness interventions and the requisite funding.Reference Jacobsen13
A gap of preparedness was seen when there was a delay in identifying the first cases in the H1N1 pandemic, despite the fact that studies had shown that accessibility of standard operating procedures and effective training programs are effective, even though they might be complicated, time consuming, and costly.Reference Adini, Goldberg, Cohen and Bar-Dayan14 The benefits of previous investments in preparedness, mainly labs and data analytics capabilities, were shown when the US Centers for Disease Control and Prevention (CDC; Atlanta, Georgia USA) quickly developed a test for COVID-19.Reference Smith and Fraser15 The earlier SARS outbreak had also highlighted the importance of training staff to work outside their area of familiarity and to have access to psychological first aid and coping assistance.Reference Maunder, Leszcz and Savage10 Planning for a likely pandemic should ideally include preparing clinicians to identify and treat irregular cases of the infectious disease, as well as national preparedness for its spread.Reference Leiba, Dreiman and Weiss16
Simulation can help assess the capability of health care providers (HCPs) to follow established protocols. It has revealed errors in preparedness plans in the case of an influenza pandemic.Reference Cardenosa, Domínguez and Carratalà17 In situ simulation involving scenarios that integrate health care workers from multiple disciplines into a clinical environment has been shown to improve preparedness for infrequent, high-risk situations.Reference Fent, Blythe, Farooq and Purva18 Simulated cases can detect gaps in preparedness plans drawn up by medical staff, and may be of great value as awareness of errors and their correction can lead to refined plans and improvements in response to any case that could presage a pandemic.Reference Leiba, Dreiman and Weiss16 Doctrine, infrastructure, equipment, knowledge and education of health care workers, simulation, and drills can all comprise elements of a “preparedness pyramid” to be used in mass-casualty scenarios.Reference Leiba, Dreiman and Weiss16 Simulation-based training has been used by many medical specialties to establish competencies in the skills essential to medical practice. It can mimic various clinical scenarios that are designed to address specific objectives, and it can integrate practical knowledge allowing trainees to assess their performance in a virtual setting wherein they can identify the areas that need improvement and build self-confidence when dealing with similar, real-life cases.Reference Leiba, Dreiman and Weiss16 One of the invaluable advantages of simulation is its ability to replicate rare and highly infectious disease cases in a safe, yet challenging environment without jeopardizing anyone’s safety.Reference Lateef19-Reference Munshi, Lababidi and Alyousef21 Furthermore, a high-fidelity simulator can assess the progress of learners, from novice to expert level, in accordance with Miller’s pyramid for clinical competency, which is an essential pillar of training and assessment.Reference Lopreiato, Downing and Gammon22
The WHO has issued guidance for the prevention and protection strategies that health care workers treating COVID-19 patients should adopt in order to protect themselves, and to prevent further transmission of the virus to other vulnerable people.Reference Peng, Ho and Hota23 Particularly for those performing aerosol-generating procedures, the WHO issued guidance on the required level of PPE, which includes respirators, eye protection, long-sleeved gowns, and water-resistant aprons. In addition, WHO guidelines state that the management of patients with suspected or confirmed COVID-19 should involve clear communication while minimizing the number of people in the same room.Reference Fregene, Nadarajah and Buckley4 Many methods such as lectures, seminars, workshops, exercises, and drills have been used by health care professionals to enable them to better prepare for and manage emergencies. Drills and exercises have shown to be the most effective ways of preparing for emergencies.Reference Adini, Goldberg, Cohen and Bar-Dayan14
The hospital in this study is a relatively new academic hospital, and hence had no prior experience of pandemics. Additionally, some of its staff had no previous experience in dealing with influenza epidemics or pandemics. For that reason, this study was conducted with the aim of identifying defects that pose potential problems, testing the hospital’s systems, and modifying its standard processes for patients with COVID-19 presenting at the emergency department (ED). Another objective was to improve the ED staff confidence in managing such patients, and to increase their skills in basic and advanced airway management, as well as knowledge of proper PPE techniques.
Materials and Methods
The study was deemed exempt from the institutional review board at Princess Nourah bint Abdulrahman University (Riyadh, Saudi Arabia; log number 20-0145). The target population was ED physicians, nurses, emergency medicine technicians (EMTs)/paramedics, and respiratory therapists (RTs).
This is a quasi-experimental study that was conducted from February through May 2020 in a hospital’s ED to assess the overall performance of the entire ED team. The hospital is a 300-bed tertiary academic care center. Mangers of the ED recognized the urgent need to be prepared for COVID-19. A total of 20 unannounced mock codes were carried out in ED resuscitation and isolation rooms. Mock codes were conducted over days and nights to ensure they were experienced by all staff and in different situations. A simulated scenario that began at the main entrance with a suspected case of COVID-19 was managed until the time of intubation. The scenario began with a person playing the role of a patient who walks through the main entrance, who is replaced by a manikin (SimMan 3G; Laerdal Medical; Stavanger, Norway) once reaching the treatment room.
Mock codes were conducted by five expert emergency physicians who had prior experience of simulation training. A 20-minute mock code was followed by around 40 minutes of a standardized debriefing session that discussed the reaction phase, the encounter, and concluded with a summary of those procedures that were done well and those that needed to improve (the Plus Delta method). The debriefing considered the scenario objectives: to review the triage and respiratory zone processes peculiar to COVID-19 suspected cases, donning/doffing, and airway management. The ED board was used to document what went well and things to improve. Results were immediately reviewed and recurring themes, systemic errors, and potential problems identified.
A checklist was designed to measure performance skills of random, on-duty teams. The skills fell into three categories, concerning: (1) donning, (2) basic and advanced airway skills, and (3) doffing. The checklist was reviewed by experts in emergency medicine and critical care for content validity, and was piloted with ED teams before starting the experiment. A trained observer, who is a physician certified in emergency medicine – or two observers for some mock codes – scored each exercise. Different experts observed the mock code.
A survey was distributed before and after the 20 mock code drills. Addressed to ED HCPs (physicians, nurses, EMTs/paramedics, and RTs), its aim was to assess: (1) ED providers’ perception of in situ simulated codes (mock codes) as a tool for preparing them to deal with COVID-19 patients; (2) their perceived level of knowledge of the hospital’s triage processes for COVID-19 patients and respiratory zone flow processes for suspected or confirmed COVID-19 cases; and (3) their level of comfort when dealing with stable and unstable COVID-19 patients and their performance of airway procedures on those patients. Before starting the experiment, the survey was reviewed for its content validity by four experts from emergency medicine, piloted with ED staff, then modified to take into account their comments.
Statistical Analysis
The two sets of questionnaires were coded and the data collected were entered into Excel files (Microsoft Corp.; Redmond, Washington USA). All categorical variables were presented as frequencies and percentages, while continuous variables were presented means (95% confidence interval). In the checklist file, an overall performance score was calculated as the sum of the mock codes’ evaluation of 22 skills, with two points recorded if the item was performed, one point recorded if the item was partially performed, and zero points if the item was not performed. As the maximum score was two per item, and 44 for all items, individual and overall scores were transformed into a 100-point scale for easy interpretation. Differences in performance score according to the characteristics of mock codes and expert evaluators were examined using the Mann-Whitney test. Differences in performance score over consecutive mock codes were examined using the Jonckheere-Terpstra trend test. In the survey file, responses of the HCPs to survey questions before and after attending the mock codes were examined using the McNemar-Bowker test. Additionally, the changes in the survey outcomes (knowledge and confidence of HCPs) subsequent to experiencing the mock codes were examined according to the characteristics of HCPs using Fisher’s exact test for categorical change and the Kruskal-Wallis test for percentage change. The latter indicates the average number of items that improved, relative to all six items evaluated, with a positive percentage indicating improvement and a negative percentage indicating worsening. All P values were two-tailed. P values <.05 were considered significant. SPSS software (release 23.0; IBM Corp; Armonk, New York USA) was used for all statistical analyses.
Results
A total of 14 evaluators assessed 22 skills performed by HCPs during 20 COVID-19 mock codes conducted at the ED of the hospital. The skills checklist had a high reliability with Cronbach’s Alpha of 0.794. As shown in Table 1 and Figure 1, a wide range of differences in the performance of different skills was observed, with an average performance of 59.2%. Skills with better than 70.0% performance included: request for chest x-ray (88.0%); intubation done by the most experienced ED physician (84.5%); correct sequence and procedure of PPE with hand hygiene (79.0%); re-auscultation to confirm the tube position (74.0%); and pre-oxygenation with O2 flows <6L/minute for five minutes, with correct face mask selection (71.0%). Skills with worse than 40.0% performance included: considering early conversion to laryngeal mask airway in case of ventilation difficulty (19.5%); using closed suctioning system and standby ventilator while suctioning in case suctioning is required (28.0%); keeping nasal cannula at 5L/minute for apneic oxygenation (35.0%); and standby ventilator before removal of face mask for intubation (35.0%).
Evaluation of the Performance of 22 Skills during 20 Mock Codes for Managing Suspected Cases of COVID-19 at the Hospital ED
Abbreviations: BMV, bag mask ventilation; ED, emergency department; LMA, laryngeal mask airway; MD, medical doctor; PPE, personal protective equipment; RN, registered nurse; RT, respiratory therapist.
a Higher score means better performance. Individual score was calculated as the sum of evaluation with two points recorded if the item was done, one point recorded if the item was partially done, and zero points recorded if the item was not done. The percentage represents observed score (actual performance) relative to maximum possible score (best performance).
Evaluation of the Performance of 22 Skills during 20 Mock Codes for Managing Suspected Cases of COVID-19 at Hospital Emergency Department.
Twelve (60.0%) mock codes were performed in the resuscitation room and eight (40.0%) in the isolation room with no difference in performance according to mock code location (P = .593). A total of 13 (65.0%) mock codes were evaluated by two evaluators and seven (35.0%) mock codes were evaluated by one evaluator with no difference in performance according to the number of evaluators (P = .947). Approximately 64.3% of evaluators were male, with male evaluators performing better in mock codes than female evaluators (66.4% [SD = 11.7%] versus 48.4% [SD = 19.5%]; P = .005). Figure 2 shows a significant trend of improving performance with an increase in the consecutive number of mock codes (P <.001). For example, the first five mock codes showed a performance level of less than 40.0% while the second-half of mock codes showed performance levels ranging between 55.7% and 79.5%.
Overall Performance Score during 20 Mock Codes for Managing Suspected Cases of COVID-19 at Emergency Department.
Note: The trend was significantly increasing (P = .001 using Jonckheere-Terpstra test).
As shown in Table 2, the majority of HCPs working in the ED of the hospital who participated in the COVID-19 mock codes were emergency nurses (64.8%), followed by emergency physicians (20.4%), paramedics/EMTs (9.3%), and finally RTs (5.6%). Experience ranged from between a few months to 30 years, with an average of 8.2 (SD = 5.5) years. Approximately 59.3% of HCPs had previous experience in dealing with communicable diseases (eg, MERS-CoV and SARS) and only 37.0% attended any in situ simulation mock codes for COVID-19.
Characteristics of the Health Care Providers Working at the Hospital ED
Abbreviations: ED, emergency department; SARS, severe acute respiratory syndrome.
a Unless mentioned otherwise.
Health care providers responded to the survey questions before and after intervention. Only one of six items showed significant improvement after attending mock codes. Approximately 57.4% HCPs felt comfortable dealing with suspected/confirmed, unstable COVID-19 cases after mock codes compared with 33.3% before mock codes (P = .033). Additionally, 44.4% of HCPs felt comfortable performing airway procedures for suspected/confirmed COVID-19 cases after mock codes compared with 29.6% before mock codes, but the difference did not reach statistical significance (P = .062). The remainder of the items did not change appreciably after mock codes. For example, 61.1% of HCPs before mock codes, and 66.7% after mock codes, believed that they had a good knowledge level of the hospital ED triage processes for suspected/confirmed COVID-19 cases (P = .581). Feeling comfortable dealing with suspected/confirmed, unstable COVID-19 cases showed the highest improvement (24.1%), while having a good knowledge level of the ED triage process had the lowest improvement (3.7%).
Emergency nurses and HCPs with moderate experience had the greatest improvement in perceived knowledge and confidence dealing with COVID-19 cases after attending COVID-19 mock codes (15.7% and 23.5%, respectively). The difference was approaching significance for nurses (P = .060) but not significant for experience (P = .481).
Discussion
The hospital ED staff were not prepared for the new requirement of dealing with sick patients with suspected/confirmed COVID-19, which would be expected given that 40.7% of the hospital staff had no prior experience with communicable diseases. As was found before intervention, only 33.3% felt comfortable dealing with suspected/confirmed, unstable COVID-19 cases before mock codes. This number jumped to 57.4% after mock codes (P = .033). In another study, of the participants in the training who consisted of more than 1,250 hospital staff, 76.0% felt better able to cope after the session, compared with only 35.0% who felt prepared to deal confidently with the pandemic before the training.Reference Aiello, Khayeri and Raja7
The small improvement in perceived knowledge of ED processes related to COVID-19 patients subsequent to the mock codes (66.7 % compared to 61.1%) could be explained by the fact that there was less emphasis on ED processes and discussion during debriefing compared to other objectives. However, when results are analyzed by profession, ED nurses showed a greater improvement (15.7%) in perceived knowledge and confidence dealing with COVID-19 cases after attending the COVID-19 mock codes.
The performance of teams, as measured by the previously described checklist, was variable (Figure 1). However, the overall team performance was improving over time. In the first five mock codes, performance was less than 40.0%, while during the second-half of the mock codes, performance ranged between 55.7% and 79.5% with a significant P value (P <.001), as shown in Figure 2. One of the critical items that showed significant improvement over time was correct sequence and procedure of PPE with hand hygiene (79.0%). Abualenain, et al described a similar finding in a study using simulation-based training for Ebola PPE for HCPs.Reference Abualenain and Al-Alawi24 They found that simulation-based training improved performance of HCPs on a PPE checklist from 37.0% to 86.0%. They ended their study with a conclusion that high-fidelity simulation is effective and practical for training HCPs on the use of PPE.Reference Abualenain and Al-Alawi24
Interestingly, performance in the current study did not change with location (resuscitation or isolation room), regardless of the fact that teams were hesitant to treat sick patients in an isolation room, which had never previously been undertaken. Size of the room did not affect the entrance of the ventilator, crash cart, or x-ray machine, as had been assumed. Similar findings were reported in a study measuring cardiopulmonary resuscitation outcomes before and after mock codes that were carried out in different units of the hospital. It reported no significant differences based on location.Reference Clarke, Julie and Yao25 Another interesting finding is the higher performance in mock codes with male evaluators compared with female evaluators (66.4% [SD = 11.7%] versus 48.4% [SD = 19.5%]; P = .005). This can perhaps be partially explained by the gender bias in evaluations that has been shown in literature, though not consistently.Reference Rand, Hudes, Browner, Wachte and Avins26
In addition to the above-mentioned benefits of in situ simulation, additional benefits of detecting defects and safety threats were exhibited. These defects were generally similar to the findings of another in situ simulation study done in an intensive care unit to measure readiness for COVID-19 patients.Reference Fregene, Nadarajah and Buckley4 Another study done by Lababidi, et al described the authors’ experience of simulating a full-scale disaster drill in a big tertiary hospital to test preparedness of HCPs for a COVID-19 outbreak.Reference Lababidi, Alzoraigi and Almarshed27
Their debriefing report recommended reinforcing PPE training and ensuring the availability of different sizes of PPE.Reference Rand, Hudes, Browner, Wachte and Avins26 Some defects were significant, such as critical equipment availability and the transporting of beds that were too wide to fit through doors, while others were simple and easy to fix, such as the location of biohazard bins. A list of examples of defects identified during the mock codes and immediate correctives measures taken is shown in Table 3.
Examples of Defects Identified during Mock Codes and Corrective Measures Taken After Simulation Mock Codes
Abbreviations: EMT, emergency medical technician; PPE, personal protective equipment.
Limitations
Conducting multiple, unannounced mock codes in clinical areas, especially crowded areas such as EDs, is challenging and prone to many factors that might affect the applicability of such a mock codes program. As the study was quasi-experimental, there is also potential limitation of generalizability of results.
Scheduling ED staff and grouping them into fixed teams was not feasible, which limited the measurement of individual team or individual subgroup performance. The observed improvement in team performance could be partly due to directives given on how to care for and manage the protocols of COVID-19 patients. However, no major interventions or training programs were introduced to ED staff during the few months of this study.
Conclusion
In situ simulation is a vital task, regardless of constraints imposed by time and place, as it is clear that being prepared for pandemics is of utmost importance. Such preparation, done in a safe environment, helps establish the performance of the different units, as has been shown in the case of the hospital’s ED department, where in situ simulation mock codes were able to identify the most common safety threats. Repeated mock codes showed improvements in ED staff confidence in dealing with suspected or confirmed COVID-19 patients. In situ simulation is found to be an effective method for increasing the readiness of EDs to deal with the COVID-19 pandemic and other infectious outbreaks. The effect of those mock codes on ED patient outcomes, however, has not been much studied and is an important area for future research.
Acknowledgment
The authors would like to thank ED physicians, nurses, EMT/paramedics, and RTs at the King Abdullah bin Abdulaziz University Hospital (KAAUH; Riyadh, Saudi Arabia) for their participation and incredible enthusiasm. The authors also acknowledge the support of the KAAUH senior management for simulation training.
Conflicts of interest/funding
This research was funded by the Deanship of Scientific Research at Princess Nourah Bint Abdulrahman University, Riyadh, Saudi Arabia through the Fast-track Research Funding Program. The authors have no conflicts of interest to declare.
Open access
