The European Centre for Disease Prevention and Control (ECDC) and the World Health Organization (WHO) recommend regular simulation exercises (SimEx) to train, evaluate, and adapt the existing emergency response. In addition to identifying gaps in emergency plans and policies, it allows staff members to practice their roles, familiarize them with procedures within a secure environment, improve team building, motivation, (cross-border or cross-sectional) communication as well as coordination, and reduce costs in an actual response.1–3
PANDEM-2 (https://pandem-2.eu) is a Horizon 2020 project funded by the European Union (EU), focusing on the development of new IT and training solutions for efficient, EU-wide pandemic management. To address the real-world needs of public health agencies for pandemic management, an online Dashboard for pandemic preparedness and response, incorporating additional IT tools for pandemic forecasting, visual analytics, and resources management, as well as supporting material (e.g. information on biosafety measures for first responders), was developed.
Within this project, a functional exercise (FX) was conducted within the Dutch and German national Public Health Emergency Operations Centres (PHEOC) over 2 days.4 Its goal was to train the coordinated response to a large-scale pandemic in Europe testing the PANDEM-2 solutions and other available tools to alert and share information (Appendix 1).
ECDC and WHO have developed detailed guidance on how to plan SimEx in public health settings,1, 5 but few reports are published on the implementation of public health emergency SimEx.6 This report addresses this gap by describing the steps taken in planning, conducting, and evaluating the 2 day FX.
Methods
The Robert Koch Institute (RKI) established the exercise management team (EMT) which led the design and implementation of the FX with the cross-sectoral PANDEM-2 planning team: 6 national public health institutes (PHIs), a hospital specialized in high-consequence infectious diseases, 3 national first responder organizations, 2 university departments of virology and computer science, a software company, an ethics and data protection company, a public relations and training company, as well as a research institute.
The SimEx cycle by ECDC for public health settings1 was used to guide the planning and implementation of the FX (Figure 1).
Figure 1. The simulation exercise cycle by the European Centre for Disease Prevention and Control (ECDC).
Foundation
The foundation part included tasks that enabled a successful exercise, which was set by the project mandate, including the aim and objectives. Derived from this, the planning timeline and allocation of the budget and other resources was developed.
An FX was chosen since it creates an exercise situation as close to a real-life event as possible (eg, time pressure) without the deployment of resources making it more cost- and time-efficient. Hence, it is more likely to produce a realistic response from the participants, leading to more representative results.1, 5
Design and Development
In the design and development part, the content, components, and necessary materials for the exercise were developed. The planning team held 5 videoconferences during which the roles and responsibilities, the technical set-up, the scenario, and the participants were discussed.
The first step was the creation of a plausible scenario script which was based on a fictitious pandemic influenza caused by a H5N1 subtype first detected in birds. This pathogen was chosen because it is associated with high hospitalization rates and mortality, particularly in young adults, which would require a high surge capacity in intensive care and for ventilators during a pandemic.7 The resulting scenario covered all phases from peacetime to response to post-outbreak debrief (Figure 2).
Figure 2. Timeline of pandemic influenza scenario caused by a fictitious virus.
An inject list was developed based on the script. Injects are building blocks of the exercise with information about the developing situation. The list provided an overview of the exercise schedule and detailed sequential information about each inject, such as pandemic phase, inject number, sending time, the time in the scenario, duration, type, summary, and the anticipated response. Two separate lists were written with the same structure but adapted to the context of the Netherlands and Germany. The EMT had regular exchanges with the PANDEM-2 technology partners to facilitate the alignment of the Dashboard functions and the development of synthetic data used for the FX.
Next, a variety of injects, such as fictitious emails, were developed. Moreover, the exercise agenda, briefing material (eg, participant handbook), and exercise templates (eg, for situation reports, problem log sheet, contact list) were created.
After consultation with members of the ECDC and Directorate General for Health and Food Safety (DG SANTE), the EMT decided to use the Early Warning and Response System (EWRS) Simulation Exercise (SimEx) Module8 for a realistic simulation of cross-border communication during the FX.
Conduct
The conduct part included setting up the venues, briefing activities, and the conduct of the 2 exercise days, which were structured into welcome, participant briefing, running the exercise, debriefing (conducted by Fraunhofer), and closing remarks.
Exercise Check and Briefing
A detailed check of all relevant technical systems was performed with RKI and RIVM a day before and half an hour before the exercise. In the days before the exercise, a final check of all injects for validity and correctness was performed. Email drafts, as well as a detailed dissemination timeline, were created to mitigate the chance of errors when sending injects out during the exercise. The participant briefing activities entailed a preparatory email, including the participant handbook as well as an introductory presentation by a member of the EMT shortly before the official start of the exercise.
Exercise Management Team
The EMT consisted of 4 people who sent out the injects, monitored and managed the speed of the exercise, and additionally, 1 facilitator per country who could answer questions in person. All members of the EMT were in close contact via a group chat.
Venue and Streaming Set-Up
The participants took part in the exercise from a conference room in their respective national PHI where they were joined by 1 RKI facilitator and evaluation staff from Fraunhofer. The outbreak response of both countries was followed by the EMT and observers in 2 separate video-streams. The relevant screen was shared in order to observe the live interaction of the participants with the Dashboard and the EWRS SimEx platform.
Injects
The injects gave the participants information on the evolving scenario and triggered specific tasks to complete within their country’s context. Tasks included using the PANDEM-2 tools or preparing communication statements. Besides public health related injects (eg, EWRS SimEx platform, vaccination strategy, or case management), there were injects also focusing on security aspects (eg, outbreak in prison). A great variety of relevant stakeholders was simulated by the EMT via 1 functional email address, such as the Local Public Health Authorities, Ministry of Health, veterinary health sector, press, citizens, and communication by the WHO and ECDC.
Evaluation
The fourth step, evaluation, was led by Fraunhofer and systematically identified the strengths and weaknesses in the pandemic preparation and response processes and IT solutions.
During a debriefing right after the exercise, participants gave their first feedback on the tools and the exercise in a semi-structured group discussion. In the following weeks, interviews were held with the participants, the EMT, and project partners who followed the FX. Besides feedback on the IT solutions, interviewees were asked to give feedback on the content and organization of the FX, according to 6 topic areas. Participants gave feedback on the documentation, scenario, goals, and tested IT solutions. Observers gave additional feedback on the FX organization, logistics, and participant commitment.
Improve
The evaluation results feed into the last step, improvement, where the tested PANDEM-2 tools are adjusted according to the feedback provided by participants and observers.
Results
Participation and Process
The FX took place on March 15-16, 2023. The participants were 11 public health experts of the PHI of the Netherlands (RIVM) and 6 from Germany (RKI). The multidisciplinary teams entailed epidemiologists, public health experts/policy advisors, clinicians, and communication officers.
As planned, the exercise was actively supported by 6 PHI involved in cross-border communication via the EWRS SimEx platform, as well as 1 hospital specialized in high-consequence diseases, and 2 first responder organizations, who were available to give expert advice. Up to 25 observers were following the SimEx via the online live-stream, including members of the Advisory Board (representatives of the ECDC, Austrian Red Cross, and Irish Defense Forces), the German Federal Ministry of Health, the Dutch Ministry of Health, and DG HERA.
During the course of the exercise, the participants received all 44 injects, of which 25 injects included the use of the Dashboard, and 5 included the EWRS SimEx platform. Two security and 6 communication-related injects were received.
Overall, the exercise went according to schedule with injects sent out on time. The EMT monitored the participants’ response via the live-streams and close contact with the facilitators. The 2 participant teams showed all crucial anticipated responses, and fulfilled key roles previously proposed by the EMT, such as a team lead, and persons responsible for the Dashboard or EWRS.
Key Evaluation Results
In total, 16/17 participants were interviewed. Additionally, the 4 EMT members and 17 other project partners were interviewed.
Aim and Objectives
Both participants and project partners agreed the objectives of the FX had been met and the 2 teams were able to demonstrate and test the PANDEM-2 solutions as well as their pandemic response.
Scenario and Commitment of Actors
Participants and observers alike judged “in general, [the] scenario [as] realistic.” Participants thought it was a “good training” opportunity and showed great commitment in engaging in the scenario. It was perceived as “maybe a little too close to COVID to be original/challenging.” There was a realistic scope of cross-border communication and collaboration (“not more in the scenario than we already do”), and more challenging, novel injects (eg, security injects) that fostered more innovative team responses (“some injects involved unexpected questions”). For future exercises, emphasizing a whole-of-society approach was proposed.
Documentation and Briefing
The handbook previously disseminated to the participants was appreciated as a useful preparation material, especially the summary of the key information necessary for the exercise (“cheat sheet helped […]”). Few observers suggested that a summary of the scenario script would have helped them to better follow the exercise.
Organization, Management, and Logistics
In general, the planning and logistics of the FX was considered appropriate by observers. Other than times when the participants did not share the screen they were working on, the observers appreciated being able to follow the different teams.
Discussion and Conclusion
With this FX, both the coordinated response of 2 PHEOCs to a large-scale pandemic in Europe and innovative IT-solutions could be tested.
Innovative aspects of the FX entailed the use of a shared, adaptable IT platform that has the potential to support future cross-border pandemic preparedness training and collaboration among EU member states. The scenario, particularly due to the use of the PANDEM-2 solutions, was experienced as an opportunity to exercise a realistic pandemic response. Further unique features were the live-streaming of the participants’ response and interaction with the newly developed IT tools, which helped to follow and evaluate the FX. Moreover, the active engagement of other project partners and the use of the EWRS SimEx platform contributed to a realistic simulation of international risk communication.
A practical limitation of the FX was posed by the feasible number of active participants, leaving the remaining stakeholders to be represented by the EMT. For future exercises, observers could benefit from a summary of the scenario script. Further, the use of the EWRS SimEx platform could be emphasized. Also, a whole-of-society approach focusing on collaborative efforts with socioeconomic sectors, particularly one including vulnerable groups, could be considered for future exercises.9
Overall, this FX demonstrates the benefit of regular, national stress tests at member state level to strengthen EU cross-border coordination and communication of pandemic management. Following the exercise, members of the participating PHEOC expressed their motivation to plan and conduct SimEx in their institutes on a regular basis, and support these in partner countries. This will improve the capacity of PHI to respond to the next pandemic.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/dmp.2024.298.
Acknowledgments
We would like to thank Epiconcept, Modus Create, the Public Health Agency of Sweden, the National Institute of Public Health Romania, Radboud University Medical Center, Austrian Red Cross, the National Institute of Medical Emergency, Italian Red Cross, the National Health Institute Doctor Ricardo Jorge, Trilateral Research, the Finnish Institute for Health and Welfare and Carr Communications as part of the PANDEM-2 consortium for supporting the planning and implementation of the exercise.
Additionally, we would like to thank Inge van Jaarsveld MD from the local public health authority, GGD Gelderland-Zuid, in the Netherlands, for reviewing the inject list.
Author contribution
CH, UG, KP, CS, RC and MS designed, planned, and conducted the functional exercise. MC and JH contributed to concept development and planning. AW supported the conduct of the exercise. BK and MO conducted the evaluation. CS prepared the first draft of the manuscript and all authors reviewed and edited the manuscript. All authors approved the final manuscript for publication.
Funding statement
The project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 883285. The material presented and views expressed here are the responsibility of the author(s) only. The EU Commission takes no responsibility for any use made of the information set out.
Competing interests
None.
Appendix 1: Aim and Objectives of the FX
Aim
To explore the coordinated response to a large-scale public health event with pandemic potential in Europe using the tools available to alert and share information.
Objectives
-
• To test the functionalities of the PANDEM-2 IT system (e.g. 7-day-incidence, situation report, hospital capacities) and other solutions (e.g. modelling tools, PANDEM source)
-
• To exercise in a multisectoral collaboration
-
• To perform a national risk assessment
-
• To conduct risk communications at a national level and between EU MS (e.g. PANDEM-2 pandemic communication toolkit)
-
• To strengthen the EU network of pandemic managers
-
• To evaluate and improve cross-border coordination and communication of pandemic management
-
• To improve the capacity of the operational system to respond to the next pandemic.
The Planning Team agreed on this aim and these objectives for the FX.
Open access
