Background
The rapid spread and scale of the novel coronavirus disease (COVID-19) pandemic, which started in early 2020, created substantial challenges for coordination and oversight of response efforts. The requirement for surge activities such as diagnostic testing, contact tracing, and distribution of personal protective equipment alongside tracking new cases, and individuals in quarantine or isolation, as well as other tasks, put a substantial strain on health authorities and other stakeholders. The multisectoral nature of the response further stretched communication and coordination capabilities in many settings.
Numerous jurisdictions activated Emergency Operations Centers (EOCs) to facilitate COVID-19 response coordination, some of which were already specialized for public health functions (Public Health EOCs, or PHEOCs). 1 The World Health Organization (WHO) has supported the development and operationalization of PHEOCs since 2012, with emergency preparedness and response functions described as core capacity requirements under the International Health Regulations (2005). In 2012, WHO launched EOC-NET, a network designed to promote best practices and standards, support PHEOC capacity strengthening, and strengthen collaboration between response partners for more effective use of PHEOCs. 2 Global interest in implementing sub-national, national and regional PHEOCs grew markedly in the wake of the 2014–2016 West Africa Ebola virus disease epidemic, particularly based on observations that Nigeria’s EOC, established as part of the national polio eradication program, likely served a key role in containing the spread of Ebola in Nigeria after an imported case led to secondary transmission. Reference Shuaib, Musa and Muhammad3 These efforts to implement PHEOCs resulted in a wealth of guidance documents and resources to support design, development and operationalization of PHEOCs across different countries and political-economic contexts, often accompanied by in-service training and on-site technical assistance.
Travel restrictions were 1 of the earliest and longest lasting public health control measures implemented by governments to control the spread of COVID-19. Reference Grépin, Ho and Liu4 Together with restrictions on the numbers of people gathering in person, as well as stay-at-home (“lockdown”) orders in some settings, these measures severely limited donors, and implementing partners to provide technical assistance or trainings in person, necessitating design and execution of alternative modalities for provision of support.
In 2014 to 2015, the US Centers for Disease Control and Prevention (US CDC) collaborated with the Ministry of Health in Baghdad Iraq, to develop a series of training materials, and resources to support the design, development, and implementation of a new PHEOC. Due to travel self-directed and government personnel traveling to Baghdad at the time, the materials were created to be self-directed, and uploaded onto a compact disc (CD) to share with Ministry counterparts. With ministry personnel able to access the materials off-line and at their own pace, the nicknamed “EOC in a Box” contributed to establishing a working EOC within the National Medical Operations Centre in the Ministry. 5 At the outset of the COVID-19 pandemic, US CDC experts recognized an opportunity to leverage the “EOC in a Box” materials and virtual environment to support broader global efforts to strengthen EOC and PHEOC capacities to respond to the burgeoning crisis. This paper describes the establishment of a new public health emergency management website, www.epidemic-em.org, as well as a pilot virtual training conducted in South Africa, which utilized materials on the website to train COVID-19 responders in principles of public health emergency management.
Methods
Core resources and Website Launch
In order to begin populating the www.epidemic-em.org website, materials were extracted from the “EOC in a Box” CD and consisted of a series of 7 PowerPoint slide decks covering the planning, design, and development/ operation of a PHEOC. These slide decks were then reviewed and updated to ensure alignment with the latest international guidance and standards for PHEOCs, notably WHO’s Framework for a Public Health Emergency Operations Centre, 6 and associated handbooks. 7,8 In addition, the content was revised to ensure users would be able to apply the concepts specifically to actions and activities relevant to the COVID-19 response. Materials such as templates, checklists, standard operating procedures, and other resources were identified through internet searches and examination of online document libraries to complement the slide decks and provide additional guidance to users. To expand accessibility, where available, materials in languages other than English were also collected, and all the slide decks and most supporting resources were translated into Spanish. Spanish was prioritized as the first language for translation as, at the time, COVID-19 case numbers were surging in Latin America, and the project prioritized the availability of resources to assist with the response there. Future translations into additional languages, pending funding availability have been planned.
With these initial materials in place, the website was launched in June 2020.
Case Studies
To facilitate understanding of the role of PHEOCs and their application of all-hazards public health emergency response principles in responding to COVID-19, as well as to demonstrate the use and application of available resources, an electronic questionnaire was created and disseminated to colleagues working on public health emergency management in Malaysia, Nigeria, and Guinea. These countries were selected as examples of countries in different political and economic contexts and geographical settings that have developed PHEOCs. The intent of this purposeful selection of countries was to demonstrate how PHEOCs have been developed and operated, including the resources and guidance materials used, across different contexts. Input from in-country contributors was also referenced to describe how each country’s PHEOC network had been utilized during the COVID-19 response, and where future gaps and opportunities for strengthening response efforts, across all types of public health hazards lay.
A copy of the questionnaire used to develop the case studies is provided in the Supplemental Material.
Pilot Virtual Training
To complement the passive availability of materials and resources on the website, the team explored how the website’s content and virtual environment could be leveraged for more active training purposes. By mid-2021, South Africa had experienced several waves of high COVID-19 incidence and had identified a need for enhanced application of emergency management principles, particularly at the sub-national level to sustain the response. A needs-assessment questionnaire (see Supplemental Material) was developed to identify specific topic areas for a pilot virtual training, targeting COVID-19 responders at the sub-national level. The needs assessment responses highlighted 4 priority topics for training: (1) Strengthening application of Incident Management System (IMS); (2) Data and resource management; (3) Coordination and communication; and (4) Leadership and advocacy.
The topics were organized into a 4-module, 5-week mixed synchronous and asynchronous virtual training program (Figure 1). For each module, learning objectives aligned to the needs assessment responses were developed. Content was created in the form of short video-recorded lectures, as well as interviews with global subject matter experts, to be uploaded onto the www.epidemic-em.org website. Where possible, closed captions in videos were included for increased accessibility. Selected readings complemented the recorded videos, in the form of peer-reviewed papers, articles, or other written pieces related to the module topic. Each module also had a short pre- and post-module quiz, which could be downloaded from the website for completion. While not compulsory, completion of the pre- and post-module quizzes was strongly encouraged and linked to receiving a final certificate of completion at the end of the course.
Schedule plan for the 4-module virtual pilot training program.
Synchronous sessions (designated as “SYNC” in the figure) ran in real time, with students and instructors attending together from different locations. Asynchronous sessions (designated as “ASYNC” in the figure) occurred when students accessed class materials during different hours and from different locations on their own time.
Participants were identified through the South African International Health Regulations (IHR 2005) contact list and were asked to attend a short orientation and introductory Zoom session in Week 1. Contacts were encouraged to share the link to training with relevant colleagues to increase reach of the training program. Participants were also asked to complete an initial pre-training assessment relating to their knowledge, understanding, and ability to apply public health emergency management (PHEM) and IMS principles and concepts, as well as manage and train others. They were expected to watch the pre-recorded video lectures asynchronously during the week, whenever convenient for them, and attend a 1-hour “live” Zoom discussion session at the end of each week. The synchronous Zoom session provided a short recap of the module’s content, allowing participants to ask questions about the topic, and also included a group discussion exercise to encourage deeper engagement with the material. We recorded all Zoom sessions and shared the links with the participants to allow viewing by those who were not able to attend live.
A short post-training assessment and evaluation survey followed the final module. It could be completed anonymously via online Google form or emailed back to the facilitator. In addition to repeating the knowledge, understanding and ability perception questions from the pre-training assessment, the evaluation survey sought participant feedback on the modality, quality, and utility of the training, and included a blank section for additional qualitative feedback. (see Supplemental Material for a copy of the post-training assessment and evaluation survey).
Ethical Considerations
The protocol for collection of PHEOC data for the case studies via questionnaire was submitted to Georgetown University’s Institutional Review Board (IRB) for review (STUDY00002843). It was designated as “not human subjects’ research,” and thus did not require oversight from the IRB.
Results
Website
The www.epidemic-em.org website is fully open access, with all resources publicly available for download either individually or by module. The main home page contains the primary self-directed training content, organized into 8 modules:
-
1) What is an EOC and why do we need one?
-
2) How do we organize a response?
-
3) How do we design and equip our EOC?
-
4) How do we activate our EOC?
-
5) How do we operate our EOC?
-
6) How do we deactivate our EOC?
-
7) How do we keep the EOC prepared?
-
8) How do we review performance during a response?
The first 7 of these modules were part of the original launch, and the final 1 was added in 2021 to reflect the growing interest and application of intra-action reviews at the national and sub-national level during the COVID-19 pandemic. A drop-down menu under each of these module headings provides access to 1 or more slide decks for self-directed learning; these presentations are available in both English and Spanish, with French, Russian, and Arabic translations planned for the near future. The drop-down menu also lists any supporting guidelines, templates, checklists, or other resources that may be helpful in implementing the actions described in the slide deck. Most materials are available in both English and Spanish, and where external resources were already available in other languages, those versions have been included as well.
A link at the top of the page provides access to the other website technical content. This includes the training page, which houses all the content developed as part of the pilot training program, downloadable versions of the pre- and post-module quizzes for each topic as well as the pre-, and post-training assessment as well as evaluation survey, and all the recorded lectures and interviews from subject matter experts. In most cases, the videos include closed captions in English and Spanish. The case study page lists the country case studies that have been completed to date, which can be read directly on the website or downloaded. The Guinea country case study is available in both English and French. On each page there is a link to a contact page, which provides an online form for providing feedback on the website, to explore potential collaborations, or to request technical assistance.
User Analytics
As of June 2022, the website received 4221 visits from users in 139 countries and territories, representing all WHO regions (Figure 2). The 5 countries with the greatest number of sessions (“session” meaning 1 user coming to the website (any page); users could return to the website and visit a page which would constitute a new visit and session) were United States (1047), Nigeria (304), Malaysia (203), and China (194), as well as South Africa (166). The site had a total of 2630 unique users (determined by IP addresses) by June 2022, with around 50% of these users located in the United States, China, and Malaysia, as well as Nigeria, India, Japan, and Germany (Table 1).
Website analytics showing cumulative number of user visits per country (as of June 2022).
Top 20 countries with the most unique users (as of June 2022)
*Fifty-four users did not have a location set on their Internet protocol (IP) address.
Pilot Virtual Training
The pilot virtual training took place in November – December 2021. The number of attendees for the live synchronous sessions varied; more than 30 attended the introductory orientation session, and 19 provided email addresses to be contacted directly throughout the course. Eighteen participants completed the pre-training assessment question on job position, which revealed a diverse range of job titles and technical areas (Figure 3). These participants represented both national and sub-national/provincial posts.
Composition of job titles and positions of the participants, per responses to the pre-training assessment (N = 18).
Six participants completed both the pre- and post-training assessment. The comparison of Likert scale responses between the pre- and post-training assessment showed overall improvement in the knowledge, understanding and confidence in using and applying PHEM principles and working, or managing others, within an IMS structure (Figure 4). In the post-training assessment, all participants reported that they either “Agree” or “Strongly agree” with the statements relating to their own understanding or abilities with respect to PHEM and IMS, as well as their ability to manage others. The only questions where participants responded “Neither agree nor disagree” in the post-training assessment related to knowledge in development and implementation of drills and exercises and in capability of training others in PHEM principles. The responses still represented an increase in knowledge and capability compared to the pre-test scores.
Comparison of pre- and post-training assessment responses relating to knowledge, understanding and confidence with applying PHEM and IMS concepts (N = 6).
The evaluation survey was completed by 10 participants. All 10 respondents indicated either “Agree” or “Strongly agree” to statements about whether participation in the training will allow them to be more effective in their role; the appropriateness of the balance between synchronous and asynchronous (recorded) materials; the knowledge and respectfulness of the facilitators; the relevance of the examples and content to the participants’ professional content; the appropriateness of the content to the participants’ skill level and experience; and if they would recommend the course to others. Two participants responded, “Neither agree nor disagree” to the statement related to the relevance of the content of the training to their day-to-day role (the other 8 respondents to this question were split equally between “Agree” and “Strongly agree”).
The qualitative feedback provided by the participants was strongly positive, and particularly reflected the participants’ appreciation of the flexibility afforded by the virtual engagement model as well as the utility of combining both recorded material with synchronous sessions for further discussion. For example, 1 participant noted that “the balance of pre-recorded sessions and the live interaction on a weekly basis worked well. The facilitators were great at engaging the attendees during the weekly sessions. It really is a great way of attending a course amidst the business of daily work.”
Another commented that “the synchronous content was practical and helped to solidify the asynchronous content. The facilitators were respectful of time and so it was manageable and interesting – easy to engage. The format was the right amount of balance.”
Other participants similarly noted that the “weekly meetings were a good complement to the modules” and that they “liked this design with interactive communications.”
Discussion
The successful launch and broad user uptake of the www.epidemic-em.org website highlight the opportunities prompted by the COVID-19 pandemic to expand virtual access to training materials and rethink models of technical assistance. Generally, users viewed the overall approach including the pilot virtual training, very favorably, and were not only open to the idea of virtual engagement but reported significant advantages to the mixed asynchronous and synchronous modality. This suggests that virtual training for PHEM and IMS is an opportunity area for expansion, particularly if we can create additional tools on the website to help users identify their gaps and needs and locate appropriate materials and resources accordingly. Few trainings covering PHEM-focused EOCs have such detailed asynchronous offerings, which fill an important gap as personnel activated during an emergency who lack the necessary background in EOCs may not have the ability to leave their deployment to participate live in a training. The synchronous sessions were also a valuable opportunity for participants to discuss and apply the asynchronous education, enhancing their understanding and capacity to utilize the material in their own EOCs.
The pilot virtual training program began to explore these opportunities with response partners, through an initial needs assessment and the design and delivery of a tailored and fully virtual training experience adapted to the needs of local responders across different sites and sectors. As an example, the National Institute for Communicable Diseases in South Africa, a division of the National Health Laboratory Service utilized the resources from the website, augmented with resources from WHO, and added local examples as well as an additional module on exercises, and delivered the content using in-house EOC staff via Project ECHO. Further engagements of this kind would likely identify other target areas for capacity strengthening, resulting in new modules, and diversifying the training content availability via the website.
It was anticipated that the “Contact” page on the website would result in requests for technical assistance. However, as of December 2022, only 1 such request had been received. In that case the website did provide a helpful conduit for initial communication. Additional advertisement and sharing of information about the website and the resources it contains, for example through national Public Health Surveillance Bulletins and WHO’s EOC-NET, could potentially bring it to more users’ attention and create more opportunities for information exchange and assistance. The intent to expand the reach of the website and materials is to facilitate free and easy access to important resources which can amplify the benefits of the usage of the materials to EOCs operating in public health emergencies around the world.
The launch of the www.epidemic-em.org website and associated case studies and virtual training had several limitations. First, the project only had a limited budget for translations, which meant that materials could only initially be translated into Spanish. The addition of more languages will help make the website and resources more accessible to a wider group of end-users, and translations in French, Arabic, Russian, and possibly Chinese would be added in the future, to cover all United Nations’ languages. Second, given the acute crisis conditions during the time period of the website’s development, the content was developed with an emphasis on activities and actions that would be specifically relevant to the COVID-19 response. However, as all the materials are based on underlying core public health emergency management principles and best practices, even these examples are still broadly applicable to other epidemics or even non-infectious disease public health emergencies. To this end, the materials embody an all-hazards approach. A non-pandemic offering of the training could also allow for training to reach more participants, so the content deployed at the onset of an emergency ensure that all appropriate audiences are coordinating within EOCs. Informal feedback from website users suggested that additional guidance, potentially in the form of a self-completed checklist or questionnaire, might assist users to access the right materials to match their needs. The evaluation of the pilot training program was limited by the small number of respondents who completed both the pre- and post-training assessments. Furthermore, increases in knowledge and capability were based on self-reported responses in the post-training assessment and were not tested objectively.
Conclusions
The authors’ experience in developing and launching www.epidemic-em.org and using it for a pilot virtual training program has demonstrated that web-based approaches offer a valuable addition to existing tools for PHEM capacity strengthening and technical assistance. Further development and evaluation of the website to enhance impact and utility for end-users will be ongoing.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dmp.2024.36
Acknowledgements
The contents of this work are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services. We thank the CDC Foundation and the Bill & Melinda Gates Foundation for the funding support, and the Talus Analytics team for their technical support and professionalism in building the website. We would like to acknowledge the many professionals working in public health emergency management and EOCs around the world who contributed to the tools and resources referenced on the www.epidemic-em.org website. We extend particular thanks to all our case study contributors from Guinea, Malaysia, and Nigeria, and particularly to Dr. Amadou Traoré, Prof. Badrul Samad, and Ms. Winifred Upkonu. Finally, we would like to thank all the participants of the virtual training program in South Africa, who took time out of their busy schedules to contribute and share their own knowledge and experiences.
Author contributions
JAB, AG, and CJS conceived the project; JAB, BM, RR, and IVA, as well as RH, and CJS researched, reviewed, and created website content, with technical input and review from AG and SK; JAB, NG, MJG, and CJS designed, implemented, and evaluated the virtual training program; JAB, RH, and CJS drafted the manuscript, and all authors reviewed and approved the final version.
Sources of support
This work was funded through the support of the CDC Foundation (award number 1085.16), through a COVID-19 emergency response grant provided by the Bill & Melinda Gates Foundation.
Competing interests
The authors declare none.
Abbreviations
COVID-19, Novel Coronavirus Disease (2019); EOC, Emergency Operations Center; IMS, Incident Management System; PHEM, Public Health Emergency Management; PHEOC, Public Health Emergency Operations Center; US CDC, United States Centers For Disease Control And Prevention; WHO, World Health Organization
Open access
