Climate change has led to an increase in the number and severity of natural disasters across the globe.Reference Benevolenza and DeRigne 1 California in particular, has experienced more frequent wildfires and a longer wildfire season. More frequent, intense, and destructive wildfires have prompted public health departments in California to enhance disaster response plans as evacuations and shelter needs increase.
Public health nurses (PHNs) are essential to local health departments’ disaster preparedness response. Public health nurses in the United States are bachelor- or graduate-prepared registered nurses (RNs) who practice in a variety of settings and roles focusing on disease prevention and health promotion at a population level. Although only 2 states provide a certification for PHNs, PHN scope and standards of practice 2 and practice-based competenciesReference Campbell, Harmon and Joyce 3 describe the population-based work PHNs perform at the individual, community, and systems level, defining it as a specialty within nursing. Public health nurses utilize skills in community health assessment, program planning and evaluation, health education, case management, cross-sector collaboration, advocacy, and policy development to address the structural determinants of health of communities and populations. Although a PHN working in a local health department would also be a disaster service worker required to respond to disasters, all PHNs are uniquely suited to respond to disasters to provide mass care and address population-level issues.
Given their skills, training, and, for some, their designation as disaster service workers, PHNs must feel prepared and confident to respond to meet population health needs. Historically, there has been a lack of education around disaster preparedness, response, and recovery in nursing education, and practicing nurses cite a lack of preparedness.Reference Veenema, Lavin and Griffin 4
There is a scarcity of literature investigating the effectiveness of disaster training and education programs for PHNs, with only 1 article discussing a disaster preparedness training program specifically for PHNs.Reference Chiu, Polivka and Stanley 5 Investigators conducted a literature review of over 200 articles in PubMed, CINAHL, and Google Scholar utilizing search terms of disaster preparedness training, public health nurses, public health, and education. Only 10 studies were included in the review. Two used full online formats, 6 used a fully in-person format, and 2 used a blended online and in-person format. Only 3 studies described training for nurses. The programs for nurses included disaster preparedness and response training for PHNs, nurse managers, and nurses who worked in the emergency department.
Both quantitative and qualitative methods were utilized to measure the effectiveness of the programs. Overall, the quantitative studies showed significant improvement in knowledge and skills competencies.Reference Alan, Harmanci Seren and Eskin Bacaksiz 6 –Reference White-Lewis, Berens and Lighter 10 Qualitative studies and mixed-method studies revealed positive feedback on the programs, increased confidence in the ability to respond to a disaster, situational awareness, and attitudes toward disaster preparedness education.Reference Chiu, Polivka and Stanley 5 , Reference Alim, Kawabata and Nakazawa 11 –Reference Sanchez Cristal, Metcalf and Kreisberg 14
Academic and professional nursing organizations recognize how crucial this education is to ensuring population health during a disaster. The new American Association of Colleges of Nursing (AACN) Essentials, which dictates essential content to be included in nursing education, includes public health preparedness. 15 Additionally, the Future of Nursing 2020-2030 report underscores the crucial importance of having a national nursing workforce prepared with the knowledge, skills, and abilities to respond to disasters.Reference Wakefield, Williams and Menestrel 16 Finally, the International Council of Nurses identifies a core set of competencies as a universal standard for all nurses who are deployed outside of their usual work environment to respond to a disaster. 17
Acknowledging the critical need for the PHN workforce to be trained to work in disaster shelters, a regional association of health officers in Northern California solicited expertise to develop training for regional PHNs. The project investigators were awarded a contract to develop and implement the training. Ensuring effective training was particularly critical as the development of the training included a “train the trainer” toolkit for the training to be replicated and modified for individual local health jurisdiction use. The investigators opted to use a flipped classroom approach as a strategy to promote critical thinking in training participants. The purpose of this study was to evaluate the effectiveness of disaster training for PHNs using a flipped classroom approach.
Methods
Description of Intervention
The educational program was designed to meet the training needs of regional local health departments’ public health nurses. Before designing the intervention, a needs assessment conducted by regional PHN directors identified the topics to be covered, which included sheltering basics, infection control, psychological first aid, disaster resources, physical assessment, pharmacology, and basic medical interventions. The needs assessment also identified challenges, including an overstretched workforce, a PHN workforce across the 13 counties with variable direct care and shelter deployment experience, varying organizational cultures across health departments, and keeping participants engaged with a large volume of content. As participants had varied clinical and work experience and could benefit from a self-paced course format, and because investigators wanted to provide opportunities to actively engage in the material, the project investigators opted for a flipped classroom approach.
The flipped classroom is an innovative learning strategy that has been shown to be effective in the development of critical thinking and clinical judgment.Reference Betihavas, Bridgman and Kornhaber 18 , Reference Dehghanzadeh and Jafaraghaee 19 Aligning with adult learning principles in which learners are active participants in their learning experience, learners in a flipped classroom are introduced to course material prior to class through assigned readings, modules, and other materials.Reference Betihavas, Bridgman and Kornhaber 18 The main objective of pre-class preparation is for learners to be familiar with the content beforehand so that application and synthesis can occur in class, bridging the gap between theory and practice.Reference Dehghanzadeh and Jafaraghaee 19 This approach also facilitated the integration of all 3 learning domains of Bloom’s taxonomy.Reference Anderson, Krathwohl and Bloom 20 Participants completed online modules, which promoted self-directed learning and captured the cognitive domains of Bloom’s taxonomy. Participants subsequently completed an in-person training that included hands-on skills application, allowing them to engage with the concepts and to capture the affective and psychomotor domains of Bloom’s taxonomy.
The online training consisted of approximately 5 hours of content divided across 6 modules. Topics covered introduction to emergency management, the role of the nurse in a general population shelter, introduction to health services in a general population shelter, common approaches to physical assessment, basic pharmacology and medical management, and psychological first aid. Participants could pace themselves through the training. Each module concluded with a quiz, and participants had to score at least 80% to continue to the next module. For a detailed description of the online training, please refer to a prior publication.Reference Nouredini, Holbrook and Ladua 21 Participation in the online training was a prerequisite for the in-person training, and the majority of participants completed the online modules just prior to the in-person session.
The 7-hour in-person training consisted of a targeted recap of the online content, a large-group tabletop exercise on infection control, small-group tabletop exercises integrating topics from the online and in-person training, and hands-on practice with physical assessment and interventions. For physical assessment stations, standardized patients were used for a scenario-based review of assessment skills. For skills stations, the university’s skill lab was utilized, and stations included respiratory treatments (inhalers, nebulizers, oxygen), diabetes treatments (blood glucose monitoring, insulin administration, and glucagon administration), and emergency treatments (epinephrine and naloxone administration). Participants were evaluated using checklists developed or modified by nurse educators. The use of real-life scenarios, tabletop exercises, and case studies facilitates teamwork and collaboration, leading to higher levels of cognition and an internalization of knowledge, skills, and values.Reference Billings and Halstead 22
The online training was advertised to PHNs through the 13 regional PHN directors and was also available on a local emergency preparedness training site. However, it was available to anyone through an online continuing education platform. The in-person training was only offered to staff from the regional local health departments. Each department identified PHNs to attend and was guaranteed 2 slots for PHNs per training, with the remaining slots available on a first-come, first-served basis. Not every local health department was able to send PHNs to attend the in-person training, which was only offered for 2 days.
Participants
Participants were eligible to enroll in the training if they were a PHN in one of the 13 regional counties or were otherwise authorized by their local health department to participate. Nurses were recruited primarily through their PHN directors and through a local training and exercise website. Forty-two participants attended the in-person portion of the training. Only 3 participants dropped out due to personal issues or illness; the remaining seats were not filled by local health departments. Participants who completed the online training and attended the in-person portion were included in the evaluation.
Data Collection
Participants completed a pre- and post-evaluation for each online module as well as at the beginning and end of the in-person training day. All measures of change in knowledge and confidence were measured using a Likert scale of 1 to 5. For the online training, a score of 1 indicated that the participant had no knowledge of the topic, whereas a score of 5 indicated they were extremely knowledgeable on the topic. For the in-person training, a score of 1 indicated that the participant had never performed the skill or was not confident in performing the skill, whereas a score of 5 indicated that the participant felt that they could teach the skill. Two open-ended questions were also included in the evaluation to assess what participants found to be most helpful in preparing to deploy to a shelter and to identify topics that should be covered in future training.
Statistical Methods
Analyses were completed using STATA v15.1. Investigators measured descriptive statistics using mean, standard deviation, and percentages. Investigators used paired t-tests to determine the mean score of the items in the pre- and post-surveys. Investigators also included a measure of effect size using Cohen’s d. Effect size is a way to standardize the mean change in score to compare it to other educational interventions. Finally, investigators analyzed open-ended questions using thematic analysis.
Results
Demographics
Participants were diverse in terms of overall nursing and public health experience. The mean years working as a RN was approximately 12.3. The mean years of experience as a PHN was 6.7. Approximately 64% of participants had never been deployed during a disaster, whereas 7% had reported being deployed to 5 or more disasters. The primary reason that PHNs attended this training was due to their interest in the topic of disaster preparedness. See Table 1 for additional demographic data.
Demographic characteristics of the participants (n=42)
Online Training
For the online training, participants completed 6 modules related to a range of areas considered essential to the ability to provide disaster shelter care. For the online modules, investigators measured mean change and standardized mean change in self-perceived knowledge and found every module demonstrated a significant change in mean self-appraised knowledge. The 3 variables with the greatest pre-/post-change in mean were disaster laws (mean change of 1.97, Cohen’s d = 2.65), providing psychological first aid (mean change of 1.69, Cohen’s d = 2.51), and assessing and responding to drug withdrawal (mean change of 1.69, Cohen’s d = 2.5). See Table 2 for online data.
Online training module: key components and descriptive information (n = 41)
* Sample size by module: variability in participant numbers.
In-Person Training
For the in-person training, investigators found a statistically significant increase in self-perceived confidence across all measures. The topics with the greatest pre-/post-change in mean were in administering emergency medications such as naloxone (mean change of 1.72, Cohen’s d = 1.5), epinephrine (mean change of 1.53, Cohen’s d = 1.4), and glucagon (mean change of 1.5, Cohen’s d = 1.4). The data were not normally distributed for the skills for the in-person training, as participants had a high baseline confidence in many of the skills and had a low baseline confidence in administering naloxone, epinephrine, and glucagon. See Table 3 for a complete list of in-person training results.
In-person training: key components and descriptive information (n = 41)
Participant Perception of the Training
Among participants who completed the online training, 50% of participants strongly agreed and 34.2% agreed the training helped prepare them for working in a shelter. Among nurses who completed both online and in-person trainings, 78% of participants strongly agreed and 17.1% agreed that they felt better prepared to work in the shelter. When asked which format should be used in future disaster shelter training, 78% of participants reported that the hybrid online and in-person training format should be utilized (see Table 4).
Comparing participant perceptions of training (after online training vs. after online and in-person training)
According to the investigators’ thematic analysis of the open-ended questions, the overarching theme was how much the participants appreciated the interactive nature of the training. Of all the activities completed, participants found the tabletop activities and the opportunity to practice their assessment and clinical skills as being the most beneficial aspect of the training. In terms of what participants would like to have included in future training, the most common response was more first aid training related to injuries sustained during a fire (burns, wound care, smoke inhalation) and having the opportunity to set up a shelter.
Discussion
Prior to launching this training, the study investigators determined that to understand if the training was effective, there should be a minimal change of at least 1 point on the Likert scale across all learning objectives in addition to all findings being statistically significant with a P value <0.05. Furthermore, investigators also wanted to ensure that the effect size was greater than Cohen’s d of 0.8, indicating a large effect size. Most learning objectives were able to meet all 3 criteria, thus indicating that the training was effective.
The health assessment training had the least amount of change in both knowledge and confidence. However, in the open-ended questions and comments section of the evaluation, many participants specifically stated that the assessment training was very helpful or the most valuable part of the training. Given the variability in clinical experience, it could be that the assessment skills refresher was more relevant to nurses with the least recent or least extensive clinical experience.
In education research, knowledge refers to the understanding and awareness that an individual possesses regarding a particular subject or domain. However, self-efficacy refers to an individual’s belief in their ability to perform a specific task or achieve a particular goal. Across all measures, there was a noticeable improvement in knowledge, which corresponds with cognitive domain in Bloom’s taxonomy. Interestingly, the measures of knowledge were actually higher than the measures of confidence that were measured during the in-person training. This indicates that increased knowledge does not always translate into increased confidence. Measuring confidence is important in that it is a strong predictor of self-efficacy.Reference Malureanu, Panisoara and Lazar 23 To ensure that nurses feel confident, educators should address the affective and psychomotor domains, which is exactly what occurred during the in-person training.
Similar to the 2 prior studiesReference Chiu, Polivka and Stanley 5 , Reference Sanchez Cristal, Metcalf and Kreisberg 14 that included a training that utilized both self-paced modules and in-person learning, this study’s results indicate that the utilization of both online modules and in-person training is useful. Seventy-eight percent of participants who completed the entire training preferred a hybrid approach for future training, indicating this format was indeed the best one to meet the needs of PHNs. Of note, no nurses felt the training should be provided online only. Although the online training did increase participant knowledge and 84.2% of participants did feel it better prepared them to work in a disaster shelter, the addition of the in-person training increased the effectiveness of the training as a whole. After completion of the in-person training, 95.1% of participants felt better prepared to work in a disaster shelter setting. Although scores did meaningfully improve, the end score was still lower than what would be ideal for a PHN deploying to a shelter. To see the desired effect, the training may need to be reinforced through annual training or field experience.
This model could be easily adapted to meeting the disaster responses training needs for other types of responders, including physicians, emergency personnel, and community health workers. In addition, this model does not have to be limited to disaster response content. It could be adapted to meet training needs for other emerging or evolving public health issues in which staff need new training or reinforcement of prior training.
Limitations
Although the results of the training are promising, self-selection bias may have skewed the results. Very few nurses attended the training because they were mandated to, and most reported an interest in the topic of disaster response. Investigators also did not account for the number of nurses who started the online training but either did not complete the online training or did not attend the in-person training. Investigators could not control for recency bias because all participants rated themselves immediately after completing the training. It is unknown how knowledge and confidence levels change over time.
Finally, the evaluation of the training was based on the perceived knowledge and confidence of participants. Although the analysis relied on subjective data, all study participants did have to meet objective measures. Participants had to score an 80% or higher on quizzes to advance through modules in the online training. Their clinical skills were objectively evaluated by experienced nurse educators utilizing a standard checklist during the in-person training.
Conclusion
Training needs for public health nurses are ongoing, particularly in the context of disaster preparedness and response, and it is critical to identify the most effective method for continuing education in this learner population. The flipped classroom approach to continuing nurse education may be a helpful model for other topics as well, particularly for learners with a variation in experience. The curriculum developed as part of this research may, in combination with other training, be an effective and convenient way of ensuring the PHN workforce is trained to respond to disasters.
Data availability statement
The data that support the findings of this study are available on request from the corresponding author, KH.
Acknowledgments
The authors would like to thank Zerlyn Ladua, MSN, FNP, for sharing her expertise in public health emergency management and the role of public health nurses in disaster preparedness and response.
Author contribution
KH: was responsible for manuscript preparation, data collection, and project management; MR and RM: were responsible for conducting a literature review; SN: was responsible for study conceptualization and methodology, data collection, data analysis, and project management; CS: was responsible data management and table development; MC: was responsible for formatting the abstract, tables, and references and for preparation of the manuscript.
Funding statement
This project was funded by a grant from the Urban Areas Security Initiative that was awarded through Alameda County contract (C-19658).
Competing interest
None
Ethical standard
This study was approved by the Internal Review Board at California State University East Bay.
Open access
