Hostname: page-component-8448b6f56d-42gr6 Total loading time: 0 Render date: 2024-04-23T20:05:51.732Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises

Published online by Cambridge University Press:  15 September 2015

Foluso Agboola ,
Dorothy Bernard ,
Elena Savoia  and
Paul D. Biddinger
Foluso Agboola*
Affiliation:
Division of Policy Translation and Leadership Development, Harvard T.H. Chan School of Public Health, Boston, MassachusettsUSA
Dorothy Bernard
Affiliation:
Division of Policy Translation and Leadership Development, Harvard T.H. Chan School of Public Health, Boston, MassachusettsUSA
Elena Savoia
Affiliation:
Division of Policy Translation and Leadership Development, Harvard T.H. Chan School of Public Health, Boston, MassachusettsUSA Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MassachusettsUSA
Paul D. Biddinger
Affiliation:
Division of Policy Translation and Leadership Development, Harvard T.H. Chan School of Public Health, Boston, MassachusettsUSA Department of Emergency Medicine, Massachusetts General Hospital, Boston, MassachusettsUSA
*
Correspondence: Foluso Agboola, MD, MPH Division of Policy Translation and Leadership Development Harvard T.H. Chan School of Public Health 401 Park Drive Boston, Massachusetts 02115 USA E-mail: fagboola@hsph.harvard.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Introduction

Exercises that simulate emergency scenarios are accepted widely as an essential component of a robust Emergency Preparedness program. Unfortunately, the variability in the quality of the exercises conducted, and the lack of standardized processes to measure performance, has limited the value of exercises in measuring preparedness.

Methods

In order to help health organizations improve the quality and standardization of the performance data they collect during simulated emergencies, a model online exercise evaluation toolkit was developed using performance measures tested in over 60 Emergency Preparedness exercises. The exercise evaluation toolkit contains three major components: (1) a database of measures that can be used to assess performance during an emergency response exercise; (2) a standardized data collection tool (form); and (3) a program that populates the data collection tool with the measures that have been selected by the user from the database. The evaluation toolkit was pilot tested from January through September 2014 in collaboration with 14 partnering organizations representing 10 public health agencies and four health care agencies from eight states across the US. Exercise planners from the partnering organizations were asked to use the toolkit for their exercise evaluation process and were interviewed to provide feedback on the use of the toolkit, the generated evaluation tool, and the usefulness of the data being gathered for the development of the exercise after-action report.

Results

Ninety-three percent (93%) of exercise planners reported that they found the online database of performance measures appropriate for the creation of exercise evaluation forms, and they stated that they would use it again for future exercises. Seventy-two percent (72%) liked the exercise evaluation form that was generated from the toolkit, and 93% reported that the data collected by the use of the evaluation form were useful in gauging their organization’s performance during the exercise. Seventy-nine percent (79%) of exercise planners preferred the evaluation form generated by the toolkit to other forms of evaluations.

Conclusion

Results of this project show that users found the newly developed toolkit to be user friendly and more relevant to measurement of specific public health and health care capabilities than other tools currently available. The developed toolkit may contribute to the further advancement of developing a valid approach to exercise performance measurement.

AgboolaF, BernardD, SavoiaE, BiddingerPD. Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises. Prehosp Disaster Med. 2015;30(5):503–508.

Keywords

disaster planningemergency preparedness exerciseexercise evaluationmeasuresperformance improvement
Type
Brief Reports
Information
Prehospital and Disaster Medicine , Volume 30 , Issue 5 , October 2015 , pp. 503 - 508
Copyright
© World Association for Disaster and Emergency Medicine 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Federal Emergency Management Agency. National Preparedness Cycle. FEMA Web site. https://www.fema.gov/national-preparedness-cycle. Accessed April 22, 2015.Google Scholar
2. World Health Organization. Hospital Safety Index: Guide for Evaluators. WHO; 2008. http://www.paho.org/english/dd/ped/SafeHosEvaluatorGuideEng.pdf. Accessed April 22, 2015.Google Scholar
3. Djalali, A, Hosseinijenab, V, Hasani, A, et al. A fundamental, national, disaster management plan: an education based model. Prehosp Disaster Med. 2009;24(6):565-569.CrossRefGoogle ScholarPubMed
4. Dausey, DJ, Moore, M. Using exercises to improve public health preparedness in Asia, the Middle East, and Africa. BMC Res Notes. 2014;7:474.CrossRefGoogle ScholarPubMed
5. Agboola, F, McCarthy, T, Biddinger, PD. Impact of emergency preparedness exercises on performance. J Public Health Manag Pract. 2013;19(Suppl 5):S77-S83.Google Scholar
6. Biddinger, PD, Savoia, E, Massin-Short, S, Preston, J, Stoto, MA. Public health emergency preparedness exercises: lessons learned. Public Health Rep. 2010c;125(Suppl 5):100-106.Google Scholar
7. Fowkes, V, Blossom, HJ, Sandrock, C, Mitchell, B, Brandstein, K. Exercises in emergency preparedness for health professionals in community clinics. J Community Health. 2010;35(5):512-518.CrossRefGoogle ScholarPubMed
8. Kollek, D, Cwinn, AA. Hospital emergency readiness overview study. Prehosp Disaster Med. 2011;26(3):159-165.Google Scholar
9. Nelson, C, Lurie, N, Wasserman, J. Assessing public health emergency preparedness: concepts, tools, and challenges. Annu Rev Public Health. 2007;28:1-18.Google Scholar
10. Savoia, E, Preston, J, Biddinger, PD. A consensus process on the use of exercises and after action reports to assess and improve public health emergency preparedness and response. Prehosp Disaster Med. 2013;28(3):305-308.Google Scholar
11. Harvard School of Public Health Preparedness and Emergency Response Research Center. Emergency Preparedness Exercise Evaluation Tool. http://phasevtechnologies.com/studies/lamps/index.php. Accessed April 22, 2015.Google Scholar
12. Parker, AM, Nelson, C, Shelton, SR, et al. Measuring Crisis Decision Making for Public Health Emergencies. Santa Monica, California USA: Rand Corporation; 2009. http://www.rand.org/pubs/technical_reports/TR712. Assessed April 22, 2015.Google Scholar
13. Agboola, F, Dauksewicz, B, McCarthy, T, Biddinger, PD. Evaluation of Public Health Emergency Preparedness Exercises: New Research and Tools for Exercise Planners. Orlando, Florida USA. 2013 Preparedness, Emergency Response, and Recovery Consortium and Exposition. http://www.perrc.org/sites/default/files/PERRC%20Session%2009.pdf. Assessed April 22, 2015.Google Scholar
14. Lazar, EJ, Cagliuso, NV Sr, Gebbie, KM. Are we ready and how do we know? The urgent need for performance metrics in hospital emergency management. Disaster Med Public Health Prep. 2009;3(1):57-60.CrossRefGoogle ScholarPubMed