Hostname: page-component-8448b6f56d-c4f8m Total loading time: 0 Render date: 2024-04-23T07:45:35.277Z Has data issue: false hasContentIssue false
  • English
  • Français

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

Published online by Cambridge University Press:  13 November 2018

Shoukat H. Qari ,
Hussain R. Yusuf ,
Samuel L. Groseclose ,
Mary R. Leinhos  and
Eric G. Carbone
Shoukat H. Qari*
Affiliation:
Office of Science and Public Health Practice, Office of Public Health Preparedness and Response, United States Centers for Disease Control and Prevention, Atlanta, Georgia
Hussain R. Yusuf
Affiliation:
Division of Health Informatics and Surveillance, Office of Public Health Scientific Services, United States Centers for Disease Control and Prevention, Atlanta, Georgia
Samuel L. Groseclose
Affiliation:
Office of Science and Public Health Practice, Office of Public Health Preparedness and Response, United States Centers for Disease Control and Prevention, Atlanta, Georgia
Mary R. Leinhos
Affiliation:
Office of Science and Public Health Practice, Office of Public Health Preparedness and Response, United States Centers for Disease Control and Prevention, Atlanta, Georgia
Eric G. Carbone
Affiliation:
Office of Science and Public Health Practice, Office of Public Health Preparedness and Response, United States Centers for Disease Control and Prevention, Atlanta, Georgia
*
Correspondence and reprint requests to Shoukat H. Qari, Centers for Disease Control and Prevention, Office of Public Health Preparedness and Response, 1600 Clifton Rd. NE, MS K-72, Atlanta, GA 30333 (e-mail: sqari@cdc.gov).
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives

The US Centers for Disease Control and Prevention (CDC)-funded Preparedness and Emergency Response Research Centers (PERRCs) conducted research from 2008 to 2015 aimed to improve the complex public health emergency preparedness and response (PHEPR) system. This paper summarizes PERRC studies that addressed the development and assessment of criteria for evaluating PHEPR and metrics for measuring their efficiency and effectiveness.

Methods

We reviewed 171 PERRC publications indexed in PubMed between 2009 and 2016. These publications derived from 34 PERRC research projects. We identified publications that addressed the development or assessment of criteria and metrics pertaining to PHEPR systems and describe the evaluation methods used and tools developed, the system domains evaluated, and the metrics developed or assessed.

Results

We identified 29 publications from 12 of the 34 PERRC projects that addressed PHEPR system evaluation criteria and metrics. We grouped each study into 1 of 3 system domains, based on the metrics developed or assessed: (1) organizational characteristics (n = 9), (2) emergency response performance (n = 12), and (3) workforce capacity or capability (n = 8). These studies addressed PHEPR system activities including responses to the 2009 H1N1 pandemic and the 2011 tsunami, as well as emergency exercise performance, situational awareness, and workforce willingness to respond. Both PHEPR system process and outcome metrics were developed or assessed by PERRC studies.

Conclusions

PERRC researchers developed and evaluated a range of PHEPR system evaluation criteria and metrics that should be considered by system partners interested in assessing the efficiency and effectiveness of their activities. Nonetheless, the monitoring and measurement problem in PHEPR is far from solved. Lack of standard measures that are readily obtained or computed at local levels remains a challenge for the public health preparedness field. (Disaster Med Public Health Preparedness. 2019;13:626-638)

Keywords

emergency preparedness program assessmentperformance improvementpublic health emergency preparedness and responsedisaster planning
Type
Systematic Review
Information
Disaster Medicine and Public Health Preparedness , Volume 13 , Issue 3 , June 2019 , pp. 626 - 638
Copyright
Copyright © 2018 Society for Disaster Medicine and Public Health, Inc. 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

1. US Centers for Disease Control and Prevention. Public Health Emergency Preparedness (PHEP) CooperativeAgreement. https://www.cdc.gov/phpr/readiness/phep.htm. Accessed August 9, 2018.Google Scholar
2. US Centers for Disease Control and Prevention. Public Health Preparedness Capabilities: National Standards for State and Local Planning. http://www.cdc.gov/phpr/capabilities/dslr_capabilities_july.pdf. Published March 2011. Accessed August 9, 2018.Google Scholar
3. Nelson, C, Lurie, N, Wasserman, J. Assessing public health emergency preparedness: concepts, tools, and challenges. Annu Rev Public Health. 2007;28:1-18.Google Scholar
4. Altevogt, BM, Pope, AM, Hill, MN, et al. Research priorities in emergency preparedness and response for public health systems: a letter report. Washington DC: National Academies Press; 2008.Google Scholar
5. Qari, SH, Abramson, DM, Kushma, JA, et al. PERRCs: early returns on investment in evidence-based public health systems research. Public Health Rep. 2014;129(Suppl 4):1-4.Google Scholar
6. Leinhos, M, Qari, SH, Williams-Johnson, M. Preparedness and emergency response research centers: using a public health systems approach to improve all-hazards preparedness and response. Public Health Rep. 2014;129(Suppl 4):8-18.Google Scholar
7. Savoia, E, Rodday, AM, Stoto, MA. Public health emergency preparedness at the local level: results of a national survey. Health Serv Res. 2009a;44(5 Pt 2):1909-1924.Google Scholar
8. Leep, CJ. 2005 National profile of local health departments. J Public Health Manag Pract. 2006 Sep-Oct; 12(5):496-498.Google Scholar
9. Davis, MV, Mays, GP, Bellamy, J, et al. Improving public health preparedness capacity measurement: development of the local health department preparedness capacities assessment survey. Disaster Med Public Health Prep. 2013 Dec;7(6):578-584. doi: 10.1017/dmp.2013.108.Google Scholar
10. Davis, MV, Bevc, CA, Schenck, AP. Declining trends in local health department preparedness capacities. Am J Public Health. 2014a Nov;104(11):2233-2238.Google Scholar
11. Bevc, CA, Davis, MV, Schenck, AP. Temporal trends in local public health preparedness capacity. Front Public Health Serv Syst Res. 2014;3(3).Google Scholar
12. Davis, MV, Bevc, CA, Schenck, AP. Effects of performance improvement programs on preparedness capacities. Public Health Rep. 2014b;129(Suppl 4):19-27.Google Scholar
13. Dalnoki-Veress, F, McKallagat, C, Klebesadal, A. Local health department planning for a radiological emergency: an application of the AHP2 tool to emergency preparedness prioritization. Public Health Rep. 2014;129(Suppl 4):136-144.Google Scholar
14. Hunter, JC, Yang, JE, Petrie, M, et al. Integrating a framework for conducting public health systems research into statewide operations-based exercises to improve emergency preparedness. BMC Public Health. 2012a Aug 20;12:680.Google Scholar
15. Agboola, F, Bernard, D, Savoia, E, et al. Development of an online toolkit for measuring performance in health emergency response exercises. Prehosp Disaster Med. 2015;30(5):503-508.Google Scholar
16. Glik, DC, Eisenman, DP, Donatello, I, et al. Reliability and validity of the Assessment for Disaster Engagement with Partners Tool (ADEPT) for local health departments. Public Health Rep. 2014; 129(Suppl 4):77-86.Google Scholar
17. Hunter, JC, Crawley, AW, Petrie, M, et al. Local public health system response to the tsunami threat in coastal California following the Tōhoku Earthquake. PLoS Curr. 2012b;4:e4f7f57285b804. doi: 10.1371/4f7f57285b804 Google Scholar
18. Hunter, JC, Yang, JE, Crawley, AW, et al. Public health response systems in-action: learning from local health departments’ experiences with acute and emergency incidents. PLoS One. 2013 Nov 13; 8(11):e79457. doi:10.1371/journal.pone.0079457 Google Scholar
19. Hunter, MD, Hunter, JC, Yang, JE, et al. Public health system response to extreme weather events. J Public Health Manag Pract. 2016 Jan-Feb;22(1):E1-10. doi: 10.1097/PHH.0000000000000204 Google Scholar
20. Stoto, M. Measuring and assessing public health emergency preparedness. J Public Health Manag Pract. 2013;19(Suppl 2):S16-21.Google Scholar
21. Zhang, Y, Lopez-Gatell, H, Alpuche-Aranda, CM, et al. Did advances in global surveillance and notification systems make a difference in the 2009 H1N1 pandemic?–A retrospective analysis. PLoS ONE. 2013;8(4):e59893.Google Scholar
22. Stoto, MA. The effectiveness of U.S. public health surveillance systems for situational awareness during the 2009 H1N1 pandemic: a retrospective analysis. PLoS ONE. 2012;7(8):e40984.Google Scholar
23. Klaiman, T, O’Connell, K, Stoto, M. Local health department public vaccination clinic success during 2009 pH1N1. J Public Health Manag Pract. 2013;19(4):E20-E26.Google Scholar
24. Klaiman, T, O’Connell, K, Stoto, MA. Learning from successful school-based vaccination clinics during 2009 pH1N1. J Sch Health. 2014;84:63-69.Google Scholar
25. Piltch-Loeb, R, Kraemer, JD, Nelson, C, et al. A public health emergency preparedness critical incident registry. Biosecur Bioterror. 2014a May-Jun;12(3):132-143. doi: 10.1089/bsp.2014.0007 Google Scholar
26. Piltch-Loeb, RN, Nelson, CD, Kraemer, JD, et al. A peer assessment approach for learning from public health emergencies. Public Health Rep. 2014b;129(Suppl 4):28-34.Google Scholar
27. Stoto, MA, Nelson, C, Higdon, MA, et al. Lessons about the state and local public health system response to the 2009 H1N1 pandemic: a workshop summary. J Public Health Manag Pract. 2013 Sep-Oct; 19(5):428-435. doi:10.1097/PHH.0b013e3182751d3e Google Scholar
28. Savoia, E, Agboola, F, Biddinger, PD. Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012 Aug ; 9(8):2949-2963. doi: 10.3390/ijerph9082949. Epub 2012 Aug 16.Google Scholar
29. Barnett, DJ, Balicer, RD, Thompson, CB, et al. 2009 assessment of local public health workers’ willingness to respond to pandemic influenza through application of the extended parallel process model. PLoS ONE. 2009;4(7):e6365.Google Scholar
30. Witte, K. Putting the fear back into fear appeals: The extended parallel process model. Commun Monogr. 1992;59:329-349.Google Scholar
31. Barnett, DJ, Thompson, CB, Errett, NA, et al. Determinants of emergency response willingness in the local public health workforce by jurisdictional and scenario patterns: a cross-sectional survey. BMC Public Health. 2012;12:164.Google Scholar
32. Barnett, DJ, Thompson, CB, Semon, NL, et al. EPPM and willingness to respond: the role of risk and efficacy communication in strengthening public health emergency response systems. Health Commun. 2014;29(6):598-609.Google Scholar
33. Errett, NA, Barnett, DJ, Thompson, CB, et al. Assessment of psychological preparedness and emergency response willingness of local public health department and hospital workers. Int J Emerg Ment Health. 2012;14(2):125-133.Google Scholar
34. Savoia, E, Massin‐Short, S, Higdon, MA, et al. A toolkit to assess Medical Reserve Corps Units’ performance. Disaster Med Public Health Prep. 2010;4(3):213-219.Google Scholar
35. Horney, JA, Markiewicz, M, Meyer, AM. Regional public health preparedness teams in North Carolina: an analysis of their structural capacity and impact on services provided. Am J Disaster Med. 2011 Mar-Apr;6(2):107-117.Google Scholar
36. Schuh, RG, Eichelberger, TR, Stebbins, S, et al. Developing a measure of local agency adaptation to emergencies: a metric. Eval Program Plann. 2012;35(4):473-480.Google Scholar
37. Potter, MA, Schuh, RG, Pomer, B, et al. The adaptive response metric: toward an all hazards tool for planning, decision support, and after‐action analytics. J Public Health Manag Pract. 2013 Sep-Oct;19(Suppl 2):S49-54. doi:10.1097/PHH.0b013e318296214c Google Scholar
38. Asch, SM, Stoto, M, Mendes, M, et al. A review of instruments assessing public health preparedness. Public Health Rep. 2005;120(5):532-542.Google Scholar
39. Kirschenbaum, A. Measuring the effectiveness of disaster management organizations. Int J Mass Emerg Disasters. 1994;22(1):75-102.Google Scholar
40. Links, JM, Schwartz, BS, Lin, S, et al. COPEWELL: a conceptual framework and system dynamics model for predicting community functioning and resilience. Disaster Med Public Health Prep. 2017 Jun 21:1-11. doi: 10.1017/dmp.2017.39 Google Scholar
41. Savoia, E, Agboola, F, Biddinger, PD. A conceptual framework to measure systems’ performance during emergency preparedness exercises. Int J Environ Res Public Health. 2014 Sep 17;11(9):9712-9722. doi: 10.3390/ijerph110909712 Google Scholar
42. Jhung, MA, Swerdlow, D, Olsen, SJ, et al. Epidemiology of 2009 pandemic influenza A (H1N1) in the United States. Clin Infect Dis. 2011;52(Suppl 1):S13-26.Google Scholar