Skip to main content

Quality Indicators for Evaluating Prehospital Emergency Care: A Scoping Review

  • Ian Howard (a1) (a2), Peter Cameron (a3), Lee Wallis (a2), Maaret Castren (a4) and Veronica Lindstrom (a1) (a5)...

Historically, the quality and performance of prehospital emergency care (PEC) has been assessed largely based on surrogate, non-clinical endpoints such as response time intervals or other crude measures of care (eg, stakeholder satisfaction). However, advances in Emergency Medical Services (EMS) systems and services world-wide have seen their scope and reach continue to expand. This has dictated that novel measures of performance be implemented to compliment this growth. Significant progress has been made in this area, largely in the form of the development of evidence-informed quality indicators (QIs) of PEC.


Quality indicators represent an increasingly popular component of health care quality and performance measurement. However, little is known about the development of QIs in the PEC environment. The purpose of this study was to assess the development and characteristics of PEC-specific QIs in the literature.


A scoping review was conducted through a search of PubMed (National Center for Biotechnology Information, National Institutes of Health; Bethesda, Maryland USA); EMBase (Elsevier; Amsterdam, Netherlands); CINAHL (EBSCO Information Services; Ipswich, Massachusetts USA); Web of Science (Thomson Reuters; New York, New York USA); and the Cochrane Library (The Cochrane Collaboration; Oxford, United Kingdom). To increase the sensitivity of the literature, a search of the grey literature and review of select websites was additionally conducted. Articles were selected that proposed at least one PEC QI and whose aim was to discuss, analyze, or promote quality measurement in the PEC environment.


The majority of research (n=25 articles) was published within the last decade (68.0%) and largely originated within the USA (68.0%). Delphi and observational methodologies were the most commonly employed for QI development (28.0%). A total of 331 QIs were identified via the article review, with an additional 15 QIs identified via the website review. Of all, 42.8% were categorized as primarily Clinical, with Out-of-Hospital Cardiac Arrest contributing the highest number within this domain (30.4%). Of the QIs categorized as Non-Clinical (57.2%), Time-Based Intervals contributed the greatest number (28.8%). Population on Whom the Data Collection was Constructed made up the most commonly reported QI component (79.8%), followed by a Descriptive Statement (63.6%). Least reported were Timing of Data Collection (12.1%) and Timing of Reporting (12.1%). Pilot testing of the QIs was reported on 34.7% of QIs identified in the review.


Overall, there is considerable interest in the understanding and development of PEC quality measurement. However, closer attention to the details and reporting of QIs is required for research of this type to be more easily extrapolated and generalized.

Howard I , Cameron P , Wallis L , Castren M , Lindstrom V . Quality Indicators for Evaluating Prehospital Emergency Care: A Scoping Review. Prehosp Disaster Med. 2018;33(1):4352.

Corresponding author
Correspondence: Ian Howard, BEMC, MPhil Department of Clinical Science and Education, Södersjukhuset Karolinska Institutet Stockholm, Sweden E-mail:
Hide All

Conflicts of interest: none

Hide All
1. Myers, JB, Slovis, CM, Eckstein, M, et al. Evidence-based performance measures for Emergency Medical Services systems: a model for expanded EMS benchmarking. Prehosp Emerg Care. 2008;12(2):141-151.
2. Maio, RF, Garrison, HG, Spaire, DW, et al. Emergency Medical Services outcomes project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33(4):423-432.
3. O’Connor, RE, Slovis, CM, Hunt, RC, Pirrallo, RG, Sayre, MR. Eliminating errors in Emergency Medical Services: realities and recommendations. Prehosp Emerg Care. 2002;6(1):107-113.
4. Moore, L. Measuring quality and effectiveness of prehospital EMS. Prehosp Emerg Care. 1999;3(4):325-331.
5. Dunford, J, Domeier, RM, Blackwell, T, et al. Performance measurements in Emergency Medical Services. Prehosp Emerg Care. 2002;6(1):107-113.
6. Studnek, JR, Garvey, L, Blackwell, T, Vandeventer, S, Ward, SR. Association between prehospital time intervals and ST-elevation myocardial infarction system performance. Circulation. 2010;122(15):1464-1469.
7. Takahashi, M, Kohsaka, S, Miyata, H, et al. Association between prehospital time interval and short-term outcome in acute heart failure patients. J Card Fail. 2011;17(9):742-747.
8. Pons, PT, Haukoos, JS, Bludworth, W, Cribley, T, Pons, KA, Markovchick, VJ. Paramedic response time: does it affect patient survival? Acad Emerg Med. 2005;12(7):594-600.
9. Pons, PT, Markovchick, VJ. Eight minutes or less: does the ambulance response time guideline impact trauma patient outcome? J Emerg Med. 2002;23(1):43-48.
10. Blackwell, TH, Kline, JA, Willis, JJ, Hicks, GM. Lack of association between prehospital response times and patient outcomes. Prehosp Emerg Care. 2009;13(4):444-450.
11. Stirling, CM, O’Meara, P, Pedler, D, Tourle, V, Walker, J. Engaging rural communities in health care through a paramedic expanded scope of practice. Rural Remote Health. 2007;7(839):1-9.
12. Woollard, M. The role of the paramedic practitioner in the UK. J Emerg Prim Heal Care. 2006;4(1):1-9.
13. O’Meara, PF, Tourle, V, Stirling, C, Walker, J, Pedler, D. Extending the paramedic role in rural Australia: a story of flexibility and innovation. Rural Remote Health. 2012;12(1978):1-13.
14. Cooper, S, Grant, J. New and emerging roles in out of hospital emergency care: a review of the international literature. Int Emerg Nurs. 2009;17(2):90-98.
15. Spaite, DW, Maio, RF, Garrison, HG, et al. Emergency Medical Services Outcomes Project ( EMSOP ) II: developing the foundation and conceptual models for out-of-hospital outcomes. Ann Emerg Med. 2001;37(6):657-663.
16. Garrison, HG, Maio, RF, Spaite, DW, et al. Emergency Medical Services Outcomes Project III (EMSOP III): the role of risk adjustment in out-of-hospital outcomes research. Ann Emerg Med. 2002;40(1):79-88.
17. Maio, RF, Garrison, HG, Spaite, DW, et al. Emergency Medical Services Outcomes Project (EMSOP) IV: pain measurement in out-of-hospital outcomes research. Ann Emerg Med. 2002;40(2):172-179.
18. Siriwardena, AN, Shaw, D, Donohoe, R, Black, S, Stephenson, J. Development and pilot of clinical performance indicators for English ambulance services. Emerg Med J. 2010;27(4):327-331.
19. Mainz, J. Defining and classifying clinical indicators for quality improvement. Int J Qual Heal Care. 2003;15(6):523-530.
20. McGlynn, E, Asch, SM. Developing a clinical performance measure. Am J Prev Med. 1998;14(3 Suppl):14-21.
21. Rubin, HR, Pronovost, P, Diette, GB. From a process of care to a measure: the development and testing of a quality indicator. Int J Qual Health Care. 2001;13(6):489-496.
22. Ballard, DJ. Indicators to improve clinical quality across an integrated health care system. Int J Qual Health Care. 2003;15(Suppl 1):i13-i23.
23. Arksey, H, O’Malley, L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19-32.
24. Levac, D, Colquhoun, H, O’Brien, KK. Scoping studies: advancing the methodology.
25. National Quality Forum. Measures, Report, and Tools.
26. Agency for Healthcare Research and Quality. AHRQ Web site. https:/
27. National Quality Measures Clearinghouse. National Quality Measures Clearinghouse Web site.
28. Donabedian, A. The quality of care. How can it be assessed? JAMA. 1997;260(12):1743-1748.
29. Norris, RM. A new performance indicator for acute myocardial infarction. Heart. 2001;85(4):395-401.
30. Sobo, E, Andriese, S, Stroup, C, Morgan, D, Kurtin, P. Developing indicators for EMS evaluation and quality improvement. Jt Comm J Qual Improv. 1992;27(3):138-154.
31. Grudzen, CR, Liddicoat, R, Hoffman, JR, Koenig, W, Lorenz, KA, Asch, SM. Developing quality indicators for the appropriateness of resuscitation in prehospital atraumatic cardiac arrest. Prehosp Emerg Care. 2007;11(4):434-442.
32. Daudelin, DH, Kulick, ER, Amore, KD, Lutz, JS, Barrientos, MT, Foell, K. The Massachusetts Emergency Medical Service Stroke Quality Improvement Collaborative 2009 - 2012. Prev Chronic Dis. 2013;10.
33. Rosengart, MR, Nathens, AB, Schiff, M. The identification of criteria to evaluate prehospital trauma care using the Delphi technique. J Trauma. 2007;62(March):708-713.
34. Santana, MJ, Stelfox, HT. Development and evaluation of evidence-informed quality indicators for adult injury care. Ann Surg. 2014;259(1):186-192.
35. Patterson, PD, Lave, JR, Martin-Gill, C, et al. Measuring adverse events in helicopter Emergency Medical Services: establishing content validity. Prehosp Emerg Care. 2013;18(1):35-45.
36. Stelfox, HT, Bobranska-Artiuch, B, Nathens, A, SE, S. A systematic review of quality indicators for evaluating pediatric trauma care. Crit Care Med. 2010;38(4):1187-1196.
37. Hoogervorst, EM, van Beeck, EF, Goslings, JC, Bezemer, PD, Bierens, JJLM. Developing process guidelines for trauma care in the Netherlands for severely injured patients: results from a Delphi study. BMC Health Serv Res. 2013;13(1):79.
38. Oostema, JA, Nasiri, M, Chassee, T, Reeves, MJ. The quality of prehospital ischemic stroke care: compliance with guidelines and impact on in-hospital stroke response. J Stroke Cerebrovasc Dis. 2014;23(10):2773-2779.
39. Myers, JB, Slovis, CM, Eckstein, M, et al. Evidence-based performance measures for Emergency Medical Services systems: a model for expanded EMS benchmarking - Myers benchmarking in EMS. Prehosp Emerg Care. 2008;12(1):141-151.
40. Valenzuela, TD, Spaite, DW, Meislin, HW, Clark, LL, Wright, AL, Ewy, GA. Emergency vehicle intervals versus collapse-to-CPR and collapse-to-defibrillation intervals: monitoring Emergency Medical Services system performance in sudden cardiac arrest. Ann Emerg Med. 1993;22(11):1678-1683.
41. Colwell, C, Mehler, P, Harper, J, Cassell, L, Vazquez, J, Sabel, A. Measuring quality in the prehospital care of chest pain patients. Prehosp Emerg Care. 2009;13(2):237-240.
42. Greenberg, MD, Garrison, HG, Delbridge, TR, et al. Quality indicators for out-of-hospital Emergency Medical Services: the paramedics’ perspective. Prehosp Emerg Care. 1997;1(1):23-27.
43. O’Meara, P. A generic performance framework for ambulance services: an Australian health services perspective. J Emerg Prim Heal Care. 2005;3(3.
44. Stelfox, HT, Straus, SE, Nathens, A, Bobranska-Artiuch, B. Evidence for quality indicators to evaluate adult trauma care: a systematic review. Crit Care Med. 2011;39(4):846-859.
45. Dunford, J, Domeier, RM, Blackwell, T, et al. Performance measurements in Emergency Medical Services. Prehosp Emerg Care. 2002;6(1):107-113.
46. Nakayama, DK, Saitz, EW, Gardner, MJ, Kompare, E, Guzik, E, Rowe, MI. Quality assessment in the pediatric trauma care system. J Pediatr Surg. 1989;24(2):159-162.
47. Gitelman, V, Auerbach, K, Doveh, E. Development of road safety performance indicators for trauma management in Europe. Accid Anal Prev. 2013;60:412-423.
48. Patterson, PD, Probst, JC, Moore, CG. Expected annual emergency miles per ambulance: an indicator for measuring availability of Emergency Medical Services resources. J Rural Health. 2006;22(2):102-111.
49. Greenberg, MD, Garrison, HG, Delbridge, TR, Miller, WR, Mosesso, VN, Roth, RN, Paris, PM. Quality indicators for out-of-hospital Emergency Medical Services: the paramedics’ perspective. Prehosp Emerg Care. 1997;1(1):23-27.
50. Bevan, G, Hamblin, R. Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK. J R Stat Soc Ser A Stat Soc. 2009;172(1):161-190.
51. Nakayama, DK, Gardner, MJ, Waggoner, T. Audit filters in quality assurance in pediatric trauma care. J Pediatr Surg. 1993;28(1):19-25.
52. Willis, CD, Gabbe, BJ, Cameron, PA. Measuring quality in trauma care. Injury. 2007;38(5):527-537.
53. Spaite, DW, Valenzuela, TD, Meislin, HW, Criss, E, Hinsberg, P. Prospective validation of a new model for evaluating Emergency Medical Services systems by in-field observation of specific time intervals in prehospital care. Ann Emerg Med. 1993;22(4):638-645.
54. Pozner, CN, Zane, R, Nelson, SJ, Levine, M. International EMS systems: the United States: past, present, and future. Resuscitation. 2004;60(3):239-244.
55. Roudsari, BS, Nathens, AB, Arreola-Risa, C, et al. Emergency Medical Service (EMS) systems in developed and developing countries. Injury. 2007;38(9):1001-1013.
56. Adnet, F, Lapostolle, F. International EMS systems: France. Resuscitation. 2004;63(1):7-9.
57. Lindström, V, Bohm, K, Kurland, L. Prehospital care in Sweden. Notfall + Rettungsmedizin. 2015;18(2):107-109.
58. O’Brien, J, Shortell, S, Hughes, E, et al. An integrative model for organization wide QI - lessons from the field. Qual Manag Healthc. 1995;3(4):19-30.
59. Øvretveit, J, Gustafson, D. Evaluation of quality improvement programmes. Qual Saf Health Care. 2002;11(3):270-275.
60. Margolis, P, Provost, LP, Schoettker, PJ, Britto, MT. Quality improvement, clinical research, and quality improvement research-opportunities for integration. Pediatr Clin North Am. 2009;56(4):831-841.
61. Perla, RJ, Provost, LP, Murray, SK. Sampling considerations for health care improvement. Qual Manag Health Care. 2014;23(4):268-279.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Prehospital and Disaster Medicine
  • ISSN: 1049-023X
  • EISSN: 1945-1938
  • URL: /core/journals/prehospital-and-disaster-medicine
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score