Introduction
The prehospital assessment is crucial in the mobilization of receiving teams and in-hospital resources for trauma,Reference Anantha, Painter and Diaz-Garelli1,2 stroke,Reference Acker Iii, Pancioli and Crocco3,Reference McKinney, Mylavarapu, Lane, Roberts, Ohman-Strickland and Merlin4 or cardiac catheterization.Reference Kurz and Rozas5 Vital signs are an essential component in the care of the undifferentiated patient in the prehospital setting, especially when there is an influx of multiple patients where accurate triage is of upmost importance. Triage algorithms based on vital signs could be incorporated within early warning systems to identify patients at the greatest risk of clinically important outcomes, such as mortality or need for intensive care unit (ICU) admission.Reference Dorsett, Kroll, Smith, Asaro, Liang and Moy6–Reference Raux, Sartorius, Le Manach, David, Riou and Vivien10 The identification of patients with critical illness in the prehospital setting using structured notification mechanisms can help in promptly delivering treatment after emergency department (ED) arrival and optimize the use of hospital-based resources, both in daily operations and during mass-casualty events.
The are opportunities to improve the use of vital signs by deriving ranges linked to clinically important outcomes. There is a paucity of knowledge about the relationship between vital signs and the provision of prehospital life-saving interventions (LSIs). A strategy incorporating the association of prehospital vital signs with risk for the provision of prehospital LSIs could improve the accuracy of evaluating patients in need of advanced care prior to or immediately following arrival to the ED. Prior work, for example, has suggested that vital signs are important in trauma triage,Reference Newgard, Fischer and Gestring11 in the identification of respiratory distress or failureReference Brown and Prasad12 and hemorrhagic shock,Reference Liu, Holcomb, Wade and Salinas13 all of which may warrant the performance of an LSI by Emergency Medical Services (EMS). One prior study suggested that prehospital vital signs are predictive of the receipt of an LSI among injured patients,Reference Holcomb, Salinas, McManus, Miller, Cooke and Convertino14 with another failing to demonstrate a clinically important association between these.Reference Liu, Holcomb, Wade and Salinas15 If a meaningful association of vital signs with prehospital interventions can be broadly established, this may be leveraged to generate clinically meaningful and maximally predictive data-driven cut points for the interpretation of vital signs by EMS clinicians.
The goal of this study was to develop cut points for vital signs among adults transported to the hospital by EMS based on their association with prehospital LSIs.
Methods
Study Dataset
A retrospective, cross-sectional study of EMS encounters from the 2022 National Emergency Medical Services Information System (NEMSIS; Salt Lake City, Utah USA; version 3.4.0) was performed; NEMSIS is a comprehensive national collection of patient care reports from United States’ EMS agencies. The 2022 dataset included information on EMS responses from 13,946 EMS agencies across 54 states and territories. Because of the use of publicly available, de-identified data, this study was considered as nonhuman subjects research by the Ann & Robert H. Lurie Children’s Hospital of Chicago Institutional Review Board (Chicago, Illinois USA).
Study Population
The target population of interest was adults (≥18 years) in the 2022 NEMSIS dataset transported from the scene to the hospital. The following exclusions were made: encounters without documented age, encounters that did not originate as 9-1-1 calls, and encounters not cared for by an Advanced Life Support (ALS) or critical care unit. The latter exclusion was applied on the basis that most of the outcome measures for LSI were out of the scope of practice for a Basic Life Support (BLS) clinician.
Data Extraction
From the dataset, demographic details including encounter age, sex, urbanicity, census region, dispatch complaint, vital signs, medications, and procedures were extracted. Dispatch complaint was extracted using pre-defined codes provided by NEMSIS.16
Outcomes
The primary outcome was performance of a prehospital LSI, which are outlined in Table 1. The term LSI is intended to capture interventions that, when performed according to standard clinical guidelines, may “reverse a critical condition or rapidly improve hemodynamic stability.”Reference Jarvis, Hamilton, Taigman and Brown17 The definition of LSI was adapted from prior work reported by Jarvis, et alReference Jarvis, Hamilton, Taigman and Brown17 within the constraints of the provided data elements in NEMSIS. As it was not anticipated that vital signs would have substantial merit as a primary predictor of ST-elevation myocardial infarction, stroke, or vaginal deliveries, they were not incorporated into the definition of LSI. As secondary outcomes, LSI types were categorized into eleven categories based on the clinical syndrome targeted by the LSI, as follows: tachydysrhythmia, cardiac arrest, airway, seizure/sedation, toxicologic, bradycardia, foreign body removal, vasoactive medication, hemorrhage control, needle decompression, and hypoglycemia. The occurrence of LSI was considered as a dichotomous outcome variable.
Criteria Used Within the Outcome of Life-Saving Intervention (LSI)
Exposures
The following predictors were evaluated: heart rate (HR), respiratory rate (RR), systolic blood pressure (SBP), oxygen saturation, and Glasgow Coma Scale (GCS). The first vital sign charted for each encounter was used, based on the rationale that these are likely to be most useful in shaping EMS assessments and are representative of a patient’s clinical status prior to prehospital interventions, including the need to call for an additional EMS unit, a higher level of EMS care for performance of an LSI, or guidance from medical oversight. Additionally, these may be potentially integrated into predictive models to help identify higher-risk EMS encounters.
Analysis
Encounter characteristics were described. Encounters were then divided into training (75%) and test (25%) partitions. For HR, RR, and SBP, cut point analyses were performed twice: first among encounters with a vital sign less than the overall population median to identify an optimal low measure for that vital sign (eg, bradycardia), then second among encounters with a vital sign above the population median (eg, tachycardia). Cut point selection techniques were used to prioritize specificity, keeping sensitivity constrained to at least 25%. An explicit target for specificity was not specified. The rationale behind this approach was to prevent an overly sensitive threshold from generating excessive false positive alerts, which would reduce the overall utility of the alert system. By establishing a minimum sensitivity requirement of 25%, a meaningful number of true positive cases was ensured to be captured, while allowing specificity to be maximized within that constraint. This approach helped strike a balance between detecting important cases and minimizing unnecessary alerts, thereby improving the practical usability of the system.Reference Amaral18
Using both the training and the test partitions, the association of the developed cutoffs for the occurrence of an LSI overall and within each of the 14 defined subgroups was evaluated. The following measures of diagnostic accuracy were described: sensitivity, specificity, positive predictive value (PPV), negative predictive value (NVP), and positive and negative likelihood ratios. To evaluate the additive role of vital sign abnormalities in predicting LSI, a multivariable prediction model using the five studied measures when using the established cut points was developed, with findings described using odds ratios (OR) with 95% confidence intervals (CI). Besides other vital signs, no other variables were incorporated into the model. The analysis was limited only to complete case data. Findings were described using the area under the receiver operator characteristic curve (AUROC) for the overall outcome of LSI, and for each individual component of this outcome measure. The AUROCs were categorized as follows: >0.90, high accuracy; 0.70-0.90, moderate accuracy; and 0.50-0.70, low accuracy.Reference Fischer, Bachmann and Jaeschke19 Constructed calibration plots were developed to visualize the relationship between predicted probabilities and observed outcomes. Analyses were performed using the rms (v6.7-0),Reference Harrell20 cutpointr (v.1.1.2),Reference Thiele and Hirschfeld21 and epiR (v2.0.62)Reference Stevenson, Nunes and Sanchez22 packages in R, version 4.3.2 (R Foundation for Statistical Computing; Vienna, Austria).
Results
Study Inclusion
A total of 53,179,492 encounters were included within the 2022 NEMSIS dataset. After excluding encounters with a missing age (n = 7,952,109), children (n = 3,078,979), encounters not resulting from a 9-1-1 call (n = 9,891,511), inter-facility transports (n = 653,215), those that did not result in hospital transfer (n = 9,466,202), and BLS calls (n = 3,877,710), a total of 18,259,766 encounters were included. The median age was 63 years (IQR 44-77 years) and 51.8% were female. Characteristics of the study sample, both overall and stratified by training and test partitions, are provided in Table 2. The most documented vital signs were HR (99.4%) and SBP (98.2%). Complete vital signs data were available in 87.4% of cases.
Sample Demographics, Vital Signs Availability, and Outcomes
Note: Overall and stratified into test and training samples. Numbers in cells represent N (%) or median [interquartile range].
Abbreviations: AICD, automated implantable defibrillator; CVA, cerebrovascular accident; HR, heart rate; RR, respiratory rate; SBP, systolic blood pressure; LSI, life-saving intervention.
Outcomes
An LSI was documented in 6.3% of encounters. The most common LSIs were for airway (2.2%), toxicologic interventions (1.6%), and cardiac arrest (1.1%). Proportions of study interventions were similar between the training and test samples.
Identification of Optimal Vital Signs
Vital sign ranges obtained after applying cut point criteria are provided in Table 3. When applied to the test sample, the sensitivity was highest with GCS (49.5%; Table 4). The specificity of individual vital signs exceeded 85% for all measures. When using individual outcome measures (Supplementary Tables 1-5; available online only), the performance of vital signs criteria for each varied widely. The PPV for HR was highest for cardiac arrest and use of vasoactive medications. The PPV for RR was highest for airway management and use of vasoactive medications. The PPV for SBP was highest for airway management and cardiac arrest. The PPV for oxygen saturation and GCS were highest for airway management and toxicologic emergencies.
Vital Sign Ranges Identified Using the Youden Approach Derived from the Training Partition
Accuracy of the Vital Sign Outside of Normal Ranges for Predicting an Outcome of LSI in the Test Partition
Abbreviations: Sens, sensitivity; Spec, specificity; PPV, positive predictive value; NPV, negative predictive value; LR (+), positive likelihood ratio; LR (-), negative likelihood ratio; LSI, life-saving intervention.
Combination of Vital Signs
Among encounters in the test sample with complete vital signs data (n = 3,987,838), a total of 1,430,260 had at least one abnormal vital sign (35.9%). For an outcome of LSI, the presence of any abnormal vital sign had a sensitivity of 75.1%, specificity of 66.6%, PPV of 12.5%, NPV of 97.7%, positive likelihood ratio of 2.3, and negative likelihood ratio of 0.4. The highest univariable OR for an LSI was for a low RR and abnormal GCS (Table 5). In multivariable models, all vital sign cutoffs were associated with an LSI, with the greatest effect size noted with a low RR (OR = 10.83; 95%CI, 10.46-11.20) and a low GCS (8.77; 95%CI, 8.69-8.86). The AUROC for this outcome in the test sample was moderate (0.78; 95%CI, 0.78-0.78; Table 6). The visualized calibration curve for this model demonstrated a tendency for an LSI to occur less frequently at higher predicted probabilities of the outcome (Supplementary Figure 1; available online only). This AUROC varied widely among the specific outcome measures. The AUROC for all subgroups were classified as moderate, except for foreign body removal, hypoglycemia, and hemorrhage, which were low.
Univariable and Multivariable Odds of LSI Using All Vital Signs for Encounters Within the Test Partition
Abbreviation: LSI, life-saving intervention; OR, odds ratio; CI, confidence interval.
Performance of the Model for LSI, Overall and Within Each Secondary Outcome in the Test Partition
Abbreviation: LSI, life-saving intervention; AUROC, area under the receiver operator characteristic curve.
Discussion
A retrospective analysis of a large, nationally representative EMS database was performed to evaluate the association of prehospital vital signs with LSI among 18.2 million adult EMS transfers to the hospital, resulting in the development of data-driven cut points for HR, RR, SBP, oxygen saturation, and GCS. These had moderate discrimination for the identification of LSI, with an AUROC of 0.78 when combined within a multivariable logistic regression model. The specific performance of these cut points varied based on the individual outcome. These findings can inform future work focused on the development of prehospital protocols for the identification of patients at higher risk of critical illness and injury. Enhanced prehospital triage and notification of patients with high or low likelihood of needing an LSI can be beneficial to 9-1-1 systems, receiving hospitals, and especially during mass-casualty and disaster events to optimize the use of on-scene and hospital-based resources. Despite achieving moderate performance, these findings underscore the limitations of relying solely on vital signs to assess the risk of out-of-hospital emergencies requiring an LSI.
These findings expand upon prior knowledge evaluating the association between prehospital vital signs and LSI by evaluating non-traumatic emergent conditions. These findings increase the external validity of prior work using a larger sample size. A retrospective single-center study of injured patients in Texas, USA demonstrated that vital signs, which included radial pulse character in categories of absent, weak, normal, motor, and eye components of GCS, and SBP, predicted prehospital LSI with an excellent performance (AUROC of 0.97 among 381 patients).Reference Holcomb, Salinas, McManus, Miller, Cooke and Convertino14 In another study of 195 injured adults, the investigators compared the vital signs of HR, SBP, shock index, pulse pressure, and GCS, with findings suggesting that vital signs poorly differentiated patients who did and did not receive an LSI.Reference Liu, Holcomb, Wade and Salinas15
While vital signs, when combined, demonstrated moderate performance for the prediction of LSI, limitations remain when considering these in a clinical context. No cut point of vital signs demonstrates adequate performance for their use in isolation, highlighting the importance of considering these together and in combination with other clinical factors. Nevertheless, some important inferences can be made based on the findings from this study. The sensitivity and PPV of receiving LSI were the highest with GCS and RR, respectively. This suggests that alteration in consciousness and respiratory status is more predictive of the need for LSI than other vital signs. Similar findings have also been reported in sepsis screening criteria.Reference Singer, Deutschman and Seymour23 These results are consistent with prior work that evaluated the association of vital signs with clinical deterioration in the in-hospital setting, defined as the composite outcome of cardiac arrest, transfer to ICU, and in-hospital mortality.Reference Churpek, Adhikari and Edelson24 One multi-center cohort study including approximately 270,000 in-patients at five hospitals in Chicago (Illinois USA) found that RR had the largest AUROC for clinical deterioration among HR, systolic and diastolic blood pressure, oxygen saturation, and temperature, although the study did not include GCS or other consciousness measurement as one of predictors.Reference Churpek, Adhikari and Edelson24 Other work has demonstrated a strong independent association for GCS measured in the prehospital setting and the ED with clinically important patient outcomes.Reference Sartorius, Le Manach and David8,Reference Udekwu, Kromhout-Schiro, Vaslef, Baker and Oller25–Reference Bautista, Lenhardt and Yang27
The present study underscores both the potential utility and inherent limitations of data-driven vital sign cut points in the prehospital setting. The use of empirically derived cut points based upon the association with LSI represents an advancement over conventional vital sign ranges. To create vital sign cut points that are broadly meaningful, these were derived among patients considering many potential LSIs and by using a broad inclusion approach. In the multivariable model, all empirical cut points of vital signs were associated with an LSI. The AUROC for the overall outcome of LSI was categorized as moderate, with performance for individual vital signs varying more broadly. As measures that are used primarily to screen for patients with instability, vital signs should never be the sole means to identify patients at higher risk, but the differentiation between patients with and without abnormal vital signs can be optimized beyond their current application to enhance their predictive value. Optimally derived vital signs cutoffs demonstrated substantial tradeoffs between measures of diagnostic accuracy. Vital sign cut points favoring a higher specificity were preferred for this investigation, considering the potential use of these to identify the need for additional resources at a scene or for hospital-based alert systems to activate resource availability immediately upon arrival. Both of these potential uses represent resources with limited availability, thus a lower false positive rate would be beneficial for real-world applicability. Furthermore, while the derived cut points are informative, they are not definitive predictors of the provision of subsequent LSI. This is reflective of the complex nature of prehospital care, where a multitude of factors beyond vital signs influence the decision to perform an LSI. As a research implication, these findings indicate that further understanding of the clinical decision of EMS clinicians to carry out LSI is warranted, potentially using mixed methods approaches.
These findings indicate several future research directions. Firstly, future research should focus on prospective validation of the derived cut points of vital signs in diverse prehospital settings and evaluate their impact on patient outcomes when used as a part of decision support tools. Secondly, the integration of these cut points of vital signs with other clinical data, such as presenting clinical conditions or complaints, could be explored to enhance the predictive power for identifying need for LSIs. Thirdly, further efforts are needed to identify the optimal method to measure vital signs in the prehospital setting. A prior study in 2014 reported that wireless, portable vital sign monitor systems showed better prediction for LSI for trauma patients compared to standard vital sign monitors.Reference Liu, Holcomb, Wade, Darrah and Salinas28 Given advances since the study period, further studies using current information technology would be important to develop an innovative method to measure prehospital vital signs. Additionally, future research may consider the potential utility of serial vital signs in detecting patient deteriorationReference Churpek, Adhikari and Edelson24 or evaluate the role of vital sign variability in predicting clinical deterioration. One study, for example, demonstrated an association with HR complexity when used as a continuous measurement and the need for an LSI among injured helicopter-transported patients.Reference Cancio, Batchinsky and Salinas29
Limitations
The findings from this study are subject to limitations. The NEMSIS dataset was built using retrospective data. As with all registry studies, data integrity, validity, and ascertainment bias are potential limitations. However, use of the national-representative dataset with standardizing and aggregating EMS responses was intended to minimize this limitation. The analysis was limited by the use of vital signs as the predictor. It is possible that trends of vital signs might describe trajectory of physiologic status and could predict LSIs more accurately.Reference Churpek, Adhikari and Edelson24 This analysis was limited to the first vital signs because of their broad availability, their presence prior to most study interventions, and because this measure is most frequently used in prior work.Reference Ramgopal, Horvat, Macy, Cash, Sepanski and Martin-Gill30–Reference Lindskou, Ward, Søvsø, Mogensen and Christensen32 These findings may also be subject to confounding by indication, as clinicians were not blinded to vital signs and likely incorporated them into their decision making, making it difficult to disentangle the independent effect of vital signs from the decision to perform an LSI. In some cases, interventions may have been initiated before vitals were recorded, potentially influencing this analysis. Finally, there is no existing consensus on LSI and its definition varies across studies.Reference Holcomb, Salinas, McManus, Miller, Cooke and Convertino14,Reference Liu, Holcomb, Wade and Salinas15,Reference Jarvis, Hamilton, Taigman and Brown17 Despite these limitations, the findings from this work represent an important advancement in the use of prehospital vital signs in the identification of higher-risk adults transported to the hospital by EMS.
Conclusion
A large, nationally representative registry of prehospital encounters was used to identify data-driven cut points for prehospital vital signs based on an outcome of LSI. Empirically derived cut points for HR, RR, SBP, oxygen saturation, and GCS moderately discriminated the need for LSI, with a combined model demonstrating an AUROC of 0.78 in a test dataset. The study suggests that alterations in consciousness and respiratory status were the most important predictors of LSIs and may inform the development of prehospital protocols for identifying patients at risk.
Conflicts of interest
The authors have no competing interests to disclose.
Supplementary Materials
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X25001542
Open access
