Introduction
The Institute of Medicine (National Academy of Medicine; Washington, DC USA) identified clinical decision support (CDS) tools as vital to the electronic health record (EHR) in 2003. Reference Mills1 Clinical decision support is a set of tools to support clinical decision making and may include alerts, guidelines, order sets, templates, and summaries. Ideally, knowledge and data can be generated and made available to clinicians during care delivery. 2 Ultimately, CDS tools should be developed using evidence-based clinical practice guidelines (CPGs), when available. Evidence demonstrates the use of CPGs leads to improved patient care and outcomes, and subsequently, to increased survival rates and reduced cost. Reference Gurney and Miller3 However, despite the evidence showing improved outcomes, adherence to CPGs in practice is lower than expected. Reference Shaloma, Shahara, Parmetb and Lunenfeld4 Potential barriers to adoption include alert fatigue, Reference Khalifa and Zabani5 poor-quality evidence base for CDS creation, Reference Sutton, Pincock, Baumgar, Sadowski, Fedorak and Kroeker6–Reference Sim, Gorman and Greenes8 and user acceptance and integration into existing workflows. Reference Alexiuk, Elgubtan and Tangri9–13 Increasing compliance with CPGs through the use of CDS is one method to achieve these improved outcomes. The use of CDS can “significantly impact improvements in quality, safety, efficiency, and effectives of health care.” 2 Shalom, et al evaluated the effect of CDS on the correctness and completeness of clinicians’ use of a CPG to guide care. Performance of the CPG-recommended actions increased from 41% to 93% when CDS was used. Reference Shaloma, Shahara, Parmetb and Lunenfeld4 The authors concluded CDS has the potential to decrease errors of omission and redundant actions. Reference Shaloma, Shahara, Parmetb and Lunenfeld4 A CDS tool designed to communicate with the electronic patient record and work with systems already in place has significant potential to improve the quality and timeliness of patient care. Reference Mills1
Military medicine defines en route care as the “provision of care during patient movement between the roles of care,” 14 which includes both ground and rotary or fixed-wing air transports. Projections for air transport requirements in future multi-domain operating environments include movement of larger casualty volumes compared to recent conflicts. Currently, Critical Care Air Transport (CCAT) team capabilities support movement of three high-acuity patients or six stabilized lower-acuity patients. This maximum patient load is based on the current operating environment where CCAT teams utilize paper-based charting and patient monitors which lack connectivity to documentation or CDS systems. The use of real-time CDS integrated with patient monitors is one possible technological solution to increase the decision-making capacity of en route critical care teams and enable these teams to safely transport larger patient volumes. Specifically, CDS can provide cognitive offload to military clinicians in austere medical environments by simplifying workflows for complex data analysis and standardized care algorithms in the setting of competing medical and operational priorities. Reference Worsham, Gonzalez and Kucia15 This differs from usual care (UC) for military transport teams utilizing Joint Trauma System (JTS) Clinical Practice Guidelines and handbooks. These materials, available in paper or PDF formats, include specialized cognitive aids developed for various transport platforms and mission environments. Reference Denger, McCafferty and Neal16–Reference Burkhardt, Cheney and Leib18 The CPGs provide a tiered intervention strategy for addressing elevated intracranial pressure (ICP) during transport, and military providers are trained according to these guidelines.
The Battlefield Assisted Trauma Distributed Observation Kit (BATDOK; Air Force Research Laboratory [AFRL]; Wright-Patterson Air Force Base, Ohio USA) is a government, off-the-shelf mobile software solution, which is the software tool for patient documentation and care delivery at point-of-injury and en route care. 19 The BATDOK can wirelessly and autonomously aggregate streaming vitals from multiple patient monitors (eg, the Zoll Propaq [Zoll Medical Corp.; Chelmsford, Massachusetts USA]) and populate documentation forms. The BATDOK also has the capability to customize patient profiles with early warning notification preset to specific hemodynamic parameters and integrate CDS in the form of suggested tasks based on current CPGs.
The purpose of this study is to evaluate the impact of a BATDOK with the traumatic brain injury (TBI) CDS tool adapted from the JTS Clinical Practice Guideline for TBI and neurosurgery Reference Denger, McCafferty and Neal16 on brain injury guideline adherence during simulated en route critical care air transport and to evaluate users’ perceptions of the tool. This guideline applies to combat casualties with moderate to severe TBI, defined as a Glasgow Coma Scale of 12 or less; TBI was chosen because CPGs provide clearly defined hemodynamic and laboratory parameters to manage increased ICP in a tiered management approach. It was hypothesized that the use of BATDOK with TBI CDS would improve adherence to current CPGs for simulated patients with severe TBI and that users would evaluate the tool favorably.
Methods
Study Design
This study was approved by the 59th MDW Institutional Review Board (IRB; Joint Base San Antonio-Lackland, Texas USA): FWH20240015E.
A prospective, randomized, cross-over study was conducted with simulated en route critical care missions consisting of twelve, two-provider teams caring for two critically injured combat casualties. Twenty-four participants were recruited for this study. Each two-provider team completed two simulations, one simulation utilizing CDS and one simulation employing UC practices.
Participants
Twenty-four en route critical care eligible United States military service members were recruited to participate in this study. Eligibility criteria for participants included active clinical practice in a specialty eligible for en route care, including physicians, nurses, and technicians. Military en route care has broad eligibility for physician specialties such as emergency medicine, surgery, and critical care, as well as for nurses and respiratory therapists who deliver emergency and critical care in garrison settings. Exclusion criteria included physical limitations causing inability to perform care tasks in a simulated scenario.
Simulation Protocol
Participants were paired into teams of two before providing care to two simulated combat casualties in a simulation room modeled after a fixed-wing aircraft used for en route CCAT missions. The simulated care setting was the receipt of two combat casualties from a forward facility where they had undergone damage control resuscitation and surgery. Care delivery in the simulation focused exclusively on in-flight management. Care teams completed two sequential simulations with each simulation spanning a 10-minute fixed duration. Prior to each simulation, the team was given a brief patient report to summarize the care received prior to air transport and to simulate a standard patient handoff when transferring care from one provider to another. The first simulated casualty had a severe TBI, meeting criteria for use of the JTS CPG for TBI. The second simulated patient had polytrauma from an improvised explosive device blast injury with clinical concern for a wound infection in the pre-scenario handover. Participants provided care to both patients to model a multi-provider and multi-casualty transport; however, the study outcomes focused on interventions to the TBI patient. In both simulations, the TBI patient experienced elevated ICP requiring intervention by the team. Of note, the simulations were designed such that even if teams performed correct interventions to the simulated TBI patient, the ICP would remain elevated. Persistent elevated ICP is not uncommon in actual patients with severe TBI, thus the simulation scenarios were designed to force providers to continually reassess and implement additional measures to treat an on-going or worsening elevation in ICP. Adherence to the JTS CPG would result in tiered treatments culminating in the most aggressive interventions at conclusion of the simulation. After the first simulation scenario, participants exited the simulation room, were informed that two hours of uneventful flight had passed, and then returned to the simulation room to continue care for the same two patients. Participants were informed that patient physiologic changes and responses to interventions would be accelerated to evaluate medical decision making. The scenario duration was kept relatively short to accommodate the limited time military clinicians had available to participate. Additional scenario details are available as a Supplemental File.
The research team developed a passive decision-tree style TBI CDS tool within BATDOK based on JTS CPG for Neurosurgery and Severe Head Injury. This JTS CPG mirrors civilian management guidelines in which interventions are tiered and should be performed in a certain order. Reference Ratcliff, Morrison and Tran20 The TBI CDS was adapted from the CCAT cognitive aid cards and provided a stepwise if/then flowchart for recommended next steps to manage elevated ICP (Figure 1a). Reference Burkhardt, Cheney and Leib18
(A) TBI CDS Flowchart for Steps to Manage Elevated ICP and (B) Printed Reference Materials Currently Used During En Route Care Missions.
Note: 1A represents information available via the BATDOK TBI CDS tool, while 1B represents information available during Usual Care (UC).
Abbreviations: TBI, traumatic brain injury; UC, usual care; CDS, Clinical Decision Support; ICP, intracranial pressure; BATDOK, Battlefield Assisted Trauma Distributed Observation Kit.
Figure 1 Long description
Panel A: TBI CDS Flowchart for Steps to Manage Elevated ICP. The flowchart starts with monitored traumatic brain injury (TBI) and checks if intracranial pressure (ICP) is greater than 22. If yes, it proceeds to level and zero the ICP monitor. If the monitoring device is not at the appropriate level and zeroed, it instructs to level and zero the ICP monitor. If the head of the bed is not at 30 degrees and the C-collar is not positioned appropriately, it instructs to place the head of the bed at 30 degrees and adjust the C-collar. If the patient does not have adequate pain control and sedation, it instructs to give analgesia and sedation. If the patient does not have an extraventricular drain (EVD), it instructs to drain cerebrospinal fluid (CSF) as per neurosurgery (NSRG) recommendations. If the EVD is set to drain or the maximum drainage has been reached, it instructs to drain EVD per NSRG recommendations and recheck. If a 3 percent saline bolus has not been administered within the last hour, it instructs to give a 250 cc bolus of 3 percent NS and start the 3 percent NS drip at 50 cc per hour. If cerebral perfusion pressure (CPP) is not greater than 60, it instructs to treat hypovolemia if present and give norepinephrine. If neuromuscular blockade (paralysis) has not been administered, it instructs to give vecuronium 10 mg. Panel B: Printed Reference Materials Currently Used During En Route Care Missions. The reference materials include guidelines for ensuring euvolemia, treating intracranial hypertension, and general management of severe head injury. It specifies the use of blood products and normal saline, vasopressors, and protocols for managing intracranial hypertension and cerebral hypoperfusion.
Teams were randomized to whether they completed the first simulation as control group (UC) or treatment group (CDS). Teams then crossed over to the other group for the second scenario. The crossover design was chosen to allow each team to serve as its own control, maximizing the statistical power of the sample and ensuring robust comparisons without needing more participants. Usual care teams utilized standard patient care equipment and CCAT decision aids based on current practice patterns of the participants (ie, their own experience and printed reference materials currently used during en route care missions; Figure 1b). The CDS group had standard patient care equipment, as well as a tablet with BATDOK software containing the TBI CDS system developed by the research team.
Outcome Measures
The primary outcome was percentage of completion of critical actions by simulation conclusion. The secondary outcome was the percentage of interventions that were performed in the correct order. Exploratory outcomes included time to completion of critical actions, whether interventions followed the recommended tiered approach as outlined in the JTS CPG (and BATDOK with the TBI CDS, if available to them), and participant evaluation of tool usability.
Data Collection
During simulations, trained observers identified and timed when teams completed individual critical actions needed to address elevated ICP, as defined by JTS CPG for Neurosurgery and Severe Head Injury. Reference Denger, McCafferty and Neal16 Based on the clinical scenario and simulated patient conditions, critical actions were as follows: (1) level and zero ICP monitor; (2) adjust cervical collar; (3) administer analgosedation; (4) drain cerebrospinal fluid (CSF) from the extra-ventricular drain (EVD); (5) administer 3% hypertonic saline (HTS) bolus followed by continuous infusion; and (6) administer neuromuscular blockade/paralytic agent. If an intervention was never completed by the team, it received a time correlating with the end of simulation (600 seconds/ten minutes). Because the second simulation represented a continuation of the first, not all scenarios required completion of all steps, such as adjustment of the cervical collar.
Participant feedback regarding the use of BATDOK was collected by administering a brief questionnaire after participating in the simulation, which included the System Usability Scale (SUS), a validated subjective Likert scale commonly utilized to assess usability of technology solutions. Reference Hyzy, Bond and Mulvenna21 The ten items on the SUS are scored on a scale from one (strongly disagree) to five (strongly agree). Five of the ten items are reverse-coded (ie, worded as negative rather than positive statements) to mitigate response bias. The words “BATDOK Decision Support Platform” were substituted for the questionnaire’s phrase “this system.” Items included statements such as “I thought that the BATDOK Decision Support Platform was easy to use” and “I found the BATDOK Decision Support Platform unnecessarily complex” (Supplementary Data Figure 1). Median and interquartile ranges were evaluated, as were the proportion of participants rating each item as “positive” or “very positive.”
Data Analysis
Descriptive statistics were used to summarize participant characteristics, percent completion of critical actions, percent of critical actions performed in the correct sequence, and time to completion of critical actions. Categorical variables were reported as frequencies and percentages and continuous variables were reported as means with standard deviations (SD) or medians with interquartile ranges (IQR).
When evaluating time to completion of critical actions, times were used regardless of whether actions were deemed to be in the correct or incorrect order. The “correct sequence” was defined using the tiered approach outlined in the JTS CPG Reference Denger, McCafferty and Neal16 and used completion times to determine the order of completion. Only completed critical actions were used to determine the percentage of actions performed in the correct sequence. Because the second/crossover condition was completed as a continuation of the same simulation (eg, teams would complete the UC scenario and subsequently the CDS scenario, or vice-versa), the research team only evaluated the last four interventions (analgosedation, draining CSF, saline bolus, and administration of paralytic) for the second/crossover condition. The researcher team compared the percentage of critical actions completed and percent of interventions completed in the correct sequence between the UC and CDS simulations using the paired nonparametric Wilcoxon Signed Rank test (to account for the crossover nature of the study).
The SUS scores were calculated by first reversing the scores of negatively worded items to be on the same scale as positively worded items (eg, the response “1: strongly disagree” for the item “I found the system unnecessarily complex” became a “5: very positive” response regarding system complexity). Likert ratings were converted to a 0-100 scale by subtracting one from each item (creating a zero-to-four range), summing the results, and multiplying by 2.5. Reference Brooke22 The percent of participants responding to each item with either very negative, negative, neutral, positive, or very positive answers was also calculated.
Results
In total, 24 participants grouped into 12 teams completed simulations. Participants were primarily physicians (n = 14; 58.3%), with intensive care unit (n = 13; 54.2%) and/or emergency department (n = 10; 41.7%) experience (Table 1). The remaining participants included nurses (n = 9; 37.5%) and respiratory therapists (n = 1; 4.2%). One-third had prior CCAT training, but none had deployed as part of a CCAT team. Demographics for those initially assigned either to UC versus CDS were similar (Table 1). Of the 12 teams, nine (75%) had at least one physician and six (50%) had at least one member with CCAT training.
Participant Demographics and Self-Reported Deployment Experience by Initial Simulation Group
Table 1. Long description
Data for 24 participants are displayed overall and based on usual care vs clinidal decision support first groups. Information includes military status, job description and current work environment, physician specialty training, and previous CCAT training or deployment.
Abbreviations: UC, Usual Care; CDS, Clinical Decision Support; CCATT, Critical Care Air Transport; ED, Emergency Department; ICU, Intensive Care Unit; BATDOK, Battlefield Assisted Trauma Distributed Observation Kit.
There were no significant differences in the primary outcome, percentage of critical items completed. Teams completed a median 83.3% (IQR 79.2%-100.0%) of the critical items in the UC scenario, compared to 91.7% (IQR 83.3%-100.0%) in the CDS scenario (P = 0.58; median difference 4.2%; 95% CI of the difference -8.1% to 16.4%). Teams with and without a physician completed the same percentage of critical items overall (median 83.3%; IQR 66.7%-83.3% for both groups). Teams with a CCAT-trained member (median 83.3%; IQR 66.7%-83.3%) also completed a similar percentage of critical items as those without a CCAT-trained member (median 83.3%; IQR 83.3%-83.3%; Supplemental Table 1).
All teams successfully completed the first intervention (“level and zero ICP monitor;” median time to completion 84.5 seconds; IQR 47.5-164 seconds) in their initial scenario, and all teams completed the third intervention (“administer analgosedation;” median time to completion 154.5 seconds; IQR 93-212 seconds) and the fourth intervention (“drain CSF;” median time to completion 275.5 seconds; IQR 165-328.5 seconds) in both scenarios. Administration of paralytic (the sixth and final intervention) was the most commonly missed critical action (50% completion in both UC and CDS simulations; median time to completion 515 seconds; IQR 420.5-571 seconds).
Figure 2 demonstrates team delivery of interventions in the tiered, stepwise progression outlined by the JTS CPG. In the UC scenario, teams completed a median of 55% (IQR 40-100%, range 0-100%) of interventions in the correct algorithm sequence, compared to a median of 100% (IQR 100-100%, range 40-100%) of interventions in the correct sequence in the CDS scenario (median difference 40%; 95% CI of the difference 5.9% to 50.8%). Most teams (10/12; 83.3%) completed all interventions in the correct sequence during the CDS simulation, compared to less than one-half (5/12; 41.7%) of teams during the UC simulation.
Visualization of Intervention Order by Teams.
Note: Interventions completed in the correct sequence are marked with closed diamonds and those completed out of sequence are marked with open diamonds. Interventions that were not completed are represented by a gray bar.
Abbreviations: ICP, intracranial pressure; CSF, cerebrospinal fluid.
Figure 2. Long description
Four charts demonstrate the time to completion of each recommended intervention, stratified by team and usual care vs clinical decision support. The charts differentiate between interventions performed in the correct and incorrect order.
The mean SUS score for the CDS platform was 77.6 (SD = 16.0), which was above the benchmark for a “good” SUS rating of 68. Reference Ratcliff, Morrison and Tran20 Every item on the scale received a “positive” or “very positive” response from the majority of participants (Figure 3). When comparing questions based on “positive” and “very positive” versus “neutral,” “negative,” and “very negative” responses, the most positively rated items were related to a desire to use the system frequently (91.7% of participants) and ease of use (91.7%). The least positively rated items were regarding the complexity of the system (62.5%), confidence in using the system (66.7%), and the amount of training required to use the system (66.7%).
System Usability Scale (SUS) Responses on Individual Items (n = 24).
Note: Boxes indicate median score, with brackets outlining Interquartile Range (IQR).
Discussion
Interpretation
Provision of the BATDOK TBI CDS did not result in differences in frequency of the primary (percentage of completion of critical actions) or one of the secondary (time to completion of critical actions) outcomes. However, it resulted in significantly increased provision of interventions in the tiered, stepwise progression outlined by the JTS CPG. For example, teams in the UC arm adjusted the simulated patient’s cervical collar late or not at all; this is a low-risk and non-invasive step recommended early in tiered interventions. Conversely, all six teams in the CDS group completed this task, with 5/6 completing it as the second task and 1/6 as the third. Teams demonstrated significant variation in time to completion of various tasks, which increases the number of teams needed to see statistical differences. Follow-up work with more teams completing the simulation(s) entirely with or without CDS may provide a better understanding of the impact of the tool on completion of guideline-concordant tasks. In addition, the simulations were of relatively short duration with a relatively small number of simulated patients. The use of BATDOK TBI CDS tool during longer simulations with higher patient loads, and therefore higher cognitive strain, may show benefit not seen in these initial simulations. This study successfully integrated novel technology into a multi-patient, multi-provider high-fidelity simulation environment, and these initial findings support integrating CDS features within operational EHR systems such as BATDOK to enhance cognitive performance during high-load missions. This study facilitated human factor evaluations and human-system integration testing within high-fidelity simulation environments. These simulations are essential for forward military and prehospital medicine contexts, where randomized controlled clinical trials are often precluded by regulatory hurdles, low patient volumes for specific conditions, and the logistical challenges of a geographically dispersed clinician network.
Clinical Implications
Working memory is the amount of information available for the execution of cognitive tasks and is limited to a small number of elements that can be retained for a very short period of time (seconds). Reference Miller23–Reference Young, Van Merrienboer, Durning and Ten Cate25 Clinicians who are less familiar with care for TBI, and those caring for a high number of patients with frequent demands on their attention, may especially benefit from a cognitive aid that reduces the need for use of working memory. A significant challenge common to combat care and civilian ground and air operations is the requirement for providers to manage low-frequency, high-complexity conditions while simultaneously navigating the constraints of austere environments and the operational rigors of transport medicine. In military settings, this is further complicated by intermittent or unavailable medical oversight. While CPGs are intended to standardize care, they currently rely on static paper or digital formats. Optimizing the intelligence of these cognitive aids can enable forward clinicians to leverage clinical decision support (CDS), even in high-tempo environments characterized by significant non-clinical demands. Reference Clark and Severn26 Advancing the capabilities of CDS has been identified as a priority for both military medicine and civilian prehospital leadership. Reference Worsham, Gonzalez and Kucia15,Reference Bauter27
This CDS tool is fairly simple in nature and avoids several of the potential barriers to user acceptance (alert fatigue, not evidence-based, and user acceptance and integration). There are currently no alerts associated with its use, avoiding alert fatigue. Reference Khalifa and Zabani5 It is based on the JTS CPG which is evidence-based while also incorporating the nuances of care in a combat environment. Finally, the tool is incorporated into software already used during en route care, allowing users easy access, and provides a “Yes/No” stepwise flowchart to guide next steps, minimizing complexity. Given the literature on improved guideline adherence and provision of clinical care with CDS instruments, Reference Mills1,2,Reference Shaloma, Shahara, Parmetb and Lunenfeld4 provision of CDS in the en route care environment is a logical step toward improving outcomes in this high-risk environment, especially if future conflicts require the simultaneous transport of higher volumes of critically ill and injured patients. Additional testing is needed in longer scenarios to evaluate the impact of CDS in more dynamic environments, incorporating task switching between patients and operational demands as would be required in en route clinical care. These more complex scenarios were deferred for these initial simulations in order to obtain baseline competency and user feedback on interface, usability, and functionality of the BATDOK TBI CDS tool and to reduce extraneous load, or the mental effort needed to process information that is poorly presented, visually difficult to understand, or difficult to access. Overall, participants gave “good” usability reviews with a validated instrument, suggesting that integration of BATDOK with CDS into clinical care would be met favorably by clinicians in the en route environment. Further work is needed to create additional CPGs and to evaluate BATDOK with tailored CDS effectiveness for other patient conditions encountered in the en route care environment.
Limitations
Several limitations of this approach warrant consideration. As this was the first study evaluating the effectiveness of the BATDOK TBI CDS in an en route care simulation, the number of simulations completed may have limited the statistical power. The study team did not require participants to complete technical tasks with complete fidelity, which may have altered participants’ time to completion and some of the secondary endpoint measures, as well as decreasing cognitive load, impacting these findings. Some participants completed their first simulation under CDS conditions, diluting the measured effect of the BATDOK TBI CDS tool compared to UC. While all clinician roles used in this study are potentially eligible for CCAT deployment, not all clinicians had completed CCAT training, which may have impacted the findings. This study focused on a simulated military fixed-wing transport scenario, so additional research is needed to evaluate generalizability to civilian ground and air operations.
Conclusion
Initial exploration of the impact of BATDOK TBI CDS tool did not show significant changes on time to provision of critical steps but did show favorable user perceptions of the tool. These types of CDS tools are anticipated to be necessary to prepare clinicians for high-volume en route patient transport missions.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1049023X26108966
Acknowledgments
The study team acknowledges the 59th Medical Wing Critical Care Air Transport Team Pilot Unit and the United States Army Institute of Surgical Research Burn Center (JBSA Fort Sam Houston, Texas USA) for their time and expertise supporting the study.
Author Contributions
LLF contributed to project design, data analysis, and manuscript preparation. AA, NH, LM, CB contributed to project design, data collection, and data analysis. KD contributed to data analysis. GB contributed to project design, technical support, and critical revision. WTD contributed to study design, IRB approval, data collection and analysis, and manuscript preparation.
Funding Statement
This study was supported by the Defense Health Agency-Restoral, project number: ST1189.
Competing Interests
No conflicts of interest declared.
Use of AI Technology
None was used in the creation of this manuscript.
Presentation
This study was presented at the 2025 Military Health System Research Symposium.
Disclaimer
The views expressed are those of the author and do not reflect the official views or policy of the Department of War, or its Components. This study was conducted under a protocol reviewed and approved by the 59 MDW Human Research Protections Program and in accordance with the approved protocol. The Views of the manufacturer, (BATDOK®, or any other manufacturer mentioned here) are not necessarily the official views of, or endorsed by, the US Government, the Department of War, or the Department of the Air Force. No Federal endorsement of BATDOK® or any other manufacturer mentioned here is intended.
Open access
