What’s Next?

However, much of the initial excitement surrounding the use of generative AI has not always held up under rigorous evaluation. For instance, when different queries related to bystander cardiopulmonary resuscitation (CPR) were tested, nearly one-half of the responses contained information that was unrelated to CPR. ^{Reference Murk, Goralnick, Brownstein and Landman3} Similarly, our studies on disaster triage have demonstrated variable accuracy, with some results showing less than 50% accuracy when using AI for triage. ^{Reference Franc, Hertelendy, Cheng, Hata and Verde4}

The lack of accuracy should not be surprising, given the context of how these models are trained. Large language models are based on probabilistic approaches to generating text, and they do not inherently possess specific content knowledge on the topics they address. Furthermore, the initial training of many of these models relied heavily on human responses, which can introduce biases, toxicity, and confabulations. While the field is evolving rapidly and many measures are being implemented to reduce these errors, they do persist.

Unfortunately, the low barrier to using these chat-based AI tools has led to their almost casual integration into clinical workflows. Unlike the introduction of a new medication or medical device, which would typically require a formal certification process, these AI tools can be readily accessed and applied by anyone with an internet browser and interest. As a result, these tools are often used without thorough investigation into their efficacy, safety, and accuracy.

What Did COVID-19 Teach Us?

How should we approach this sudden influx of new technology? In fact, this is not an entirely new phenomenon. During the coronavirus disease 2019 (COVID-19) pandemic, new technologies and treatments emerged rapidly and were swiftly integrated into the medical ecosystem. For example, although the concept of mRNA vaccines had been explored since the early 21st century, the COVID-19 pandemic spurred unprecedented research and development in these vaccines. ^{Reference Polack, Thomas and Kitchin5} Through well-designed clinical trials, the safety and effectiveness of these vaccines were clearly demonstrated, and their influence on the course of the pandemic is undeniable.

However, rapid technological developments do not always lead to meaningful changes. During the COVID-19 pandemic, we learned that initial enthusiasm and even preliminary empirical results could suggest that a treatment is safe and effective. For example, ivermectin was initially reported anecdotally to be effective against COVID-19 and saw wide-spread off-label use. ^{Reference Marcolino, Meira and Sernizon Guimarães6} A similar pattern was observed with other treatments, such as famotidine. ^{Reference Cheema, Shafiee and Teymouri Athar7} However, when rigorous controlled trials and systematic reviews were conducted, little evidence supported the effectiveness of these treatments. Many treatments that initially seemed promising were later found to be ineffective, leading to an unprecedented number of journal retractions during the COVID-19 era. ^{Reference Van Noorden8}

Moving Forward

While there is no doubt that AI tools are important and that their investigation in prehospital and disaster medicine is essential, their implementation should be moderated by the same methods we use for adopting new medications, devices, procedures, or technologies. Therefore, AI-guided tools should undergo the same statistically rigorous evaluation that we would apply to a new drug or medical device. The statistical tools needed for this type of evidence-based research are well-known to the medical community, and there are many options available for succinct and concise study designs.

Special Collection Announcement

Prehospital and Disaster Medicine’s (PDM’s) mission statement is to publish innovative, high-impact, evidence-based research in prehospital and disaster medicine. The PDM Editorial Team is pleased to announce a forthcoming special collection dedicated to “Evidence-Based Artificial Intelligence in Prehospital and Disaster Medicine.”

PDM invites the submission of original, evidence-based research articles that explore the applications of AI within the critical fields of prehospital medicine and disaster medicine.

All submissions must follow the PDM instructions for authors and conform to the formatting and content requirements for either the Original Research or Innovation Report article types, as detailed in a recent Editorial. ^{Reference Franc9}

Please note that as this is a collection on evidence-based medicine, the collection will not consider expert opinions, frameworks, editorials, review articles, calls for action, or commentaries. All manuscripts will undergo our standard peer-review process, as detailed on the journal’s website, ¹⁰ and must adhere to the ethical guidelines outlined. ^{Reference Franc and O’Mathúna11}

Selected manuscripts will be evaluated by an international panel of judges. The top three selected manuscripts will be featured in exclusive interviews with the authors, which will be published on the World Association for Disaster and Emergency Medicine (WADEM) website and social media channels. The authors of the first-place manuscript will also be invited to give a webinar about their research project, sponsored by PDM and WADEM. Submissions will be accepted from September 1 - December 1, 2025 with publication commencing on/after January 1, 2026.

We eagerly anticipate your high-impact contributions to this rapidly evolving field.

Author Contribution

JMF performed the conception, writing, and final approval of the manuscript.

Use of AI Technology: No AI technology was used.