Professional association conference proceedings summarize presentations discussing advances in treatment or technologies without the interaction between the presenter and audience. A professional association conference summary may be provided for attendees, those interested in the subject, or members of the professional association or society to add to their reference library. An extension of these summaries may outline future endeavors or pursuits by the organizing professional association or society. Consensus opinions via survey can be derived by activities at designated professional association conferences with the intent to issue specific statements to promote advances in clinical guidelines, treatment, or therapies.
The utilization of the scientific Delphi methodology at a conference of similarly educated and trained medical researchers and practitioners/responders to provide consensus statements as the sole objective of a professional association conference has proven to be a fruitful endeavor.Reference Ryan, Kako and Garner 1 These professional association conferences are designed based on the Delphi methodology, from the creation of the Delphi statements using an iterative process or discussion, to the conference participant in-person rounds of anonymous linear numeric responses of each statement until consensus or stability of answers is achieved. The direct output of these professional association conferences can be examined in the future to create clinical guidelines endorsed by conference attendees or the professional association or society that hosted or sent representatives to the conference.
The Society for Disaster Medicine and Public Health (Society) 10-year anniversary conference (SDMPH) took place on December 4th and 5th 2023 at the American Geophysical Union Conference Center in Washington D.C. The theme was “10 years on: Building the Discipline of Disaster Medicine and Public Health.” The sessions (Table 1) presented aspects of disaster risk reduction, disaster response, and the effects on vulnerable populations following these themes:
-
• Climate change and disaster response
-
• Conflict medicine and population health
-
• Advanced technology, artificial intelligence, decision aids, devices and equipment and their effect on mass casualty incident response
-
• Disaster medicine education, training, and competencies which are culturally and geographically relevant
SDMPH Conference Presentations
The distinction between the prior professional association conferences designed specifically as a Delphi conference and the SDMPH is that the education and instruction opportunity permitted each presenter to learn the attendee’s interest or appreciation of the relevance of their research and to guide future direction of their research, education, training, and competencies through the scientific Delphi methodology after the conference.
Methodology
The Delphi technique is a systematic process of forecasting using the collective opinion of panel members.Reference Pill 2 In this study, the Delphi expert panel included in-person and virtual SDMPH participants, as well as Society members who did not attend the SDMPH. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine.Reference Keeney, Hasson and McKenna 3 The Delphi method assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically or logistically difficult, or evidence is conflicting.Reference Shang 4 Delphi studies are modified by how the initial first round statements are created. This includes the use of open-ended questions to panel members; through an iterative process amongst authors, focus groups, or panel members; or other processes. Modified Delphi (mD) study statements were created by the SDMPH presenters and then an internal focus group of lead authors discussed and edited these statements to meet the format of Delphi statements for the Stat59 statistical analysis platform (Stat59 Services Limited, Alberta, Canada). The initial 32 statements submitted by presenters included statements from the Public Health Extreme Events Research (PHEER) network (Presentation # 3) and Telemedicine (Presentation # 9) (Table 1). These statements were excluded from consideration due to the Internal Focus Group determining that the total number of statements should ideally be less than 40 and both topic statements would attain consensus. In the end, statements were edited to fit the Delphi format leading to 38 statements to be presented to the Delphi Panel of SDMPH participants (Table 2).
DELPHI Statement Review Process
SDMPH participants, in-person and virtual, were asked to participate as mD experts in a 3-round mD survey study seeking consensus of statements regarding research activities, training, education, and/or competencies related to topics presented at the SDMPH (n = 127). Also, Society members who were not attending were asked to participate in the Delphi (n = 230). Introductory emails were sent November 21, 2024 explaining the project objectives and the mD to these potential mD experts.
mD experts that agreed (n = 37, 25 SDMPH participants and 12 Society members not participating) were sent an email from the Stat59 mD organizational program with a link to the Stat59 website consent page. Each mD expert registered an account, validated it, and was sent a new email to log into their secure webpage to begin the first mD expert consensus round no later than 1359 GMT 4 December 2023, the start of the SDMPH. mD experts that had not logged into the system were asked to verify their access and log in and asked to notify the author if they had not received the introductory email, with instructions on how to ensure future emails were received.
Once the mD experts logged in, they were provided with a formal explanation of the mD methodology and informed consent was obtained. For informed consent (Supplemental Digital Content link to Stat59 Consent Page), participants were notified that they were anonymous volunteers who could withdraw at any time, that participation or withdrawal would not impact their employment, and that their data was secure (Supplemental Digital Content link to Stat59 Security Page). mD experts received a reminder email to complete this first round before the round closed.
The next page was the list of 38 statements with instructions to rate each statement on a 7-point linear numeric scale, where 1 = strongly disagree and 7 = strongly agree. With this initial set of statements, the mD expert was asked to answer 4 demographic questions. Consensus amongst mD experts was defined as a standard deviation ≤1.0. Statements that attained consensus after this first mD expert round were included in the final report.
Each mD expert received an email from the Stat59 program on December 9, 2023 notifying them that the second round was open; reminder emails were sent before the close of the second round (December 19, 2023) from the author that asked participants to log back into their Stat59 page that showed the mean response of all the mD experts for each statement that did not attain consensus and their own response for that specific remaining statement; they were then asked to reconsider their 7-point linear numeric scale for these remaining statements.
This process was repeated after the second mD expert round with statements that attained consensus included in the final report. The statements that did not attain consensus were advanced to the third and final round with the mD experts asked to reconsider these statements. Each mD expert received an email from the Stat59 program January 8, 2024 notifying them that the third and last round was open and reminder emails before the close of the third round January 18, 2024, similar to the above reminder emails. The third mD expert round produced statements that attained consensus to add to the first and second round consensus statements in the final report. Remaining statements after this third round were the final statements that did not attain consensus.
The University of South Florida Institutional Review Board (Tampa, Florida, USA) determined STUDY006415 met the criteria for exemption.
Results
As summarized in Figure 1, 37 mD experts (25 SDMPH participants and 12 Society members not participating) confirmed their participation, established a unique account on the Stat59, and completed the first mD expert round (Table 3). The majority of experts were from North America; worked at a university, or were involved in research, education, and training; were physicians; and had over 10 years of experience.
Modified Delphi expert panel demographics
RESULTS
Twelve statements attained statistical significance with a standard deviation ≤1.0 after this first mD expert round, and achieved consensus (Table 4, first round, first section in bold). The 26 statements that did not attain statistical significance, with standard deviation >1.0, were advanced to the second mD expert round.
Statements Attaining Consensus
Thirty-five mD experts completed the second mD expert round, with 15 of the 26 statements that advanced to the second mD expert round achieving consensus (Table 4, second round middle section). The remaining 11 statements were unable to attain consensus and advanced to the third mD expert round.
Thirty-four mD experts completed the third and final mD expert round, 8 of the remaining 11 statements achieved consensus, with a total of 35 statements achieving consensus. (Table 4 third round, last section in bold). The remaining 3 statements were unable to attain consensus after 3 mD expert rounds and were not recommended for consideration for future research guidance (Table 5).
Statements for not attaining consensus
Limitations
The PHEER and Telemedicine statements that were not included in the study impacted the overall number of statements but had no other bearing on the study.
The Delphi method seeks to arrive at group consensus by the aggregate of a panel of experts who rate a statement on a linear numeric scale. Franc et al. concluded that the sampling distribution tends to normality for sample sizes greater than or equal to 5. For Delphi studies using a 1-7 linear numeric scale, the use of the standard deviation for assessment of consensus and the mean for ranking is acceptable. Use of these parametric tests presents minimal risk of false consensus or of false ranking when compared to the non-parametric statistics of interquartile range and mean. This suggests that the linear 1-7 scale analyzed using the mean and standard deviation should become the preferred statistical method for Delphi studies.Reference Franc, Hung and Pirisi 5
Study participants were skewed to North America and were physicians. The impact on the data cannot be determined. Certainly, having more input from non-physicians with less experience and from low-middle income countries is the goal of future mD studies to impact statements that could potentially attain consensus to guide future disaster medicine and public health preparedness education, research, and training.
The objective of the distribution of mD experts was to represent those involved with the education, training, and competencies of those who seek disaster risk reduction, disaster medicine response, or recovery as evidenced by their participation in the SDMPH or Society. The distribution of these mD experts favor those who had the means to attend the SDMPH or join the SDMPH, potentially impacting the study lacking the input from those from a resource-limited setting who could not attend or join.
An essential component of the Central Limit Theorem is that the average of sample means will be the population mean, or if 1 finds the average of all of the standard deviations in the sample, then 1 will find the actual standard deviation for the population. 6 This will hold true regardless of whether the source population is normal or skewed, provided the sample size is sufficiently large (usually ≥30, the number of mD experts in this study per round averaged 29.67). 7 The application of the Central Limit Theorem to this study infers that the 35 statements that attained consensus can be recommended to guide presenters with their current and future research.
Discussion
The theme of the SDMPH “Building the discipline of Disaster Medicine and Public Health” is to retain the lessons learned after an exercise or an actual incident for the participating agencies or organizations to improve their unique education, training, competencies and response. The first day of the SDMPH focused on climate change and conflict in disaster medicine. The second day featured presentations on ethics and artificial intelligence, devices and equipment, equipment and training.
Short-term and long-term exposure to ambient air particle pollution cause a myriad of health and ecological effects (statements #1, 2). 8 , Reference Rajak and Chattopadhyay 9 In particular, short-term exposures lasting hours to a few weeks and long-term exposures lasting months to years adversely affect cardiopulmonary health as evidenced by increased premature mortality, and hospitalization for cardiovascular disease (#1, 2).Reference Chen, Warrington and Dominici 10 , Reference Su, Li and Xu 11 Individuals with prevalent heart disease, namely ischemic heart disease and heart failure, are at higher risk for having adverse health effects after exposure (#2).Reference de Bont, Jaganathan and Dahlquist 12 Wildland fire smoke is an important and growing source of particulate air pollution and is increasing overall average concentrations of fine particulate matter in most of the continental U.S. or at least arresting the overall improvement of air quality that has occurred over the next several years (#3).Reference Cascio 13 Yet, few health care professionals educate their patients to modify their exposure risk highlighting the potential benefit of education (#4, 5).Reference Rappold, Reyes and Pouliot 14 , Reference Engebretson, Hall and Blades 15 When questioned, the attendees showed strong agreement that the education of health care professionals should include training on the care of people affected by wildfire smoke and that health care systems should increase the care for patients affected by wildfire smoke (#17, 18). Similarly, study participants agreed that a transdisciplinary approach is required to anticipate and mitigate the impact of emerging risks of floods on public health systems due to compromised treatment and care for people with non-communicable diseases (#9, 23).
Humanitarian surgical care (HSC) in 21st-century armed conflict poses a complex nexus of challenges related to factors including shifting conflict dynamics, security constraints, and proliferation of unconventional actors within the humanitarian space.Reference Wren, Wild and Gurney 16
Study participants agreed that a focused approach is required to improve the quality of surgical care delivered to civilians in asymmetric conflict with forward surgical services with intact echelons of care to reduce mortality and morbidity (#19). The study participants agreed that HSC and the trauma stabilization point (TSP) should be considered to manage the full range of population needs including not only trauma-related but also non-traumatic emergencies (e.g., cesarean section), acute medical conditions, and uncomplicated minor injuries in addition to asymmetric warfare trauma (#12, 20, 24, 25).Reference Chu, Stokes, Trelles and Ford 17
Study participants agreed that coordination between military and other armed groups must be developed based on the humanitarian trauma system needs of non-combatant citizens, military, and other armed groups (#26). At present, surgical training in high-resource settings is not well-suited for this breadth.Reference Lin, Dahm and Kushner 18 Study participants agreed that capacity and capability of the humanitarian trauma system should be based on available expertise of humanitarian responders and the local health care delivery system (#27). Standardized pre-deployment training and competency assessment in core HSC procedures may contribute to ensuring quality of care. To adequately benchmark care quality, it is necessary to establish a minimum dataset (MDS) with legitimate-use sharing agreements (#21).Reference Wild and Wren 19 Development of risk-adjusted perioperative mortality calculators for resource-constrained environments may support this process.Reference Wild, Stewart and LeBoa 20 Due to the high-threat environments in which HSC often occurs, there has been increased engagement from non-traditional actors.Reference Gurney, Graf and Staudt 21 Establishing transparent credentialing and training in international humanitarian law (IHL) among all actors providing HSC in conflict may help ensure protection of humanitarian principles (#22).
Over the last 10 years ethical challenges presented and the rise of artificial intelligence provided disaster medicine researchers fertile subject matter to pursue. Three statements related to ethical issues with artificial intelligence (AI) in disasters attained consensus. Statement #34 noted that the ethical issues with AI for mass casualty incidents have not been thoroughly analyzed as examples are growing of specific situations where AI has been deployed during disasters without due consideration of its ethical considerations. These include cases where concerns about privacy and confidentiality of data occurred.Reference Bhardwaj 22 Another concern relates to the quality of datasets used to train AI.Reference Velev and Zlateva 23 , Reference Fan, Zhang and Yahja 24
Statement #11 about biases in datasets raising serious ethical concerns attained consensus and affirmed the importance of research into this issue. Given that many disasters occur in low-income countries, issues of justice need research related to AI. Nature carried a headline accompanying the release of a 2024 World Health Organization (WHO) report into the ethics of AI stating, “Medical AI could be ‘dangerous’ for poorer nations.”Reference Adam 25 Statement #33 attained consensus, agreeing that AI may not benefit those impacted by disasters in low-income countries as much as those in higher-income countries. Further research is needed to examine and mitigate such disparities.
Many other ethical issues remain under-investigated with analysis urgently needed to avoid undermining AI’s potential benefits. With regards to the need for encryption of patient identifiers in AI tools, the study participants attained consensus with strong support (mean 6.6, SD 0.8) (#2). The study participants showed moderate support (mean 5.5, SD 0.9) for the need for strict statistical validation of AI tools prior to implementation (#29). In contrast, study participants showed weak support (mean 3, SD 1) for the replacement of traditional printed disaster plans with AI guided just in time interfaces (#30). Though each statement attained consensus, additional studies in concert with developing ethical solutions should foster more dedicated research of AI models.
Defense Advanced Research Projects Agency’s (DARPA’s) In the Moment (ITM) program will leverage insights from the statements that did [#6] and did not attain consensus (Not #1, 2). ITM research will assess how human decision-makers’ perceptions of an AI algorithm’s competence, value alignment and/or other factors might make a human decision-maker more likely to delegate a decision to the AI algorithm. ITM will also assess how the decision to delegate may vary depending on environmental conditions and/or personal context. DARPA shared insights from the study with the program’s performers. The insights will also inform the program’s Ethical, Legal, and Societal Implications (ELSI) processes through panel discussions at ITM workshops. The workshops are part of DARPA’s outreach to policymakers and practitioners.
Development, creation, and subsequent implementation of devices in the disaster medicine setting impose additional layers of ethical and practical considerations. The value of point-of-care ultrasound technology in the prehospital and resource-scarce environment is evident (#32) but finding ways to share this technology and knowledge through effective training programs has challenges that include sustainability and quality assurance once the training programs have concluded. One potential feasible solution to this challenge is the use of virtual meeting platforms and remote image review as quality assurance measures once the technology is implemented in a community (#16). These measures provide a means to also gauge impact of training programs and should be incorporated into training programs during the early planning stages in coordination with the host communities as standard practice. With the implementation of programs, sustainability should be an important consideration, but consensus was not attained on whether sustainability should impact the decision to implement a point-of-care ultrasound training program in specific communities (Not #3). This may be partially due to the ethical dilemma of withholding lifesaving technology that could impact mortality from a community during a time of need, such as a sudden-onset disaster or conflict.
Historically, after action reports (AARs) of exercises and incidents with long scene or transport times have identified gaps determining which patients require lifesaving interventions (LSI) and priority transport with the demand of patients exceeding the supply of monitoring devices. As the discipline of disaster medicine has grown over the last 10 years, government agencies cognizant of these AARs have dedicated research to address these gaps. Study participants support the design and creation of patient monitoring devices utilized in an MCI to enhance LSI and priority transport (#3, 4). Scene safety, ingress and egress routes, the placement of casualty collection points, advanced medical posts, transportation staging, incident command, and identification of patients in the continuous triage process of an MCI can be assisted by drones (#31). Data collected and stored by patient monitoring devices can be analyzed retrospectively to identify trends to improve prehospital processes during MCI responses (#5).
The NIGHTINGALE research and innovation project funded by the European Union’s Horizon 2020 program convened experts in technology, research, medical practitioners, and leading organizations in Europe that specialize in MCI response. The objectives include to: (1) upgrade evaluation of injured and affected populations (triage) using digital identification, traceability, fast diagnosis, continuous monitoring, and accurate classification of medical conditions; (2) optimize pre-hospital LSI and enhance utilization of assets, resources and capacities using AI-based solutions; and (3) enable shared response across emergency services and communication between emergency teams and with patients by developing augmented reality tools for first responders. This is consistent with the statement that prehospital MCI response training should be delivered to all first responders (#28).
High-fidelity simulation MCI response training will lead to evidenced-based practices for education and training of both civilian and conflict responders to enhance trainees’ stress tolerance and will accelerate trainees’ professional identity development (#8, 14, 15). As the discipline of disaster medicine has advanced, the recognition that responders are able to focus their attention through practices such as mindfulness breathing has led to research with military medical students investigating whether mindfulness-based practice can improve performance while under stress.Reference Cole and Bronstein 26 The Sendai Framework calls for disaster medicine education to be incorporated into standard health professional medical curriculums consistent with the study participants’ acknowledgement that high-fidelity MCI response simulation training increases readiness for deployment (#7, 13).Reference Sakurai and Sato 27 In addition, future studies should measure MCI response readiness outcomes during simulation within both civilian and conflict settings.
Scenario based simulation for MCIs and HCRs should become more accessible along with training on the use of rapid systematic assessment tools. Together these would maximize health care readiness for deployment to help ensure dynamic data-driven decisions are made with high confidence levels. A tool that could provide the template to support such a capability is the United Nations Public Health System Resilience Scorecard (Scorecard) (#10). The Scorecard is versatile, scalable, and could be easily modified. It is well suited to gather insights systematically and rapidly on needs and priorities using a consensus-based approach to determine which priority actions and needs from earlier planning should be kept or removed prior to deployment. The Scorecard process is designed to ensure a multidisciplinary team discussion and systemwide exploration of healthcare mission readiness, providing a high level of confidence of what is needed for success.
Conclusion
This study demonstrates how an mD conducted at a professional association conference with the objective to provide a forum for presenters conducting research in disaster risk reduction, disaster medicine response, and recovery can provide consensus statements to guide the presenter’s current and future research to advance education, training, and competencies.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/dmp.2024.251.
Author contributions
ESW: concept, delphi, edit statements, wrote manuscript and abstract, and maintained manuscript review and edits, references; JC, FB: concept, edited statements, reviewed and edited manuscript; HW, RC, JR, JF: presented at the SDMPH, submitted and edited statements, reviewed and edited manuscript; TB: presented at the SDMPH, edited statements, reviewed and edited manuscript; DO, WC: presented at the SDMPH, submitted and edited statements, reviewed manuscript; AW, MV: edited statements, reviewed and edited manuscript; MC, DE: presented at the SDMPH; EH: IRB lead; AJ: concept and oversight.
Acknowledgements
With our sincere appreciation for their time and expertise, we thank the experts who consented to be acknowledged and those that did not consent: Rita Burke, Terry Donat, Romeo Fairley, Brian Flynn, David Glotzer, James Gosney, Heather Hayanga, Mariana Helou, Mayumi Kako, Deborah Kim, Allison Latif, Melissa Leming, Shawn Metzner, Veselin Mitrović, Chaverle Noel, Frederick Nolting, Georgina Nouaime, Chiadi Onwuka, Matteo Paganini, Luis Rios, Jamla Rizek, Jeff Rubin, Elias Saade, Cheryl Schmidt, Yasir Shafiq, Mark Shah, Eman Shaikh, Angelika Underwood, Nunzio Zotti. With profound humility and gracious appreciation: We thank Alice O’Donnell for her support and focus. We graciously thank Dr. James James for his vision and dedication to the SDMPH. We thank Dr. Frederick “Skip” Burkle for his altruism and perspicacity as we stand on your shoulders.
Open access
