Introduction
Diversity, equity, and inclusion (DEI) are imperative in supporting intersections of health science education, public health, and research. It is well documented that diverse teams outperform homogenous teams in business settings, however, these findings also extend to healthcare settings [Reference Hunt, Yee, Prince and Dixon-Fyle1,Reference Rock and Grant2]. Authors Gomez and Bernet summarized diversity research specific to healthcare to better understand the relationship between diversity, innovation, patient health outcomes and financial performance. Their meta-analysis affirms that diverse teams are related to improved patient care. In addition, the authors found that diverse healthcare teams demonstrate increased innovation, communication, and risk assessment. Finally, the authors confirmed that diversity-friendly environments support change management. Within healthcare these domains are central to health equity and workplace belonging [Reference Gomez and Bernet3]. The fields of epidemiology and public health have made significant contributions to best practices in defining and using race and ethnicity in research; however, many opportunity gaps remain surrounding diversity, equity, and inclusion in the field [Reference Borrell and Crawford4]. Following the COVID-19 pandemic [Reference Fortuna, Tolou-Shams, Robles-Ramamurthy and Porche5], the murder of George Floyd [Reference Krieger6], and a rise in anti-Asian hate [Reference Nguyen, Criss and Dwivedi7] there was a recommitment across biomedical and health professional institutions to develop diversity initiatives to address anti-racism, diversity education and curriculum, and equity in recruitment and retention [Reference Collins, Adams and Aklin8].
In 2020, the Public Health National Center for Innovations launched a task force to review the 10 Essential Public Health Services (EPHS) centered on equity and removal of systemic and structural barriers (e.g., poverty, racism, gender discrimination, ableism, and other forms of oppression) that result in health inequities [Reference Castrucci9]. The eighth EPHS mission was to “Build and support a diverse and skilled public health workforce.” Potential steps to address diversity and inclusion in public health include culturally congruent education and training for professionals and leadership and nurturing diverse future public health practitioners. The public health and health sciences field, however, lacks diverse racial and ethnic representation and remains overwhelmingly White [Reference Alang, Hardeman and Karbeah10–Reference Sellers, Leider and Gould12]. There have been efforts to incorporate diversity and structural competency, defined as the ability to recognize and intervene on structural factors that impact health inequalities, as core tenants in medical education, but these teachings are still needed for those in public health research [Reference Harvey, Neff and Knight13,Reference Metzl and Hansen14].
Diversity training can be defined as a distinct set of instructional programs to facilitate positive intergroup interactions, reduce prejudice and discrimination, and enhance skills, knowledge, and motivation to interact with diverse individuals [Reference Pendry, Driscoll and Field15]. There are several core aspects of successful diversity training. A meta-analysis of 260 independent studies found greater positive effects when diversity training was targeted to awareness and skills development, conducted over time, and utilized a comprehensive approach [Reference Bezrukova, Spell, Perry and Jehn16]. Rooted in organizational development theory, diversity training sessions that utilize a comprehensive approach to identifying and intervening on structural factors are more efficacious than trainings that focus primarily on individual attitudes. A comprehensive diversity training includes support from senior leadership, is tailored to the specific organization, connected to the group’s operating goals, inclusive of employees from all levels, and facilitated by content experts [Reference Bendick, Egan and Lofhjelm17,Reference Shandera, Matsick, Hunter and Leblond18].
We conducted a literature review within the Journal of Clinical and Translational Science and using the keywords “diversity, equity, inclusion, and curriculum.” The results identified 6 relevant articles that focused on curriculum development for trainees [Reference James, Venable and Caro19,Reference Enders, Golembiewski, Orellana, Silvano, Sloan and Balls-Berry20], utilization of diversity committees [Reference Balls-Berry, Rubio and Aguilar-Gaxiola21,Reference Enders, Wurth, Orellana and Menser22], and faculty development [Reference Rubio, Norman and Mayowski23,Reference Strekalova, Qin and Sharma24]. Enders and colleagues evaluated faculty and staff attitudes to a Health Science Research (HSR) departmental Diversity, Equity, and Inclusion (DEI) plan. Authors found that many respondents expressed support for the proposed initiatives of a blinded promotion review process and DEI training. Interestingly, authors observed that attitudes toward the proposed initiatives differed by faculty and staff subgroups. Specifically, more faculty and staff from racial and ethnic minority groups supported inclusion nudges and blinded promotion reviews compared to White and non-Latinx individuals [Reference Enders, Golembiewski, Pacheco-Spann, Allyse, Mielke and Balls-Berry25]. Our curriculum furthers the existing body of literature by leveraging our departmental diversity committee to inform our workshop curriculum for a health science academic department that is inclusive of faculty and non-faculty researchers.
We developed and implemented a four-part DEI curriculum for health research faculty, staff, and researchers at an academic health science institution. Four 2-hour workshops combined didactic and experiential activities rooted in transformative learning and critical race theories [Reference Ford, Griffith, Bruce and Gilbert26–Reference Van Schalkwyk, Hafler and Brewer28]. Transformative learning theory suggests that paradigm shifts occur through critical self-reflection, acquiring new knowledge, building competencies, and integrating new schemas [Reference Van Schalkwyk, Hafler and Brewer28,Reference Kaufman, Mann and Swanwick29]. Critical race theory is grounded in four pillars including race consciousness, contemporary orientation, centering in the margins rather than the mainstream, and theory-informed actions, also known as praxis [Reference Ford and Airhihenbuwa27]. Combined, they provide an excellent framework to support a curriculum that provides evidence-based diversity content and reveals the often invisible forms of inequity and tools to support equity and allyship at individual and structural levels.
The workshop series was designed to increase foundational knowledge of core diversity concepts and improve skills to support an inclusive population health science academic department. We describe the curriculum development and implementation and evaluate the program’s efficacy.
Methods
Setting
The 250-person Department of Epidemiology and Biostatistics was comprised of 51% faculty, 41% staff (including non-faculty academics), and 8% trainees (doctoral students or postdoctoral trainees). Self-reported race and ethnicity was 58% White, 27% Asian and Asian American, 6% Black and African American, and 5% Hispanic/Latino(a); more staff than faculty identified as a member of an underrepresented racial and ethnic minority (URM) group (27% vs. 7%, p < 0.0005). Sixty-two percent were women.
Workshop development and curriculum content
The DEI committee, consisting of 14 department members (6 faculty, 7 staff, and 1 student), developed the four-part DEI curriculum for a 250-person department of epidemiology and biostatistics. The committee partnered with a campus DEI professional (LHW) to develop the curriculum and serve as the lead facilitator. She developed all workshop content using feedback from an all-departmental needs assessment survey and conversations with committee members. Committee members assisted with workshop logistics and facilitated small group activities. The goal was to increase overall understanding of core DEI concepts and skills for inclusive behaviors. Our methodological framework was based on the Six-Step Approach to curriculum development [Reference Thomas, Kern, Hughes and Chen30].
Step 1: Problem identification and general needs assessment
Current scholarly work on diversity training and workshops in public health and academic health science institutions reveals underutilization of a multisession approach. In addition, extant literature mainly focused on education or didactic information, leaving out opportunities to examine experiential and skill-based curriculums [Reference Gwayi-Chore, Del Villar and Fraire31,Reference Harrison-Bernard, Augustus-Wallace, Souza-Smith, Tsien, Casey and Gunaldo32].
Step 2: Targeted needs assessment
The month before developing the workshop curriculum, we administered a short survey to evaluate the level of DEI knowledge and attitudes and to assess the department members’ content preference. Of the 100 respondents, 50% were staff, 34% faculty, and 16% trainees. Most (71%) had been in the department for less than 5 years and 62% did not have prior DEI training. Among those with previous training (n = 34), participation was based on personal interest (65%), a requirement (44%), or professional development (47%). Supplemental Table 1 describes respondents’ preference for workshop session topics and perceived disparities across workplace areas. Information and feedback from the needs assessment survey informed the workshop series topics and curriculum content.
Presession participant characteristics, overall and by department of epidemiology and biostatistics diversity, equity and inclusion workshops, 2019–2020
Note: Numbers may not sum to session total due to missing response values.
URM = underrepresented minority; LGBTQI = Lesbian, Gay, Bisexual, Transgender, Queer, Intersex.
* p-value for Fisher’s exact test of independence for each participant characteristic across sessions.
Step 3: Goals and objectives
Each of the four workshop sessions was preceded by a design phase where the DEI facilitator and committee members met bi-weekly to review past session feedback and co-develop curriculum content (slides, facilitator guides, and handouts), plan implementation (advertising, engaging leadership), and address logistical details regarding participants’ needs (venue, time, and food). Feedback, observations, and recommendations gathered from the previous session’s survey results were considered to inform curriculum content.
Workshop content
Session 1 aimed to establish a foundation for the workshop series by generating enthusiasm and buy-in, reviewing needs assessment data, and amplifying the departmental commitment to diversity. To facilitate content delivery to a large audience, we held the session immediately after an all-departmental meeting. The first session focused on foundational DEI concepts, unconscious bias, and cultural humility with large and small group activities. Learning objectives included: (a) understand departmental knowledge and perceived value of DEI; (b) communicate departmental diversity goals and initiatives and garner support; (c) introduce foundational diversity concepts; and (d) apply foundational concepts and frameworks to the situational and cultural context of an academic research department.
Session 2 was an extended workshop focused on microaggressions, remarks or behaviors that express hostile, derogatory, and invalidating messages based on race, gender, sexual orientation, and other diverse social identities [Reference Ehie, Muse, Hill and Bastien33]. It included a didactic presentation on microaggressions and allyship, a worksheet guided self-reflection on microaggression experiences, and anonymized case scenarios of department members’ microaggression experiences presented in guided small group activities. The facilitator emphasized that microaggressions are determined by those who experience harm and stressed the importance of impact vs. intent. The responsibility of the person observing the microaggression to intervene or resolve the situation, rather than the person experiencing the microaggression, was emphasized as part of allyship. Learning objectives included (a) demonstrate the importance of inclusive language by pronouns; (b) understand what microaggressions are and how to spot them; (d) identify and begin applying skills and strategies to engage as an ally.
Session 3 was a hands-on workshop building upon session two that emphasized skill-building to address workplace microaggressions. A didactic presentation on allyship focused on moving from “bystander to upstander” was provided, followed by several small group role-playing activities to practice applying skills [Reference Ho, Chong and Narayan34]. Materials and activities focused on the misconception that the person who is the target of a microaggression is responsible for addressing the harm or giving feedback to others and, instead, stressed the role that bystanders can play as allies to disrupt microaggressions and/or give feedback to those responsible for the harm. Small group activities utilized session 2 case scenarios and tools and materials for disrupting microaggressions [Reference Harrison-Bernard, Augustus-Wallace, Souza-Smith, Tsien, Casey and Gunaldo32]. Learning objectives included: (a) demonstrate foundational knowledge of diversity concepts: allyship, power, privilege, and microaggressions; (b) expand awareness about the impact of microaggressions; and (c) practice skills and intervention strategies as an “upstander” to address microaggressions.
Session 4 was rescheduled from a in person session to a virtual session in June 2020 due to the COVID-19 pandemic. Content was modified to address the nation’s racial justice reckoning after George Floyd’s murder and the pandemic’s impact on perpetuating racial disparities. Past foundational work provided an opportunity to come together during this powerful time, process the events, and apply knowledge and skills gained across sessions 1-3. The session was titled “Exploring power, privilege, and race in the COVID-19 era: Opportunities for Allyship.” It began with a reflective exercise, where participants could anonymously share how they were affected by recent events. After a brief didactic review, concepts of power, privilege, and allyship were applied to the co-concurrent topics of anti-black racism and the COVID-19 pandemic. Trained moderators used discussion prompts to facilitate small group discussions via virtual break-out rooms. Participants were pre-assigned to small groups based on their self-identified racial and ethnic identity, with a separate group for department leaders to allow for more candid discussion. Learning objectives included (a) review foundational diversity concepts: power, privilege, and allyship; (b) apply foundational diversity concepts amid COVID-19, shelter-in-place, and events of racial violence; and (c) identify strategies for allyship.
Step 4: Educational strategies
Workshop content was designed to include a mix of didactic, large-group, and small-group activities to meet the specific session objectives. The DEI facilitator (LHW) provided content expertise. LHW leads diversity education initiatives, serving as a facilitator, keynote speaker, and mentor for emerging diversity champions.
The curriculum content was developed in partnership with the departmental DEI committee members to ensure that content applied to the academic research work environment. Participants received handouts outlining each session’s specific content with space for self-reflections and notes for future reference. Additional materials included: definitions of key terms, resources for reporting discrimination and harassment, and resources for additional DEI learning.
Step 5: Implementation
Workshop administration
To meet our goal of increasing DEI topic knowledge and shifting departmental culture, we sought to engage many departmental members at each workshop session. Participation was voluntary and open to affiliated students, staff, and faculty. To encourage attendance, we advertised the sessions frequently by email, flyers, and announcements at departmental meetings, enlisted support from department leaders, and sent individual invitations to influential members across research groups, projects, and social circles. In-person workshops prior to the pandemic offered breakfast. Lastly, session times were selected to complement core departmental activity and class time to avoid conflicts. Division heads and the Department Chair attended all sessions. Session 1 was held in-person directly following an all-department meeting. To increase accessibility for department members working in different locations, sessions 2–3 were conducted in person with an option for remote participation. Session 4 was conducted solely via Zoom due to local shelter-in-place edicts. Sessions 2–4 didactic portions were recorded for future use.
Step 6: Evaluation
We administered a pre-session survey and a post-session survey at each workshop session measuring: participant characteristics (i.e., workplace position, supervisor status, time in department, gender identity, URM membership, LGBTQIA + identity, current DEI knowledge and skill level, perception of departmental disparities, and willingness/ability to address and respond to DEI in the workplace). Post-surveys also included open-ended questions to solicit feedback about what worked well, what we could do better and suggestions for DEI topics and experiences for future sessions. Post-survey feedback was reviewed after each session, allowing for some modifications to subsequent session facilitation (e.g., participant requests for additional time in small groups or further clarification on DEI terminology). Surveys were anonymous; personal identifiers were not collected.
Data analyses
Attendees varied across the four workshop sessions. Furthermore, given that all survey data—both pre- and post-session and across sessions—were anonymous, linking responses to specific individuals was not possible. Consequently, we conducted all analyses using unpaired methods.
We evaluated differences in participant characteristics across sessions using Fisher’s exact test of independence. To evaluate the impact of each session, we calculated summary statistics for all learning evaluation measures pre- and post-session and compared these statistics within a session. For binary measures, we calculated counts and percentages, assessing differences using Fisher’s exact test. For ordinal Likert scale measures, we first determined the distribution of responses. When responses followed a normal distribution, we calculated means and standard deviations and assessed differences between pre- and post-session responses using unpaired t-tests. In cases where Likert scale responses were not normally distributed, we calculated medians and interquartile ranges and assessed within-session differences using Wilcoxon rank-sum tests. To evaluate the impact of the training series across the four sessions, we analyzed both binary and ordinal survey responses in two ways. Initially, Fisher’s exact test was applied to assess any differences in the distribution of post-session responses across sessions. Then, trend analyses were conducted using the chi-square trend test for binary responses and the Jonckheere-Terpstra trend test—a non-parametric, rank-based approach—for ordinal responses.
All quantitative analyses were conducted using Stata 16 statistical software [35]. Two authors analyzed the open-ended qualitative data to evaluate other workshop benefits. Results were summarized using reflexive thematic analysis [Reference Braun and Clarke36]. First, results were coded to identify session successes, failures, and recommended future topics. Next, codes were reviewed to generate themes using an inductive, iterative approach. After reviewing organized data, two authors (AR and LHW) identified and discussed themes with a third author (MM) until reaching agreement regarding workshop benefits. Our study received exempt status from the University of California, San Francisco’s Institutional Review Board.
Results
Workshop attendance varied by session; range 45 (session 3) to 82 (session 4) (Table 1). Across all sessions, most attendees were staff (64%, including non-faculty academics), and only 25% were faculty, a notable contrast to the department’s overall composition of 41% staff and 51% faculty. Most attendees identified as female (74%) and 28% identified as a member of an URM group, reflecting the composition of the department; 17% identified as LGBTQI+. Characteristics of the workshop attendees did not significantly differ by session (Table 1).
Knowledge and perceptions of DEI
We compared post-session to presession responses (i.e., within-session differences). We observed statistically significant increases in perceived knowledge about DEI in each of the four sessions (Table 2). Significant pre-post session improvement was observed in having “the skills to improve DEI” in the workplace in sessions 2–4. No statistically significant differences were noted in pre-post session responses for the other ability/willingness to respond domains (i.e., agreement they are responsible for improving DEI or that they plan to work to improve DEI in their workplace).
Knowledge and perceptions of current diversity, equity and inclusion, pre vs. post for each session, department of epidemiology and biostatistics diversity, equity and inclusion workshops, 2019-2020
p-values for comparisons of binary variables (i.e. perceived disparities in the workplace) between pre and post based on Fisher’s exact test; p-values for comparisons of continuous variables (i.e. Likert scales) between pre and post based on two sample t-tests if normally distributed responses and Wilcoxon rank-sum tests if non-normally distributed; (paired tests not used because individual-level identifiers were not collected).
IQR = interquartile range; SD = standard deviation; DEI = diversity, equity, and inclusion.
* 4-point Likert scale ranging from 1 “Not knowledgeable at all” to 4 “Very knowledgeable.”
** 4-point Likert scale ranging from 1 “Strongly disagree” to 4 “Strongly agree.”
We also assessed differences in post-session responses across the four sessions (i.e., between-session differences) and found a statistically significant difference for perceived disparities in mentoring that supports mentees from diverse backgrounds (p = 0.041), perceived disparities in progression and promotion (salary) (p = 0.011), and perception that the lack of DEI is a problem within the workplace (p = 0.034). A linear trend across sessions was observed for perceived disparities in opportunities for leadership positions (p = 0.026), indicating a growing perception of disparities in this area (Table 3).
Post-workshop knowledge and perceptions of diversity, equity and inclusion topics, overall and by sessions, [UNIVERSITY] diversity, equity, and inclusion workshops, 2019–2020
DEI = diversity, inclusion, and equity.
* p-values based on Fisher’s exact test for any difference in the response distribution between sessions.
** p-values based on test for trend across the four sessions (chi-square test for trend for dichotomous responses; Jonckheere–Terpstra non-parametric, rank-based test for trend for ordinal responses, with exact p-value from Monte–Carlo permutations).
Finally, we investigated whether the observed differences in perceived disparities across sessions, namely mentoring, progression and promotion, and opportunities for leadership positions, varied depending on identification with an URM group. We found that the observed significant differences across sessions were among the URM participant groups, but not among non-URM groups (Supplemental Table 2).
Open-ended feedback
Themes for what worked well were consistent across sessions. Having an excellent facilitator was the most common theme, particularly in the first two sessions. Respondents praised the facilitator as an engaging, dynamic, and funny; one attendee said she made “a difficult topic easier to handle.” Other common positive themes included small group discussions and good real-life examples, both of which increased across sessions. Themes for what could have improved varied across sessions. The most common themes for improvement were the need for more tangible skills (session 1), more time and better time management for discussions (sessions 2 and 3), and the need for more sessions (session 4). Desire for more small-group interaction time was noted across all sessions. The most requested topic for future sessions was how to personally improve/promote DEI in the workplace (session 1) and microaggressions, responding to conflicts, and allyship (sessions 2–4).
Discussion
We designed and implemented a successful departmental DEI workshop series at a large academic institution. Our goals were to increase the overall understanding of core DEI concepts and increase skills to improve DEI. The series was one step in our larger departmental approach to changing departmental culture and deconstructing academic systems for enhanced inclusion and diversity.
Our evaluation identified several positive findings indicating the workshops’ success. There was an overarching increase in the workshops’ effectiveness in promoting DEI knowledge and skill development. A more in-depth analysis revealed that perceptions of DEI disparities in the workplace were only significant for underrepresented participants. Finally, open text feedback affirmed that the workshop design aligned with researched best practices for comprehensive DEI initiatives and approaches [Reference Bezrukova, Spell, Perry and Jehn16]. In addition, we identified ways to improve future DEI efforts in our department which may inform DEI curriculum development and implementation at similar institutions.
The finding that the DEI workshops had positive effects on knowledge and skill development was consistent with existing literature on the efficacy of diversity training, especially literature that lauds the positive effect that diversity training has on cognitive learning [Reference Bezrukova, Spell, Perry and Jehn16]. This also suggests a benefit to attending one or multiple workshops. Future diversity training may model a curriculum series of DEI topics to build upon learning and provide more opportunities to practice skills and demonstrate sustained commitment to DEI initiatives. Additionally, while attendance at all DEI workshops in a curricula series should be encouraged, participants will benefit from attending one or more workshops.
Analyses assessing perceived disparities in the workplace indicated significant increases across sessions; however, further analysis revealed that these findings were only significant for URM participants. Over the course of the DEI workshops, URM participants reported perceived disparities in mentoring for mentees from diverse backgrounds, perceived disparities in professional career progression and promotion, and confirmed the perception that the lack of DEI is a problem within the workplace. A possible understanding can be drawn from the theory of stereotype threat [Reference Casad and Bryant37]. Participating in the DEI workshop series may have unearthed the previously internalized experiences of URM attendees (e.g., “I can’t say for sure that I am being treated differently from my non-URM peers, but I feel overlooked for professional opportunities”), thus transforming perceived disparities from an individual challenge to a collective experience of inequity (e.g., “While it is still disheartening to hear that other URM junior faculty and researchers struggle with mentorship, it’s helpful to know that I’m not the only one”) [Reference Ajayi, Rodriguez and de Jesus Perez38].
Open text feedback also illuminated workshop strengths on the efficacy of comprehensive diversity training that were consistent with the literature, including themes such as facilitation by experts, content tailored to the organization, inclusion of employees from all levels, support from senior leadership, and connection to the group’s operating goals [Reference Bendick, Egan and Lofhjelm17]. In addition, the department had a previously established diversity committee made up of both faculty and staff members who independently participated in campus professional development for DEI foundations. The department chair supported the committee and attended every workshop in the DEI training series. Together, these factors likely contributed to the overall success of the DEI training series.
There were limitations to our study. First, our data do not allow us to assess the workshop’s impact on behaviors or whether participants practiced skills introduced in the workshop. Also, we could not discern unique attendees from the total and how many attended all four sessions. Changes in behavior can take longer to observe which may explain why we did not see significant improvements in attendees’ agreement that they are responsible for improving DEI within their workplace or that they plan to work to improve DEI in their workplace. Discussions on challenges transferring learning to behavior often cite the need for longitudinal initiatives, leadership accountability, and strategies to support new behaviors [Reference Kirkpatrick and Kirkpatrick39]. Our department is conducting a 1-year post-series questionnaire to evaluate longer-term changes in knowledge and skills and whether participants have applied DEI skills. On a broad institutional level, the academic university implements many of the best practices listed in Enders and colleagues’ recommendations for a comprehensive DEI action plan, such as a biannual climate survey and incorporating best practices in addressing bias in search committees. However, because we operate in a very large matrixed health science institution, it is imperative that DEI action plans are further refined at the departmental level to address unique needs and maximize impact [Reference Enders, Golembiewski and Orellana40].
Overall, curating a multipart DEI workshop series successfully improved participants’ knowledge and attitudes and aligned with core values of culturally congruent training for public health practitioners and researchers. As professionals, we educate the next generation of leaders and engage in service, research, and public policy that frequently shape narratives surrounding historically underrepresented and vulnerable communities. Additionally, as structural racism influences health outcomes but also the scholarly and academic knowledge about populations, etiology, and health disparities, diversity education and equity, initiatives must become centered in these endeavors. DEI initiatives are a collective responsibility, but often rely on the energy provided by historically excluded groups. While our attendees were largely representative of our department demographics, future efforts should be mindful of the need to increase engagement of faculty, especially from male and/or White identified faculty members. While our curricula was developed to engage a broad audience of academic researchers, one insight learned is the importance of balancing these collaborative sessions with additional affinity-style spaces and curricula to address the needs for various subgroups (e.g., staff, faculty, trainees, URM, women, LGBTQI +, etc.)
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/cts.2024.6
Funding statement
No funding was received to assist with the preparation of this manuscript.
Competing interests
The authors have nothing to declare.
Open access
