Learning by Doing: Using an Undergraduate Research Lab to Promote Diversity and Inclusion

ABSTRACT Undergraduate research labs have long been recognized as having educational and professional benefits, but much less attention has been given to how they can promote diversity and inclusion. Without a conscientious effort to promote these goals, labs are likely to replicate and perpetuate existing inequalities. This article discusses our experiences and lessons from launching an undergraduate research lab in a political science department at a research-oriented state university. It concludes with suggestions for other departments interested in starting undergraduate research labs. Promoting diversity and inclusion by working with undergraduates is unlikely if faculty are recruiting students individually outside of a lab, if the burden is on students to approach faculty, or if labs do not take practical steps to make this happen.

I nspired by our department's "Learning by Doing" initiative, we created an undergraduate research lab that seeks to improve experiential learning for students by prioritizing diversity and inclusion. Our department's initiative stresses the importance of involving students in hands-on research as both a learning tool and an important part of their professional education and acquisition of practical, real-world skills. We contend that a research lab is an ideal way to use experiential learning to encourage diversity and inclusion.
We first examine the state of racial, ethnic, and gender biases in the discipline, followed by a review of the benefits of undergraduate research labs. Although undergraduate research labs have long been recognized as having educational and professional benefits, much less studied is how these labs can promote diversity and inclusion. We explain the specific objectives of our lab and how we seek to improve existing lab models by promoting diversity and inclusion. Then we describe our experiences in launching our lab in 2020. We conclude with suggestions for other departments that may be interested in creating undergraduate research labs at their own institution. We argue that these considerations are urgently needed. Without taking practical steps to advance diversity and inclusion, labs are likely to reinforce existing inequalities.

DIVERSITY AND INCLUSION IN POLITICAL SCIENCE
Addressing diversity, equity, and inequality is one of the most pressing challenges for academia today because gender, racial, and ethnic bias manifests at all stages of the research process. Djupe, Sokhey, and Smith (Forthcoming) document the institutional factors that contribute to significant disparities across gender and race in academic careers in the social sciences. For instance, women often are underrepresented in the publication of journal articles (Djupe, Smith, and Sokhey 2019;Saraceno 2020;Teele and Thelen 2017) and in books (Samuels and Teele 2021). Furthermore, women face several challenges to career advancement in academia. They are under-cited (Chakravartty et al. 2018;Dion, Sumner, and Mitchell 2018) and less likely to be promoted to associate and granted tenure (Hesli, Lee, and Mitchell 2012). The COVID-19 pandemic has only increased the need for promoting diversity in political science because it reinforced and intensified existing structural disadvantages that women face, especially those who also are parents (Breuning et al. 2021;Shalaby, Allam, and Buttorff 2021).
There also is a significant and less-studied gap in the inclusion of nonwhite scholars who are Black, Indigenous, and/or People of Color (Garcia and Alfaro 2021). Chakravartty et al. (2018) argued that the white-male paradigm is the predominant lens through which academic work is viewed. As a result, nonwhite scholars are underrepresented in publications, citations, and editorial positions. Furthermore, Dupree and Boykin (2021) demonstrated how there are psychological, interpersonal, and structural factors that impede racial equity in academia, including racial ignorance, stereotype expression, and the under-resourcing of scholars of color. Racial biases intersect with gender biases when women of color face incredible challenges from being hyper-observed and also ignored by the white-male-dominated profession of academia (Michelson and Lavariega Monforti 2021).
As is evident, there are clear biases in political science that recent events have only spotlighted. To combat these racial and gender gaps, it is important to improve diversity and inclusion in the discipline. As Cassese, Bos, and Schneider (2014) highlighted, women are more likely to discuss gender-related topics in scholarly work. Dion, Sumner, and Mitchell (2018) further showed that the more gender diverse a subfield is, the smaller the gender gap in citations is, with women more likely to cite work from female authors. Dupree and Boykin (2021) argued that one source of racial bias in academia is the lack of resources available to these scholars. If the discipline can improve inclusion and outreach to these communities, it can make progress on closing these gaps and diversifying the discipline. To promote inclusion in the discipline and lift up a diverse community of political scientists, we must invest in our undergraduates from historically underrepresented groups to generate a new, diverse generation of scholars.

THE BENEFITS OF UNDERGRADUATE RESEARCH LABS
Having undergraduates work with faculty on research is widely thought to improve educational outcomes (Becker 2019). Research opportunities improve student retention especially among those most at risk of dropping out (Gregerman et al. 1998) and generate interest in topics (Russell, Hancock, and McCullough 2007). Research experiences enhance students' understanding of concepts (Ishiyama and Breuning 2003); promote the ability to think critically, synthesize ideas, and learn independently (Becker 2019;Ishiyama 2002); and improve their success after graduation (Hathaway, Nagda, and Gregerman 2002;Mabrouk 2009). Social science research labs also have been shown to promote learning and produce high-quality data (Becker 2019;Glazier and Bowman 2019;Lei and Chuang 2009;Stover 1979).
Much less explored is how a lab can be an ideal initiative through which to promote diversity and inclusion. Although diversity and inclusion are important goals for many universities and colleges, diversity does not automatically translate into inclusion in academic and campus life or success after college (Tienda 2013). A survey of introductory economics students revealed that women and underrepresented racial and ethnic minority students report less relevancy of the course material to their own life, less belonging, and less belief that economics is something at which they could excel (Bayer et al. 2020). Having a more diverse student body and courses on diversity and inclusion are obviously important, but they should be part of larger efforts on campus.
Research also shows that students would benefit from diversity experiences outside of the classroom and, in particular, more interpersonal interactions with racial diversity. Bowman (2011) showed that these types of diversity experiences improve civic attitudes and behaviors. Based on a quantitative meta-analysis, he concluded that "Colleges and universities must work not only to maintain a racially diverse student body but also to facilitate meaningful interactions among students from different racial backgrounds" (Bowman 2011, 49). If we want to advance diversity and inclusion, classes are not enough.
Undergraduate research labs can advance diversity and inclusion goals by prioritizing the admission and mentoring of underrepresented students. Labs also can foster community through professionalization and socialization events that encourage students to form personal connections with one another and faculty. If some of the lab's substantive research relates to diversity and inclusion, it advances knowledge and interest about these issues and sends a powerful signal about the university's values. Furthermore, without a conscientious focus on diversity and inclusion, labs are likely to exacerbate existing inequalities.

OUR UNDERGRADUATE RESEARCH LAB IN 2020
We began with a pilot lab in summer 2019. Six students were recruited from courses taught by Dr. Sarah Wilson Sokhey, including two women and four people of color. 1 Jeffrey Nonnemacher was the graduate research assistant for the lab, assisting with the management of the lab and the research project. Students worked together on a project that coded news articles and then presented their work to the political science department in fall 2019. Based on results of the pilot, we received departmental and university funding to establish a research lab.
Our STUDIO Undergraduate Research Lab officially launched in spring 2020. 2 We admitted undergraduates as lab members who were matched with faculty projects and paid to work as research assistants. Lab members attended professionalization and socialization events and were encouraged to present their research at our university and other venues as well as to submit their work for publication. We began in person and transitioned to be fully remote in March 2020 due to the COVID-19 pandemic. We had several specific objectives, as follows: • Goal 1: Promote diversity and inclusion.
• Goal 2: Develop professional skills including teamwork; present research in professional contexts (e.g., conferences within the department, university, and larger community); and publish the results of research to help students be Without taking practical steps to advance diversity and inclusion, labs are likely to reinforce existing inequalities.
academically successful and competitive in applying for other internships and jobs. • Goal 3: Foster knowledge about the social sciences including theoretical, empirical, methodological, and policy-orientedwork.
In spring 2020, we invited applications from undergraduate students and asked faculty and PhD students in the political science department to submit proposals for projects on which students could work. Of 21 applicants, 19 students were accepted, including nine women and three people of color. 3 The admitted students were assigned to five different research projects, which included work on a human rights dataset, climate change, immigration policy, and trade liberalization. Two events were held in person before the pandemic necessitated the lab going fully remote.
To promote teamwork and a better research output, we assigned two or more students to a single project whenever possible. For instance, three students were assigned to update a human rights dataset to enable them to check for intercoder reliability. In other projects, students coded the text of news articles and/or compiled news articles and primary and secondary sources. Students were paid a minimum hourly wage and were expected to work about 5 hours a week.
Based on diversity in admissions and feedback from faculty, graduate students, and undergraduate students, the lab accomplished its objectives reasonably well in spring 2020 despite the challenges of the pandemic. Faculty and graduate students working with the lab overwhelmingly reported that they were satisfied with the quality of research assistance; only one faculty member was not.
Six students responded to an anonymous exit interview after spring 2020. 4 In their positive feedback, students stated the following: • "I was able to understand better how research projects are conducted." • "It made me learn that there are many more quantitative methods than I originally thought." • "I believe that even just being around others who were conducting research gave me a greater idea of how academic research is conducted and the work associated with it. Additionally, the quantitative/qualitative dichotomy was helpful." • "It was super fun!" Suggestions for improvement included the following: • "I think more joint events (of course, COVID makes things a little more difficult) where members and faculty discuss their research process and what undergrad researchers are doing. I believe hearing how others are conducting their research would have been helpful, even if the projects were quite different." • "Face-to-face lab was certainly better. It seemed more connected and organized. I am not sure if this is unique to me, but it seemed that when things went remote, the whole process was a little deflated. Communication with my professor and other undergrad researchers all but ended when summer began." • "I am not sure if this was my fault or not, but…the [lab] essentially ended when the school year did. I was told those involved in the project I was working on would have a meeting relatively quickly after the school year ended and I never received another communication. I, obviously, could have emailed and do regret I did not. Even still, it seemed strange to me that it did end in that way." As these comments reflect, communication and community building suffered from the transition to going remote. In fall 2020, in an effort to better promote diversity and inclusion, we recruited more broadly and asked whether students had a work-study award. As a result, 103 students applied. One faculty member narrowed down the top 40 candidates; another faculty member and two PhD students helped choose which students to accept from those 40 candidates. The main criteria for admittance were enthusiasm and how well a student's research interests fit with proposed projects. Of the 23 students accepted, there were 16 women, five people of color, and three international students. 5 We accepted 20 students as "honorable mentions" and invited them to attend lab events.
We had 13 project proposals of which only two were denied because of concerns that they did not sufficiently promote "learning by doing." The accepted projects included two doctoral projects and topics covering COVID-19 in Bolivia, COVID-19 in Russia, diplomatic relations, European Union legislative activity, policy diffusion, statistical software in undergraduate research methods, climate change and indigenous communities, Qatar's immigration policy, misinformation in the United States, human rights, and media in the Middle East. The research assignments included collecting news articles and primary and secondary sources, coding reports, and scraping websites.
Although we would have preferred to be in person, the remote format in fall 2020 appeared to work well. Students and faculty had regular communication and attended remote events; many students returned to work with the lab in spring 2021 when it was still fully remote. Our events in fall 2020 included two sessions on academic and professional writing facilitated by Dr. Levente Szentkirályi (University of Colorado Boulder), a session on publishing in two departmental journals for undergraduate authors, and a session with Dr. Betsy Sinclair (Washington University-St. Louis) about civic engagement. Attendance ranged from two to 15 students.
Much less explored is how a lab can be an ideal initiative through which to promote diversity and inclusion. Although diversity and inclusion are important goals for many universities and colleges, diversity does not automatically translate into inclusion in academic and campus life or success after college.
Students overwhelmingly reported good experiences in fall 2020. Their favorite remote events were the two sessions on writing, which were well attended. Some students indicated, understandably, that they did not want to attend the remote events because of "Zoom fatigue"; however, most indicated that they were interested even if they could not join. One student wrote to us that she "…really enjoyed the [STUDIO] experiment. I feel the warmth of the professors and get along well with my team members. We meet every week, and I look forward to sharing something. That was the happiest thing during the epidemic." When asked if they learned more about social science research, all of the students from fall 2020 responded that they did. 6 In spring 2021, 17 of our lab members continued working with the lab. Of those who left, three students had completed their work after two semesters with the lab and one student had graduated. No one indicated that they left because of a negative experience. We admitted five new students and accepted two new research projects, including one that addressed citizenship and civil war. As such, our projects spanned five regions of the world and included work on American politics, comparative politics, political theory, international relations, public policy, methodology, and pedagogy.
Although our experiences are recent and necessarily based on limited observations, we think our lab has a good start in promoting diversity and inclusion. In spring 2021, we scheduled nine events for student presentations. The first event was three women presenting their work on updating a quantitative human rights dataset including analysis generated using R. Our second event included two women and one person of color presenting their data collection about Syrian migration policy shocks. Our third event was three women presenting, all of whom emphasized the importance of including computer programming and coding skills in introductory political science methods courses. Given that our institution is predominantly white and that women are underrepresented in methodological fields, we found these events to be tremendously encouraging.

SUGGESTIONS FOR LABS TO PROMOTE DIVERSITY AND INCLUSION
Instructors also should "learn by doing": we must start taking practical steps to be more diverse and inclusive if indeed that is our goal. Our lab model can be adopted even with limited resources. We started with one faculty member recruiting a group of students and one graduate research assistant, and we used a limited budget to expand this to a lab-based model. Our main financial cost is paying students for their work. When budgets are especially limited, employing work-study students may make hiring them more feasible. The administrative work includes recruiting students and faculty, making acceptance decisions, matching students and faculty, organizing events, and regularly checking in with participants.
We hope that this example provides a roadmap for others and advances the discussion about how to meet the urgent need for more diversity and inclusion on our campuses. Without taking practical steps to promote diversity and inclusion, there is every reason to believe that labs will continue to replicate existing inequalities. We draw on the expertise of those who have long studied and written about diversity and inclusion. Based on our experiences and advice we have received, we offer the following suggestions.

Recruit Broadly
We strongly suggest recruiting broadly for both students and faculty proposals. We do not require applicants to be political science majors, we hold events to recruit students, and we ask faculty to reach out to students both individually and through class announcements. We are grateful for advice received at the American Political Science Association teaching conference in fall 2020 to emphasize the opportunity to develop nonacademic professional skills to recruit a more diverse set of applicants. Moving forward, we will ask whether students are part of an alliance organization at our university that includes programs intended for underrepresented groups of students. 7

Use Work-Study
In recruiting students broadly, it is important to consider the role of work-study positions. Consider reserving some positions in a lab for students who have received work-study awards. These students already have completed a Free Application for Federal Student Aid (FAFSA) form that determined a financial need. Hiring work-study students also allows for a cost-effective way to include more students who otherwise would not have this opportunity. Many work-study positions do not include educational opportunities that combine socialization and professionalization events and that prioritize diversity and inclusion. To include work-study students, it also is important to remember how and when these students are finding positions.

Make Applying Easier
Undergraduate research labs should make it easy for students to apply and should prioritize enthusiasm over previous experience. A disadvantage of initiatives such as the Undergraduate Research Opportunity Program (UROP) is that they place the burden on students to seek out faculty. Existing research strongly suggests that this will benefit already privileged students rather than drawing in underrepresented or first-generation college students who, for various reasons, may be less likely to know about such programs and are less likely to approach faculty, especially at a large research university. We intentionally designed our lab to make it easier for students who are enthusiastic and eager for this type of experience to apply even if they lack clear research interests or personal connections with faculty. In the recruiting process, we suggest emphasizing that this opportunity does not require experience. 8

Recruit Diverse Faculty and Provide Mentor Training
It is important to have as diverse a group of faculty members as possible and faculty with different specializations and areas of expertise. To this end, in fall 2021, we included a call for faculty Instructors also should "learn by doing": we must start taking practical steps to be more diverse and inclusive if indeed that is our goal.
research projects that address issues related to diversity and inclusion. 9 Moreover, faculty should be trained in mentoring practices that promote diversity and inclusion. Brown and Montoya (2020, 784) stated that "Mentoring, however, does not shield these groups from inherent biases in these structures and can itself perpetuate oppression" and advocated for a "more intersectional and actionoriented model of mentorship that moves beyond an emphasis on survival and toward empowerment and transformation." In fall 2021, we introduced a series of training sessions for faculty participating in the lab to provide concrete ways to be better mentors who more effectively promote diversity, equity, and inclusion. This is especially important because underrepresented faculty often are burdened by working as "universal donors of mentorship" who are disproportionately tasked with advising certain students (Brown and Montoya 2020;Jimenez et al. 2019). We want to expand knowledge about better mentoring.

Communicate Clearly
Cox (2019) explained that assignments must be explicit about tasks to be effective. A lab is uniquely poised to provide support for clear faculty communication with students by encouraging and requiring regular check-ins and lab events. Communication is especially important if students are enthusiastic but lack experience. We should not assume that students already understand the jargon and the often-unspoken norms of our field.

Build Community
Allen-Ramdial and Campbell (2014) noted that institutional culture may include shared values and beliefs that respect diversity; however, this may not align well with the institutional climate that affects a student's sense of belonging. Social events and lab-specific training events have the potential to improve climate for historically underrepresented students. This includes promoting student presentations and publishing. A lab can build community and mentor students, making them aware of further opportunities outside of the classroom. The lab should be a stepping stone to future success.

Pursue Systematic Assessment
To date, our assessment is based largely on feedback from faculty and students in the first two semesters of our lab. This is insufficient for building a long-term program that effectively promotes diversity and inclusion. In fall 2021, our lab included a research project with an undergraduate research assistant working on developing more systematic assessment tools. These included better exit surveys and suggestions for a review by those who are experienced in diversity and inclusion issues.

CONCLUSION
The preliminary evidence suggests that our undergraduate research lab model can improve on previous efforts. We hope other departments will consider creating labs that prioritize diversity and inclusion so that initiatives to improve higher education do not perpetuate existing inequalities. Our students deserve no less.

ACKNOWLEDGMENTS
For funding the students involved in this project, we are grateful to the University of Colorado Boulder's UROP and the political science department's "Learning by Doing" initiative. We also are extremely grateful to the undergraduate students who participated in this research lab. They are amazing and already are going on to do great things. Finally, we thank Dr. Adrian Shin, who stepped in as the temporary director of the STUDIO lab in spring 2020 and who continues to help make STUDIO a success. Notably, Dr. Shin is responsible for coming up with "STUDIO" as the name of our lab. ▪ N O T E S 1. Three students applied independently to the Undergraduate Research Opportunity Program (UROP) and each received funding of $3,000. The faculty member received a team grant from UROP that partially financed the hiring of three more students. Additional funds were provided from the political science department, which made it possible to pay each of the six students $3,000 for their summer work. 2. The STUDIO lab website is www.colorado.edu/research/studiolab.