Adverse effects on research

Trying to pursue science and activism simultaneously will make it more challenging to conduct the kind of rigorous and unbiased research that Follmer et al. seek. Early philosophers of science, such as Francis Bacon and William Whewell (Losee, Reference Losee2010; Snyder, Reference Snyder2023), argued that the primary aim of empirical science is to understand and build knowledge about the natural world. This is done by introducing systematic data collection procedures, ideally planned in advance. Many of the practices that we use today in our scientific research were put in place to reduce subjectivity and mitigate the myriad biases people are prone to in the acquisition of knowledge. For example, null hypothesis significance testing (although admittedly flawed) helps to introduce an objective standard by which researchers are able to make claims, and standardized measurement adds a consistently applied procedure in collecting observations that can be replicated by other researchers. As another example, the contemporary open science movement is designed to reduce bias and subjectivity by preregistering procedures and increasing transparency, both limiting a researcher’s ability to make unwarranted claims based on ad hoc, idiosyncratic interpretations of results.

Researchers ideally begin a project with an open mind about what the results might be. Investigations ought to be a search for truth, not an attempt to find evidence to support one’s personal beliefs and values. When we see our research as doing the work of activism, we accentuate some of the very biases that scientific methods were intended to attenuate, not the least of which is confirmation bias. It is clear from Follmer et al.’s article that they presume to know the results of the studies that they propose. That is what allows them to conclude that research (along with activism) will “stimy these regressive policies” (p. 42). How could one draw this conclusion without presuming that future results will support their existing beliefs? From the activist perspective, empirical evidence is wielded as a tool for advocacy rather than a means for testing a research question in good faith.

To presume the results of tests of DEI policies and programs is to ignore the possibility of being wrong. However, it is common in science (including DEI research) that the proposed positive effects of an intervention fail to materialize, and unintended negative consequences emerge (Sitzmann & Schwartz, Reference Sitzmann and Schwartz2024). Leslie (Reference Leslie2019) outlines several possible unintended consequences of diversity initiatives, and these go beyond the backlash effects among majority group members that Follmer et al. mention in their section “Motivations for Dismantling DEI Policies.” For instance, at the risk of stating the obvious, diversity is difficult to manage (Leslie et al., Reference Leslie, Flynn, Foster-Gimbel and Manchester2023; van Knippenberg & Schippers, Reference van Knippenberg and Schippers2007). More diversity does not necessarily mean better business outcomes or innovation (van Knippenberg et al., Reference van Knippenberg, Nishii and Dwertmann2020). In some studies, diversity has been shown to have negative effects on group performance (van Knippenberg et al., Reference van Knippenberg, De Dreu and Homan2004). Only when it is effectively managed can diversity (particularly deep-level diversity) improve business outcomes and innovation (Chen et al., Reference Chen, Hsu, Lee and Mack2024; Homan et al., Reference Homan, Gündemir, Buengeler and van Kleef2020).

As another example, in research funded by SIOP’s anti-racism grant, Derricks and her colleagues found that organizational recruitment efforts targeting Black Americans “backfire by activating concerns about being tokenized” (Carle, 2022, para. 20). Thus, it is not just backlash effects among majority group members that are worth considering; it is that DEI policies and programs could unintentionally be detrimental to the very people that they are aimed to support. These findings are consistent with work by Gray et al. (Reference Gray, Spector, Lacey, Young, Jacobsen and Taylor2020) on unhelpful help, which involves well-meaning attempts to help someone that, in reality, makes things worse. Similarly, a recent paper found that “well-intentioned or not, asking members of marginalized groups to approach discussions of intergroup issues and policies through the specific lens of their personal experiences may be subtly undermining” (Vorauer & Petsnik, Reference Vorauer and Petsnik2023, p. 135).

Good intentions often go awry as all interventions have unintended consequences (Watts et al., Reference Watts, Gray and Medeiros2022). Seeing oneself as both activist and scientist likely closes one’s mind to the possibility that a certain DEI program or policy can have unintended negative consequences or might not be effective at all.

Suppressing results that do not support the cause

Trying to pursue science and activism simultaneously will increase the temptation to suppress unfavorable evidence. Follmer et al. argue that “I-O psychologists have a responsibility to stand up against these attacks [on DEI] … the community cannot stand by idly” (p. 41). Although we agree that I-O psychologists have a role to play in informing the debate around DEI efforts, conflating research and activism can lead researchers who see themselves as activists to approach their work with a predetermined conclusion in mind rather than letting the evidence speak for itself. When one approaches research with a closed mind, contraindicating evidence can be discounted in two ways.

First, it is likely to be seen as counterproductive to promoting one’s cause. Consequently, researchers who see their work as a form of activism are likely to be tempted to suppress any evidence they gather that does not align with their preconceptions. This creates a sort of self-imposed file drawer problem, leading to an inaccurate state of knowledge in the published literature. The issue of suppressing evidence extends beyond the data that one gathers oneself. It also encompasses efforts to censor contraindicating evidence gathered by others. The second author of this commentary recalls an incident in the 1980s with one of his students who submitted her thesis results to a journal. Today, we have dozens of studies showing physical attractiveness is an advantage in hiring, but hers was one of the first such studies. The journal rejected her paper with one reviewer arguing that this line of research should not be published because results could harm the feminist cause. Along these lines, recent research suggests that scientific censorship is “often impelled by prosocial concerns” (Clark et al., Reference Clark, Jussim, Frey, Stevens, Al-Gharbi, Aquino, Bailey, Nicole Barbaro, Baumeister, April Bleske-Rechek, Buss, Ceci, Del Giudice, Ditto, Forgas, Geary, Geher, Haider, Honeycutt and von Hippel2023, p. 1).

Second, adopting an activist perspective can make it easy to rationalize questionable research practices like p-hacking. If one “knows” that a certain relationship exists, it seems reasonable to massage and manipulate the data until statistical significance is found. Further, it is easy to ignore disconfirming evidence by finding reasons to discount it while overinterpreting evidence that appears to support one’s preconceptions.

Credibility

Trying to pursue science and activism simultaneously will reduce the credibility of DEI research. Follmer et al. suggest that the accumulation of scientific evidence in favor of DEI programs and policies will help to persuade individuals and organizations of their value and this, in turn, will reverse the recent wave of anti-DEI legislation. Let us entertain the possibility that over the next 5 to 10 years, such favorable evidence accumulates rapidly and unambiguously. Simply because this evidence has been amassed does not mean that the stakeholders Follmer et al. described will believe it. In order to conduct science that is trustworthy and persuasive to the kind of stakeholders who are promoting anti-DEI legislation and policies, I-O psychologists should strive to be beyond reproach. We ought to earn trust by being seen as serious scientists who conduct our research in good faith and who are presenting unbiased results to the best of our abilities. Calling ourselves activists is anathema to that goal. Doing so will undermine our credibility as trusted sources who can be relied upon to provide accurate scientific information.

Recommendations for conducting influential research

We echo Follmer et al.’s call for more DEI research, but we recommend that it be divorced from activism. We have six suggestions for building credibility as we conduct research on DEI topics.

1. 1. Be careful with terminology. Follmer et al. use the term “DEI” throughout their article to refer to various policies, programs, and initiatives, ranging from support for abortion access to diversity training, affirmative action, and inclusive recruitment practices, but they do not provide a clear, explicit definition of what DEI encompasses. Indeed, we see recent instances in the literature of multiple interpretations of this term (e.g., Thomason et al., Reference Thomason, Opie, Livingston and Sitzmann2023; Waldman & Sparr, Reference Waldman and Sparr2023). The lack of specificity can lead to confusion and make it difficult to compare findings across studies. We encourage researchers to consider using terminology that is as specific as possible in referring to particular policies, programs, and ideas. Indeed, it may be preferable to avoid politicized terms like DEI in research reports in favor of more precise language that reflects the activity or program component being investigated.

2. 2. Focus research on specific activities or programs that can be compared among studies, such as blind resume screening, diversity training content, mentoring program structure, or inclusive leadership development.

3. 3. Pay attention to the context in which findings are presented. For example, reporting that there is 25% attrition in STEM doctoral programs for women is hard to interpret in isolation. Meaningful comparisons are needed to provide a frame of reference for interpretation. We would reach a different conclusion about the meaning of women’s attrition if men’s attrition is 10% than if it is 50%.

4. 4. Be as quick to report on positive effects as negative. There is little that undermines credibility faster than only showing results that support your side. Researchers should consider preregistering their studies. This helps to prevent the selective reporting of results and enhances the credibility of the research.

5. 5. The stakeholders for DEI efforts are everyone, so the impact should be shown on all groups, not just those considered marginalized. The inclusion of everyone in studies of the impact of DEI policies and programs makes it possible to show whether everyone benefits, only some groups benefit, or no one benefits. Researchers should also examine potential differential effects across groups, as some initiatives may benefit certain groups more than others. This can provide a more nuanced understanding of the effects of DEI efforts and inform the development of targeted interventions. It also provides context for interpreting results as in Point 3 above.

6. 6. Consider engaging in adversarial collaborations. Adversarial collaboration involves researchers with differing views or perspectives working together on a study to test their competing hypotheses or theories (Clark et al., Reference Clark, Costello, Mitchell and Tetlock2022; Kahneman, Reference Kahneman2003). By bringing together researchers who may have different expectations about the effects of DEI initiatives, adversarial collaborations can help to ensure that studies are designed and conducted in a more balanced manner. This approach can also increase the credibility of findings, as it demonstrates a commitment to impartial investigation and a willingness to engage with alternative viewpoints.

SIOP as an organization has an important stake in being seen widely as a trustworthy authority on workplace issues. To build trust with the public and a range of stakeholders, we should make clear that SIOP is concerned first and foremost with providing the best scientific evidence in an unbiased and objective way.