WHAT MAKES A GOOD CODE OF CONDUCT?

The purpose of a code is twofold: (1) it discourages negative behavior from happening, and (2) it responds to negative behavior if it does occur. How do we know if a code achieves its purpose? Our study focuses on the content of the codes as indicators of their potential to stop harassment from occurring and to respond appropriately to harassment when it does occur. That is, we evaluated the content of the code, not conference-level outcomes. In an ideal world, we would link the policy text to changes in actual behavior. However, the first goal makes it difficult to evaluate the success of a code—it would be difficult to find evidence that someone intended to use a conference to harass others but then changed their mind after reading the code.Footnote ⁵ If a code is successful in its second goal by encouraging reporting, its adoption may lead to an increase in the number of reported incidents. This could make it appear that the number of incidents has increased (i.e., a failure of the code) when, in fact, it is more reporting of the same number of incidents (i.e., a success of the code). We also are limited by a lack of data on incidents and outcomes because those data are held by conference organizations, which rightly are concerned about privacy and confidentiality.

What can we learn, then, about code effectiveness solely from code content? In general, we know little about whether written policies translate into actual decreases in harassment. In a recent review of the literature on sexual harassment in higher education, Bondestam and Lundqvist (Reference Bondestam and Lundqvist2020, 406) found that “there is almost no evidence-based research on the actual effects of policies on, for example, decreasing prevalence of sexual harassment.” As Stubaus (Reference Stubaus2023, 66) wrote: “More research is needed on whether formal sanctions are effective in preventing repeat harassment and, if they are, which sanctions are most effective….” Despite this lack of research, we can extrapolate a framework on what theoretically might make codes more effective in terms of both preventing and responding to harassment by reviewing best practices from nonacademic sectors.Footnote ⁶ The US Equal Opportunity Commission (n.d.), for example, lists steps for preventing harassment in small businesses that include informing employees about harassment, identifying reporting contacts, and efficiently investigating complaints. The International Labour Organization (ILO) suggests codes as one form of a policy document to reduce harassment (2022, 30). The ILO (2022, 28) recommends that policies to reduce harassment include definitions and examples, information on how to report complaints, and provisions for fair investigations that protect confidentiality.

In addition to recommendations for policies designed to limit harassment, we consider recommendations for how to design an effective code, even if that code is applied to a non-harassment domain. For example, the corporate social responsibility (CSR) literature often addresses the content and effectiveness of codes in changing business behaviors. To compare the content of CSR codes in international franchising, Preble and Hoffman (Reference Preble and Hoffman1999, 247–49) searched for the presence of ethical statements (equivalent to our “definitions” dimension) and enforcement mechanisms (equivalent to our “enforcement/adjudication” dimension). Another study of CSR code content noted that codes need to specify how to report misbehavior to a “competent body” (Béthoux, Didry, and Mias Reference Béthoux, Didry and Mias2007, 84). Researchers also study how CSR code users (i.e., employees) perceive elements of effective codes. Employees express the importance of language that requires code violations reporting, mechanisms that protect anonymity, and consistent enforcement (Schwartz Reference Schwartz2004, 334). When managers are asked how to increase the “teeth” of codes to reduce discrimination in the workplace, they contend that enforced sanctions result in more effective codes (Petersen and Krings Reference Petersen and Krings2009).

As further evidence of what makes an effective code, we can review critiques of codes—a “good” code should have in it what is lacking in “bad” codes. For example, consider the recent adoption of a code of conduct by the US Supreme Court. Many experts have criticized the “lack of an enforcement mechanism” in the document (see, e.g., Liptak Reference Liptak2023). A similar argument was made by Spar (Reference Spar1998) regarding monitoring mechanisms in corporate codes. These critiques suggest that enforcement is a crucial component in an effective code.

The common elements from these sources indicate that preventing harassment with codes involves informing and responding. Conference attendees must know which behaviors are prohibited; they must have an effective means of reporting troubling behaviors; and they must know that their reporting will lead to consequences. In other words, for a code to be effective in curbing discrimination and harassment, it should (1) define what constitutes discrimination and harassment; (2) provide reporting mechanisms—ideally, mechanisms that protect confidentiality and privacy; and (3) specify how reports are adjudicated and enforced, including potential consequences of code violations. The fourth section defines and evaluates specific measures concerning these three dimensions.

WHICH TYPES OF CONFERENCES ARE MORE LIKELY TO HAVE CODES?

Which factors explain why some conferences have a code whereas others do not? In general, better-resourced conferences should be more able to invest time and energy in developing a code. In addition, the leadership composition of a conference may matter, specifically if there are more women in leadership roles or if there are other formalized means for putting harassment issues on the agenda. This section describes six potential correlates for the likelihood of having a code: conference age; size; presence of permanent staff; women in leadership roles; presence of status, professional ethics, or diversity committees; and conference mode (i.e., online or offline).

First, we considered the age of a conference, with an expectation that older, more established conferences will be more likely to have a code. Older conferences may have a larger base of resources from which to draw. They also may be more aware of the issue of harassment at conferences. For example, they may have had past scandals that prompted action. We measured the number of years that a conference has been in existence.

Second, we expected that larger conferences will be more likely to have a code. Again, the primary logic supporting this expectation is the availability of resources to develop a code. In addition to having more resources, larger conferences may have more need for a code. It is more likely that most participants in smaller conferences know, or know of, one another. These smaller conferences may rely more on informal mechanisms to mitigate harassment, such as not admitting known harassers or relying on “whisper networks” to indicate who is to be avoided. We measured the size of conferences by the number of days of their most recent conference.Footnote ⁷

Third, we tested whether the presence of permanent conference staff is correlated with a code. This is an extension of the logic of conference size but focuses on the financial resources of a conference. We expected that conferences with the resources to hire permanent staff also are more likely to have the resources to draft a code.

Fourth, we tested for the presence of women in leadership roles. From the publicly available documentation about the effort to address harassment in political science (e.g., Binder Reference Binder2019; Brown Reference Brown2019; Sapiro and Campbell Reference Sapiro and Campbell2018), it is clear that one driving force behind the creation of codes is individuals with a strong interest in deterring harassment at conferences. Those who work to bring the issue forward often are women; for example, in the case of APSA, a group of “senior women” wrote a letter of concern to organization leadership in 2015 (Sapiro and Campbell Reference Sapiro and Campbell2018, 5). We systematically evaluated the presence of women in leadership by researching every person listed in a leadership roleFootnote ⁸ to observe which pronouns were used to describe them, assuming that “she/her/hers” are used for women.Footnote ⁹ Of the surveyed conferences for which we could find information, 90% had at least one woman in a leadership role; the median percentage of women in leadership was 50%.Footnote ¹⁰ We used percentage of women in leadership as our measure in the quantitative analysis.

Fifth, we searched for the presence of a relevant status, professional ethics, or diversity committee (binary measure in analyses) in the organizations holding a conference.Footnote ¹¹ Committee work is another way that individuals can bring the issue of harassment to the conference agenda. The presence of such committees speaks to both the resources available to the conference and the directives from leadership to address these issues.

Sixth, the conference mode also may affect the prevalence of codes. Strictly online conferences may be less likely to have a code than offline conferences. Online conferences do not create opportunities for physical harassment; therefore, organizers may not see the need for a code.

We emphasize that these factors are not the only possible drivers for code adoption and creation. Further research into the exact details of how codes are created, maintained, and enforced is warranted to better understand how they come into existence.

CONFERENCES, CODES, AND CONTENT ANALYSIS

We compiled a list of 177 conferences and workshops from political science and adjacent fields. To be included in the study, a candidate event had to meet at least one of two main criteria as well as additional secondary criteria. First, the event could be open to submissions, meaning that presenters are not by invitation only and that we could find an online record of an open “call for proposals” or a contact link for scholars to self-nominate to present. Second, attendance at the event must have been open to all who wanted to register; this excluded, for example, departmental speaker series because attendance generally is limited to department or university members. The second criterion means that we included events with invitation-only speakers if the event was open to all attendees. Most of the included conferences met both criteria; all met the second criterion. As additional criteria, we considered only those events based in the United States, and the event had to have a website.

We searched for conferences in seven areas of political science: general, American politics, comparative politics and area studies, international relations, methodology, political theory, and regional and subgroups. The first six areas are standard subfields in political science. We used the last type, regional and subgroup conferences, to refer to either regional conferences that were open primarily to scholars based in a certain region or those that were designed for groups of scholars based on specific identities. Because of the increasingly interdisciplinary nature of our field, we also included prominent conferences from adjacent fields (e.g., sociology, economics, and history) and area studies (e.g., the African Studies Association). Table 1 lists the number of identified conferences in each area. Online appendix A contains the full list of conferences and a description of how we compiled our final list. We gathered data on the factors potentially associated with the likelihood of a code from conference websites.

Table 1

Numbers of Conferences and Workshops by Type

We searched the website of each conference for a code from the most recent gathering. To be counted as a code, the document had to contain provisions regarding behavior at conferences. Documents that counted as codes by our definition were not always officially labeled as such. Other relevant document titles included “Statement of Diversity and Inclusion,” “Professional Conduct Policy,” “Ethics Statement,” “Policy Against Harassment,” and “Conference Rules & Guidelines.”

Codes, when found, were labeled manually by the two authors of this study. In the initial round of coding, the average inter-rater reliability as measured by Cohen’s Kappa was 0.63 for 21 closed-ended questions, which is in the range of a substantial yet still unsatisfying agreement between the two annotators. We therefore undertook multiple rounds of annotating and included a final correction and consensus-building step. Ultimately, we reached 100% agreement on all of the annotations (see online appendix B for a full description of the labeling process, including example language from codes that met our definitions). The content we labeled for is listed in table 2.

Table 2

Content Variables and Annotation Questions

RECOMMENDATIONS AND CONCLUSION

The increase of codes at political science conferences demonstrates that there are many who want to make our field more welcoming for all. Although codes are not a panacea to cure all academic conferences ills, they are a step that organizers have taken to reduce discrimination and harassment. Approximately 19% of the surveyed political science conferences had a code. We found evidence that larger, better-resourced, offline conferences are more likely to have a code. We also found that conferences with relevant committees are more likely to have a code. We found no association between the prevalence of women in leadership and the presence of a code—perhaps because most of the conferences have a high proportion of women in leadership. We note that these descriptive findings are from bivariate analyses without accounting for likely confounding among factors.

Prior literature suggests that more effective codes will contain definitions, information about reporting channels, and procedures for enforcement. We found variability in the content of political science codes on these dimensions. The analyzed codes did well in identifying prohibited behaviors but they did less well on reporting mechanisms and procedures for enforcement. We found that few codes had a way to report complaints to bodies external to the conference association. Few codes also had a truly external ombudsperson who could provide information and advice to potential complainants.

One theme that emerged from our study in terms of both code prevalence and content is the role of resources in creating and enforcing codes. Ending conference harassment will require time, energy, and expense, whether via a code or through other mechanisms. Maintaining external reporting channels, hiring ombudspersons, and staffing committees are not inexpensive tasks. Relying on volunteer labor has a significant opportunity cost for those who serve. Our general findings and the theme of resources lead us to make the following suggestions to those conferences that are considering a code of conduct:

• • Draft codes to include clear definitions, reporting, and enforcement/adjudicating.

• • Use prior examples as a model for new codes, but first determine whether their provisions fit what the conference is willing or able to do.

• • Ensure that codes are backed by sufficient resources for enforcement and maintenance.

• • Create standing status, ethics, and/or diversity committees to reevaluate, modify, and enforce codes of conduct.

As a discipline, there are ways that large conference organizations and wealthier parties can support measures to improve our understanding of what works and our ability to provide it. We offer the following suggestions:

• • Organizations should be transparent about how codes of conduct are created and implemented because examples from one organization shape developments in others.

• • Large, well-established conferences should consider how they can use their leadership position to improve codes by sponsoring research and workshops to share information among conference organizers.

• • We should consider pooling resources among conferences or across the discipline to maintain external ombudspersons, reporting channels, and enforcement procedures.

In addition to these normative suggestions, this study has prompted important questions for future research. We want to see more research on actual code effectiveness. In the CSR literature, research has found a significant relationship between code quality and businesses’ organizational cultures (Erwin Reference Erwin2011). Does the same hold for conference codes? Researching how a code of conduct can change behavior at conferences may involve partnering with organizations to determine whether enacting a code changes reporting and/or incidents of harassment. Qualitative work on the history of how codes are enacted and how they work in practice would complement our quantitative findings on the diffusion of codes among organizations. Finally, there are many other factors that could explain the prevalence of codes or the prevalence of certain elements in them. Future work could consider the role of scandal, for example, or the role of non-white conference leadership. Our study provides an important first step in understanding the prevalence and content of codes of conduct in political science.