Introduction
The 2019 British general election witnessed 50 members of the House of Commons not running for re-election. While it is not unusual for some MPs to stand down, the reasons stated for doing so were novel. Abuse, threats and the normalization of a culture of intimidation, especially on social media, were explicitly mentioned by mostly female MPs (Gorrell et al., Reference Gorrell, Bakir, Roberts, Greenwood and Bontcheva2020). This highlights yet another barrier aspirants seeking public office face, and that the height of this barrier may depend on candidate characteristics.
Existing studies offer inconclusive insights to the gender gap in online incivility, with some suggesting female politicians to receive more uncivil content (Collignon and Rüdig Reference Collignon and Rüdig2020; Gorrell et al. Reference Gorrell, Bakir, Roberts, Greenwood and Bontcheva2020), while others reporting the opposite (Erikson, Håkansson, and Josefsson Reference Erikson, Håkansson and Josefsson2021; Greenwood et al. Reference Greenwood, Bakir, Gorrell, Song, Roberts and Bontcheva2019; Southern and Harmer Reference Southern and Harmer2021; Ward and McLoughlin Reference Ward and McLoughlin2020). This may partly be due to sample selection effects, the difference between using elite surveys versus reliance on observed incivility on social media, and many studies selecting on the dependent variable (i.e., uncivil comment). The lack of cross-national data and the omitting of several individual level variables, such as politicians’ online and offline prominence, may further contribute to the mixed record of evidence. This paper aims to address these gaps in the literature by examining the dynamics of gender-based online incivility in a cross-national setting and by comparing male and female politicians with varying levels of prominence. Most importantly, we take one of the first steps in exploring the gender differences in the type of uncivil language in a cross-national setting.
We theorize that the underlying cause of the gender gap in the proportion and type of incivility can be due to bias derived from one’s gender (gender discrimination logic) as well as due to their status and recognizability (prominence logic), with prominence potentially affecting men’s and women’s experiences to a different extent. We use machine learning models to analyze more than 23 million tweets directed at electoral candidates and members of national parliaments from Germany, Spain, the United Kingdom, and the United States at two separate time points: 2014 and 2016–17. Although Twitter was an important campaign tool during these time periods, there were also significant variations in individual politicians’ Twitter use. Our approach thus presents a unique opportunity to investigate what affects the proportion and type of incivility and how the effect of traditional predictors of incivility, such as prominence and candidate viability, may vary depending on politician’s gender and broader context at a time when Twitter was a widely used social media campaigning tool.
While our data offers little consistent evidence of a direct gender effect, women and men at different career stages and at varying levels of Twitter-prominence receive different levels of incivility. By juxtaposing the Twitter experiences of highly prominent members of the legislature and politicians enjoying Twitter-celebrity status to their lesser-known colleagues in four advanced democracies, this study moves beyond existing literature, which largely focuses on highly established politicians in a single-country case study.
Understanding the extent to which widely-used social media platforms, such as Twitter at the time, democratize public discourse by reducing existing inequalities between politicians — or, on the contrary, demobilize certain social groups — has important democratic consequences. Uncivil behavior aimed at female candidates, regardless of their prominence, likely limits women’s choice of possible communication more than men’s. More dramatically, extensive and severe attacks that female politicians experience from the moment they enter politics and social media may not only result in a more cautious online-persona, but also damage women’s willingness to participate in politics (Krook Reference Krook2017). The implications of differential experiences on social media thus go beyond currently active politicians. Increasingly unpleasant climate on social media could affect differently young women’s and men’s eagerness to participate in politics. As such, our findings point to potential policy implications, where a further need to regulate social media content, more similarly to traditional media, may be necessary to ensure the equality of political representation.
Incivility on Social Media
Social media are widely used for information consumption and political communication. Political elites, among others, have widely adopted Twitter as a tool for personalization, mobilization and promotion (Barberá and Zeitzoff Reference Barberá and Zeitzoff2017; Enli and Skogerbø Reference Enli and Skogerbø2013), though strong cross-party and cross-national differences exist (van Vliet, Törnberg, Uitermark and Reference Van Vliet, Törnberg and Uitermark2020). By allowing for (semi)public profiles and asymmetric following of others, Twitter has provided an opportunity to directly communicate with “powerful” individuals who were previously beyond most users’ reach. Furthermore, Twitter was one of the first platforms to offer an opportunity for politicians who traditionally struggled for equal and equitable coverage in the traditional media (Aaldering and Van Der Pas Reference Aaldering and Van Der Pas2020; Greene and Lühiste Reference Greene and Lühiste2018; Hayes and Lawless Reference Hayes and Lawless2016; Lühiste and Banducci Reference Lühiste and Banducci2016) to bypass the gatekeepers and control their own message. But while Twitter enabled new ways to reach the public and the elites, there have also been concerns about the platform’s misaffordances likely inhibiting its interactive and democratic potential. Indeed, abundance of anecdotal evidence suggest that Twitter has over time become an environment that is rife with incivility and where prominent users are often viciously attacked (Friedersdorf, Reference Friedersdorf2015).
Despite the agreement of the existence of incivility in the online sphere, past research debates the very concept of incivility. Some see it as violation of norms of politeness (Mutz Reference Mutz2015); others see it as violation of democratic norms (Papacharissi Reference Papacharissi2004), with recent conceptual refinements noting that intolerance — rather than incivility — better describes the second perspective making it, at the same time, also more easily distinguishable due to the democratically poisonous effect of behaviors that violate political equality (Rossini Reference Rossini2022). These two approaches to incivility are rarely examined in the same studies, leading to some conceptual and empirical muddling few have tried to disentangle (for exceptions see Muddiman Reference Muddiman2017).
The conceptual debate notwithstanding, researchers document an increase in online incivility and intolerance (Rossini Reference Rossini2022; Theocharis et al. Reference Theocharis, Barberá, Fazekas and Popa2020). Twitter (now X) in particular, which has undergone major structural changes and even saw the mass firing of its content moderation team, allows one not only to remain largely anonymous, but restricts interaction toward short and direct messages, facilitating blunter communication (Sydnor Reference Sydnor2018, 6). Furthermore, while the communication can be easy and rapid, attacks and abuse are largely unpunished due to Twitter’s lack of capacity (and, more recently, willingness) to deal with it in a timely manner. Hence, the causes of online incivility, independent of the gender being targeted, lie at least partially on anonymity and the online disinhibition effect (Suler Reference Suler2004).
Gender Discrimination Logic and Prominence Logic
Not everyone suffers from online incivility to the same extent and in the same way. Women parliamentarians are reportedly the number one target of online psychological violence, including “sexist and misogynistic remarks, humiliating images, mobbing, intimidation and threats” (Inter-Parliamentary Union 2016, 6) — language that falls within various definitions of incivility and intolerance in the literature. Moreover, attacks on female representatives tend to be defined by the age, ethnicity, and length of service, with young and minority ethnic representatives suffering more incivility, especially when first elected (Ward and McLoughlin, Reference Ward and McLoughlin2020). Yet, few studies have empirically and systematically mapped gender differences in online incivility on a large cross-national scale.
As a result, current record of evidence is mixed, with reports focusing on female parliamentarians only emphasizing the high prevalence of attacks toward them. For example, the overwhelming majority of the UK women MPs have experienced aggressive behavior and received physical threats (Collignon and Rüdig Reference Collignon and Rüdig2021; Krook Reference Krook2017), in particular those belonging to ethnic minorities (Demos 2016). Moreover, when asked about online incivility in candidate surveys, female respondents not only report higher levels of attacks, but are also more likely than men to express fear (Collignon and Rüdig Reference Collignon and Rüdig2020, Reference Collignon and Rüdig2021) or perceive limits to their room for maneuver (Erikson, Håkansson, and Josefsson Reference Erikson, Håkansson and Josefsson2021; Tenove et al. Reference Tenove, Tworek, Lore, Buffie and Deley2023) as a reaction to their experiences. Unsurprisingly, a number of female representatives have been forced to quit social media after harassment campaigns (Inter-Parliamentary Union 2016) or decided to leave parliamentary politics altogether (Gorrell et al. Reference Gorrell, Bakir, Roberts, Greenwood and Bontcheva2020).
Yet, most studies using elite surveys with varying response rates, where empirical evidence relies on self-reported instances of abuse (see for example Collignon and Rüdig Reference Collignon and Rüdig2020, Reference Collignon and Rüdig2021; Krook Reference Krook2017), differ in their conclusions from recent research documenting online incivility by either hand-coding or classifying the tweets targeted at the politicians (Greenwood et al. Reference Greenwood, Bakir, Gorrell, Song, Roberts and Bontcheva2019; Southern and Harmer Reference Southern and Harmer2021; Ward and McLoughlin Reference Ward and McLoughlin2020). Studies looking at a wider range of different types of politicians show that male politicians, on average, receive more uncivil tweets than their female contenders (Greenwood et al. Reference Greenwood, Bakir, Gorrell, Song, Roberts and Bontcheva2019; Southern and Harmer Reference Southern and Harmer2021; Ward and McLoughlin Reference Ward and McLoughlin2020). Yet, none of these findings are uniformly one-directional, with women being more likely stereotyped by identity (men by party) and to be questioned in their position as an MP (Southern and Harmer Reference Southern and Harmer2021). Moreover, Ward and McLoughlin (Reference Ward and McLoughlin2020) point to differences in the intensity of incivility: while men, overall, receive a larger proportion of uncivil tweets, women and minority politicians are more likely to experience hate speech.
To complicate matters further, Rheault, Rayment, and Musulan (Reference Rheault, Rayment and Musulan2019), using machine learning models to predict incivility in about 2.2 million messages addressed to Canadian and US politicians on Twitter, find evidence that women — not men —are more heavily targeted by uncivil messages. But these findings are conditional too, with gender gap in receiving online incivility being moderated by candidate visibility. Their study, thus, provides a first indication that prominence might have an important role to play in who gets attacked online — a key consideration we discuss below.
While past empirical evidence offers mixed results, theories of gender role expectations and gender-based discrimination (Eagly and Karau Reference Eagly and Karau2002) suggest that female politicians may be susceptible to more online incivility than their male counterparts. While participation of women in politics is more common than ever before, stereotypes about traditional gender roles persist. As such, female candidates can still be seen as overstepping the private sphere when getting involved in politics, and thus subjected to more questioning of their presence in (online) public debates, which often comes with abuse and harassment (Eagly and Karau Reference Eagly and Karau2002; Krook and Restrepo Sanín Reference Krook and Sanín2016; Rheault, Rayment, and Musulan Reference Rheault, Rayment and Musulan2019). This may particularly be the case in the context where traditional gender roles are more prevalent.
While past research offers a mixed record of evidence, our theorizing building on gender role expectations and gender stereotyping lead us to hypothesize:
H1 Female politicians receive a higher proportion of uncivil tweets in comparison to their male contenders, all else being equal.
Yet not all female and male politicians have the same likelihood of experiencing incivility on social media (Rheault, Rayment, and Musulan Reference Rheault, Rayment and Musulan2019), with prominence being described as the primary driver of online attacks (Theocharis et al. Reference Theocharis, Barberá, Fazekas, Popa and Parnet2016). For example, most party leaders and cabinet ministers enjoy greater name recognition than the average candidate running for a first — not to mention, second-order election — which likely translates to higher Twitter visibility and, potentially, to more attacks. But this type of status-led prominence is not the only driver of visibility. Certain politicians’ capacity to strategically leverage Twitters’ affordances and reach “celebrity politician” status therefore underlines a recent phenomenon that can neither be ignored,Footnote 1 nor conflated with traditional notions of prominence. We therefore propose that both status-led prominence and Twitter-celebrity status (measured crudely by number of followers) likely affects politicians’ Twitter experiences.
H2a More prominent politicians receive a higher proportion of uncivil tweets in comparison to less prominent and less known politicians.Footnote 2
However, does prominence affect male and female politicians the same way or does an implicit gender bias manifest through prominence affecting men’s and women’s Twitter experiences to a different extent? Building on gender role theory and emerging evidence from most recent research, we theorize that prominence likely affects women’s experiences of abuse more so than men’s. Recent scholarship examining political violence against politicians more broadly — not just online — suggests that the most pronounced gender gap in violence is observed among politicians high in the political hierarchy (Håkansson Reference Håkansson2021; Herrick et al. Reference Herrick, Thomas, Franklin, Godwin, Gnabasik and Schroedel2021). Both of these studies rely on elite survey data from local level elections, thus demonstrating how female mayoral candidates in Sweden (Håkansson Reference Håkansson2021) and female mayors in the US (Herrick et al. Reference Herrick, Thomas, Franklin, Godwin, Gnabasik and Schroedel2021) are the most likely to experience meaningful levels of violence and psychological abuse. Furthermore, Collignon and Rüdig’s (Reference Collignon and Rüdig2021) candidate survey data from the 2019 UK general election suggest that candidate gender remains a significant predictor of abuse, even when controlling for candidate viability. These findings, however, all rely on self-reported instances of harassment and abuse, which men and women may not always be equally inclined to do. As such, substantiating past survey findings with data from other observational sources is needed. And while there is some evidence from the US and Canadian context suggesting women with higher follower-count to receive more uncivil messages on Twitter than their comparable male counterparts (Rheault, Rayment, and Musulan Reference Rheault, Rayment and Musulan2019), the potentially gendered effect of status-led prominence on online incivility is yet to be established outside the North American context.
Gender role theory offers some reasons why women and men of similar prominence may experience different levels of online incivility. Due to the ever-persistent traditional gender role expectations, female politicians are often seen as violating their perceived status (Eagly and Karau Reference Eagly and Karau2002). By taking this argument further, Håkansson (Reference Håkansson2021) suggests that positions higher in the political hierarchy not only demand more assertiveness and confidence but are also more associated with power-seeking behavior than lower-level positions. Hence, if attacks toward women in politics are driven by a dislike of female politicians, women at higher positions of power are more visible and more likely to be perceived as violating the gender role expectations, leading us thus to a more nuanced hypothesis:
H2b Prominence has a larger differential effect on the proportion of online incivility received by female politicians than by male politicians.
Does Uncivil Content Differ by Gender?
While past social media research on the topic has relied on counting tweets that fall into operational definitions of incivility or intolerance, most women politicians who have publicly reported online incivility note that attacks toward them are not merely about “how much,” but more about the nature of incivility (Erikson, Håkansson, and Josefsson Reference Erikson, Håkansson and Josefsson2021).
Some past social media research has differentiated less severe attacks from more severe types of incivility. “Milder” versions include name-calling, mockery, character assassination, and belittling or insulting others (Borah Reference Borah2012; Sobieraj and Berry Reference Sobieraj and Berry2011), while the use of homophobic, racist, and sexist language exemplify “heavier” attacks (Munger Reference Munger2016; Papacharissi Reference Papacharissi2004). Incivility, thus, is understood as a continuum in which civil language lies on one end, impoliteness or mildly uncivil language, such as sarcasm and insults, lie somewhere in the middle, and strongly uncivil language, such as racial slurs and obscenity, at the other end of the scale (Sydnor Reference Sydnor2018). Specifically, Papacharissi (Reference Papacharissi2002, 267) argues that incivility differs from mere impoliteness in that it demonstrates offensive behavior toward particular social groups (such as women), thereby disrespecting collective democratic traditions. While such definitions do not explicitly distinguish identity-based attacks from other examples of incivility, they tend to consider racist or sexist abuse at the more severe end of uncivil communication. Others have argued that the consideration of uncivil behavior alone is insufficient for understanding how uncivil discourse might be threatening democratic values. Rossini (Reference Rossini2022) suggests a conceptual distinction instead, where incivility — often occurring alongside meaningful discursive engagement — should be distinguished from intolerance that tends to occur in specific and more homogeneous discussions around minorities and civil society.
Literature on gender-based political violence offers further considerations for operationalizing such attacks. Most definitions in this literature consider (1) aggressive acts aimed largely or solely at women in politics, (2) because they are women, often using gendered means of attack, and (3) with the goal of deterring their participation to preserve traditional gender roles and undermine democratic institutions as types of violence against women in politics (Krook Reference Krook2017). While it refrains from classifying some forms of abuse as more severe than others, existing scholarship nevertheless distinguishes between hostile and benevolent sexism (Chen et al. Reference Chen, Pain, Chen, Mekelburg, Springer and Troger2020; Glick and Fiske Reference Glick and Fiske2018). Benevolent sexism is defined as a behavior that perpetuates stereotypical attitudes toward women (“women should stay at home”), thus directly linking to Papacharissi’s (Reference Papacharissi2002) definition of incivility, but also demonstrates a lack of tolerance — as per Rossini’s (Reference Rossini2022) account — for specific social groups (women) who challenge the stereotypically male-dominated political arena (Krook Reference Krook2017).
Yet not all attacks that occur against women in politics need to be necessarily gendered as “women can be victims of happenstance or when generalized political violence harms men and women in roughly equal proportions and in the same way” (Bardall, Bjarnegård, and Piscopo Reference Bardall, Bjarnegård and Piscopo2019, 932). Whether a specific act of incivility online qualifies as gendered or not would depend on whether: (1) the perpetrators have gendered motives and use violence to preserve men’s control of politics; (2) gendered forms result in men and women experiencing the abuse in different ways; or (3) the subjective meaning-making processes of the recipients of abuse lead to gendered impacts (Bardall, Bjarnegård, and Piscopo Reference Bardall, Bjarnegård and Piscopo2019).
Against this background, we expect female politicians to receive different and/or additional types of incivility than those directed at men. While past research does not offer consistent evidence that men receive less severe attacks than women, we expect the latter to be more susceptible to gendered means of incivility. Examples of gendered attacks may also include, but are not limited to, references to sexual identity or morality (i.e., accusations of being a bad wife or a bad mother, name-calling, belittling, etc.). And while these may not include words that are often classified as offensive (making them difficult to capture empirically — for examples see Siegel Reference Siegel, Persily and Tucker2020), they may nevertheless be harmful and have serious repercussions for one’s career. Considering this, we expect gendered means of incivility to be less relevant for men, as men’s participation in politics is considered more “normal” (Eagly and Karau Reference Eagly and Karau2002; Meeks Reference Meeks2019). However, this does not mean that male politicians would never be susceptible to gendered incivility. Moreover, besides gendered attacks, some men (and women) may experience other types of identity-based incivility, either due to their racial or ethnic identity, age, disability, or sexual orientation.
We thus generate the following hypothesis:
H3 Uncivil language received by female politicians is more gendered than that received by male politicians.
Data & Methods
Data Collection & Case Selection
We collect a diverse set of replies to and mentions of politicians on Twitter from four countries, resulting in a unique, multi-layered and extremely rich multilingual and cross-national dataset. Our cross-national approach represents a novelty in social media research, which has primarily focused on single-country case studies (most commonly on the US), often including only high-profile politicians.
The first layer includes data from the 2014 European Parliament election in Germany, Spain, and the UK, where we collected Twitter account details for each identified candidate before the elections (January–April 2014).Footnote 3 Kantar PublicFootnote 4 used the list of candidates and accounts to collect all Twitter communication centered around the candidates during and after the election campaign (May 5, 2014–June 1, 2014) using the Twitter firehose. This includes a total of 341 MEP candidates in the UK, 266 in Spain, and 163 in Germany.
As EP candidates are often regarded as second-tier politicians, who are less known and thus potentially less exposed to vitriolic comments, we supplement these data with incumbent MPs in these three countries and the members of the US Congress. For each of the countries, we compiled a list of sitting legislators and manually identified whether they had an active Twitter account as of February 2016 (September 2016 for the US).Footnote 5 This includes a total of N = 487 accounts in the US, N = 740 in the UK, N = 344 in Spain, and N = 375 in Germany.Footnote 6 For these politicians, we collected the data directly via the Twitter Streaming API: in the US, October 17, 2016–October 15, 2017;Footnote 7 and in the European countries, December 19, 2016–October 15, 2017. The data include all tweets that are either direct replies to a politician’s post or that mention any politicians’ Twitter accounts,Footnote 8 a total of 15,456,186 tweets in the US, 4,017,468 tweets in the UK, 3,117,873 tweets in Spain, and 779,443 tweets in Germany.
Besides including a wide variety of politicians from different countries, our data also spans a long time period, including a series of less or more politicized events, like the EP elections (May 2014), the 2016 US elections, the debate following the Brexit referendum in the UK (June 2016), the government formation process in Spain (October 2016), and a series of state-level (May 2017: Schleswig-Holstein and North Rhine-Westphalia) and federal elections in Germany (September 2017). The lengthy time span allows us to explore incivility in both periods of heated debate and during politically quieter times. Furthermore, the selection of countries in our sample does not only vary with regards to levels of incivility on Twitter, but also in terms of electoral and party systems. We include countries with strong multiparty systems that use proportional representation (Germany and Spain), where focus on individual candidates and personalities is less pronounced, in comparison to the more majoritarian/plurality systems used in the UK and the US. This allows us to present first descriptive circumstantial evidence of potential cross-system differences in politicians’ experiences on social media and allow for a more nuanced examination of female and male politicians’ Twitter experiences. While our case selection includes countries with more and less prevalent traditional gender norms, our sample is missing a country with very high levels of gender equality (i.e., Sweden or Finland). As such, our findings are likely generalizable to most industrialized Western democracies apart from Northern European countries.
Measuring Incivility
To classify tweets as uncivil, we manually labeled nearly 30,000 random sample of tweets,Footnote 9 which we then used to train a machine learning classifier. The manual coding scheme was developed by the authors for a larger project, containing various categories related to tweet content, including incivility. Further information on the manual labelling process, including coder training, is presented in Appendix A.
We then used the 29,474 human-coded tweetsFootnote 10 to train machine learning models (logistic regression) that predict incivility based on the words in a tweet. The text is preprocessed by replacing named entities, mentions, numbers, and URLs with placeholders (e.g., PERSON, MENTION, NUMBER, URL); removing punctuation and stopwords; and lemmatization of the words. Hence if these tweets contain any relevant textual information, they were annotated by humans based on the relevant text. If they do not contain any relevant textual information (i.e., they only include images or links) we exclude them from our analysis. Subsequently, unigrams are transformed into a numerical matrix using term frequency-inverse document frequency (tf-idf). Prediction performance is reported using 10-fold cross validation: we split the data in 10 subsets and use nine subsets for training and one for testing. This is repeated 10 times, so that each subset is used exactly once for testing. The model hyperparametersFootnote 11 were optimized with five-fold cross validation on the training data. The out-of-sample performance of the machine learning classifiers is summarized in Table 1. Area Under the ROC Curve (AUC) measures the performance of a classifier in predicting the probability of a tweet being uncivil, independent of the frequency of the classes.Footnote 12 The F1 score measures the accuracy of a classifier in assigning the actual label (uncivil or not) for a certain probability threshold.Footnote 13 The F1 scores for all countries are low,Footnote 14 indicating that the classifiers are not sufficiently accurate in assigning the correct label per tweet. However, high AUC (above 80%), indicates that the classifier probabilities are relatively trustworthy in all countries, except for Germany.Footnote 15 Considering the prediction uncertainty, we base further analysis on the predicted probability of a tweet being uncivil and exclude Germany from the main quantitative analysis.Footnote 16
Table 1. Out-of-sample performance of machine learning classifiers to predict incivility
Notes: For the US, we included synthetic labels (see details in Yannis Theocharis et al. Reference Theocharis, Barberá, Fazekas and Popa2020) for an additional set of 16,000 tweets.
AUC measures the probability that the model ranks a random positive example higher than a random negative example (Provost and Fawcett Reference Provost and Fawcett2013). A perfect AUC is 1, while 0.5 indicates random performance.
F1 is the harmonic mean of precision and recall, with values ranging from 0 to 1. The F1 metric depends on the distribution of the classes. A baseline model always predicting “uncivil” would score 0.33 for the European sample (20% uncivil tweets) and 0.55 for the US sample (38% uncivil tweets). All F1 scores in the table exceed baseline performance and our US F1 score is comparable to that of others using the same Twitter dataset (Davidson, Sun, and Wojcieszak Reference Davidson, Sun and Wojcieszak2020).
Recall is the percentage of uncivil tweets (as determined by human annotators) that are correctly identified by the model as uncivil.
Precision is the percentage of tweets predicted by the model to be uncivil that are actually uncivil, according to human annotators.
Another advantage of using the probability-based measure of incivility is that it provides a little bit more nuance to our otherwise binary classifier (civil/uncivil). Unfortunately, classifying multiple types of incivility would have required significantly more training data, and capturing more nuanced categories would likely have resulted in even lower classifier accuracy. However, to address the issue of differentiating between various forms of incivility, we supplement our main quantitative analysis with inductive text analysis of uncivil tweets. This additional analysis gives us both deeper insights into the different types of incivility as well as provide a crude concept validity check.
Analysis
Who Experiences More Incivility?
We first examine if incivility varies across gender and whether other factors moderate the gender effect.Footnote 17 We use machine learning output to compute the outcome variable, capturing the probability of a politician to receive uncivil comments, by averaging the probability of being uncivil across all tweets addressed at a given politician.Footnote 18 The dependent variable is thus the average probability of tweets addressed to a politician being uncivil.
Derived from our theorizing, the main explanatory variables are genderFootnote 19 and prominence. We operationalize prominence by distinguishing the status of the politician from their Twitter visibility. In both datasets, Twitter visibility is measured by the (logged) number of followers.Footnote 20 To capture politician’s status-led prominence, we distinguish between Member of the House and Member of the Senate in the US. In Europe, we distinguish between a top tier (MPs who are government ministers), a second tier (MPs who do not hold positions in the government), a third tier (2014 EP candidates in safe party list positions), and a fourth tier (EP candidates who had little or no chance of being elected) politicians.Footnote 21
We also control for ideology and ideological extremism. Ideology is measured with a party-level left-right score, captured by Chapel Hill Expert Survey’s (CHES) (Bakker et al. Reference Bakker, Hooghe, Jolly, Marks, Polk, Rovny, Steenbergen and Vachudova2020) in Europe and by the party label (Democrats and Republicans) in the US. In Europe, we capture ideological extremism with the absolute value of the difference between the left-right CHES placement and the theoretical midpoint of the scale (i.e., 5), while in the US it is measured as the absolute value of DW-NOMINATE (a metric of ideology estimated using roll-call votes in Congress; see Lewis et al. Reference Lewis, Poole, Rosenthal, Boche, Rudkin and Sonnet2021). We further include controls for government party (only available for European data), age (only available for the US), and the percent of total tweets addressed to a politician that are replies. All the explanatory variables are re-scaled to take values between 0 and 1, allowing for substantive interpretation of the effects.
We employ linear models to account for the continuous nature of our outcome variable and for ease of interpretation.Footnote 22 As the set of predictors varies slightly from the US to Europe, we run separate models and due to the structure of the European data (i.e., politicians nested in parties and countries), we run a series of multilevel models with country fixed effects. In both cases, we first report models, where the only explanatory variable is gender (Model 1 in Tables 2 and 3). To test our main hypotheses about the effect of gender (H1) and status-led and Twitter-based prominence (H2a), Model 2 in Table 2 and Models 2 and 3 in Table 3 include variables capturing politician’s gender, their status-led prominence, and their Twitter-visibility as well as controls for ideology and extremism (Model 2 in Tables 2 and 3).Footnote 23 To test our Hypothesis about whether prominence affects women and men’s experiences on Twitter the same way or differently (H2b), we present Model 3 in Table 2 and Model 4 in Table 3 that include the relevant interaction effects.
Table 2. Predicting incivility, US
Notes: Outcome variable denotes the probability of a tweet being uncivil averaged across tweets of a given politician and is measured on a 0 to 100 scale. Linear estimations with standard errors in parentheses.
*** p < 0.001, ** p < 0.01, * p < 0.05.
Table 3. Predicting incivility, only Spain and UK
Note: Outcome variable denotes the probability of a tweet being uncivil averaged across tweets of a given politician and is measured on a 0 to 100 scale. Linear estimations with standard errors in parentheses.
*** p < 0.001, ** p < 0.01, * p < 0.05.
Focusing on our central variable of interest — gender — we do not find consistent support for H1. In both the US (Table 2) and Europe (Table 3), gender is not a statistically significant predictor across models that include controls (except Model 4 in Table 3), tentatively suggesting that there is no consistent difference in the proportion of incivility received by female and male politicians. The only exceptions are Model 1 and 4 in Table 3. In a model with no controls (Model 1, Table 3), European men have a slightly higher likelihood of receiving proportionally more uncivil tweets than women. Yet, the effect size is small, and it disappears once controlling for prominence — the most important predictor of incivility, thus supporting H2a. However, when including interaction effects between gender and prominence (Model 4, Table 3), the sign of the main effect of gender flips and becomes significant, indicating that prominence affects women’s and men’s likelihood to receive online incivility in varying ways in our European sample. Notably, women in less prominent positions and with fewer Twitter followers are more likely to receive uncivil content than comparable male politicians.
In the US, the more objective prominence measure (i.e., the difference between being a Senator or a House Member) does not seem to matter, while Twitter visibility has a strong positive and statistically significant effect (Model 2, Table 2). Politicians with the highest number of followers are expected to receive, on average, 19 percentage points more uncivil content in comparison to a politician with the fewest followers. In Europe, Twitter visibility matters, too, though the difference is relatively smaller (only approximately 6 percentage points [Model 3, Table 3]). Yet, status-led prominence matters, too, with MPs and Cabinet ministers receiving, on average, respectively approximately 6 and 8 percentage points more uncivil content in comparison to MEP candidates (Model 2, Table 3). The broader range of politicians in our European sample may explain why status-led prominence only matters in Europe.
While the prominence does not moderate the effect of gender in the US, we find the opposite effect to our expectation in Europe (H2b) (Model 4, Table 3), where Twitter visibility substantially increases the proportion of incivility male politicians receive, without having any statistically significant impact on the incivility received by women (see Figure 1).Footnote 24 To put this into perspective, in our sample the median number of uncivil tweets that female politicians with few followers (i.e., less than 100) receive is seven compared to one for men with a similar number of followers. In contrast, a popular male politician (i.e., more than 10,000 followers) receives in median approximately 100 more uncivil tweets than a popular female politician (529 compared to 418). Our findings thus suggest that while male politicians need to worry more about Twitter incivility once they reach higher political visibility, their female colleagues are exposed to uncivil content from the moment they enter Twitter. Notably, in a situation of receiving relatively few tweets, even just a couple of uncivil comments are likely to be noticed, and thus potentially having a stronger impact on the receiver.
Figure 1. Conditional impact of gender on incivility depending on Twitter visibility, European countries.
Regarding other control variables, politicians representing right-leaning parties, regardless of their gender, are more likely to receive more uncivil content, and more ideologically extreme politicians in the US get substantially more incivility online. On both continents, direct replies in comparison to mentions attract more incivility.
Differences in the Type of Incivility
We use various text analysis methods for exploring any gender differences in the uncivil language addressed to politicians (H3). While in-depth qualitative analysis is beyond the scope of this paper, the inductive text-analysis tools used below allow us to explore if the specific words used in uncivil tweets that are targeted at women and men vary from one another. First, we use keyness measureFootnote 25 to identify words that are significantly related to one gender. Second, we perform additional analysis by training classification models that predict incivility split by gender. In both cases, we present a couple of example tweets where there are words with the highest keyness (most distinct from male versus female sample) or the words most likely to appear in tweets classified as uncivil.Footnote 26 Both analyses suggest that the language associated with uncivil content varies by gender. Put simply, the specific words in uncivil tweets aimed at male and female politicians are not the same, with words indicating more gendered attacks being more prevalent in uncivil tweets sent to women.Footnote 27
To compute the keyness of a word w, we perform an association test using the χ 2 value to the frequency of w in the target corpus (uncivil tweets targeted to women) versus the frequency of w in the reference corpus (uncivil tweets received by men). We find that around 1% of the words in all uncivil tweets have a significant keyness, meaning they are significantly related to one gender rather than the other (see Table 4).
Table 4. Number of words with p-value below 0.05 for χ 2 value of words from target group (uncivil tweets to women) compared to reference group (uncivil tweets to men)
Figure 2 displays the words with highest keyness scores in uncivil tweets, some of which allude to identity-based language use, while not necessarily being intuitively predictive of incivility (i.e., “lady,” “woman,” “sir,” and “mr” in the US or “frau” and “herr” in Germany). As a next step, we examine the individual tweets where the words with the highest keyness scores are used.
Figure 2a. Word keyness plot for uncivil tweets by gender. Black bars are associated with female and grey bars with male gender: UK.
Figure 2b. Word keyness plot for uncivil tweets by gender. Black bars are associated with female and grey bars with male gender: Spain.
In the UK, the words with the highest keyness scores in uncivil tweets targeted at female politicians include, for example, “child,” “leader,” and “politician,” while those received by men use words, such as “fuck,” “cunt,” and “old.”Footnote 28 Examining the tweets where these words appear suggests that the uncivil tweets sent to UK male politicians include many swear-words and name-calling:
“@SadiqKhan Fuck off ya fake”
“@jeremycorbyn You’re a fucking twat”
“@BorisJohnson You’re a cunt. A sweaty, dipping, lying, absolute fucking cunt. Fuck off back to whichever swamp you somehow crawled out of”
At the same time, some of the uncivil tweets targeted at UK female politicians, where high-keyness words appear, tend to question their quality as a politician or their suitability for role models:
“@HackneyAbbott @SMCommision as a whole black children are given the wrong role models. Expectations are deliberately set low.”
“@HarrietHarman and you’re a danger to children!!”
“@YvetteCooperMP Now there’s the truth of how incompetent & traitorous are our politicians, msm et al.….”
We see some similar examples of gendered incivility targeted at female politicians in the US, too: highest scoring keyness words for women include, for example, “lady,” “woman,” and “hag,” while those received by men contain words, such as “tax,” “sir,” and “coward.” In both countries, some words (i.e., “child,” “immigration,” “tax”) also allude to women and men being targeted in relation to specific issue areas. Unlike in the UK, many of the uncivil tweets targeted at the US female politicians are also heavy on profanity and name-calling — features that were more prevalent in offensive tweets sent to British men. Moreover, in the US, our sample of uncivil tweets includes references to female politicians’ appearance and the questioning of some Republican female politicians’ ability of and credentials in representing women. The tweets below exemplify some of the more extreme instances of gendered incivility targeted at US female politicians, with high-keyness words:
“@MaxineWaters you are a lying, crazy old hag who doesn’t belong in out House of Reps. Get out”
“@NancyPelosi @HouseGOP Nancy, your profile image is more photoshopped than mine, you nasty old anti-American hag!”
“@lisamurkowski You should be ashamed of yourself. You have capitulated to the male dominance governing [US flag] we need STRONG YOUNG women to run this country. Get yourself to a retirement home soon”
When examining uncivil tweets with high keyness score words sent to US male politicians, we observe some more policy-related tweets (i.e., about taxation) than mere personal attacks and name-calling. However, there are also several offensive tweets, including name-calling:
“@PeterRoskam NO TAX REFORM UNTIL WE SEE FAKE PRESIDENT TRUMP’S TAX RETURNS!”
“@SenJohnMcCain You should be proud. Too bad others will lose what you’re getting from the taxpayers. Judas.”
It is somewhat harder to identify any clear gendered incivility patterns in Germany and Spain. While the specific words with the highest keyness scores vary across genders in Spain, both male and female politicians receive uncivil tweets related to corruption, questioning their dignity and suitability to partake in politics:Footnote 29
“@DolorsMM @cope_es And your family keeps stealing from the social security system. Public embarrassment”
“@Rafa_Hernando @PPopular I think you are a problem for society. You’re in the government and all corrupt. Either you explain why or you resign.”
“@GLlamazares What you are doing is unbelievable. Don’t ever say that you are left-wing. You are a traitor.”
Similarly to Spain, uncivil tweets targeted at German politicians appear to use fewer deliberately abusive words than in the US and the UK. Hence the impoliteness sounds comparatively polite. This may partly explain why our classifier performs poorly with the German sample:
“@petertraube Thank you, Mr. Outrage-leader. But please also tell your stepmother and the courtier of the interior. They don’t answer my calls…”
“@MdB_Stroebele This woman is hard to top when we talk about double standards. An ice-cold fishing-rod!”
While it is difficult to observe coherent themes in the tweets with high keyness-scoring words targeted at German male politicians, some of the communication classified as uncivil toward women involves name-calling and questioning female politicians’ ability to understand politics. The tweets below exemplify some of the more explicitly gendered uncivil tweets targeted at German female politicians:
“@SteinbachErika Confused old women, who constantly send their nonsense via Twitter to alleviate their difficult-to-treat persecution mania.”
“@katjakipping Leftist dreamer didn’t pay attention in history class and blanketly calls people Nazis. This must have consequences.”
To further explore any gender differences in the type of incivility, we perform additional analysis by training classification models that predict incivility split by gender.Footnote 30 More specifically, we divide the data to tweets toward male politicians versus tweets toward female politicians, and train separate classification models to predict incivility for each group (see Appendix E, Table E1). The coefficients of these models provide further insights to the predictive terms for incivility (see Table 5).Footnote 31 While there is a core set of uncivil words — similarly to words with highest keyness scores — there are large gender differences in the top predictive terms. For example, out of the total 100 terms in the UK (50 for men and 50 for women), only 22 terms overlap, further suggesting that the type of incivility likely varies dependent on the recipient.
Table 5. Words most likely to be associated with incivility for each gender
Figure 2c. Word keyness plot for uncivil tweets by gender. Black bars are associated with female and grey bars with male gender: Germany.
Figure 2d. Word keyness plot for uncivil tweets by gender. Black bars are associated with female and grey bars with male gender: US.
A cursory reading of various examples provides further circumstantial evidence of female candidates receiving more morality-focused words (vile, shame, shameful, disgraceful, hypocrite, deluded), including references to personal attacks (kill, destroy, rape), rather than simple swear words. However, words indicating a reference to appearance or identity (i.e., old, white, disabled) are amongst the top predictive words for both men and women. The following tweets targeted at Labour MP Diane Abbott and US Representative for California’s 43rd congressional district Maxine Waters exemplify some of the morality-focused incivility:
“@HackneyAbbott Your a disgrace women !!! Vile person”
“@MaxineWaters ur one stupid low life biotch”
At the same time, top five tweets with the highest incivility prediction scores (99%), mostly consisting of various insults and swear words, in the UK were all targeted at men.
“@jeremycorbyn fuck off you, you stupid cunt”
“@jeremycorbyn @realDonaldTrump fuck off you stupid cunt”
“@Nigel_Farage @UKIP FUCK FUCK SHIT CUNT DAAAAAMMN MOTHERFUCKER”
“@Nigel_Farage fuck the fuck off you fucking bigoted twat”
“@Jeremy_Hunt @NEAmbulance fuck off you stupid tory cunt”
Similarly, in Spain, the US and Germany, very few terms overlap across genders (with the least overlapping terms in Germany: 14). Moreover, the very word politician is amongst the top predictive terms for women both in Germany and the UK, but not for men, potentially indicating that female politicians are attacked with a reference to their capacity as politicians.
Discussion
Social media came with a big promise: a personal publicity channel, allowing traditionally disadvantaged candidates, like women, to bypass media gatekeepers and communicate their message in their own terms (Tucker et al. Reference Tucker, Theocharis, Roberts and Barberá2017). Focusing on theoretical ideas stemming from political communication and politics and gender research, we hypothesized that this prospect is potentially lost due to the gender differences in the quantity and type of incivility received by politicians. We tested these assumptions using a unique cross-national and multilingual dataset, capturing the Twitter experiences of politicians in different contexts and of varying levels of prestige. Our analysis relies on data from a period when Twitter was an important tool for political campaigning, but when different politicians in different countries used it to a varying extent. Also, Twitter was content moderated at a time, with some highly abusive content likely removed before the users saw it. As such, our data provide good variation for examining different types of politicians’ social media experiences as well as perhaps somewhat more conservative test than more recent data from X (formerly Twitter).
Our quantitative analysis points to indirect, rather than direct, gender differences in the proportion of online incivility received, with prominence moderating the effect of gender. More specifically, while Twitter-visibility matters on both sides of the Atlantic, status-led prominence only matters in Europe. This may partly be due to our US sample lacking in “low” level politicians, but more likely it reflects the more professionalized, personalized, and polarized nature of American politics. Particularities of the political systems are important here as European politicians (even cabinet members and the MPs, especially in PR systems) are considerably less visible on Twitter than their US counterparts, with European female politicians having both substantially fewer followers and receiving significantly fewer mentions and replies.Footnote 32
These observed trends are instructive and the inclusion of cases from outside North America, allow us to present more nuanced findings. The fact that European female politicians do not experience proportionally more incivility online could point to other structural disparities. For example, female politicians in Europe could avoid building an extensive social media presence, especially because of their initial more negative experiences on the platform — as evidenced by our analysis. For example, a politician who receives only a small number of responses, might actually read and remember them. Hence, the impact of such messages could be more influential and affect politicians’ future behavior more than in the case of someone receiving thousands of responses every day. Based on what we know from the US, an increased professionalization and personalization of European politics could result in a diminished gender gap in social media popularity also in Europe. However, the question of whether such changes will eradicate the gender gap in incivility across all groups or, contrarily, will only expose European female politicians to even more attacks, withstands. An even darker prediction would see the more professionalized and personalized campaigns in Europe resulting in an even greater gender gap in the willingness to run for high level office.
We also acknowledge that our findings regarding the interaction of gender and prominence go against some past research, which suggest that the most high-profile, rather than less prominent, women receive the most attacks both online and offline (Collignon and Rüdig Reference Collignon and Rüdig2021; Herrick et al. Reference Herrick, Thomas, Franklin, Godwin, Gnabasik and Schroedel2021; Rheault, Rayment, and Musulan Reference Rheault, Rayment and Musulan2019). However, most of this body of work relies on self-reported evidence from elite surveys, rather than content analysis of social media data. As such, due to the prevalence of gender roles and norms and its likely impact also on men, elite survey data may suffer from male politicians under reporting the incivility and attacks they receive. Additionally, male politicians enjoying high levels of prominence may be less likely to manage their own social media accounts, which could lead to further under reporting of incivility. More importantly, our results may reveal women’s unwillingness to engage more actively on a platform where they receive incivility the moment they enter politics or Twitter. Furthermore, our findings are in line with some past research, that suggest young and minority representatives get the most abuse (Ward and McLoughlin Reference Ward and McLoughlin2020).
Our second major puzzle concerned the nature of online incivility in a cross-national setting. In line with findings from qualitative research (Erikson, Håkansson, and Josefsson Reference Erikson, Håkansson and Josefsson2021; Sobieraj Reference Sobieraj2020), revealing that online incivility targets women’s identities, and consists of rape threats and attacks on their appearance, we find that uncivil tweets directed to female politicians focused also on morality, personal attacks, appearance, and them being (un)suitable politicians. Hence, we found circumstantial evidence of gendered language in the tweets classified as uncivil targeted at women, while male politicians received incivility that was heavy on profanity and name-calling. The fact that women politicians are more likely to receive gendered and intolerant attacks, rather than random incivility and name-calling, points to potentially serious consequences. Communicative environments in which references to one’s very identity are used as a potential silencing tool may reveal a broader political culture rife with intolerance. As such, our findings point to potential policy implications, where a further need to regulate social media content more similarly to traditional media may be necessary to ensure the equality of political representation.
Importantly, our study goes beyond anecdotal evidence, by providing valuable insights into quantitative and qualitative gender differences in uncivil language targeted at politicians in a variety of contexts. The application of machine learning methods enables us to analyze millions of tweets in four different countries but does not come without limitations. Our choice to work with logistic regression classifiers is motivated by the ease of interpretation — the words or topics with the largest coefficients are the most predictive of uncivil language — and higher or similar performance to other classifiers.Footnote 33 Even though our models are competitive with other available alternatives, their performance is still far from perfect, especially for Germany. As mentioned above, this might be due to more subtle incivility. Similarly, while the use of binary classification does not automatically allow us to easily distinguish problematic tweets from profanity, a more qualitative reading of the tweets classified as “uncivil” showed we nevertheless captured a range of different types of abuse. Yet, by solely focusing on word usage, the method may miss some of these forms of incivility. More advanced NLP methods, which consider the context for each occurrence of a given word would have limitations, too, as these do not yet understand all subtleties of natural language. This reaffirms what other political scientists studying hateful language have stressed: automated machine learning methods have difficulties in capturing the wide variety of more subtle but nevertheless harmful uncivil language (Siegel Reference Siegel, Persily and Tucker2020), some of which may be more often used against women than men. This is not surprising, as — based on the ICR scores — even humans struggle with consistently identifying uncivil communication. Regardless of these limitations, our results show important cross-national differences in the amount and type of incivility targeted at female and male politicians.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S1743923X25100111.
Open access
