Implicit hate datasets

The task of classifying hateful texts has led to an avalanche of hate detection datasets and models (Schmidt and Wiegand, Reference Schmidt and Wiegand2017; Tahmasbi and Rastegari, Reference Tahmasbi and Rastegari2018; Vidgen and Derczynski, Reference Vidgen and Derczynski2020). Before discussing the literature, it is imperative to point out that issues with generalizable (Yin and Zubiaga, Reference Yin and Zubiaga2021), biasing (Balayn et al. Reference Balayn, Yang, Szlavik and Bozzon2021; Garg et al. Reference Garg, Masud, Suresh and Chakraborty2023), adversary (Masud et al. Reference Masud, Singh, Hangya, Fraser and Chakraborty2024b), and outdated benchmarks (Masud et al. Reference Masud, Khan, Goyal, Akhtar and Chakraborty2024a) are prevalent for hate speech detection at large and forms an active area of research.

Focusing on implicit hate datasets, we searched the hate speech database (Vidgen and Derczynski, Reference Vidgen and Derczynski2020) with the keyword “implicit” as an indicator of whether the label set contains “implicit” labels and obtained $4$ results. DALC (Caselli et al. Reference Caselli, Schelhaas, Weultjes, Leistra, van der Veen, Timmerman and Nissim2021b) is a Dutch dataset consisting of $8k$ tweets curated from Twitter, labeled for the level of explicitness as well as the target of hate. Meanwhile, ConvAbuse consists of 4k English samples obtained from in-the-wild human-AI conversations with AI chatbots. Each conversation is marked for the degree of abuse (1 to -3) and directness (explicit or implicit). The other two datasets are also in English. AbuseEval (Caselli et al. Reference Caselli, Basile, Mitrović, Kartoziya and Granitzer2020) is $14k$ , Twitter labeled for “abusiveness” and “explicitness.” On the other hand, ImpGab (Kennedy et al. Reference Kennedy, Atari, Davani, Yeh, Omrani, Kim, Coombs, Havaldar, Portillo-Wightman, Gonzalez, Hoover, Azatian, Hussain, Lara, Cardenas, Omary, Park, Wang, Wijaya, Zhang, Meyerowitz and Dehghani2022) consists of 27k posts from Gab, which contain a hierarchy of annotations about the type and target of hate.

Meanwhile, from the ACL Anthology (we looked at the results from the first two pages out of 10), we discovered four more datasets. LatentHatred is the most extensive and most widely used implicit hate speech dataset. It consists of $21k$ Twitter samples labeled for implicit hate as well as $6$ additional sub-categories of implicitness. It also contains free-text human annotations explaining the implied meaning behind the implicit posts. Along similar lines, SBIC (Ocampo et al. Reference Ocampo, Sviridova, Cabrio and Villata2023c) is also a collection of $44k$ implicit posts curated from online platforms with human-annotated explainations. However, unlike complete sentences in LatentHatred SBIC focuses on single-phrased explainations. Further, SBIC does not have a direct marker for the explicitness of the post, and by default, all posts are implicit. For specific target groups and types of hate speech, such as sexism (Kirk et al. Reference Kirk, Yin, Vidgen and Röttger2023) or xenophobia against immigrants (Sánchez-Junquera et al. Reference Sánchez-Junquera, Chulvi, Rosso and Ponzetto2021), researchers have also explored imploying multiple-level annotations as a means of obtaining granular label spans as explainations for the hateful instance. It serves as an alternative to free-text annotations, allowing for more structured and linguistic analysis (Merlo et al. Reference Merlo, Chulvi, Ortega-Bueno and Rosso2023) of implicitness. Further, building upon the multimodal hate meme dataset MMHS150K (Gomez et al. Reference Gomez, Gibert, Gomez and Karatzas2020), proposed a multimodal implied hate dataset (Botelho, Hale, and Vidgen, Reference Botelho, Hale and Vidgen2021) with the different types of implicitness occurring as a combination of the text and image.

More recently, the ISHate (Ocampo et al. Reference Ocampo, Sviridova, Cabrio and Villata2023c) dataset has been curated by combining existing hate speech and counter-hate speech datasets and relabeling the samples for explicit–implicit markers, consisting of $30k$ samples labeled as explicit, implicit, subtle, or non-hate. It is interesting to note that in their analysis, the authors do not showcase how the different datasets interact with each other in the latent space. We hypothesize that the performance improvements in hate detection are obtained not as a result of modeling but due to the fact that these samples are obtained from distinct datasets, that is, distinct distributions. For example, counter-hate datasets do not contribute to the non-hate class. Meanwhile, the majority of implicit hate samples come from LatentHatred and ToxiGen (Hartvigsen et al. Reference Hartvigsen, Gabriel, Palangi, Sap, Ray and Kamar2022). The latter is a curation of around 1 M toxic and implicit statements obtained by controlled generation.

Modeling implicit hate speech in NLP

Despite a large body of hate speech benchmarks, the majority of datasets fail to demark implicit hate. Even during the annotation process, fine-grained variants of offensiveness Founta et al. (Reference Founta, Djouvas, Chatzakou, Leontiadis, Blackburn, Stringhini, Vakali, Sirivianos and Kourtellis2018); Kulkarni et al. (Reference Kulkarni, Masud, Goyal and Chakraborty2023); Kirk et al. (Reference Kirk, Yin, Vidgen and Röttger2023) like abuse, provocation, and sexism are favored over the nature of hate, that is, explicit vs implicit. As the annotation schemas have a direct impact on the downstream tasks (Rottger et al. Reference Rottger, Vidgen, Hovy and Pierrehumbert2022), the common vouge of binary hate speech classification, while easier to annotate and model, focuses on explicit forms of hate. It also comes at the cost of not analyzing the erroneous cases where implicit hate is classified as neutral content. This further motivates us to examine the role of PLMs in three-way classification in this work.

Given the skewness in the number of implicit hate samples in a three-way classification setup, data augmentation techniques have been explored. For example, (Ocampo et al. Reference Ocampo, Sviridova, Cabrio and Villata2023c) employed multiple data augmentation like substitution and back translation and observed that only when multiple techniques are combined did they surpass the finetuned HateBERT in performance. Adversarial data collection (Ocampo, Cabrio, and Villata, Reference Ocampo, Cabrio and Villata2023a) and LLM-prompting (Kim et al. Reference Kim, Park, Namgoong and Han2023) have also been explored for augmenting and improving implicit hate detection.

Language models are being employed not only to augment the implicit hate corpora but also to detect hate (Ghosh and Senapati, Reference Ghosh and Senapati2022; Plaza-del Arco, Nozza, and Hovy, Reference Plaza-del Arco, Nozza and Hovy2023). With the recent trend of prompting generative large language models (LLMs), hate speech detection is now being evaluated under zero-shot (Nozza, Reference Nozza2021; Plaza-del Arco et al. Reference Plaza-del Arco, Nozza and Hovy2023; Masud et al. Reference Masud, Singh, Hangya, Fraser and Chakraborty2024) and few-shot settings as well. An examination of the hate detection techniques under fine-grained hate speech detection has revealed that traditional models, either statistical (Waseem and Hovy, Reference Waseem and Hovy2016; Davidson et al. Reference Davidson, Warmsley, Macy and Weber2017) or deep learning-based (Badjatiya et al. Reference Badjatiya, Gupta, Gupta and Varma2017; Founta et al. Reference Founta, Chatzakou, Kourtellis, Blackburn, Vakali and Leontiadis2019), are characterized by a low recall for hateful samples (Kulkarni et al. Reference Kulkarni, Masud, Goyal and Chakraborty2023). To increase the information gained from the implicit samples, researchers are now leveraging external context.

Studies have mainly explored the infusion of external context in the form of knowledge entities, either in the form of knowledge-graph (KG) tuples (ElSherief et al. Reference ElSherief, Ziems, Muchlinski, Anupindi, Seybolt, Choudhury and Yang2021) or Wikipedia summaries (Lin, Reference Lin2022). However, both works have observed that knowledge infusion at the input level lowered the performance on fine-grained implicit categories. An examination of the quality of knowledge tuple (Yadav et al. Reference Yadav, Masud, Goyal, Akhtar and Chakraborty2024) infusion for implicit hate reveals that KG tuples fail to enlist information that directly connects with the implicit entities, acting more as noise than information. Apart from textual features, social media platform-specific features like user metadata, user network, and conversation thread/timeline can also be employed to improve the detection of hate and capture implicitness in long-range contexts (Ghosh et al. Reference Ghosh, Suri, Chiniya, Tyagi, Kumar and Manocha2023). However, such features are platform-specific, complex to curate, and resource-intensive to operate (in terms of storage and memory to train network embeddings). From the latent space perspective, researchers have explored how the infusion of a common target group can bring explicit and implicit samples closer (Ocampo, Cabrio, and Villata, Reference Ocampo, Cabrio and Villata2023b), aiding in the detection of the latter. While the idea is intuitive since implicit hate and explicit slurs are specific to a target group, here, the extent of overlap in the case of multiple target groups or intersectional identities is not adequately addressed.

Distance-metric learning

Akin to supervised classification task in NLP, all the setups reviewed so far use an encoder-only BERT-based + cross entropy (CE) for finetuning. Therefore, in our study, BERT + CE acts as a baseline. Despite its popularity, CE’s impact on the inter/intra-class clusters is suboptimal (Liu et al. Reference Liu, Wen, Yu and Yang2016). Since classification tasks can be modeled as obtaining distant clusters per class, one can exploit clustering and distance-metric approaches to enhance the boundary among the labels, leading to improved classification performance. Distance-metric learning-based methods employ either deep divergence via distribution (Cilingir, Manzelli, and Kulis, Reference Cilingir, Manzelli and Kulis2020) or point-wise norm (Chopra et al. Reference Chopra, Hadsell and LeCun2005). The most popular deep metric learning is the contrastive loss family (Chopra et al. Reference Chopra, Hadsell and LeCun2005; Schroff, Kalenichenko, and Philbin, Reference Schroff, Kalenichenko and Philbin2015; Chen et al. Reference Chen, Chen, Zhang and Huang2017). In order to improve upon the CE loss and benefit from the one-to-one mapping of the implicit hate and its implied meaning, contrastive learning has been explored (Kim et al. Reference Kim, Park and Han2022), which has only provided slight improvement.

However, like cross-entropy, contrastive loss operates on a per-sample basis; even when considering positive and negative exemplars, they are curated on a per-sample basis. Clustering-inspired methods (Rippel et al. Reference Rippel, Paluri, Dollár and Bourdev2016; Song et al. Reference Song, Jegelka, Rathod and Murphy2017) have sought to overcome this issue by focusing on subclusters per class. ADD a.k.a magnet loss (Rippel et al. Reference Rippel, Paluri, Dollár and Bourdev2016) specifically lends a good starting point to operate the shift in the intercluster distance to extend to our use case. Based on the fact that ADD has surpassed contrastive losses in other tasks, we use ADD as a starting point and improve upon its formulation for implicit detection. As the current ADD setup fails to account for the implied meaning, we infuse external information into the latent space as an implied/inferential cluster.

Hypothesis

A manual inspection of the hate speech datasets reveals that non-hate is closer to implicit hate than explicit hate. We, thus, measure the intercluster distance between non-hate and implicit hate compared to non-hate and explicit hate.

Setup

For three implicit hate speech datasets, that is, LatentHatred ImpGab and AbuseEval, we embed all the samples for a dataset in the latent space using $768$ dimensional CLS embedding from BERT. The embeddings are not finetuned on any dataset or task related to hate speech so as to reduce the impact of confounding variables. We then consider 3 clusters directly adopted from the implicit, explicit, and non-hate classes and record the pairwise average linkage distance (ALD) and average centroid linkage distance (ACLD) among these clusters.

As the name suggests, for ACLD, we first obtain the embedding for the center of each cluster as a central tendency (mean or median) of all its representative samples and then compute the distance between the centers. This distance indicates the overall closeness of the two centers, which, in our case, measures the extent of similarity between the two classes. We also assess the latent space more granularly via ALD. In ALD, the distance between two clusters is obtained as the average distance between all possible pairs of samples where each element of the pair comes from a distinct group. It allows for a more fine-grained evaluation of the latent space, as not all data points are equidistant from each other or their respective centers. Formally, consider a system with E ( $\mathbb{R}\in d$ ) points, where each point ( $e_i$ ) belongs to one of the N clusters $c^n(e_i)$ and $\mu ^n = \frac{1}{|c^n|}\sum _{e_i \in c^n}e_i$ is the cluster center. For clusters $a$ and $b$ , $ACLD^{a,b}=dist(\mu ^a,\mu ^b)$ . Meanwhile, $ALD^{a,b}=\frac{1}{|c^a| * |c^b|}\sum _{e_i \in c^a \& e_j \in c^b}dist(e^i,e^j)$ where ( $c^a(e_i)\neq c^b(e_j)$ ).

The intuition behind using both ACLD and ALD stems from the fact that online hate speech is part of the larger discourse on the Web. Thus, it is possible that at the level of individual datapoint labeling an isolated instance as hateful is hard. Furthermore, some implicit samples may be closer to the explicit hate samples in terms of lexicon or semantics. On the other hand, it is also possible for some non-hate samples to contain slurs that are commonplace and context-specific but not objectionable within the community (Diaz et al. Reference Diaz, Amironesei, Weidinger and Gabriel2022; Röttger et al. Reference Röttger, Vidgen, Nguyen, Waseem, Margetts and Pierrehumbert2021). ACLD and ALD allow us to capture these dynamics at a macroscopic and a microscopic level.

Observation

From Table 1, we observe that under both ALD and ACLD, non-hate is closer to implicit samples. As expected, ALD shows more variability than ACLD. It follows from the fact that the mere presence of a keyword/lexicon does not render a sample as hateful.

Table 1.

The L1 intercluster distances between neutral (N) and explicit hate (E)), as well as non-hate and implicit hate (I) samples based on ALD and ACLD

Stemming from these observations, we see a clear advantage of employing a distance-metric approach that can exploit the granular variability in the latent space. Adaptive density discrimination (ADD) based clustering loss, which optimizes the inter and intra-clustering around the local neighborhood, directly maps to our problem of regional variability among the hateful and non-hateful samples. Further, our observations motivate the penalization of samples closer to the boundary responsible for increasing variability. The proposed model, as motivated by our empirical observations, is outlined in Figures 1 and 2.

3.1 Background on adaptive density discrimination

Here, we briefly outline ADD, which forms the backbone of our proposed framework. ADD is a clustering-based distance-metric. It evaluates the local neighborhood or clusters among the samples after each training iteration. At each epoch, after the training samples have been encoded into vector space, ADD clusters all data points within a class into $K$ representative local groups via K-means clustering. The subclusters within a class help capture the inter/intra-label similarity around the local neighborhood. If there are $N$ classes, then each training sample will belong to one of the $N*K$ subclusters.

Given that mapping and tracking distances among all N*K groups are computationally expensive, ADD randomly selects a reference/seed cluster $I_s^c$ representing class $C$ and then picks $M$ imposter clusters from local neighborhood $I_{s1}^{c'}, \ldots, I_{sm}^{c'}$ but from disparate classes ( $ c\not = c'$ ) based on their proximity to seed cluster. To understand the concept of seed and imposter cluster better, consider the three-way hate speech classification task with implicit, explicit, and non-hate labels. As we aim to distinguish implicit hate speech better, we select one of the implicit hate subclusters as the seed. Consequently, the imposter clusters will be from explicit hate or non-hate, where implicit hate can be misclassified. ADD then samples $D$ points uniformly at random from each sample cluster. For the $d^{th}$ data point in $m^{th}$ cluster, $r_d^m$ is its encoded vector representation, with $C(.)$ representing the class for the sample under consideration. Subsequently, $\mu ^m = \frac{1}{D}\sum _{d=1}^{D}r_d^m$ acts the mean representation of $m^{th}$ cluster. Here, ADD applies Equation 1 to discriminate the local distribution around a point:

(1) \begin{equation} p^{ADD}(r_d^m) = \frac{e^{-\frac{1}{2\sigma ^2} \left \|r_d^m - \mu ^m \right \|_2^2 - \alpha }}{{\sum _{\mu ^o:C(\mu ^o)\neq C(r_m^d)} -\frac{1}{2\sigma ^2} \left \|r_d^m - \mu ^o \right \|_2^2 }} \end{equation}

Here, $\alpha$ is a scalar margin for the cluster separation gap. The variance of all samples away from their respective centers is approximated via $\sigma ^2 =\frac{1}{MD-1}\sum _{m=1}^{M}\sum _{d=1}^{D}\left \|r_d^m - \mu ^m \right \|_2^2$ .

After each iteration, as the embedding space gets updated, so does each of the subclusters; this lends to a dynamic nature to ADD. It allows for the selection of random subclusters and data points after each iteration. The overall loss is computed via Equation 2.

(2) \begin{equation} \ell (\Theta ) = \frac{1}{MD}\sum _{m=1}^{M}\sum _{d=1}^{D}-\log{p^{ADD}}(r_d^m) \end{equation}

Novel component: inferential infusion

As each output label $y_d$ belongs to one of the distinct classes ( $c_i \in C$ ), we employ the respective embeddings $r_d$ and offline K-means algorithm to obtain $K$ subclusters per class. For implicit hate samples, the latent representation of their implied/inferential counterparts $\tilde{x_d}$ is denoted as $\tilde{r_d} = R_h(\tilde{x_d})$ . If $r_1^m, \ldots, r_d^m$ are representations for $D$ samples of $m^{th}$ implicit cluster, then $\tilde{r}_1^m,\ldots, \tilde{r}_d^m$ represent their respective inferential forms. The updated inferential adaptive density discrimination ( $ADD^{inf}$ ) helps reduce the distance between $(r_d,\tilde{r_d})$ for implicit hate samples via Equation 3.

(3) \begin{equation} p^{ADD^{inf}}(r_d^m) = \frac{e^{-\frac{1}{2\sigma ^2} \|r_d^m - \mu ^m \|_2^2 - \alpha } + e^{-\frac{1}{2\tilde{\sigma ^2}} \|r_d^m - \tilde{\mu ^m} \|_2^2 - \alpha }}{{\sum _{\mu ^o:C(\mu ^o)\neq C(r_m^d)}^{} -\frac{1}{2\sigma ^2} \left \|r_d^m - \mu ^o \right \|_2^2 }} \end{equation}

Here, $\mu ^m$ ( $\sigma ^2$ ) and $\tilde{\mu ^m}$ ( $\tilde{\sigma ^2}$ ) are the mean (variance) representations of the implicit and inferential/implied form for $m^{th}$ implicit cluster, respectively.

Figure 2.

The architecture of FiADD. Input X is a set of texts, implied annotations (only for implicit class), and class labels. PLM: pretrained language model (frozen). ${R'}_{nhate}$ , ${R'}_{exp}$ , and ${R'}_{imp}$ are the representatives for seed and imposter clusters of non-hate, explicit, and implicit, respectively. ${R'}_{inf}$ represents inferential meaning for corresponding ${R'}_{imp}$ . ACE is alpha cross-entropy, and $ADD^{Inf+foc}$ is the adaptive density discriminator with inferential + focal objective.

The above equation can be broken into two parts. The first part is equivalent to ADD thus focusing on reducing the intra-cluster distance within the implicit class. The second part brings the implicit class closer to its implied meaning. Meanwhile, in the case of explicit or non-hate clusters, there is no mapping to the inferential/implied cluster, and $ADD^{inf}$ in Equation 3 reduces to ADD in Equation 1.

Novel component: focal weight

Both $ADD^{inf}$ and ADD assign uniform weight to all samples under consideration. In contrast, we have established that some instances are closer to the boundary of the imposter clusters and harder to classify (i.e., contribute more to the loss). Inspired by the concept of focal cross-entropy (Lin et al. Reference Lin, Goyal, Girshick, He and Dollár2017), we improve the $ADD^{inf}$ objective by introducing $ADD^{inf + foc}$ . Under $ADD^{inf + foc}$ , the loss on each sample is multiplied by a factor called the focused term $(1-p^{ADD^{inf}}(r_d^m))^\gamma$ . $\gamma$ , a hyperparameter, acts as a magnifier. The formulation assigns uniform weight as $\gamma \rightarrow 0$ , reducing to $ADD^{inf}$ . Analogously, the focal term is paying “more attention” to specific data points. Even without inferential infusion, our novel focal term can be incorporated as $ADD^{foc}$ as enlisted in Equation 4.

(4) \begin{equation} \ell ^{ADD^{*}}(\Theta ) = \frac{1}{MD} \sum _{m=1}^{M} \sum _{d=1}^{D}\bigg [-(1-p^{ADD^{*}}(r_d^m))^\gamma log(p^{ADD^{*}}(r_d^m))\bigg ] \end{equation}

Here, $\ell ^{ADD^{inf+foc}}$ ( $\ell ^{ADD^{foc}}$ ) captures the setup with (without) inferential objective. We utilize $p^{ADD^{inf}}$ (Equation 3) for the former and $p^{ADD}$ (Equation 1) for the latter. Despite $ADD^{foc}$ being a minor update on ADD, we empirically observe that focal infusion improves ADD.

Training pipeline

It should be noted that selecting the seed cluster and its subsequent imposter clusters is a random process for initial iterations. We assign the label with the highest loss margin for later iterations as the seed. Here, $ADD^{inf + foc}$ operates for implicit hate and overcomes the drawback of existing literature where implicit detection fails to account for implied context. It is also essential to point out that this evaluation is carried out in the local neighborhood, which is aided by the focal loss (Equation 4).

Overall loss

Apart from employing $r_d$ in $\ell ^{ADD^{*}}$ , it is also passed through a classification head $CH(r_d)$ . We combine CE with the focal inference to obtain the final loss of FiADD with $\beta$ controlling the contribution of the two losses as Equation 5.

(5) \begin{equation} \ell (\Theta )= \beta \ell ^{CE}(\Theta ) + (1-\beta )\ell ^{ADD^{*}}(\Theta ) \end{equation}

Inference

During inference, the system does not have access to implied meaning. Once the PLM is trained via FiADD, the CH performs classification similar to any finetuned PLM. Here, we rely on the latent space being modified so that the implicit statements are closer to their semantic or implied form and sufficiently separated from other classes.

Note on K-mean

As a clustering algorithm, K-means is the most generic as it does not assume any dataset property (like hierarchy) except for the semantic similarity of the samples. Further, the K-means computation happens offline in each epoch, that is, it does not consume GPU resources. In the future, we aim to employ faster versions of K-means to improve training latency. Meanwhile, the computational complexity of FiADD during inference is the same as the finetuned PLM.

Implicit hate classification datasets

Based on our literature survey of implicit hate datasets, we discard the ones that are either multilingual (DALC) or multimodal (ConvAbuse, MMHS150K), as modeling them is out of the scope of current work. Further, SBIC and ToxiGen do not offer 3-way labels; hence, they are discarded, too. From among the English datasets left for assessment, we drop ISHate as it is an aggregated dataset, and its implicit samples are already covered by LatentHatred. Finally, we have LatentHatred AbuseEval, and ImpGab as English-based text-only datasets with explicit, implicit, and non-hate labels that suit our task. For LatentHatred, we employ the first level of annotation and the existing manual annotations of implied hatred for implicit samples. Meanwhile, AbuseEval and ImpGab do not have the implied descriptions. We manually annotate the implicit samples of these datasets with their implied meaning generated as free text.

Annotation for implied hate

Implied contexts are succinct statements that make explicit the underlying stereotype. Note that the implied context cannot be considered a comprehensive explanation for implicit hate but rather a more explicit understanding of the underlying subtle connotations. For AbuseEval and ImpGab, two expert annotators (one male and one female social media expert; age range between 29 to 35) perform the annotations based on the following guidelines:

• • Implied meaning should consider the post’s author’s perspective.

• • Implied meaning should emphasize on the post’s content only.

• • Annotations must be explicitly associated with the target entity.

• • Annotations must contain a broader abusive context for the given post.

• • Annotations should balance lexical diversity and uniformity w.r.t abuse toward a target group.

Annotation agreement

For our use case, annotation agreement scores help establish how well-aligned and coherent the explicit connotations are. To carry out the assessment, annotators A and B exchange a random sample of $30$ annotation pairs. They score the pairs on a 5-point Likert scale (Likert, Reference Likert1932), with 5 being the highest agreement. For AbuseEval, we obtain a mean agreement of $4.13 \pm 1.13$ and $4.07 \pm 1.41$ for ImpGab. Table 3 lists some sample annotations and their agreement scores. Further, a third expert (a 24-year-old male) conducts an independent survey using the above metric on the other set of random $30$ samples. As per annotator C, for AbuseEval, we obtain a mean agreement of $4.55 \pm 1.09$ and $4.41 \pm 1.15$ for ImpGab. This independent assessment corroborates the annotation process, as annotator C did not participate in the initial annotations yet observed similar alignment scores.

Table 3.

Some sample posts from AbuseEval and ImpGab along with their implied annotations. We also provide the cross-annotator scores and the cross-annotator remarks

Generalizibility testing

We further consider three SemEval tasks for our generalizability analysis. Sarcasm detection (Abu Farha et al. Reference Abu Farha, Oprea, Wilson and Magdy2022) and irony detection (Van Hee et al. Reference Van Hee, Lefever and Hoste2018) are two-way classification datasets. Meanwhile, stance detection (Mohammad et al. Reference Mohammad, Kiritchenko, Sobhani, Zhu and Cherry2016) is a three-way classification. While we have implied annotations for sarcasm, they are missing for the other two datasets. Here, no additional annotations are performed.

Hyperparameters

We run all experiments on two Nvidia V100 GPUs. Three random seeds (1, 4, 7) are used per setup. We report each setup’s best performance based on overall macro-F1 out of three random seeds, where the best seed for a setup may vary. We follow an 80-20 split for the dataset across experiments (specific to the seed). In initial experiments, we observe that $ADD^{inf + foc}$ has a stronger influence on the later iterations, whereas CE influences the initial ones. Thus, to balance them throughout the training process, we put equal weightage on both using $\beta = 0.5$ . We consider $K=3$ with $M=2$ imposters for all experiments. We leave the experiments for $\beta$ and $M$ for future work. We set $100$ as the maximum K-means iterations in each training step. During finetuning, each training cycle is executed for a maximum of $5000$ epochs with all layers of PLM frozen.

PLMs

We begin our assessment with BERT (Devlin et al. Reference Devlin, Chang, Lee and Toutanova2019). For hate speech detection, we also employ a domain-specific HateBERT (Caselli et al. Reference Caselli, Basile, Mitrović and Granitzer2021a) model to establish generalizability beyond BERT embedding. HateBERT is built upon the concepts of continued pretraining on top of BERT. Here, the corpus for performing another round of unsupervised masking language modeling is obtained from potentially offensive subreddits. For the SemEval tasks, we consider BERT and XLM (Chi et al. Reference Chi, Huang, Dong, Ma, Zheng, Singhal, Bajaj, Song, Mao, Huang and Wei2022) for evaluation based on their popularity in the SemEval. The PLMs variants are “bert-base-uncased” for BERT, “xlm-roberta-large” for XLM, and “GroNLP/hateBERT” for HateBERT.

Baselines

First, we assess the improvement in the performance of $ADD^{foc}$ over vanilla ADD (Equation 4) without the influence of cross-entropy. We follow the same prediction setup adopted in ADD (Rippel et al. Reference Rippel, Paluri, Dollár and Bourdev2016), where a sample gets assigned the label based on the nearest cluster in trained latent space during inference. We choose a simple Long short-term memory (LSTM) based (Hochreiter and Schmidhuber, Reference Hochreiter and Schmidhuber1997) model for quicker experimentation and compare the original ADD formulation with class weighted ADD ( $\alpha$ -ADD) and our proposed $ADD^{foc}$ . Table 4(a) shows a significant performance improvement of $8.2$ - $10.8$ % in overall macro-F1 using $ADD^{foc}$ across all three hate datasets. We thus recommend using our $ADD^{foc}$ variant instead of vanilla ADD for future works. Interestingly, we note that $\alpha$ -ADD does not outperform $ADD^{foc}$ . Hence, it is not employed in further experiments. Further, we perform a three-way classification using BERT to compare standalone alpha cross-entropy (ACE) against standalone $ADD^{foc}$ . The results are presented in Table 4(b). We observe that ACE outperforms the standalone $ADD^{foc}$ by a substantial margin of $7.6\%$ , $1.9\%$ , and $2.6\%$ for LatentHatred, ImpGab, and AbuseEval, respectively. Based on the above two experiments, we employ ACE as our baseline. As the proposed model introduces an additional loss complimenting ACE, we thus use ACE+ $ADD^{foc}$ variants as comparative systems.

Table 4.

Baseline selection based on comparison of $ADD^{foc}$ over: (a) vanilla ADD for two-way hate speech classification via LSTM. (b) ACE for three-way hate speech classification via BERT

Table 5.

Results for two-way hate classification on BERT and HateBERT. We also highlight the highest Hate class macro-F1 that the respective model can achieve

Seed-wise analysis

Across three random seeds, two PLMs, and three datasets, we record the performance for 18 setups, each in two-way and three-way hate speech detection. We note from Tables 8 and 9 that out of the 36 combinations, only four instances register a drop in performance. It corroborates that FiADD’s improvements are not limited to a specific initialization setup. Interestingly, the setups that register failure are all under HateBERT. The results further contribute to the discussion on domain-specific PLMs in Section 6.

Table 8.

Results for two-way classification task across all three hate-speech datasets using two pretrained language models, BERT and HateBERT. Highlighted with green color are the outcomes where one of the variants of FiADD outperforms baseline ACE

Table 9.

Results for three-way classification tasks across all three hate-speech datasets using two pretrained language models, BERT and HateBERT. Highlighted with green color are the outcomes where one of the variants of FiADD outperforms baseline ACE

Figure 4.

Error analysis with (a) correctly and (b) incorrectly classified samples in three-way classification on LatentHatred. Here, scores A and B are the relative positions of implicit sample w.r.t non-hate and explicit space finetuned with ACE and $ADD^{inf + foc}$ , respectively.

Error analysis

The motivation for FiADD is that implicit is closer to non-hate than explicit hate. Employing FiADD should correct the misclassified implicit labels if this hypothesis holds. On the other hand, a false positive may occur if the example is already close to explicit subspace. Further, moving it toward explicit space can cause misclassification. We, thus, consider a positive/negative case where the predicted label for an implicit sample is correctly/incorrectly classified. To explain these two scenarios, we estimate the relative distance of the implicit sample from explicit and non-hate clusters. First, we perform K-means clustering on non-hate and explicit latent space to identify their centers. We then calculate the average Manhattan distance between the implicit samples and these local density centers. Finally, we obtain the relative score from explicit space by normalizing between 0 and 1 the average explicit distance by the sum of average distances from non-hate and explicit spaces. For example, if the sample has a distance of 3 from explicit and 6 from non-hate centers, then the normalized distance will be $3/(3+6)=0.33$ .

We highlight a positive and a negative case in Figures 4 (a) and (b), respectively. In the positive case, the implicit sample is closer to non-hate space (Point A) under the ACE objective. After employing the FiADD, its relative position moves away from non-hate and closer to explicit (point B). In contrast, for the negative case, where the implicit sample is initially close to explicit hate (point A), our objective leads to misclassification. In the future, this problem can be reduced by introducing a constraint on the distance between implicit and explicit intact.

7.1 Latent space analysis

Building upon the cluster assessment in the error analysis, where we examined only a single positive and negative sample, we now perform an overall evaluation of how $ADD^{inf + foc}$ manipulates the embedding space. Inspired by the existing literature examining the latent space under hate speech datasets (Fortuna, Soler, and Wanner, Reference Fortuna, Soler and Wanner2020) and models (Kim et al. Reference Kim, Park and Han2022; Ocampo et al. Reference Ocampo, Sviridova, Cabrio and Villata2023c), we attempt to quantify the intercluster separation via Silhouette scores.

Silhouette score

It is a metric to measure the “goodness” of the clustering technique. It is calculated as a trade-off between within-cluster similarity and intercluster dissimilarity. Consider a system with E points ( $e_i$ ), each point belonging to one of the N clusters $c^j(e_i)$ . For $e_i \in c^a$ , its Silhouette score $SS_i=\frac{max(p_i,q_i)}{q_i-p_i}$ . $p_i$ captures the intra-cluster distance of $e_i$ to all the points within the cluster it belongs; $p_i = \frac{1}{|c^a|-1}\sum _{e^j \in c^a}dist(e_i,e_j)$ . $q_i$ captures the intercluster distance of $e_i \in c^a$ to all the points in the nearest cluster to $c^a$ ; $q_i = \frac{1}{|c^b|}\sum _{e^j \in c^b}dist(e_i,e_j)$ . The Silhouette score of a setup is, thus, $SS=\frac{1}{|E|}\sum _{e_i \in E}SS_i$ . Silhouette scores are measured on a scale of -1 to 1, with -1 being the worst set of cluster assignments.

Figure 5.

2D t-SNE plots of the last hidden representations after applying K-means (K = 3) on the implicit class for AbuseEval (a, b, c), ImpGab (d, e, f), and LatentHatred (g, h, i). $\{0, 1, 2\}$ are the subcluster ids. The higher the Silhouette score, the better discriminated the clusters.

Subclustering objective

After applying the $ADD^{inf + foc}$ objective, we expect not only the per-class clusters to be sufficiently separated but also the subclusters in each class to be better segregated to match their local neighborhood better. Figure 5 shows the implicit embedding space of AbuseEval, ImpGab, and LatentHatred after applying K-means on the default BERT embedding (a, d, g), BERT finetuned with the ACE (b, e, h), and FiADD (c, f, i) on three-way hate classification. The higher the Silhouette score, the better the subclusters are separated. $0.34$ , $0.31$ , and $0.51$ are the scores for cases (a), (b), and (c), respectively, in AbuseEval. $0.38$ , $0.24$ , and $0.52$ are the scores for cases (d), (e), and (f), respectively, in ImpGab. $0.32$ , $0.29$ , and $0.32$ are the scores for cases (d), (e), and (f), respectively, in LatentHatred.

Consequently, an increase of $0.20$ , $0.28$ , and $0.03$ scores is observed when comparing FiADD with ACE for AbuseEval, ImpGab, and LatentHatred, respectively. This increase in scores validates that the local densities within a class get further refined under $ADD^{inf + foc}$ objective. As expected, ACE suboptimally treats the implicit class as a single homogeneous cluster. Interestingly, for LatentHatred the score does not improve over the default BERT, even though it improves over ACE. A deeper analysis with multiple $K$ values might help here.

Figure 6.

2D t-SNE plots of the last hidden representations obtained for the implicit class and its respective inferential (implied) set for AbuseEval (a, b, c), ImpGab (d, e, f), and LatentHatred (g, h, i). The lower the Silhouette score, the closer the surface and implied forms of hate.

Inferential infusion

Given that $ADD^{inf + foc}$ brings the surface and semantic forms of implicit hate closer, we expect a significant drop in Silhouette scores between these clusters under FiADD. Figure 6 visualizes the embedding space of default BERT (a, d, g), BERT finetuned with the ACE (b, e, h), and FiADD (c, f, i) on three-way classification for AbuseEval, ImpGab, and LatentHatred. $0.18$ , $0.18$ , and $0.03$ are the scores for cases (a), (b), and (c), respectively, in AbuseEval. $0.18$ , $0.23$ , and $0.07$ are the scores for cases (d), (e), and (f), respectively, in ImpGab. $0.14$ , $0.13$ , and $0.01$ are the scores for cases (g), (h), and (i), respectively, in LatentHatred. It is important to highlight that for both BERT and BERT + ACE, there is no explicit objective to bring the implicit and implied clusters together. Hence, they act as a baseline for comparing how well the $ADD^{inf + foc}$ objective brings the two spaces closer.

A drop of $0.15$ , $0.16$ , and $0.12$ in the Silhouette score is observed when comparing BERT + ACE with FiADD for AbuseEval, ImpGab, and LatentHatred, respectively. It corroborates that the implicit and implied meaning representations are brought significantly closer to each other by employing our model. In addition to Tables 5 and 6, the latent space analysis also quantifies our manual annotations for AbuseEval and ImpGab, as inferential infusion (supported by the manual annotations) is improving the detection of implicit hate.