3.1 The qualitative beginnings: Ad hoc evaluation strategies

When directly annotated benchmarks are unavailable, ad hoc evaluations come into play, such as qualitative analysis of specific cases to reflect established knowledge of semantic changes. For example, Sagi et al. (Reference Sagi, Kaufmann and Clark2009, Reference Sagi, Kaufmann and Clark2011) analyzed changes in semantic density within context vectors to demonstrate how the meanings of dog and deer evolved from the Middle to Early Modern English periods, consistent with shifts documented in historical linguistics resources (Campbell, Reference Campbell2013; Traugott, Reference Traugott2017). Similarly, Wijaya and Yeniterzi (Reference Wijaya and Yeniterzi2011) traced the semantic transition of mouse from an animal to a computer pointing device by capturing the topic drift during the 1980s and 1990s. Furthermore, Kim et al. (Reference Kim, Chiu, Hanaki, Hegde and Petrov2014) captured notable shifts in words such as gay and cell, as evidenced by evolving cosine similarities with their semantic neighbors, validating that the model can detect genuine semantic change.

While case illustrations offer some evidence that computational approaches to semantic change detection are plausible, these methods are often unsystematic and inadequate for comparing models trained with different workflows and hyperparameters due to limited statistical power.

3.2 Establishing public datasets within the DURel framework: Its widespread use

The SemEval 2020 shared task marked a significant advancement in the field by introducing the first public benchmarks across English, German, Swedish, and Latin (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020, Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021). More significantly, it established the DURel framework, a systematic and straightforward annotation framework for benchmark construction that has evolved to incorporate more fine-grained information to address multiple detection evaluation scenarios (Schlechtweg, Schulte im Walde, and Eckmann Reference Schlechtweg, Schulte im Walde and Eckmann2018, Reference Schlechtweg, Virk, Sander, Sköldberg, Linke, Zhang, Tahmasebi, Kuhn and im Walde2023; Kutuzov et al. Reference Kutuzov, Fedorova, Schlechtweg and Arefyev2024). This framework has consistently guided the construction of datasets for languages including Russian (Kutuzov and Pivovarova Reference Kutuzov and Pivovarova2021a, Reference Kutuzov and Pivovarova2021b), Chinese (Chen, Chersoni, and Huang Reference Chen, Chersoni and Huang2022, Reference Chen, Chersoni, Schlechtweg, Prokic and Huang2023), Spanish (Zamora-Reina et al. Reference Zamora-Reina, Bravo-Marquez and Schlechtweg2022), Slovenian (Martinc et al. Reference Martinc, Dobrovoljc and Pollak2022), Norwegian (Kutuzov et al. Reference Kutuzov, Touileb, Mæhlum, Enstad and Wittemann2022a), Japanese (Ling et al. Reference Ling, Aida, Oka and Komachi2023), and Italian (Cassotti, Basile, and Tahmasebi Reference Cassotti, Basile and Tahmasebi2024a). This subsection primarily reviews these evaluation datasets and discusses their role in enabling evaluations for GCD, and subsequently, for binary change detection.

3.2.1 The DURel framework

Inspired by the continuum of synchronic semantic proximity proposed by Blank (Reference Blank1999), Schlechtweg et al. (Reference Schlechtweg, Schulte im Walde and Eckmann2018) adapted it to diachronic contexts of lexical semantic change detection, particularly targeting graded change detection. They measured the semantic relatedness of a target word across usage pairs on a scale from 1 to 4, as illustrated in Figure 2, where 1 signifies unrelated usages (e.g., homographs) and 4 denotes semantically identical usages. Unlike the earliest dataset for GCD created by Gulordava and Baroni (Reference Gulordava and Baroni2011), which relied on out-of-context judgments, the DURel framework gathers in-context human judgments. This inclusion of contextual information not only enhances the reliability of the assessments, achieving higher annotation agreement, but also increases transparency in modeling semantic change and provides a bridge to historical usages.

Figure 2.

The DURel relatedness scale: A continuum of semantic proximity (Schlechtweg et al. Reference Schlechtweg, Schulte im Walde and Eckmann2018).

In the DURel framework, human judgments on semantic proximity of a target word across usage pairs are collected in two settings: pairs for word $w$ from the same time period, for example $sp_1 = \{(s_{t_1}^1, s_{t_1}^2), \ldots , (s_{t_1}^m, s_{t_1}^n)\}$ and $sp_2 = \{(s_{t_2}^1, s_{t_2}^2), \ldots , (s_{t_2}^m, s_{t_2}^n)\}$ ; pairs from different periods, for example $sp_{1,2} = \{(s_{t_1}^1, s_{t_2}^2), \ldots , (s_{t_1}^m, s_{t_2}^n)\}$ . The annotated scores are then aggregated using two metrics: $\Delta \text{LATER}$ and COMPARE. The former metric, $\Delta \text{LATER}$ , evaluates the shift in the mean score of $w$ between pairs from $sp_1$ (EARLIER) and $sp_2$ (LATER), as formulated in Equation (1). The absolute value of $\Delta \text{LATER}$ reflects the degree of semantic change, with larger values indicating more significant shifts. The latter metric, COMPARE, averages relatedness scores across pairs formed from sentences spanning different periods, $sp_{(1,2)}$ , as defined in Equation (2). Higher COMPARE scores indicate greater semantic stability, whereas lower scores suggest more pronounced meaning shifts.

(1) \begin{equation} \Delta \text{LATER} (w) = \frac {1}{|sp_2|} \sum _{x \in sp_2} x - \frac {1}{|sp_1|} \sum _{x \in sp_1} x \end{equation}

(2) \begin{equation} \text{COMPARE} (w) = \frac {1}{|sp_{(1,2)}|} \sum _{x \in sp_{(1,2)}} x \end{equation}

In general, COMPARE is more widely used than $\Delta$ LATER (Schlechtweg et al. Reference Schlechtweg, Schulte im Walde and Eckmann2018; Kutuzov and Pivovarova Reference Kutuzov and Pivovarova2021a; Chen et al. Reference Chen, Chersoni and Huang2022), as it better approximates the Jensen–Shannon Distance between sense frequency distributions, a more interpretable measure from the DWUGs framework explained in Section 3.2.2. This tendency is seen in increasing correlations between these two measurements ranging from 0.87 to 0.94 when sample sizes increase from 10 to 500 (Schlechtweg, Reference Schlechtweg2023, p. 64).

3.2.2 DWUGs

The original DURel framework, initially designed for graded change detection (GCD), was subsequently expanded to include binary change detection (BCD) and new measurements for both BCD and GCD based on sense frequency distribution. This expansion was facilitated by the creation of diachronic word usage graphs (DWUGs), which are populated based on human DURel-like judgments. We summarize the workflow of creating DWUGs in Figure 3.

Figure 3.

Human measurements on semantic change within the paradigm of DWUGs.

Figure 3. Long description

Panel 1: The process begins with target word w in sentence pairs. Panel 2: Human judgments are made on the semantic proximity of these words, represented by sliders. Panel 3: A directed weighted unweighted graph (DWUG) for w is created, and subgraphs for time points t1 and t2 are populated through clustering. Panel 4: Measurements for binary change and graded change are derived from the graph.

A word usage graph (WUG) measures the usage of a word within a specific period, where nodes represent individual usages, edge weights reflect the semantic proximity between usage pairs collected from annotators, and clusters are groups of similar usages formed by correlation clustering (Bansal, Blum, and Chawla Reference Bansal, Blum and Chawla2004; Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020). The clusters then represent word senses, where the emergence or disappearance of clusters across WUGs indicates the development or loss of word senses, and the cluster frequency distribution changes capture to what extent a word has changed across periods. This method offers a structured and visually intuitive framework to track word usage evolution over time (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020, Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021). Figure 4 illustrates a DWUG for the English word head, with cluster changes between subgraphs b and c suggesting semantic changes.Footnote ^d

Figure 4.

A DWUG for the English word head (a). Subgraphs b and c represent usages in two separate periods. Nodes represent the usages of the respective target words, and edges represent the median of judgments from annotators. Colors indicate different usage types.

Instead of defining word usage pairs a priori, human judgments on semantic proximity in the DWUGs are collected from randomly aggregated pairs. Although theoretically each word usage could be paired with every other for comparison, full annotation is often impractical due to the extensive workload (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020, Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021; Schlechtweg, Reference Schlechtweg2023; Schlechtweg et al. Reference Schlechtweg, Virk, Sander, Sköldberg, Linke, Zhang, Tahmasebi, Kuhn and im Walde2023). For instance, if $n$ usages are sampled, then $\frac {n(n-1)}{2}$ pairs would require annotation. Consequently, DWUGs for many languages such as English, German, Swedish, and Latin are only partially annotated (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020, Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021). Typically, these DWUGs consist of 100 sampled sentences per period, which are randomly distributed and annotated over several rounds to ensure that approximately half of the usage pairs are evaluated by more than one annotator. In contrast, full annotation becomes feasible when the sample size is smaller (Zamora-Reina et al. Reference Zamora-Reina, Bravo-Marquez and Schlechtweg2022; Kutuzov et al. Reference Kutuzov, Touileb, Mæhlum, Enstad and Wittemann2022a), such as 20 sampled usages per period (Chen et al. Reference Chen, Chersoni, Schlechtweg, Prokic and Huang2023).

Since WUGs are exploited to measure sense distributions, measurements for GCD in DWUGs quantify the overall sense distribution differences using Jensen–Shannon Distance (JSD), as formulated in Equation (3),

(3) \begin{equation} JSD(P,Q) = \sqrt {\frac {KLD(P | M) + KLD(Q | M)}{2}} \end{equation}

where $P$ and $Q$ are the normalized probability distributions of the cluster (sense) frequency distributions $D$ and $E$ , $M$ is the average distribution of $P$ and $Q$ , and the Kullback–Leibler Divergence (KLD), which are defined as:

(4) \begin{equation} \begin{aligned} KLD(P \| Q) &= \sum _{i=1}^{K} p_i \log _2\left (\frac {p_i}{q_i}\right ), \\[5pt] M &= \frac {P + Q}{2} \end{aligned} \end{equation}

According to JSD, higher values indicate greater semantic change between periods, while lower scores suggest more stable usage frequencies. However, given the potential noise introduced by the clustering step in DWUGs and the robustness of the COMPARE metric in the DURel framework, COMPARE is recommended as a secondary metric for graded change detection, as it has been shown that it approximates the JSD rather well (Schlechtweg, Reference Schlechtweg2023, p. 39).

This sense-level measurement also enabled binary change detection by observing the emergence and disappearance of clusters across WUGs from different periods, as defined in Equation (5),

(5) \begin{equation} B(w) = \begin{cases} 1 & \text{if for any sense } i, D_i \leq k \text{ and } E_i \geq n, \text{ or vice versa,} \\ 0 & \text{otherwise.} \end{cases} \end{equation}

where $D_i$ and $E_i$ are frequencies of sense $i$ in two periods, and $k$ and $n$ are lower frequency thresholds used to reduce noise in clustering.

Recently, there have been attempts to leverage large language models (LLMs) as computational annotators (Meyer et al. Reference Meyer, Elsweiler, Ludwig, Fernandez-Pichel and Losada2022; Ding et al. Reference Ding, Qin, Liu, Chia, Li, Joty and Bing2023; Tan et al. Reference Tan, Beigi, Wang, Guo, Bhattacharjee, Jiang, Karami, Li, Cheng and Liu2024) in LSC contexts (Noble, Periti, and Tahmasebi Reference Noble, Periti and Tahmasebi2024; Periti and Tahmasebi Reference Periti and Tahmasebi2024a; Yadav, Choppa, and Schlechtweg Reference Yadav, Choppa and Schlechtweg2024; Periti, Dubossarsky, and Tahmasebi Reference Periti, Dubossarsky and Tahmasebi2024c; Karjus, Reference Karjus2025; Zamora-Reina et al. Reference Zamora-Reina, Bravo-Marquez, Schlechtweg and Arefyev2025), aiming to scale up and assist human annotation in the future. Additionally, these DWUGs have been enriched with cluster labels, representing sense information derived from natural language generation (NLG) models (Giulianelli et al. Reference Giulianelli, Luden, Fernandez and Kutuzov2023; Kutuzov et al. Reference Kutuzov, Fedorova, Schlechtweg and Arefyev2024). Figure 5 displays an enriched DWUG for the English word head, where clusters are colored and labeled with generated definitions (e.g., prototypical definitions) (Giulianelli et al. Reference Giulianelli, Luden, Fernandez and Kutuzov2023). This procedure relies on the techniques of definition generation, which will be reviewed in Section 4.4.

Figure 5.

An enriched DWUGs for the English word head. Sense labels are derived from Kutuzov et al. (Reference Kutuzov, Fedorova, Schlechtweg and Arefyev2024).

Figure 5. Long description

A diagram of the enriched DWUGs for the English word head. The diagram includes multiple nodes and connections representing different senses and their relationships. The nodes are color-coded to indicate different categories: blue for the upper part of the human body, including the neck and face; orange for the person in charge of something, a leader; green for the top or extreme point of something; pink for the topmost part of a tombstone. The nodes are connected by lines, indicating relationships between different senses. The central node, labeled with the number 7, is connected to several other nodes, showing its significance in the network. The diagram illustrates how the word head can have various meanings and how these meanings are interconnected.

3.3 Sense-level datasets derived from lexicographic resources

As detailed in earlier sections, the prevalent benchmarks derived from the DURel family do not provide explicit sense-level information but rather infer senses from usage patterns. These simulated senses, obtained through clustering processes, are often criticized for introducing noise and creating sparsity issues (Schlechtweg, Reference Schlechtweg2023, p. 57). In contrast, sense-level datasets grounded in lexicographic resources provide explicit sense annotations, with earlier examples primarily designed for binary change detection (Lau et al. Reference Lau, Cook, McCarthy, Gella and Baldwin2014; Tang, Qu, and Chen Reference Tang, Qu and Chen2016; Basile, Semeraro, and Caputo Reference Basile, Semeraro and Caputo2019). Beyond binary detection, such datasets have attracted growing attention for supporting more diverse evaluation scenarios (Schlechtweg et al. Reference Schlechtweg, Zamora-Reina, Bravo-Marquez and Arefyev2024; Fedorova et al. Reference Fedorova, Mickus, Partanen, Siewert, Spaziani and Kutuzov2024b).

3.4 Other evaluation strategies and useful resources

Alternative evaluation strategies, such as leveraging datasets for downstream tasks or creating synthetic datasets, have also been explored in the literature. For instance, Frermann and Lapata (Reference Frermann and Lapata2016) evaluated their model using datasets derived from diachronic text evaluation (Popescu and Strapparava Reference Popescu and Strapparava2015), assuming that model performance on these tasks would correlate with LSC performance. Synthetic corpora are typically constructed by combining authentic data with artificially inserted changes (Kulkarni et al. Reference Kulkarni, Al-Rfou, Perozzi and Skiena2015; Dubossarsky, Weinshall, and Grossman 2017, Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019; Shoemark et al. Reference Shoemark, Liza, Nguyen, Hale and McGillivray2019). For example, Kulkarni et al. (Reference Kulkarni, Al-Rfou, Perozzi and Skiena2015) duplicated a Wikipedia dataset 20 times to simulate an absence of semantic changes and then introduced linguistic shifts in the last 10 snapshots through frequency and syntactic perturbations. In a similar approach, Dubossarsky et al. (Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019) simulated no semantic change by extracting data from each time bin and shuffling it evenly. Shoemark et al. (Reference Shoemark, Liza, Nguyen, Hale and McGillivray2019) simulated an absence of semantic change by retrieving 66 random samples from 10% of the data from a single month, and then injected controlled changes by inserting predefined pseudowords while considering frequencies and co-occurrence distributions.

Additionally, there are some LSC-related datasets that, although not directly used for LSC evaluation, also merit attention. One such dataset is The Database of Semantic Shifts in the Languages of the World (DatSemShift) (Zalizniak et al. Reference Zalizniak, Smirnitskaya, (Rousseau), Gruntov, Maisak, Ganenkov, Bulakh, Orlova, Bobrik-Fremke, Dereza, Mikhailova, Bibaeva and Voronov2024), which is the largest publicly available dataset on semantic change, covering 1,792 documented cases of semantic shifts across 516 languages (Brochhagen et al. Reference Brochhagen, Boleda, Gualdoni and Xu2023; Fugikawa et al. Reference Fugikawa, Hayman, Liu, Yu, Brochhagen and Xu2023). In this database, semantic shifts are understood as the cognitive proximity between two meanings conflated within one form, with realization types including synchronic polysemy, diachronic semantic evolution, morphological derivation, cognates, and borrowing. This resource is particularly valuable for examining typology and cross-linguistic regularities in semantic shifts (Zalizniak, Reference Zalizniak2018; Brochhagen et al. Reference Brochhagen, Boleda, Gualdoni and Xu2023; Fugikawa et al. Reference Fugikawa, Hayman, Liu, Yu, Brochhagen and Xu2023).

3.5 Summary

This section reviewed evaluation strategies for LSC models, focusing particularly on the prevalent benchmarks derived from the DURel family and sense-level datasets, as these benchmarks are closely tied to the shared tasks introduced in Section 2. A thorough understanding of how benchmarks are constructed can also inform LSC model design and facilitate performance comparisons in shared tasks. In addition, we reviewed ad hoc evaluations of words known to have changed, as well as alternative approaches such as utilizing datasets from extrinsic tasks and synthetic corpora for evaluation when direct benchmarks are unavailable or when the annotation workload is too demanding.

In general, the construction of evaluation datasets has evolved from out-of-context annotation, involving either comparison of definitions across dictionaries (Lau et al. Reference Lau, Cook, McCarthy, Newman and Baldwin2012; Tang et al. Reference Tang, Qu and Chen2013; Basile et al. Reference Basile, Semeraro and Caputo2019) or out-of-context human judgments on the degree of change (Gulordava and Baroni Reference Gulordava and Baroni2011), to in-context annotation (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020; Basile et al. Reference Basile, Caputo, Caselli, Cassotti and Varvara2020a; Schlechtweg et al. Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021). More recently, the field has moved within the in-context paradigm towards sense-level annotation, aimed at improving the interpretation of detected changes and adapting to versatile evaluation scenarios (Schlechtweg et al. Reference Schlechtweg, Zamora-Reina, Bravo-Marquez and Arefyev2024). This shift offers better transparency for both human annotation and computational modeling.

The annotation landscape is also being reshaped by the use of LLMs as computational annotators (Yadav et al. Reference Yadav, Choppa and Schlechtweg2024; Periti and Tahmasebi Reference Periti and Tahmasebi2024a; Zamora-Reina et al. Reference Zamora-Reina, Bravo-Marquez, Schlechtweg and Arefyev2025). A recognized concern is time-knowledge leakage: LLMs trained predominantly on contemporary text may project modern senses onto historical usages. The DWUG protocol partially mitigates this by design, since annotators judge the semantic relatedness of a usage pair in minimal sentential context on a 1–4 scale; diachronic period is corpus metadata, not a cue presented to the model. A residual risk is stylistic: LLMs may conflate surface-level register dissimilarity between historical and contemporary usages with genuine sense difference, particularly across longer temporal spans, and prompt strategies that direct the model to attend to propositional content rather than surface form can partially reduce this effect. On reliability, Yadav et al. (Reference Yadav, Choppa and Schlechtweg2024) find that LLM performance is poor under human-style annotation guidelines but improves substantially with optimized prompts. Zamora-Reina et al. (Reference Zamora-Reina, Bravo-Marquez, Schlechtweg and Arefyev2025) show that, with automatic prompt optimization, larger LLMs can match or exceed specialized LSCD models at the annotation level for English, but that this advantage is scale- and language-dependent: medium-sized models underperform specialist models, and for Spanish all tested LLMs fall short of DeepMistake. LLMs, therefore, offer a scalability advantage for languages or time periods where specialist models are unavailable, though prompt optimization overhead and the compute cost of large models partially offset this benefit.

Finally, binary change detection is difficult to define and carries methodological limitations when it comes to binarizing a graded signal in computational paradigms, including the DWUG framework (Schlechtweg et al. Reference Schlechtweg, Virk, Sander, Sköldberg, Linke, Zhang, Tahmasebi, Kuhn and im Walde2023). The derivation of binary change labels involves a multistep pipeline, from sparsely annotated usage graphs through graph clustering to threshold-based label assignment, and each step introduces potential instability. Specifically, BCD reliability is sensitive at three levels: graph annotation density, where sparse annotations render clustering results highly unstable (Noble et al. Reference Noble, Periti and Tahmasebi2024); clustering algorithm choice, where correlation clustering and stochastic block model variants diverge considerably, and most markedly on the words with complex or ambiguous usage patterns that are most theoretically interesting (Schlechtweg et al. Reference Schlechtweg, Tahmasebi, Hengchen, Dubossarsky and McGillivray2021; Noble et al. Reference Noble, Periti and Tahmasebi2024); and the fragility of binary labels, since binarizing a graded signal is intrinsically noisy and the label for a given word can reverse after additional annotation rounds (Schlechtweg, Reference Schlechtweg2023). We therefore recommend that researchers (1) report graded change scores alongside binary labels, as graded scores are more robust to sampling noise; (2) run clustering with multiple random seeds and report the proportion of words with stable binary labels across runs; and (3) stratify results by word-level clustering confidence (e.g., by inter-run ARI) to enable a more nuanced interpretation of benchmark performance (Noble et al. Reference Noble, Periti and Tahmasebi2024).Footnote ^f

4.1 The first generation: An exploratory era with count-based DSMs

The workflow of earlier LSC models built on count-based DSMs typically involved aligning time-specific co-occurrence matrices across periods and then comparing them through measurements such as CD (Sagi et al. Reference Sagi, Kaufmann and Clark2009; Cook and Stevenson Reference Cook and Stevenson2010; Sagi et al. Reference Sagi, Kaufmann and Clark2011; Gulordava and Baroni Reference Gulordava and Baroni2011; Tang et al. Reference Tang, Qu and Chen2013, Reference Tang, Qu and Chen2016). Although these models suffered from the curse of data sparsity and high dimensionality (Turney and Pantel Reference Turney and Pantel2010; Lenci et al. Reference Lenci, Sahlgren, Jeuniaux, Cuba Gyllensten and Miliani2023), making their performance generally weaker compared to later generations, they laid the essential foundation for subsequent developments in LSC modeling. In this subsection, we break down these earlier models into three essential components: base model, temporal aggregation, and measurements. This framework will guide our review of later generations and help explain performance breakthroughs.

4.1.1 Base model

LSC models with count-based DSMs as their backbone derive meaning representations by computing co-occurrence frequencies in the source corpus, such as between target words and their surrounding words (Sagi, Kaufmann, and Clark Reference Sagi, Kaufmann and Clark2009, Reference Sagi, Kaufmann and Clark2011; Gulordava and Baroni Reference Gulordava and Baroni2011; Tang et al. Reference Tang, Qu and Chen2013), between targets and contexts with rich linguistic features (Mitra et al. Reference Mitra, Mitra, Riedl, Biemann, Mukherjee and Goyal2014, Reference Mitra, Mitra, Maity, Riedl, Biemann, Goyal and Mukherjee2015), or by modeling target word distributions across documents (Wijaya and Yeniterzi Reference Wijaya and Yeniterzi2011). The raw co-occurrence counts are often further transformed through weighting and dimensionality reduction to improve vector quality.

Common weighting techniques include term frequency-inverse document frequency (TF-IDF) (Spärck Jones, Reference Spärck Jones1972; Turney and Pantel Reference Turney and Pantel2010), local mutual information (LMI) (Evert and Baroni Reference Evert and Baroni2007; Gulordava and Baroni Reference Gulordava and Baroni2011), pointwise mutual information (PMI) (Church and Hanks Reference Church and Hanks1990; Turney and Pantel Reference Turney and Pantel2010), and positive pointwise mutual information (PPMI) (Niwa and Nitta Reference Niwa and Nitta1994; Turney and Pantel Reference Turney and Pantel2010).

Among these, PPMI has emerged as particularly popular in LSC models (Cook and Stevenson Reference Cook and Stevenson2010; Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Rodda, Senaldi, and Lenci Reference Rodda, Senaldi and Lenci2017; Dubossarsky, Weinshall, and Grossman Reference Dubossarsky, Weinshall and Grossman2017; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019). PPMI focuses on retaining only positive word associations, filtering out negative co-occurrence values that carry little semantic signal. As defined in Equation (6) (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c), raw co-occurrence counts are transformed as follows:

(6) \begin{equation} \mathbf{M}_{i,j}^{\mathrm{PPMI}} = \max \left \{ \log \left ( \frac {\hat {p}(w_i, c_j)}{\hat {p}(w_i)\hat {p}(c_j)} \right ) - \alpha , 0 \right \} \end{equation}

where $\alpha$ is a smoothing factor to reduce bias towards infrequent words, typically set at $\mathrm{\alpha } = 0.75$ (Levy, Goldberg, and Dagan Reference Levy, Goldberg and Dagan2015; Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019). $\hat {p}$ represents the smoothed co-occurrence probabilities between target word $w_i$ and context word $c_j$ .

Dimensionality reduction is often applied to sparse count-based DSMs to enhance computational efficiency while retaining latent information (Turney and Pantel Reference Turney and Pantel2010; Lenci et al. Reference Lenci, Sahlgren, Jeuniaux, Cuba Gyllensten and Miliani2023), which often leads to performance improvements in LSC detection tasks (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016b, Dubossarsky et al. Reference Dubossarsky, Weinshall and Grossman2017; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019). Common techniques include Latent Dirichlet Allocation (LDA) (Blei, Ng, and Jordan Reference Blei, Ng and Jordan2003), Random Indexing (RI) (Karlgren and Sahlgren Reference Karlgren and Sahlgren2001), and Singular Value Decomposition (SVD) (Deerwester et al. Reference Deerwester, Dumais, Furnas, Landauer and Harshman1990; Landauer and Dumais Reference Landauer and Dumais1997).

SVD is widely used for this purpose, factorizing a high-dimensional matrix into the product of three matrices, $U \Sigma V^T$ , while retaining only the top $d$ (e.g., $d = 300$ ) singular values and corresponding singular vectors, as defined below:

(7) \begin{equation} M^{svd} = U_d \Sigma _d^\gamma , \end{equation}

where $\gamma$ is an eigenvalue weighting parameter that is used to adjust the influence of each dimension (Turney and Pantel Reference Turney and Pantel2010; Bullinaria and Levy Reference Bullinaria and Levy2012; Levy et al. Reference Levy, Goldberg and Dagan2015).

4.1.2 Temporal aggregation

Temporal information in count-based LSC models is derived from metadata in the underlying corpus, such as publication dates of news articles or publication years of books. The corpus is typically divided into time bins, with separate co-occurrence matrices constructed for each period; when trained independently, these DSMs require alignment to ensure semantic representations remain comparable across periods (e.g., Sagi, Kaufmann, and Clark Reference Sagi, Kaufmann and Clark2009, Reference Sagi, Kaufmann and Clark2011; Gulordava and Baroni Reference Gulordava and Baroni2011; Tang et al. Reference Tang, Qu and Chen2013).

A common alignment approach for raw count-based matrices is Column Intersection (CI), which retains only columns corresponding to words present in both time periods, as defined in Equation (8) (Hamilton, Leskovec, and Jurafsky, Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019).

(8) \begin{equation} \begin{aligned} M1^{CI}_{j} &= M1_{j} \quad &\text{for all } c_j \in V_{t_1} \cap V_{t_2}, \\ M2^{CI}_{j} &= M2_{j} \quad &\text{for all } c_j \in V_{t_1} \cap V_{t_2}, \end{aligned} \end{equation}

where $c_j$ represents a context word in the shared vocabulary $V_{t_1} \cap V_{t_2}$ , and $M1_{j}$ and $M2_{j}$ refer to the $j$ -th columns of matrices from periods $t_1$ and $t_2$ , respectively.

For dimensionality-reduced count-based DSMs (i.e., Bayesian probabilistic methods: Wijaya and Yeniterzi Reference Wijaya and Yeniterzi2011; Frermann and Lapata Reference Frermann and Lapata2016 and SVD methods: Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019; Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019), implicit factorizations diminish explicit co-occurrence frequencies and necessitate additional mathematical operations to ensure vectors are mutually orthogonal. These methods use the same alignment techniques as low-dimensional embeddings from prediction-based models, which will be reviewed in Section 4.2.2.

4.1.3 Measurements

With time-specific embeddings in place, semantic change across periods is measured primarily through distances between temporal embeddings (e.g., $\vec {w_{t_1}}$ and $\vec {w_{t_2}}$ ):

Cosine distance (CD). CD, defined in Equation (10), is derived from cosine similarity,Footnote ^g which measures the cosine of the angle between two vectors. When applied to word vectors from different time periods, $\text{CD} (\vec {w_{t_1}}, \vec {w_{t_2}})$ estimates the extent to which the meaning of $w$ has drifted semantically across periods, with higher values indicating greater semantic change.

(9) \begin{equation} \cos (\vec {w_{t_1}}, \vec {w_{t_2}}) = \frac {\vec {w_{t_1}} \cdot \vec {w_{t_2}}}{\sqrt {\vec {w_{t_1}} \cdot \vec {w_{t_1}}} \sqrt {\vec {w_{t_2}} \cdot \vec {w_{t_2}}}} \end{equation}

(10) \begin{equation} \text{CD} (\vec {w_{t_1}}, \vec {w_{t_2}}) = 1 - \cos (\vec {w_{t_1}}, \vec {w_{t_2}}) \end{equation}

Local neighborhood distance (LND). LND calculates the CD between two second-order similarity vectors (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016b; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019), where a second-order $s$ consists of the cosine similarity between $\vec {w}$ and each member $\vec {n_i}$ in the union of the $k$ nearest neighbors from two periods (e.g., $k \in [10,50]$ ), as defined in Equation (11) (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016b). This metric seems to be more sensitive to paradigmatic relations than to shifts in syntagmatic contexts (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016b).

(11) \begin{equation} s = \cos (\vec {w}, \vec {n_i}) \quad \forall \vec {n_i} \in N_k\big(\vec {w_{t_1}}\big) \cup N_k\big(\vec {w_{t_2}}\big) \end{equation}

(12) \begin{equation} LND\big(\vec {w_{t_1}},\vec {w_{t_2}}\big) = CD\big(s_{w_{t_1}}\!, s_{w_{t_2}}\big) \end{equation}

Beyond distance-based measurements, earlier models also employed semantic density and topic change. For example, Sagi et al. (Reference Sagi, Kaufmann and Clark2011) used the average pairwise similarity of a group of vectors, referred to as density, to infer meaning broadening or narrowing. Wijaya and Yeniterzi (Reference Wijaya and Yeniterzi2011) interpreted meaning change by analyzing topic shifts.

4.2 The second generation: A robust era with prediction-based models

The second generation of LSC models was defined by the advent of prediction-based approaches, which largely replaced the previous count-based DSMs. These new models relied on prediction rather than co-occurrence counting and demonstrated superior performance across a wide range of intrinsic and extrinsic evaluations (Baroni, Dinu, and Kruszewski Reference Baroni, Dinu and Kruszewski2014; Levy et al. Reference Levy, Goldberg and Dagan2015; Sahlgren and Lenci Reference Sahlgren and Lenci2016; Lenci et al. Reference Lenci, Sahlgren, Jeuniaux, Cuba Gyllensten and Miliani2023), as well as LSC tasks specifically (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019; Shoemark et al. Reference Shoemark, Liza, Nguyen, Hale and McGillivray2019; Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020).

Among these prediction-based approaches, Skip-gram with Negative Sampling (SGNS) quickly established itself as a particularly powerful architecture for detecting semantic shifts (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020). Despite their effectiveness, models in this generation sparked considerable debate regarding the optimal methods for temporal aggregation to minimize alignment noise (Bamler and Mandt Reference Bamler and Mandt2017; Yao et al. Reference Yao, Sun, Ding, Rao and Xiong2018; Rudolph and Blei Reference Rudolph and Blei2018; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019). In this subsection, we review these temporal aggregation methods and examine their impact on LSC performance.

4.2.1 Base model: SGNS models

The backbone of LSC models in this generation relies primarily on SGNS, which directly generates low-dimensional vectors using neural networks (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013a, Levy et al. Reference Levy, Goldberg and Dagan2015; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019; Lenci and Sahlgren Reference Lenci and Sahlgren2023; Lenci and Sahlgren Reference Lenci and Sahlgren2023). Specifically, SGNS learns $d$ -dimensional vectors for each word $w$ and context word $c$ by optimizing a shallow neural network. This optimization maximizes the probability of observed word-context pairs, estimated via their dot product ( $\vec {w} \cdot \vec {c}$ ), while minimizing the probability of randomly chosen negative samples. This process is formalized in Equation (13):

(13) \begin{equation} \arg \max _{\theta } \sum _{(w,c) \in D} \log \sigma (\vec {c} \cdot \vec {w}) + \sum _{(w,c) \in D'} \log \sigma (\!-\!\vec {c} \cdot \vec {w}), \end{equation}

where $D$ is the set of all observed word-context pairs, $D'$ represents randomly generated negative samples (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013a, Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019), and $\sigma (x) = \frac {1}{1 + e^{-x}}$ is the sigmoid function, which outputs a probability between 0 and 1. The parameter set $\theta$ represents all trainable weights in the model.

4.2.2 Temporal aggregation

As noted, low-dimensional embeddings from different semantic spaces require additional steps for alignment because models like SGNS are inherently stochastic, while those derived from SVD are nonunique. Perhaps the most straightforward solution is post-processing using techniques such as Orthogonal Procrustes (OP) (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c) to align independently trained vector spaces. Alternatively, incorporating temporal information before or during training eliminates the need for post-alignment processing (Kim et al. Reference Kim, Chiu, Hanaki, Hegde and Petrov2014; Bamler et al. Reference Bamler and Mandt2017; Yao et al. Reference Yao, Sun, Ding, Rao and Xiong2018; Rudolph and Blei Reference Rudolph and Blei2018; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019).

Orthogonal procrustes (OP). OP is a popular linear transformation technique used to align vector spaces by ensuring consistency in dimensionality, scale normalization, and orientation. As defined in Equation (14) (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c),

(14) \begin{equation} R = \mathop {\arg } \min \limits _{Q^T Q = I} \left \lVert QW^{(t_1)} - W^{(t_2)} \right \rVert _F, \end{equation}

the goal is to find the orthogonal transformation matrix $Q$ that minimizes the Frobenius norm between the transformed embeddings from time $t_1$ and the embeddings from time $t_2$ . This norm represents the sum of squared Euclidean distances between corresponding vectors in the two matrices. By minimizing this distance, OP effectively rotates the embeddings from time $t_1$ to best align with those from time $t_2$ , while preserving their relative geometric structure, including cosine similarities (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019).

Vector initialization (VI). VI involves initializing vectors based on data from an initial time period, then updating them incrementally with data from subsequent years (Kim et al. Reference Kim, Chiu, Hanaki, Hegde and Petrov2014; Dubossarsky et al. Reference Dubossarsky, Tsvetkov, Dyer and Grossman2015, Reference Dubossarsky, Weinshall and Grossman2016). The underlying assumption is that if a word $w$ appears in similar contexts between $t_1$ and $t_2$ , its vector for $t2$ will only undergo minor adjustments; however, when contexts differ more substantially, the updates will be larger (Kim et al. Reference Kim, Chiu, Hanaki, Hegde and Petrov2014; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019). For instance, Kim et al. (Reference Kim, Chiu, Hanaki, Hegde and Petrov2014) use data from 1850 to 1899 for vector initialization and start their analysis from 1990 by updating subsequent yearly data.

Word injection (WI). WI replaces target words with placeholders while using the entire corpus to learn a unified vector space (Ferrari, Donati, and Gnesi Reference Ferrari, Donati and Gnesi2017; Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019; Kaiser et al. Reference Kaiser, Schlechtweg, Papay and Schulte im Walde2020a). This method allows semantic representations for target words across different periods to be learned from the same space, thereby reducing the need for alignment.

Temporal referencing (TR). Similar to WI, TR assigns temporal labels to target words only when they function as true target words, rather than as context words (Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019). By differentiating true target words with temporal labels, TR enables training within a unified vector space using the entire corpus. This method assumes that the semantics of context words remain relatively stable over time.

Dynamic embeddings. Word embeddings jointly learned with time-stamped data, where temporal information is treated as a variable integrated directly into model architectures, are referred to as dynamic embeddings in existing studies (Tang, Reference Tang2018; Kutuzov et al. Reference Kutuzov, Øvrelid, Szymanski and Velldal2018; Tahmasebi et al. Reference Tahmasebi, Borin and Jatowt2021). Frermann and Lapata (Reference Frermann and Lapata2016) proposed a dynamic Bayesian model to capture sense evolution through changes in associated topics, where dynamic change is modeled by placing individual logistic normal priors (Lafferty and Blei Reference Lafferty and Blei2005) on sense distributions in a controlled setting of change speed. Building on this approach, Bamler et al. (Reference Bamler and Mandt2017) extended a Bayesian version of the skip-gram model (Barkan, Reference Barkan2017) by incorporating a latent time series to share contextual information across periods. Similarly, Yao et al. (Reference Yao, Sun, Ding, Rao and Xiong2018) proposed learning dynamic embeddings through block coordinate descent (Yu et al. Reference Yu, Hsieh, Si and Dhillon2012) with PPMI models. In another variation, Rudolph and Blei (Reference Rudolph and Blei2018) extended exponential family embeddings (Rudolph et al. Reference Rudolph, Ruiz, Mandt and Blei2016) by incorporating a latent variable through a Gaussian random walk, while Rosenfeld and Erk (Reference Rosenfeld and Erk2018) modified the traditional SGNS loss function to produce embeddings that evolve smoothly over time.

As a brief note, post-alignment methods have been criticized for potentially introducing noise (Bamler et al. Reference Bamler and Mandt2017; Yao et al. Reference Yao, Sun, Ding, Rao and Xiong2018; Rudolph and Blei Reference Rudolph and Blei2018; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019), yet they remain highly effective, as demonstrated by the top-performing models using OP in earlier shared tasks (Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020; Kaiser, Schlechtweg, and Schulte im Walde Reference Kaiser, Schlechtweg and Schulte im Walde2020b). While alternative techniques theoretically avoid this noise risk, they introduce their own challenges. For instance, VI heavily depends on the initialization phase to produce high-quality vectors and is sensitive to the training order when handling two subcorpora $C_1$ and $C_2$ that differ significantly in size. Both VI and WI perform better with lower dimensionality (e.g., below the common range of 200–300) (Kaiser et al. Reference Kaiser, Schlechtweg, Papay and Schulte im Walde2020a). In contrast, OP demonstrates greater consistency and robustness across different dimensionalities, particularly at higher dimensions (Kaiser et al. Reference Kaiser, Schlechtweg, Papay and Schulte im Walde2020a). TR, which assigns temporal labels to words, can significantly inflate vocabulary size, leading to increased processing times. Additionally, TR relies on the assumption that context remains stable over time, potentially introducing bias into the final results (Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019). Dynamic embeddings, while theoretically appealing, tend to be computationally expensive and complex to implement. For a more detailed exploration of these models, we recommend referring to previous surveys by Tang (Reference Tang2018) and Tahmasebi et al. (Reference Tahmasebi, Borin and Jatowt2021).

4.3 The third generation: A revolutionary era with pretrained models

This generation of LSC models builds on contextualized language models (Hu et al. Reference Hu, Li and Liang2019; Giulianelli et al. Reference Giulianelli, del Tredici and Fernández2020; Cassotti et al. Reference Cassotti, Siciliani, de Gemmis, Semeraro and Basile2023; Periti and Montanelli Reference Periti and Montanelli2024), which have broadly transformed the NLP landscape (Radford and Narasimhan Reference Radford and Narasimhan2018; Devlin et al. Reference Devlin, Chang, Lee and Toutanova2019; Radford et al. Reference Radford, Wu, Child, Luan, Amodei and Sutskever2019; Mickus et al. Reference Mickus, Paperno, Constant and van Deemter2020). These models introduce fundamental changes to LSC modeling: the pretrained nature enables zero-shot detection of semantic shifts, while the attention mechanism allows multiple context-dependent embeddings for a target word (e.g., capturing polysemy) and alleviates the temporal aggregation requirements of previous approaches (Kim et al. Reference Kim, Chiu, Hanaki, Hegde and Petrov2014; Kulkarni et al. Reference Kulkarni, Al-Rfou, Perozzi and Skiena2015; Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Rudolph and Blei Reference Rudolph and Blei2018; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019). Together, these advances allow LSC models to escape the constraints of data sparsity and alignment noise that plagued earlier generations, while enabling finer-grained semantic representations (e.g., sense- or usage-level embeddings).

However, these changes also introduce new challenges for LSC models. First, pretrained language models may be biased toward modern data, raising concerns about diachronic fine-tuning (e.g., Kutuzov and Giulianelli Reference Kutuzov and Giulianelli2020; Martinc et al. Reference Martinc, Montariol, Zosa and Pivovarova2020b; Qiu and Xu Reference Qiu and Xu2022; Periti and Montanelli Reference Periti and Montanelli2024). Second, the large-scale token embeddings generated by these models require scalable strategies for practical application. This subsection reviews both; for more comprehensive technical discussions, we refer readers to Periti and Montanelli (Reference Periti and Montanelli2024).

4.3.3 Measurements

This generation introduces a richer set of metrics. For form-based representations, CD remains effective; for usage-based representations, dedicated metrics are required to capture distributional change.

Cosine distance between prototypical embeddings (PRT). PRT measures the CD between $\bar {E}_{t_1}$ and $\bar {E}_{t_2}$ (see Equation (15)), representing the semantic drift between two prototypical embeddings (e.g., the dominant usage) of a word w across periods, $t_1$ and $t_2$ (Beck, Reference Beck2020; Martinc, Kralj Novak, and Pollak Reference Martinc, Kralj Novak and Pollak2020a; Kutuzov and Giulianelli Reference Kutuzov and Giulianelli2020; Martinc et al. Reference Martinc, Montariol, Zosa and Pivovarova2020c; Kutuzov, Velldal, and Øvrelid Reference Kutuzov, Velldal and Øvrelid2022b).

(15) \begin{equation} \text{PRT}(E_{t_1}, E_{t_2}) = 1 - \cos (\bar {E}_{t_1}, \bar {E}_{t_2}) \end{equation}

Average pairwise distance (APD). APD calculates the average distance between all pairs of token embeddings across the two sets $E_{t_1}$ and $E_{t_2}$ (Giulianelli et al. Reference Giulianelli, del Tredici and Fernández2020; Kutuzov and Giulianelli Reference Kutuzov and Giulianelli2020). As defined in Equation (16), the distance metric $d$ can be cosine distance (Periti and Tahmasebi Reference Periti and Tahmasebi2024a), Euclidean distance (Giulianelli et al. Reference Giulianelli, del Tredici and Fernández2020; Pömsl and Lyapin Reference Pömsl and Lyapin2020), or L1-distance (the Manhattan distance) (Rachinskiy and Arefyev Reference Rachinskiy and Arefyev2021a). APD captures the degree of polysemy change over time (Periti and Tahmasebi Reference Periti and Tahmasebi2024a), with higher values indicating more pronounced semantic change and lower values suggesting more stable word usage.

(16) \begin{equation} \text{APD} (E_{t_1}, E_{t_2}) = \frac {1}{|E_{t_1}| \cdot |E_{t_2}|} \cdot \sum _{e_{t_1} \in E_{t_1}, e_{t_2} \in E_{t_2}} d(e_{t_1}, e_{t_2}), \end{equation}

Jensen-Shannon distance (JSD). As noted in Section 3.2.2, JSD quantifies the difference between probabilistic usage distributions across two periods (e.g., $p_{t_1}$ and $p_{t_2}$ ) (Kanjirangat et al. Reference Kanjirangat, Mitrovic, Antonucci and Rinaldi2020; Karnysheva and Schwarz Reference Karnysheva and Schwarz2020; Rother, Haider, and Eger Reference Rother, Haider and Eger2020; Schlechtweg, Reference Schlechtweg2023).Footnote ^j To account for potential noise in these distributions, Rodina et al. (Reference Rodina, Trofimova, Kutuzov and Artemova2020) adopted the maximum square of these two distributions to detect stronger semantic shifts. Similarly, Giulianelli et al. (Reference Giulianelli, del Tredici and Fernández2020) adapted JSD using Entropy, as in Equation (17), where H represents the Boltzmann–Gibbs–Shannon entropy. This adaptation better captures changes in cluster distributions over time. In all these approaches, higher JSD values indicate significant semantic shifts, whereas lower JSD values suggest more stable usage patterns.

(17) \begin{equation} \text{JSD}(E_{t_1}, E_{t_2}) = H\left (\frac {1}{2}(p_{t_1} + p_{t_2})\right ) - \frac {1}{2}(H(p_{t_1}) + H(p_{t_2})) \end{equation}

Entropy difference (ED). ED measures the difference in normalized entropy between two usage distributions $U_{t_1}$ and $U_{t_2}$ . As defined in Equation (18), the entropy for each period is calculated, with $K$ representing the number of usage embeddings in each distribution. The overall entropy difference is then computed (Equation (19)), with a positive ED value suggesting an expansion of word meanings and a negative ED indicating meaning narrowing (Tang, Qu, and Chen Reference Tang, Qu and Chen2013, Reference Tang, Qu and Chen2016; Giulianelli, Reference Giulianelli2019).

(18) \begin{equation} \eta (U) = \log _{K} \left ( \prod _{k=1}^{K} U[k]^{-U[k]} \right ) \end{equation}

(19) \begin{equation} \Delta S = \eta (U_{t_2}) - \eta (U_{t_1}) \end{equation}

Novelty score (NS). NS quantifies changes in usage frequency, emphasizing the most significant proportional change. As defined in Equation (20) (Hu et al. Reference Hu, Li and Liang2019), NS for a usage, $ N(u)$ , is the ratio of its proportion in earlier ( $ P_{t_1}(u)$ ) and later ( $ P_{t_2}(u)$ ) time bins, with a small constant $ \alpha = 0.01$ to prevent division errors. The final score for a word, $ N(w)$ , is the maximum ratio among all usages to reflect semantic change (Frermann and Lapata, Reference Frermann and Lapata2016; Hu et al. Reference Hu, Li and Liang2019).

(20) \begin{equation} \begin{split} N(u) = \frac {P_{t_1}(u) + \alpha }{P_{t_2}(u) + \alpha }, \\[4pt] N(w) = \max \{N(u_1), \ldots , N(u_n)\} \end{split} \end{equation}

Overall, Periti and Tahmasebi (Reference Periti and Tahmasebi2024a) found that form-based approaches consistently outperformed usage-based methods across benchmarks and languages, with PRT and APD showing the strongest results.

4.4 The fourth generation: A new era approaching interpretability

While the third generation of LSC models advanced detection capabilities, one challenge remained unaddressed: how to explicitly interpret the detected semantic changes. The fourth generation addresses this challenge primarily through definitional embeddings. Specifically, these are sentence embeddings derived from in-context word definitions automatically generated by generative models (Giulianelli et al. Reference Giulianelli, Luden, Fernandez and Kutuzov2023; Kutuzov et al. Reference Kutuzov, Fedorova, Schlechtweg and Arefyev2024; Periti, Alfter, and Tahmasebi Reference Periti, Alfter and Tahmasebi2024a). This approach is still in its early stages, with only a few studies published at the time of this survey. In the following subsections, we review this generation following the same modeling pipeline as previous sections.

Notably, several studies have also leveraged in-context lexical replacement or substitution for target words to detect semantic change using contextualized models (Arefyev and Zhikov Reference Arefyev and Zhikov2020; Kudisov and Arefyev Reference Kudisov and Arefyev2022; Card, Reference Card2023; Periti et al. Reference Periti, Cassotti, Dubossarsky and Tahmasebi2024b). This approach can be viewed as an intermediate step toward the definition-based paradigm, as substitutable words provide natural interpretability.Footnote ^k For a comprehensive review of these substitution-based techniques, we refer readers to Periti and Montanelli (Reference Periti and Montanelli2024).

4.5 Summary

This section has reviewed LSC models across four generations, each defined by a dominant base model with corresponding adaptations in temporal aggregation and measurements. Each generation mirrors the broader evolution of NLP over the past two decades (Turney and Pantel Reference Turney and Pantel2010; Levy et al. Reference Levy, Goldberg and Dagan2015; Devlin et al. Reference Devlin, Chang, Lee and Toutanova2019; Lenci and Sahlgren Reference Lenci and Sahlgren2023), with each generation addressing specific limitations of its predecessors while often introducing new challenges of its own.

Early count-based DSMs demonstrated the potential for detecting semantic change in vector spaces (Sagi, Kaufmann, and Clark 2009, Reference Sagi, Kaufmann and Clark2011; Wijaya and Yeniterzi Reference Wijaya and Yeniterzi2011), but suffered from data sparsity and vector quality limitations. The second generation, primarily established by prediction-based DSMs (Mikolov et al. Reference Mikolov, Chen, Corrado and Dean2013a, Reference Mikolov, Sutskever, Chen, Corrado and Dean2013b), effectively addressed these sparsity issues and demonstrated substantial performance gains in LSC tasks (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c; Shoemark et al. Reference Shoemark, Liza, Nguyen, Hale and McGillivray2019; Schlechtweg et al. Reference Schlechtweg, McGillivray, Hengchen, Dubossarsky and Tahmasebi2020). However, these models still faced persistent challenges, particularly the noise generated during the alignment procedures necessary for temporal comparison (Bamler et al. Reference Bamler and Mandt2017; Yao et al. Reference Yao, Sun, Ding, Rao and Xiong2018; Rudolph and Blei Reference Rudolph and Blei2018; Dubossarsky et al. Reference Dubossarsky, Hengchen, Tahmasebi and Schlechtweg2019).

Table 3.

Representative model performance across LSC generations on the English SemEval-2020 / DWUG EN benchmark (Subtask 2, graded change ranking, 37 target words).

Table 3 Long description

A performance comparison table across LSC model generations on the English SemEval-2020/DWUG EN benchmark, Subtask 2 (graded change ranking, 37 target words). Notes: Gen 1 uses raw count vectors with column intersection (CI+CD); PPMI was not submitted as a standalone system in SemEval-2020. XL-LEXEME was evaluated post-hoc (Cassotti et al. 2023). Gen 1, Gen 2, and Gen 3 (BERT, no WiC) were evaluated on the full SemEval corpora; BERT+APD was evaluated on annotated samples only. Due to differences in evaluation settings across generations, cross-generation comparisons should be interpreted as indicative rather than strictly controlled.

Notes: The Gen 1 entry uses raw count vectors with column intersection (CI + CD); PPMI was not submitted as a standalone system in SemEval-2020. XL-LEXEME was not part of the original competition; the score is from a post-hoc evaluation on the same benchmark (Cassotti et al. Reference Cassotti, Siciliani, de Gemmis, Semeraro and Basile2023). Gen 1, Gen 2, and Gen 3 (BERT, no WiC) were evaluated on the full SemEval corpora; BERT + APD was evaluated on annotated samples only, which is methodologically more appropriate since the gold graded scores are derived from those samples. Due to differences in evaluation settings across generations, cross-generation comparisons should be interpreted as indicative rather than strictly controlled.

The third generation marked a major advance in LSC detection. Pretrained architectures enabled zero-shot capabilities that naturally adapt to time-specific semantic representations, thereby resolving the temporal alignment challenge that limited the first two generations. Additionally, the contextualized nature of these models introduced token-level semantic representations, offering the potential to detect semantic change at the usage- or sense- level. However, this generation also introduced its own challenges, particularly in efficiently aggregating large volumes of representations in a scalable manner and determining the most suitable measurements for analysis, as seen in its early stages (Hu et al. Reference Hu, Li and Liang2019; Cuba Gyllensten et al. Reference Cuba Gyllensten, Gogoulou, Ekgren and Sahlgren2020; Kanjirangat et al. Reference Kanjirangat, Mitrovic, Antonucci and Rinaldi2020; Karnysheva and Schwarz Reference Karnysheva and Schwarz2020). Notably, the integration of word-in-context tasks has brought substantial performance breakthroughs within this generation (Rachinskiy and Arefyev Reference Rachinskiy and Arefyev2022; Homskiy and Arefyev Reference Homskiy and Arefyev2022; Cassotti et al. Reference Cassotti, Siciliani, de Gemmis, Semeraro and Basile2023; Yadav and Schlechtweg Reference Yadav and Schlechtweg2025), establishing new state-of-the-art results across multiple languages (Periti and Tahmasebi Reference Periti and Tahmasebi2024a).

Table 3 presents representative results on the English SemEval-2020 / DWUG EN benchmark, the most widely shared evaluation setting across generations, to ground this generational account in empirical terms.Footnote ^l Gen 1 count-based models (CI + CD) score near chance ( $\rho = .022$ ), while Gen 2 SGNS + OP raises performance to $\rho = .422$ . As additional within-generation evidence from a separate benchmark, Schlechtweg et al. (Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019) show that SGNS ( $\rho = .866$ ) substantially outperforms PPMI ( $\rho = .670$ ) on the German DURel benchmark, a difference of nearly $.200$ points, further confirming the representational advantage of prediction-based over count-based methods. Moving to plain contextualised models in Gen 3 brings only a marginal improvement: the best BERT system scores $\rho = .436$ against $.422$ for the best SGNS system, a gain of just $.014$ when both are evaluated on the full SemEval corpora. However, when BERT + APD is evaluated on the annotated samples only (the more appropriate setting, since the gold graded scores are derived from those samples), it achieves $\rho = .571$ (Laicher et al. Reference Laicher, Kurtyigit, Schlechtweg, Kuhn and Schulte im Walde2021), a more substantial improvement of $.149$ over the best SGNS baseline, though this difference partly reflects the evaluation setting rather than the model alone. The decisive advance comes with WiC-integrated models: Cassotti et al. (Reference Cassotti, Siciliani, de Gemmis, Semeraro and Basile2023) reach $\rho = .757$ on English DWUG EN, and Homskiy and Arefyev (Reference Homskiy and Arefyev2022) achieve $\rho = .791$ on Russian, surpassing the best type-based system (UWB, $\rho = .417$ ) and the best non-fine-tuned contextualised system (aryzhova, ELMo+APD, $\rho = .457$ ) on the RuShiftEval leaderboard (Kutuzov and Pivovarova Reference Kutuzov and Pivovarova2021a) by $.374$ and $.334$ , respectively.

Despite their advancements, these three generations of LSC models still lack interpretability: they measure the degree of semantic change without explaining how meanings have shifted and why such changes have occurred. This limitation has spurred the emergence of the fourth generation, which seeks to enhance interpretability by integrating generative models and definitional embeddings into the LSC framework (Giulianelli et al. Reference Giulianelli, Luden, Fernandez and Kutuzov2023; Kutuzov et al. Reference Kutuzov, Fedorova, Schlechtweg and Arefyev2024). While this represents a significant step toward explainable semantic change detection, these approaches have not yet demonstrated performance advantages over the word-in-context models from the previous generation and may also face scalability challenges.

Table 4.

A summary of LSC model generations. OP = Orthogonal Procrustes, VI = Vector Initialization, WI = Word Injection, TR = Temporal Referencing, CD = Cosine Distance, LND = Local Neighborhood Distance, PRT = Prototypical Embeddings Distance, APD = Average Pairwise Distance, JSD = Jensen–Shannon Distance, ED = Entropy Difference, NS = Novelty Score

Table 4 Long description

A summary table of LSC model generations with abbreviations defined as follows: OP = Orthogonal Procrustes, VI = Vector Initialization, WI = Word Injection, TR = Temporal Referencing, CD = Cosine Distance, LND = Local Neighborhood Distance, PRT = Prototypical Embeddings Distance, APD = Average Pairwise Distance, JSD = Jensen-Shannon Distance, ED = Entropy Difference, NS = Novelty Score.

Beyond architectural choices, model selection in practice is also shaped by corpus-level confounds that interact differently with each generation. For frequency drift, Dubossarsky et al. (Reference Dubossarsky, Weinshall and Grossman2017) show that frequency confounds are mathematically inevitable in raw count representations, substantially reduced by PPMI, but reintroduced by SVD dimensionality reduction; equalizing corpus size across time bins is therefore widely recommended, and for contextualised models, sampling a fixed number of usages per word per period serves as the analogous mitigation (Schlechtweg et al. Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019). For genre and register, Hamilton et al. (Reference Hamilton, Leskovec and Jurafsky2016c) identify PPMI as especially “prone to false discoveries from global genre/discourse shifts” and contrast the genre-balanced COHA corpus with the more artifact-prone Google N-Gram data; Schlechtweg et al. (Reference Schlechtweg, Hätty, del Tredici and Schulte im Walde2019) document how register-specific noise (4.6% numerals in the cooking corpus vs. 1.2% in the diachronic corpus) requires content-word-only filtering for noisier subcorpora. For research targeting general semantic change, genre balance across periods is preferable; for domain-specific research, deliberate genre restriction is appropriate. For OCR quality, count-based and SGNS models are more sensitive to type-level noise from corrupted character sequences, while contextualised models with subword tokenisation offer partial robustness; systematic empirical comparison across model families remains an open methodological question. The appropriate model choice therefore depends primarily on the research question and available materials: when corpus quality is limited by OCR noise or severe genre imbalance, contextualised models with subword tokenisation are preferable; when large, well-balanced corpora are available, prediction-based static embeddings remain competitive and computationally efficient.Footnote ^m

5.1 Assisting with historical linguistic and lexicographic analysis

Visualizing the history of a changed linguistic unit. LSC models assist historical linguistic analysis by revealing how word meanings evolve over time. For instance, the word tenure developed new associations with faculty positions, especially during the 1990s and 2000s (Giulianelli et al. Reference Giulianelli, del Tredici and Fernández2020). Similarly, the term pathetic has shifted from its older association with passionate toward negative connotations of weakness (Hamilton et al. Reference Hamilton, Clark, Leskovec and Jurafsky2016a). The word abortion in Dutch newspapers has also undergone semantic drift, from associations with diseases like tuberculosis and the generic term pregnancy to more specific words such as contraception, sterilization, and legalization between the 1950s and 1990s (Wevers and Koolen Reference Wevers and Koolen2020). Additionally, de-adjectives such as awfully and insanely have experienced semantic bleaching, now functioning as intensifiers in phrases such as awfully nice and insanely delicious (Luo et al. Reference Luo, Jurafsky and Levin2019). Beyond individual words, LSC models can also capture changes at the level of linguistic constructions. Examples include how to death evolved into an intensifier in expressions such as pleased to death (Fonteyn et al. Reference Fonteyn, Manjavacas Arevalo, Karsdorp, McGillivray, Nerghens and Wevers2021), and how the construction Prep $NP_{IL}$ of $NP_{LM}$ , as instantiated in expressions like in the midst of the river, moved beyond conveying spatial coincidence to indicate an internal locative relation (Desagulier, Reference Desagulier2022).

Statistical regularities on semantic change. Beyond individual case-based analyses, LSC models are also instrumental in modeling semantic change more broadly. A key focus of such studies has been on validating competing hypotheses about how words evolve. For instance, Xu and Kemp (Reference Xu and Kemp2015) investigated two seemingly contradictory laws: the law of differentiation (where near-synonyms diverge over time) and the law of parallel change (where related words tend to evolve in similar ways). Their findings provided statistical support for the law of parallel change, while more recent research by Lietard et al. (Reference Lietard, Keller and Denis2023) has offered stronger evidence in favor of differentiation, suggesting that these patterns may depend on specific words or contexts. Examining how word properties influence semantic change rates represents another active line of research. For example, Hamilton et al. (Reference Hamilton, Leskovec and Jurafsky2016c) found that high-frequency words tend to evolve more slowly, whereas polysemous words change more rapidly, with contextual diversity driving these changes even when controlling for frequency. Dubossarsky et al. (Reference Dubossarsky, Weinshall and Grossman2016) observed that prototypicality (how representative a word is of its category) is inversely correlated with semantic change, with verbs changing more frequently than nouns, and nouns more than adjectives. This finding was further supported by Xu et al. (Reference Xu, Stellar and Xu2021), who demonstrated that highly prototypical emotion words are less susceptible to change. Additionally, Fugikawa et al. (Reference Fugikawa, Hayman, Liu, Yu, Brochhagen and Xu2023) found that the concreteness of a word strongly predicts the direction of its semantic change, with findings supported cross-linguistically. Dubossarsky et al. (Reference Dubossarsky, Weinshall and Grossman2017) argued, however, that the alleged statistical laws appear diminished or absent in a control condition, attributing this partly to frequency bias encoded in semantic representations.

Lexicographic analysis with the DURel tool. The DURel framework has proven useful for lexicographic analysis, particularly for identifying unrecorded word senses and informing dictionary updates (Schlechtweg et al. Reference Schlechtweg, Virk, Sander, Sköldberg, Linke, Zhang, Tahmasebi, Kuhn and im Walde2023). By constructing word usage graphs based on semantic proximity between usage pairs, DURel can highlight word senses that may have gone unrecorded (Sander et al. Reference Sander, Hengchen, Zhao, Ma, Sköldberg, Virk and Schlechtweg2024; Sköldberg et al. Reference Sköldberg, Virk, Sander, Hengchen and Schlechtweg2024). For example, Sköldberg et al. (Reference Sköldberg, Virk, Sander, Hengchen and Schlechtweg2024) used WUGs to examine 281 Swedish words recorded with a single dictionary sense. They flagged 66 headwords showing multiple clusters under LSC analysis, presenting them as candidates for expert evaluation and sense revision. Similarly, Sander et al. (Reference Sander, Hengchen, Zhao, Ma, Sköldberg, Virk and Schlechtweg2024) analyzed headwords with a single registered sense in English and German, further discussing non-recorded senses recommended by LSC models. Both studies were conducted on the DURel platform (Schlechtweg et al. Reference Schlechtweg, Virk, Sander, Sköldberg, Linke, Zhang, Tahmasebi, Kuhn and im Walde2023).Footnote ⁿ

5.2 Semantic variation across comparison settings

Community-level semantic variations. By constructing and comparing community-dependent semantic representations, researchers can detect language use differences across communities (Nguyen et al. Reference Nguyen, Doğruöz, Rosé and De Jong2016; Yang and Eisenstein Reference Yang and Eisenstein2017; Raquel, Reference Raquel2019; Lucy and Bamman Reference Lucy and Bamman2021). For example, Del Tredici and Fernández (Reference Del Tredici and Fernández2017) computationally investigated language use across online communities of programming and football using Reddit data and highlighted box (in football) and scope (in programming community) as the most prominent semantic shifts relative to their usage in general-purpose corpora. A further group of studies targets political communities using online data, reviewed in Section 5.4.

Regional variation and contact-induced use. These methods have also been applied to regional differences and contact-induced semantic shifts, using vector spaces constructed from region-specific corpora. Kulkarni et al. (Reference Kulkarni, Perozzi and Skiena2021) demonstrated that theater in the UK typically refers to performing arts, whereas in the US, it is more closely associated with sciences, literature, and anthropology. Building on alignment-based training workflows of LSC models, Miletic et al. (Reference Miletic, Przewozny-Desriaux and Tanguy2020) identified words exhibiting contact-induced meaning shifts such as exposition, terrace, and definitely in Quebec English, particularly in Montreal, Canada, where their dominant usages closely align with French cognates. In subsequent work, they applied contextualized representations (Laicher et al. Reference Laicher, Kurtyigit, Schlechtweg, Kuhn and Schulte im Walde2021) to analyze these regional usages at the sense level. For instance, exposition in Quebec English is predominantly associated with art exhibitions and exposition halls (Miletic et al. Reference Miletic, Przewozny-Desriaux and Tanguy2021). To validate these computational findings, they further quantitatively assessed these detected shifts through human acceptability ratings in contact-related social contexts (Miletić et al. Reference Miletić, Przewozny-Desriaux and Tanguy2023).

Gender and group-level language use differences. This framework also extends to gender and other group-level differences. For example, Gonen et al. (Reference Gonen, Jawahar, Seddah and Goldberg2020) employed a gender-split experiment to analyze word usage differences, finding that clutch was linked to earrings and dress in female contexts, whereas in male contexts, it was associated with dominant and layups. Similarly, Nagata et al. (Reference Nagata, Takamura, Otani and Kawasaki2023) compared word contexts across native and non-native English speakers using coverage difference, a metric that quantifies the ratio of mean word-vector norms across two corpora. Their results indicated that the dominant interpretation of near as an adverbial modifier (e.g., it has near impossible) was characteristic of the native speaker corpus, whereas such usage was absent in the non-native speaker corpus.

Language use in disciplinary discourse. This comparative framework can be extended to disciplinary discourse, where language use in specific fields is analyzed over time to identify emerging trends (Yan and Zhu Reference Yan and Zhu2018; Peterson and Liu Reference Peterson and Liu2021; Nicholson et al. Reference Nicholson, Alquaddoomi, Rubinetti and Greene2023; Baes et al. Reference Baes, Vylomova, Zyphur and Haslam2023; Deng et al. Reference Deng, Van der Meer, Tzovara, Schmidt, Bassetti and Denecke2023). For example, Yan and Zhu (Reference Yan and Zhu2018) used a word2vec model to track word- and topic-level semantic changes in biomedical literature. Their results showed that the meanings of selected terms stabilized in the 2000s compared to earlier decades, while the distance between these words decreased over time. Similarly, Baes et al. (Reference Baes, Vylomova, Zyphur and Haslam2023) examined the term trauma in psychology and found that its meaning has expanded over the past 40 years to include less severe contexts, reflecting broader shifts in disciplinary usage. In clinical contexts, Deng et al. (Reference Deng, Van der Meer, Tzovara, Schmidt, Bassetti and Denecke2023) investigated semantic drifts in terms like insomnie across various types of sleep disorders, using the Bern Sleep Database (Aellen et al. Reference Aellen, van der Meer, Dietmann, Schmidt, Bassetti and Tzovara2022). This work illustrates how semantic analysis can illuminate patient history through symptomatic language. To identify emerging trends in computer science and biomedical research, Dridi et al. (Reference Dridi, Gaber, Azad and Bhogal2019) developed Leap2Trend, a tool that learns temporal word embeddings for keywords extracted from time-stamped papers. Leap2Trend computes similarity matrices across periods and ranks them to track changes in keyword relationships, identifying emerging trends based on marked increases in similarity scores. Beyond single disciplines, McGillivray et al. (Reference McGillivray, Jenset, Salama and Schut2022) demonstrated increasing cross-disciplinary proximity, as evidenced by rising cosine similarity between discipline-specific embeddings. At the same time, they observed greater specialization within disciplines, reflected by a decreasing number of neighboring disciplines in the embedding space.

5.3 Language change and language acquisition

Child-directed language use. LSC models have been applied to examine how caregivers adapt their language across the developmental stages of children. Jiang et al. (Reference Jiang, Frank, Kulkarni and Fourtassi2022) examined how caregivers modify word contexts when addressing younger versus older children, using the LSC toolkit of Hamilton et al. (Reference Hamilton, Leskovec and Jurafsky2016c). Their results showed that verbs such as shoot, mash, skip, and pee, as well as nouns like lamp and turkey, were used with a single meaning in earlier developmental stages but later exhibited more varied contextual uses. In contrast, words related to temporal concepts, pronouns, and functional words maintained relatively stable usage patterns across developmental stages. In a related study, Prystawski et al. (Reference Prystawski, Grant, Nematzadeh, Lee, Stevenson and Xu2022) applied LSC models (Hamilton, Leskovec, and Jurafsky Reference Hamilton, Leskovec and Jurafsky2016c) to examine gender associations in the linguistic environments of children. Their findings demonstrated that word usage by both caregivers and children correlated with gender associations, though these correlations diminished from the 1970s to the 1990s, suggesting that early linguistic input and broader social changes jointly shape gender associations in childhood.

Language use across lifespan. LSC models have also been used to examine how language usage across the lifespan relates to broader patterns of semantic change. For instance, Cassani et al. (Reference Cassani, Bianchi and Marelli2021) examined the evolution of semantic coherence in English and found that words learned earlier in life tend to maintain higher semantic coherence over time. Their time-aligned computational models indicated that words related to entities and actions exhibit greater semantic stability. They linked semantic coherence to age of acquisition, consistent with Hills et al. (Reference Hills, Maouene, Riordan and Smith2010) and Braginsky et al. (Reference Braginsky, Yurovsky, Marchman and Frank2019). Building on this age-of-acquisition framework, Li and Siew (Reference Li and Siew2022); Li et al. (Reference Li, Breithaupt, Hills, Lin, Chen, Siew and Hertwig2024) demonstrated that words acquired later in life are more likely to undergo semantic change, reinforcing the link between cognitive development and semantic change. Extending beyond individual languages to cross-linguistic patterns, Brochhagen et al. (Reference Brochhagen, Boleda, Gualdoni and Xu2023) explored how human lexical creativity is shaped by both individual language development (ontogeny) and the historical evolution of languages (phylogeny). Their findings, corroborated in part through diachronic word embedding analysis, show how cognitive processes shape semantic change across both developmental and historical timescales.

5.4 Language use in political contexts

Party positions: Left and right dimensions. In political contexts, the LSC analytical framework can be adapted to examine how word usage varies across parties in a synchronic setting or how party positions shift over time (Azarbonyad et al. Reference Azarbonyad, Dehghani, Beelen, Arkut, Marx and Kamps2017; Spinde et al. Reference Spinde, Rudnitckaia, Hamborg and Gipp2021; Martinc et al. Reference Martinc, Perger, Pelicon, Ulčar, Vezovnik and Pollak2021; Karjus and Cuskley Reference Karjus and Cuskley2024). Azarbonyad et al. (Reference Azarbonyad, Dehghani, Beelen, Arkut, Marx and Kamps2017) trained party-specific vector spaces using speeches from the Conservative and Labour parties to investigate differences in word usage. For example, while the Conservatives associated democracy predominantly with unity, Labour emphasized its connection to freedom and social justice, despite the term retaining a broadly stable meaning over time. Following a similar approach, Spinde et al. (Reference Spinde, Rudnitckaia, Hamborg and Gipp2021) explored biased terms between left- and right-wing media, employing independently trained vector spaces for each party. Similarly, Martinc et al. (Reference Martinc, Perger, Pelicon, Ulčar, Vezovnik and Pollak2021) examined how quality newspapers and right-wing party-affiliated media in Slovene reported differently on LGBTIQ+ topics (e.g., homosexual marriage). Karjus and Cuskley (Reference Karjus and Cuskley2024) constructed left- and right-leaning vector spaces from categorized tweets to study linguistic divergence across the partisan divide in the United States. They observed notable semantic divergence in terms such as bs (short for bullsh^*t), left, lit (interpreted as lights or colloquially as cool, awesome), and wake up (used either literally or figuratively to mean pay attention).

Several additional studies, while not directly derived from LSC models, apply similar workflows to train party-dependent vector spaces or party embeddings. For instance, Glavaš et al. (Reference Glavaš, Nanni and Ponzetto2017); Nanni et al. (Reference Nanni, Glavas, Rehbein, Ponzetto and Stuckenschmidt2021) aligned static word embeddings across languages to determine party positions on the left–right dimension using speeches from the European Parliament. Similarly, Rheault and Cochrane (Reference Rheault and Cochrane2020) used a regression model incorporating time periods and parties to generate party embeddings, enabling the tracking of ideological shifts over time. Ceron et al. (Reference Ceron, Blokker and Padó2022) fine-tuned Sentence-BERT models using a triplet objective function to maximize similarity among within-party sentences relative to between-party sentences. In a subsequent study, Ceron et al. (Reference Ceron, Nikolaev and Padó2023) applied this model to analyze party similarities and differences across various policy domains. Their findings revealed that German parties generally agree on education and technology policies, advocating for further investment and expansion. However, their positions on military and peace and immigration and multiculturalism align more distinctly along the left–right political spectrum, with right-leaning parties favoring more militaristic approaches and restrictive immigration policies.

Political concepts: Equality, ideology, and national image. Diachronic word embeddings can reveal how political concepts have evolved over time. For example, Rodman (Reference Rodman2020) investigated how the concept of equality evolved in relation to gender, race, and international relations by analyzing newspaper articles containing equality in headlines from 1855 to 2016. Their results indicated that the prominence of social equality inversely tracked racial progress over time. In a related study, Walter et al. (Reference Walter, Kirschner, Eger, Glavaš, Lauscher and Ponzetto2021) traced antisemitic and anticommunist biases in German political discourse over time. Hengchen et al. (Reference Hengchen, Ros and Marjanen2019) examined the historical usage of nation, national, and nationalism through diachronic word embeddings, demonstrating a broadening of the vocabulary associated with national identity and discourse.

5.5 Stereotypes, cultural dynamics, and real-world events

Stereotypes and bias. Diachronic word embeddings have proven effective in capturing changes in bias and stereotypes over time, offering insights into how societal attitudes have evolved. For instance, Garg et al. (Reference Garg, Schiebinger, Jurafsky and Zou2018) used diachronic word embeddings to examine gender stereotypes, ethnic biases, and personality traits over time. By calculating the average embedding distance between target words (e.g., she and female) and various occupations (e.g., teacher and lawyer), they quantified changes in gender bias over several decades, finding persistent biases against women from 1910 to 1990. Their analysis also showed that academic positions were consistently ranked among the top Asian-biased occupations. Similarly, Jones et al. (Reference Jones, Amin, Kim and Skiena2020) used diachronic word embeddings to quantify male gender bias across different domains, finding that terms related to career and science exhibited a positive male gender bias, while terms related to family and arts showed a negative male bias. In another study, Khadilkar et al. (Reference Khadilkar, KhudaBukhsh and Mitchell2022) explored gender and social biases in Bollywood film subtitles over the past 70 years, tracking how such representations have evolved. Charlesworth et al. (Reference Charlesworth, Sanjeev, Hatzenbuehler and Banaji2023) broadened this line of research by examining stereotypes across 72 groups over time, highlighting how changes vary by group type and frequency of mentions, with sociodemographic groups exhibiting more pronounced shifts.

Semantic representations in cultural dynamics. The same methodology can illuminate cultural dimensions of society. For example, Kozlowski et al. (Reference Kozlowski, Taddy and Evans2019) used word embeddings to explore social class distinctions, specifically examining the poor and the rich over the past 200 years. They found that while markers of class shifted considerably during the economically transformative 20th century, the cultural dimensions of class remained stable, with education becoming strongly associated with affluence. Similarly, Leach et al. (Reference Leach, Kitchin and Sutton2023) studied moral concerns during the 19th and 20th centuries by examining associations among words denoting these concerns, finding a growing concern for people, animals, and the environment over time. From a cross-lingual perspective, Thompson et al. (Reference Thompson, Roberts and Lupyan2020) computationally examined cultural influences on semantic alignment across 1,010 words in 21 semantic domains among 41 languages. Their findings revealed that words related to number, quantity, and kinship show strong semantic alignment across languages, while terms associated with natural kinds, common actions, and artifacts demonstrate better alignment when languages share greater geographical proximity, history, and culture.

Personality traits in narratives. Temporal word representations of implicit personality structures in historical narratives can reveal how discourse evolves over time. For instance, Du et al. (Reference Du, Karl, Fetvadjiev, Luczak-Roesch, Pirngruber and Fischer2024) conducted a computational analysis of personality descriptions across four stages of the Epic of Gilgamesh, one of the earliest literary texts, spanning nearly 2,000 years. Their co-occurrence matrix-based analysis highlights temporal shifts in personality-descriptive terms related to social status and hierarchical relations, while also revealing the transition from a barbarian to a civilized state through linguistic markers. Similarly, Ash et al. (Reference Ash, Stammbach and Tobia2023) explored the philosophical question of what is a person by constructing diachronic semantic spaces across 200 years, focusing on dimensions of agency (e.g., planning, decision-making) and experience (e.g., feeling, hungering). Their findings showed that while women have become comparable to men in terms of agency over time, they continue to be represented as more experience-oriented in textual discourse.

Conflict and crisis-related events. Building on the observation that linguistic changes reflect broader societal trends, tracking word usage shifts can surface evidence of real-world events. For example, Kutuzov and Kuzmenko (Reference Kutuzov and Kuzmenko2016) used diachronic word embeddings preinitialized with daily news to investigate the evolving diplomatic relations between Russia and other countries, focusing on shifts in correlations among named entities. Extending this work, Kutuzov et al. (Reference Kutuzov, Velldal and Øvrelid2017) tracked a curated list of country names to explore global armed conflicts. In another instance, Stewart et al. (Reference Stewart, Arendt, Bell and Volkova2017) computationally investigated semantic shifts during the Russian-Ukrainian crisis, demonstrating how ukrop shifted from its original meaning of dill to a derogatory term for Ukrainian patriot.

Pandemic words. Similar to conflict-related shifts, LSC models have proven effective in identifying semantic changes and neologisms that emerged in response to the COVID-19 pandemic. For example, terms such as immunity and trial shifted in meaning during this global health crisis (Laurino et al. Reference Laurino, De Deyne, Cabana and Kaczer2023), while new terms such as lockdowns and maskless (referring to not wearing face coverings) emerged and spread rapidly (Würschinger and McGillivray Reference Würschinger and McGillivray2024).