This chapter focuses on the evaluation of distributional semantic models (DSMs). Distributional semantics has usually favored intrinsic methods that test DSMs for their ability to model various kinds of semantic similarity and relatedness. Recently, extrinsic evaluation has also become very popular: the distributional vectors are fed into a downstream NLP task and are evaluated with the system’s performance. The goal of this chapter is twofold: (i) to present the most common evaluation methods in distributional semantics, and (ii) to carry out a large-scale comparison between the static DSMs reviewed in Part II. First, we discuss the notion of semantic similarity, which is central in distributional semantics. Then, we present the major tasks for intrinsic and extrinsic evaluation, and we analyze the performance of a representative group of static DSMs on several semantic tasks. Finally, we explore the differences of the semantic spaces produced by these models with Representational Similarity Analysis.

This chapter discusses the major types of matrix models, a rich and multifarious family of distributional semantic models (DSMs) that extend and generalize the vector space model in information retrieval from which they derive the use of co-occurrence matrices to represent distributional information. We first focus on a group of matrix DSMs (e.g., Latent Semantic Analysis) that we refer to as classical models, since they directly implement the basic procedure to build distributional representations introduced in Chapter 2. Then, we present DSMs that propose extensions and variants to classical ones. Latent Relational Analysis uses pairs of lexical items as targets to measure the semantic similarity of the relations between them. Distributional Memory represents distributional data with a high-order tensor, from which different types of co-occurrence matrices are derived to address various semantic tasks. Topic Models and GloVe introduce new approaches to reduce the dimensionality of the co-occurrence matrix, respectively based on probabilistic inference and a method strongly inspired by neural DSMs.

Lexical semantic competence is a multifaceted and complex reality, which includes the ability of drawing inferences, distinguishing different word senses, referring to the entities in the world, and so on. A long-standing tradition of research in linguistics and cognitive science has investigated these issues using symbolic representations. The aim of this chapter is to understand how and to what extent the major aspects of lexical meaning can be addressed with distributional representations. We have selected a group of research topics that have received particular attention in distributional semantics: (i) identifying and representing multiple meanings of lexical items, (ii) discriminating between different paradigmatic semantic relations, (iii) establishing cross-lingual links among lexemes, (iv) analyzing connotative aspects of meaning, (v) studying semantic change, (vi) grounding distributional representations in extralinguistic experiential data, and (vii) using distributional vectors in cognitive science to model the mental lexicon and semantic memory.

This chapter presents current research in compositional distributional semantics, which aims at designing methods to construct the interpretation of complex linguistic expressions from the distributional representations of the lexical items they contain. This theme includes two major questions that we are going to explore: What is the distributional representation of a phrase or sentence and to what extent it is able to encode key aspects of its meaning? How can we build such representations compositionally? After introducing the classical symbolic paradigm of compositionality based on function-argument structures and function application, we review different methods to create phrase and sentence vectors (simple vector operations, neural networks trained to learn sentence embeddings, etc.). Then, we investigate the context-sensitive nature of semantic representations, with a particular focus on the last generation of contextual embeddings, and distributional models of selectional preferences. We end with some general considerations about compositionality, semantic structures, and vector models of meaning.

The distributional representation of a lexical item is typically a vector representing its co-occurrences with linguistic contexts. This chapter introduces the basic notions to construct distributional semantic representations from corpora. We present (i) the major types of linguistic contexts used to characterize the distributional properties of lexical items (e.g., window-based and syntactic collocates and documents) , (ii) their representation with co-occurrence matrices, whose rows are labeled with lexemes and columns with contexts, (iii) mathematical methods to weight the importance of contexts (e.g., Pointwise Mutual Information and entropy), ( iv) the distinction between high-dimensional explicit vectors and low-dimensional embeddings with latent dimensions, (v) dimensionality reduction methods to generate embeddings from the original co-occurrence matrix (e.g., Singular Value Decomposition), and (vi) vector similarity measures (e.g., cosine similarity).

This chapter contains a synoptic view of the different types and generations of distributional semantic models (DSMs), including the distinction between static and contextual models. Part II then focuses on static DSMs, since they are still the best known and widely studied family of models, and they learn context-independent distributional representations that are useful for several linguistic and cognitive tasks.

Neural machine translation is not neutral. The increased linguistic fluency and naturalness as the hallmark of neural machine translation sometimes runs the risk of trans-creation, which bends the true meaning of the source text to accommodate the conventionalized, preferred use and interpretation of concepts, terms and expressions in the target language and cultural system. This chapter explores the cultural and linguistic bias of neural machine translation of English educational resources on mental health and well-being, highlighting the urgent need to develop and redesign machine translation systems to produce more neutral and balanced machine translation outputs for global end users, especially people from vulnerable social backgrounds.

Access to healthcare profoundly impacts the health and quality of life of Deaf people. Automatic translation tools are crucial in improving communication between Deaf patients and their healthcare providers. The aim of this chapter is to present the pipeline used to create the Swiss-French Sign Language (LSF-CH) version of BabelDr, a speech-enabled fixed phrase translator that was initially conceived to improve communication in emergency settings between doctors and allophone patients (Bouillon et al., 2021). In order to do so, we start off by explaining how we ported BabelDr in LSF-CH using both human and avatar videos. We first describe the creation of a reference corpus consisting of video translations done by human translators, then we present a second corpus of videos generated with a virtual human. Finally, we relate the findings of a questionnaire on Deaf users’ perspective on the use of signing avatars in the medical context. We showed that, although respondents prefer human videos, the use of automatic technologies associated with virtual characters is not without interest to the target audience and can be useful to them in the medical context.