1. Introduction
Polysemy can be succinctly defined as the synchronic association of multiple related meanings with a single form (Sweetser, Reference Sweetser1990). Polysemous words typically have a prototypical meaning (i.e., their most frequent semantic interpretation) to which other senses are related in a motivated way. The complex phenomenon of polysemy has been extensively researched from a variety of disciplines such as lexicography, cognitive semantics and psychology, among others (see Falkum & Vicente, Reference Falkum and Vicente2015 for an overview of the main theories regarding polysemy). Regardless of the research focus, most authors agree that the fundamental characteristic of polysemous constructions is the natural and systematic relation of its multiple meanings. In other words, the main point of polysemy is not the multiplicity of meanings but the underlying mechanisms in charge of creating and connecting its different senses.
If the multiple readings of a particular lexical item exhibit no systematic semantic relation, that word is considered homonymous rather than polysemous. Thus, the main difference between homonymy and polysemy is the absence or presence of a systematic relation between meanings. For example, even though the word ball can be used to designate at least two different meanings, (1) a round object used in games and (2) a formal dance, it must be considered homonymous since these meanings are unrelated, that is, they show no type of connection, neither semantic nor etymological (i.e., each meaning originated from different word forms). In contrast, an example of a polysemous lexical item is the noun taste. It is mostly used with adjectives to describe the flavor of a food product (e.g., it is a basic cake, but it has a good taste). However, it can also be used to refer to someone’s discernment (e.g., he has bad taste in clothing). These senses are naturally related as they both denote likes and dislikes. In this case, the latter sense reflects the metaphorical mapping of an already existing meaning from the domain of perception to the domain of personal judgment.
One of the theories which has examined more closely the potential motivations behind the semantic extensions of polysemous lexical items is cognitive semantics. This discipline has paid a great deal of attention to the role of metaphor as a connecting mechanism between senses. Many of the senses of a polysemous word, as is the case of the discernment sense of taste, are often considered figurative interpretations of the ‘prototypical’ sense of a lexical item. These ‘non-prototypical’ related figurative meanings (e.g., metaphorical, metonymic, pejorative) are generally conceived as semantic extensions of the polysemous lexical item.
Thus, metaphor is one of the key notions behind the motivation of semantic extensions. For a substantial number of scholars in the field, metaphor is the mechanism that allows us to ground abstract concepts on embodied experience. This means that the way we perceive our surroundings shapes our conceptual system (Lakoff & Johnson, Reference Lakoff and Johnson1999). In many cases, there is a bodily based motivation behind the senses of a polysemous item which structures its meanings, and the most frequent structuring mechanisms are metaphor and metonymy (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999). An illustration of this would be the sentence I see your point, where we use the verb see metaphorically to convey comprehension instead of perception (understanding is seeing). This case appears to be motivated by the fact that most cultures conceive sight as the most reliable source of information that eventually leads us to the understanding of events. Therefore, this example shows how metaphor produces nonarbitrary associations between embodied experience and semantic extensions (Lakoff, Reference Lakoff1987).
One of the areas in which the role of metaphor in semantic extension has been more thoroughly researched is the vocabulary of perception. The main reason for this is that, even though the vocabulary of perception is used to describe how we perceive the world through our senses, it is extremely common to find perception words used metaphorically. In fact, Sweetser (Reference Sweetser1990), who pioneered research on perception verbs by diachronically exploring their multiple meanings in English, concluded that the systematic connections between our vocabularies of mind and body are essentially metaphorical in nature. For example, when we say I smell trouble there is no physical act of perception; instead, we use this verb to express our intuition that something bad is going to happen. Multiple works have focused on the analysis of perception words (especially, verbs) and their semantic extensions in many languages such as English (Kövecses, Reference Kövecses, Speed, O’Meara, Roque and Majid2019; Selmistraitis & Boikova, Reference Selmistraitis and Boikova2020; Sweetser, Reference Sweetser1990; Winter, Reference Winter2019), Spanish (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999; Jansegers et al., Reference Jansegers, Vanderschueren and Enghels2015), Italian (Monachini & Roventini, Reference Monachini, Roventini, Martin, Meijs, Moerland, ten Pas, van Sterkenburg and Vossen1994) and other languages (Burenhult & Majid, Reference Burenhult and Majid2011; Evans & Wilkins, Reference Evans and Wilkins2000; Krishna et al., Reference Krishna, Arulmozi and Mishra2022; Majid & Levinson, Reference Majid and Levinson2011; van Putten, Reference van Putten2020), amply showing the highly polysemous nature of this conceptual domain.
One of the fundamental works on the polysemy of perception verbs is the cross-linguistic analysis carried out by Ibarretxe-Antuñano (Reference Ibarretxe-Antuñano1999, Reference Ibarretxe-Antuñano2002, Reference Ibarretxe-Antuñano and Luchjenbroers2006, Reference Ibarretxe-Antuñano, Speed, O’Meara, Roque and Majid2019). In her work, she provides a cognitive semantic account of polysemy in English, Basque and Spanish perception verbs. The author describes the extended meanings that each of the five senses (i.e., vision, audition, touching, tasting and smelling) has in these three languages and summarizes the rich structure of the perception metaphors underlying them. For example, she explains that, besides its basic, physical sense of coming into contact with something, touch can be understood as ‘to affect’ both physically (my boots were here, who touched them?) and metaphorically (her story touched me deeply). Whereas the first sentence could be rephrased as who changed the position of my boots?, in the second one the verb touch specifies that what is affected is the emotional side of the speaker. Moreover, Ibarretxe-Antuñano (Reference Ibarretxe-Antuñano1999, Reference Ibarretxe-Antuñano and Luchjenbroers2006) shows how polysemy can sometimes be activated by the combination of a lexical item with other elements of the sentence which are necessary to build up that meaning. In a sentence such as ‘John hardly touched the food’ (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999, p. 69), the property of briefness is being highlighted by the adverb hardly (which here refers to a ‘small degree of something’). Without this adverb, however, the extended meaning of ‘partake’ disappears completely and the resulting sentence (John touched the food) comes to trigger the physical meaning of touch. In spite of this, we may also encounter utterances, such as X touched Y, without any additional linguistic elements activating the correct meaning of the polysemous lexical item. When the competition between the multiple meanings of a lexical item cannot be resolved, people cannot determine which of the distinct potential representations of that word should be activated. Polysemous words may lead to this situation which is known as language ambiguity. Thus, our ability to process ambiguous constructions is also of relevance when studying polysemy.
Taking this into consideration, language ambiguity has been of great interest for polysemy studies since the phenomenon of polysemy is believed to be one of the major sources of ambiguity (Nerlich & Clarke, Reference Nerlich, Clarke, Nerlich, Todd, Herman and Clarke2003; Norrick, Reference Norrick1981; Taylor, Reference Taylor, Kristiansen, Achard, Dirven and Ruiz de Mendoza Ibáñez2006). The issue of ambiguity often emerges because the different interpretations of polysemous words are manifestations of the same core meaning; however, in most polysemous expressions there is one sense that seems more appropriate in the given context (Klepousniotou, Reference Klepousniotou, Rakova, Pethö and Rákosi2007). We tend to consider that a polysemous word is ambiguous when we find it hard to choose between two or more senses that can be potentially associated with that specific form. Although people normally determine the meaning of ambiguous items by appealing to context (e.g., previous speakers’ utterances, common sociocultural background, visual input), it is not always possible to access those additional pieces of information. A classic example that illustrates this situation is the sentence ‘John touched Mary’ – X touched Y – (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999, p. 124). With no additional context, at least two of the most frequent senses of the verb touch could be considered: (1) establishing physical contact, or (2) affecting someone in an emotional manner. In short, when there is insufficient contextual information when uttering polysemous expressions, listeners cannot decide their most suitable meaning interpretation (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999).
On the whole, previous works have mostly examined the phenomenon of polysemy from a linguistic perspective (Falkum & Vicente, Reference Falkum and Vicente2015). However, the study of polysemy including ‘non-verbal’ communication modalities has not been sufficiently explored (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano, Speed, O’Meara, Roque and Majid2019). Considering that meaning can be conveyed through multiple modalities such as gesture or voice pitch, it makes sense to extend the analysis of polysemy from the purely verbal level to a multimodal perspective. In contrast to the traditional pairings of specific word forms and discrete meanings, the notion of multimodality asserts that semantic distinctions generally arise from the interplay of multiple communication modalities which can be of a different nature (i.e., ‘verbal’ and ‘non-verbal’). Hence, introducing the factor of multimodality to study the polysemy of perception verbs could be a way of expanding our previous knowledge and improving our existing models of polysemy.
One of the areas in the study of polysemy that multimodality could contribute to is the disambiguation of meaning. As mentioned before, this area has mainly been addressed by looking at the linguistic elements accompanying polysemous constructions (Hirst, Reference Hirst1987). For this reason, redirecting the research focus toward distinct communication modalities and methods to study language ambiguity could prove to be especially useful, since it adds another layer of analysis to complement verbal information. So far, how (or indeed, whether) people disambiguate the meanings of polysemous words in spontaneous communication using multimodal cues has received little attention. There are, nonetheless, some works that have attempted to combine distinct modalities when studying language ambiguity. Holler and Beattie (Reference Holler and Beattie2003) report that speakers do use gesture to clarify verbal ambiguity for the listener. They found that when speakers are asked to clarify the meaning of an ambiguous word used in the course of an extended discourse (in their work, homonyms), they are more likely to use a gesture in association with the problematic word; similar results were found by Holle and Gunter (Reference Holle and Gunter2007). Similarly, when investigating the resolution of ambiguous statements related to neutral and moral preferences, Hinnell and Parrill (Reference Hinnell and Parrill2020) showed that participants rely on gesture, when available, to understand the speaker’s ambiguous statements. Particularly, gesture appeared to be used by observers to understand the speaker’s opinion, since they were more likely to choose the idea accompanied by a gesture as the speaker’s preference, regardless of whether the linguistic cue expressed a neutral idea or a moral view that might be considered socially unacceptable. On the whole, these three studies sustain that, apart from listening to the oral message, interlocutors tend to look for meaning in the hand movements that accompany ambiguous speech (Maricchiolo et al., Reference Maricchiolo, de Dominicis, Ganucci Cancellieri, di Conza, Gnisci, Bonaiuto, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014).
Therefore, it appears that focusing on gesture together with speech could be beneficial for the analysis of polysemy since these modalities are semantically co-expressive and may transmit a different aspect of the same idea (Goldin-Meadow, Reference Goldin-Meadow2003; McNeill, Reference McNeill, Müller, Cienki, Fricke, Ladewig, McNeill and Tessendorf2013). This means that gesture can express information either on its own or when co-occurring with oral production. Co-speech gestures (i.e., gestures co-occurring with speech) may express task-relevant information not conveyed in speech; for example, sometimes children make a gesture depicting width while saying ‘this cup is bigger’ (Goldin-Meadow & Alibali, Reference Goldin-Meadow and Alibali2013). Co-speech gestures have also become a key aspect in language, cognition and communication studies because they can convey both concrete and abstract ideas. We can point to the restroom while saying Ana is in the toilet, but we can also draw circles in the air while saying the song is on repeat. Besides, co-speech gestures may be especially useful when dealing with words that have literal and figurative meanings, such as polysemous words, since they can reveal some of our covert cognitive representations (cf. Alcaraz Carrión & Valenzuela, Reference Alcaraz Carrión and Valenzuela2021; de la Fuente et al., Reference de la Fuente, Santiago, Román, Dumitrache and Casasanto2014; Núñez & Sweetser, Reference Núñez and Sweetser2006). This is mainly because while gestures convey meaning ‘globally, relying on visual and mimetic imagery’, speech does it ‘discretely, relying on codified words and grammatical devices’ (Goldin-Meadow, Reference Goldin-Meadow2003, p. 25). If someone said she is wearing a hat while moving his hand as though grasping the bill of a cap, this gesture might encourage listeners to infer that the hat referred to was particularly a baseball cap (Goldin-Meadow & Alibali, Reference Goldin-Meadow and Alibali2013). Therefore, in contrast to the traditional verbal approach, a multimodal approach focused on gesture could potentially extend our understanding of the mental processes behind polysemy.
Taking into account the aforementioned examples of polysemous perception verbs, the verb touch seems to be especially interesting, given the fact that it is the only perception verb closely tied to the domain of emotion (see Sweetser, Reference Sweetser1990 for a more detailed explanation). In fact, several works (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano and Luchjenbroers2006; Jansegers et al., Reference Jansegers, Vanderschueren and Enghels2015; Lacey et al., Reference Lacey, Stilla and Sathian2012) have already explored how the semantic content of tactile expressions and its arguments contribute to the creation of meanings such as to affect, to deal with, and so on.
While previous studies aimed at introducing different modalities (through videos and images) in language disambiguation research, no work has paid attention to the role of specific gestures in the expression of polysemous constructions in spontaneous communication. Thus, the present article explores to which extent co-speech gestures can be used to disambiguate the meaning of polysemous words. Our hypothesis, thus, is that gesture, together with linguistic cues, could also play a relevant part in the phenomenon of polysemy.
This research adopts a multimodal perspective to explore two specific senses of the perception verb touch: its physical meaning and its emotional meaning. We address the following research question: do speakers use multimodal information (namely, speech and gesture) to differentiate the physical and emotional meanings of touch? With a view to discover whether gestures are used to disambiguate polysemous words, three specific objectives emerge: (1) to compare the co-speech gestures associated with each selected meaning of touch, (2) to determine the motivation behind potential frequent co-speech gestures associated with touch and (3) to describe the relation between the linguistic context and the physical and emotional meanings of touch. In this investigation, we evidence that speakers can manifest different semantic extensions of the same lexical item through verbal (speech) and non-verbal (gesture) modalities. Based on these findings, we conclude that both speech and gesture are indeed used to distinguish the meanings of polysemous words.
2. Methodology
2.1. Dataset and tools
The audiovisual and textual information was obtained from the NewsScape Library of TV News, a multimodal repository of television news hosted in the UCLA and CWRU libraries (which gathers data from March 2004 until today). This dataset is managed by The International Distributed Little Red Hen Lab™ which is a big data science consortium for research into multimodal communication. It contains more than 500,000 hours of television news programs as well as a 4-billion-word multilingual dataset formed by television subtitles of TV news. Speech and subtitles are synchronized through force-alignment tools, which enables researchers to access the exact moment in which specific linguistic expressions were uttered. The communicative exchanges found in NewsScape are set in different communicative situations like interviews, open discussions or debates.
We also employed the corpus software CQPWeb (Hardie, Reference Hardie2012) with a subset of the NewsScape dataset (Uhrig, Reference Uhrig, Zwierlein, Petzold, Böhm and Decker2018) in order to perform more accurate linguistic searches. Specifically, we employed the NewsScape 2016 sub-corpus (269,269,133 words) which gathers data and recordings from the year 2016.
2.2. Linguistic search
We collected all the cases in which the verb touch was followed by a personal pronoun or by the words someone or somebody in NewsScape 2016. In addition, we complemented this search in the 2016 sub-corpus with a search for the specific combination touched me in the whole NewsScape Library dataset (time span: March 1, 2004 to May 1, 2021). This second search was made to balance the number of emotional and physical senses and to obtain a representative sample of gestures used in an emotional context. The expression searched was selected from a preliminary analysis which revealed that touched me seemed to be the most frequently associated construction with the emotional sense of touch.
2.3. Analysis
2.3.1. Data filtering
Considering that the aforementioned corpora are built with TV recordings, the textual and visual data stored in them tend to be quite ‘noisy’ (e.g., hits may also include voice-over cases). After the searches were done, it was necessary to discard those cases which were not suitable for our goal. So, data were manually filtered in different steps to obtain an adequate final sample. First, we removed all the cases that presented a technical issue (e.g., audio/video not available) or presented a text-speech mismatch. We also removed tagging errors (i.e., when parts of speech were not correctly identified by the system). Second, we annotated whether the query item had a physical, an emotional or an ambiguous sense, as well as cases with other senses; here, only cases with physical and emotional meanings were kept. In order to check if the meaning of touch in the speakers’ oral message had been accurately classified, a second coder analyzed a subset of the sample obtained (N = 100 gestures, 33.33% of the final dataset); there was almost perfect agreement (κ = 0.94) between coders in identifying the meaning of touch in each case (Cohen, Reference Cohen1960). Third, we noted down whether the hands of the speakers were visible or not. In particular, we classified the cases into (1) barely or nonvisible gesture, (2) partly visible gesture and (3) completely visible gesture. Cases in which the speakers’ hands were not visible (category 1) were discarded, as it could not be determined whether the speaker performed a hand gesture or not. Considering that partly visible gestures can be reconstructed by the viewer, no distinction is made between categories 2 and 3 in the analysis. Fourth, we removed repeated clips (i.e., when the exact same utterance appeared in a duplicate video) to obtain a list of unique cases. Lastly, we annotated whether the speakers performed a co-speech hand gesture or not. The end product of this filtering process was the creation of two lists of cases: one containing the gestures made with a physical sense of touch and another one containing the gestures made with an emotional sense of touch.
2.3.2. Gesture analysis
For each of the cases that contained a co-speech gesture, we annotated the following features.
Firstly, we annotated whether the gesture was related to the semantics of the linguistic expression or not, as well as those gestures whose meaning was unclear. Specifically, we considered to be semantically related co-speech gestures those which were coherent with the speakers’ utterance by adding a parallel or complementary nuance (McNeill, Reference McNeill1992). An illustration of this category would be any case in which speakers point at an element mentioned in their verbal message, such as when somebody says do not touch it and simultaneously points to the object s/he is referring to. A gesture was considered unrelated when it was not directly associated with the speakers’ words, not depicting any semantic information related to the speech. For example, in this category, we include cases such as beats which are gestures that move along with the rhythm of speech (McNeill, Reference McNeill1992) and self-adaptors which are the incidental movements individuals make when manipulating their own body or clothes, such as when someone adjusts his glasses (Ekman & Friesen, Reference Ekman and Friesen1969). There were eight problematic co-speech gestures that were labeled as unclear since they caused special controversy among the coders over whether they were related or unrelated cases. These cases were discarded from the dataset. A second coder annotated whether the co-speech gestures were related to the speaker’s utterance or not, showing a substantial agreement (κ = 0.71; Cohen, Reference Cohen1960) with the first coder. Unrelated co-speech gestures were thus discarded from the final dataset. Then, seven co-speech gestures initially labeled as related were also eliminated from the final dataset since they were ‘acted’ cases (e.g., Frankie Valli appears on KABC Jimmy Kimmel Live and while singing the words touched me, he extends his free arm forward and then changes his open palm hand shape into a fist shape); in other words, these gestures were part of a performance and not produced in a natural communicative situation.
Secondly, we annotated whether speakers touched the referent of the gesture (e.g., body parts, other people). With regards to what speakers physically touched, two categories were distinguished: other-touch (whether speakers touched an object or another person) and self-touch (whether speakers touched their own arm, chest, leg or belly). A second coder analyzed a subset of the data (N = 100 gestures, 33.33% of the final dataset) in terms of what speakers touched, showing a substantial agreement (κ = 0.73; Cohen, Reference Cohen1960) with the first coder.
2.3.3. Linguistic analysis
In relation to the linguistic elements, we examined the distribution of the pronouns following the verb touch and, taking into account that certain linguistic modifiers have been previously associated with emotional attributes in language, two additional verbal components were also annotated: the presence of intensifiers (i.e., linguistic markers which have ‘a heightening or lowering effect’ on a lexical unit; Quirk et al., Reference Quirk, Greenbaum, Leech and Svartvik1972, p. 376) and the use of negation (i.e., syntactic process through which a word or particle can change a positive construction into a negative one; Quirk et al., Reference Quirk, Greenbaum, Leech and Svartvik1972). In particular, the reason for the annotation of intensifiers was motivated by the fact that these linguistic markers have been shown to be very frequently associated with the expression of emotional experience (Argaman, Reference Argaman2009). Additionally, negation words were also taken into account because in a Sentiment Analysis study, Carrillo-de-Albornoz and Plaza (Reference Carrillo-de-Albornoz and Plaza2013) showed that negation was the most influential phenomenon when predicting the emotional meaning of the text. Since one of the two meanings of touch examined in this article is directly linked to the expression of emotion, both intensifiers and negation words could be fundamental elements that help identify the sense of the verb.
These characteristics were noted down when they modified the verb and were fewer than five words away from the query item (i.e., when they were included within the range −5:+5). The list of intensifiers and negation signals found can be seen in Supplementary Table A.1. The final dataset is publicly available at https://osf.io/v8gp3/.
3. Results
3.1. Gesture frequency
The touch + personal pronouns search (in NewsScape 2016) returned 2571 matches (60.2%) and the touched me search (in the whole NewsScape dataset) gave 1698 matches (39.8%), amounting to a total of 4269 cases (100%). From the overall amount, we removed 417 cases that presented either a technical issue or a text-speech mismatch and, subsequently, we removed 59 tagging errors. When examining closely the meaning of touch in the remaining 3793 cases, we found 30 cases with an ambiguous sense and 214 cases with other senses (i.e., neither physical nor emotional), which were then excluded. After discarding the cases that were not associated with the physical or emotional meaning of touch, the remaining dataset (3549 cases) resulted in 2194 cases expressing the physical meaning of touch and 1355 cases expressing an emotional meaning. Then, 2562 cases (1528 physical and 1034 emotional) in which the speaker’s hands were not visible were eliminated. After this step, 175 physical cases and 71 emotional cases (246 in total) were removed as they were repeated. As this work focus was on co-speech gestures, 265 cases (178 physical and 87 emotional) where speakers did not make any hand gesture were not included in our gesture sample. In order to obtain the definitive sample of co-speech gestures, 8 unclear gestures, 159 unrelated gestures and 7 acted gestures were eliminated. After that, 186 physical (4.4%) and 116 emotional (2.7%) semantically related gestures were kept in the final dataset. Hence, the total number of cases was reduced from 4269 cases to 302 co-speech gestures which amounts to 7.1% of the cases obtained from both linguistic searches (for more data curation details see Supplementary Table A.2).
Taking into account only the cases associated with the physical or emotional meaning of touch (see Figure 1), a chi-square test of independence was performed to examine the relation between the presence and absence of gesture and the meanings of touch. The relation between these variables was not significant, χ 2(1, N = 567) = 1.9124, p = .167. Hence, there was no difference in the distribution of cases with related gestures and cases without gesture when co-occurring with the meanings of touch.
Distribution of cases associated with the physical or emotional meaning of touch.
3.2. Comparing the physical and emotional senses of touch
3.2.1. Gesture referent
In the case of co-speech gestures performed with the physical sense of touch, speakers touch the referent of the gesture in 21.5% of the cases (N = 40, out of 186). Concerning the co-speech gestures performed with the emotional sense of touch, speakers identify the referent by touching it 38.79% of the times (N = 45, out of 116). With a view to discovering whether there was a significant association in the final sample between touching the referent, or not, and the type of verb sense (i.e., physical and emotional), we calculated a chi-square test and the result, χ 2(1, N = 302) = 10.558, p = .001, indicates a significant association between both variables. This means that speakers are more likely to touch the referent of the gesture when the sense of the verb is emotional than when it is physical.
Additionally, when speakers physically touch the referent of the gesture, two categories can be distinguished: other-touch and self-touch (see Table 1). Self-touch is found with both verb meanings (16 times, 18.8%, in physical cases and 45 times, 52.9%, in emotional cases), but the former, other-touch, can only be found when the verb touch implies its literal meaning (24 times, 28.2%, in physical cases). Since there were no cases in some of the cells (Table 1) we could not use the chi-square test. This is why we employed Fisher’s exact test to determine if there was a significant association between the type of the meaning (i.e., physical and emotional) and the referent that was touched (i.e., other-touch and self-touch). There was a statistically significant association between the two variables (p < .001).
Distribution of other-touch and self-touch in the final gesture dataset
Specifically, the group of gestures associated with the physical meaning shows that speakers tend to touch an object or another person (12 times, 14.1%, in each case, see Figure 2 for an example). Referring to the group of gestures associated with the emotional meaning, speakers mostly physically touch their chests (41 times, 48.2%, see Figure 3 for an example). The data collected suggests that the latter case frequently occurs together with the emotional meaning of the touch. Hereafter, this specific type of co-speech gesture is referred to as chest-touching gesture.
A chi-square test was carried out to examine the relation between the distribution of gestures where speakers touch their chest and gestures where speakers touch other things and the meanings of touch. The relation between these variables was significant, χ 2(1, N = 85) = 52.7032, p < .001. This result shows that speakers are more likely to physically touch their chest when the verb has an emotional meaning than when it has a physical meaning.
Concerning the cases of other-touch, they are noteworthy as they only seem to appear with the physical meaning of touch. The category other-touch can be easily distinguished as it involves physically touching an external element (in the following case, a different person). The other-touch example below (see Figure 2) is made while the speaker says:
-
1) ‘he came over and touched me ’ (2010-12-07 KABC The View)
In this case, Whoopi Goldberg talks about a little boy who was behaving badly and annoying her on a train trip. The most upsetting part of that situation was that the child touched her with his sticky hands. Simultaneously with this utterance, she touches her colleague’s arm with her left hand to illustrate and emphasize the fact that the naughty child dared to touch her.
Figure 2.Example 1: other-touch gesture made with a physical meaning.
Regarding chest-touching gestures, the next case is a very clear example (see Figure 3):
-
2) ‘initially it [the project] touched me ’ (2019-06-16 KCBS Entertainment Tonight)
In this clip, Halle Berry explains how she got involved in a documentary of patients with AIDS. When talking about the phone call she received to participate in that project, she makes reference to a personal experience in order to explain why she accepted the offer immediately. Then, while saying the sentence above, she raises her left hand from a resting position and opens the palm to touch her chest area.
Figure 3.Example 2: chest-touching gesture made with an emotional meaning.
3.2.2. Linguistic analysis
With regard to the linguistic utterances, we checked whether the meanings of touch could be associated with specific personal pronouns, the use of intensifiers and the use of negation words.
First, we counted the cases of each pronoun in our sample of gestures, confirming that the physical and emotional senses of touch might also be distinguished just by looking at the pronouns that appear after the verb (see Table 2). Taking into account only the touch + personal pronouns search in CQPweb (where pronouns were not specified), it can be seen that, when speakers express the physical meaning of touch, they mostly address an external referent (126 times, 79.8%) rather than refer to themselves (32 times, 20.3%). Concerning the cases associated with the emotional meaning of touch, speakers refer to themselves 11 times which amounts to 50% of the cases. That is, the frequency of the pronoun me increases when touch has an emotional meaning. The difference between the type of meaning (i.e., physical and emotional) and the type of pronoun that accompanied the linguistic expression (i.e., me and the other pronouns) is significant, χ 2(1, N = 180) = 9.3982, p = .002. Thus, the utterances conveying the emotional meaning of touch tend to be produced with the personal pronoun me.
Distribution of pronouns that appear in the touch + personal pronouns search in CQPweb
As far as the use of intensifiers is concerned, it is infrequent with the physical sense of touch and pervasive with the emotional one (Table 3). Within the group of related gestures when touch had a physical meaning, only one out of 186 cases included an intensifier, a mere 0.5%. In contrast, speakers used intensifiers in 56 of the cases in which they did a related gesture when touch had an emotional meaning; this is 48.3% of the cases.
Frequency of intensifiers and negation markers in physical and emotional meanings that co-occurred with a related gesture
Concerning negation words, they are commonly found when touch has a physical meaning but extremely rare in its emotional sense (Table 3). Negation words were present in 62 (33.3%) of the cases associated with the prototypical meaning of touch where speakers made a related gesture. In comparison, only two negation words (1.7%) were found in the list of cases linked to the emotional meaning of touch where speakers made a related hand gesture.
Taking into account the information above (see Table 4), a chi-square test was calculated to examine the relation between the distribution of linguistic markers (intensifiers and negation words) and the meanings of touch among all valid cases (i.e., non-repeated cases with visible hands). The result, χ 2(1, N = 278) = 239.1518, p < .001, shows a significant association between both variables. Thus, speakers are more likely to use negation words when the sense of touch is physical and they appear to use intensifiers when the sense of the verb is emotional.
Frequency of intensifiers and negation markers in physical and emotional meanings among valid cases
After verifying that there was a statistically significant association between these two variables, we wondered if the meaning of touch could be predicted based on the use of intensifiers and negation. For this reason, a binary logistic regression (see Table 5) was calculated to predict whether the meaning was physical or emotional based on the use of intensifiers and negations among the valid cases. The effect of intensifiers and negation words on the meaning of touch was statistically significant (χ 22 = 329; p < .001; R 2McF = 0. 347, R 2N = 0.497). In particular, the presence of intensifiers – and absence of negation – was associated with expressions with an emotional meaning and the presence of negation – and absence of intensifiers – was associated with expressions with a physical meaning.
Binary logistic regression model including Standard Estimate and 95% Confidence Interval for odds ratio
4. Discussion
4.1. Gesture frequency
Despite the fact that the distribution of the categories related gesture and no gesture among the physical and emotional meanings of touch was not statistically significant, these data also uncover some interesting points. In this work, the physical meaning and the emotional meaning of touch appear to be equally distributed in both categories (related gesture and no gesture). Thus, the difference in meaning (physical vs. emotional) does not influence the number of gestures speakers make. A potential explanation for this could be that speakers do not vary the frequency of gesture they make when they use either the physical meaning or the emotional meaning, instead they rather change the type of gesture (i.e., its formal features) they use with each verb sense. For example, when speakers express the emotional meaning of touch, they tend to physically touch their chest and they never reach out to external objects – as happens when speakers convey its physical meaning.
4.2. Gesture referent
Concerning the proportion of cases in which speakers physically touch the gesture referent, it varies depending on the meaning of touch. People touch the gesture referent more when the meaning of this perception verb is emotional (38.79%) than when it is physical (21.5%). Thus, speakers tend to visually indicate and specify the referent of their gesture – which might be crucial for determining the verb sense as it is typically the same as the ‘receiver of the action’ – when they are expressing the emotional meaning of touch. In addition to this, speakers tend to physically touch different types of gesture referents depending on the sense of touch expressed in speech. In fact, the gesture category other-touch (i.e., speakers touch an object or another person) was only found when speakers expressed the physical meaning of the verb. Even though the category self-touch (i.e., speakers touched their arm, chest, leg or belly) could be found with both meanings, it shows greater proportion in the emotional meaning. This is due to the high frequency of chest-touching gestures in emotional cases (48.2%), which is not only higher than the frequency of chest-touching gestures in physical cases (5.9%) but also higher than the overall frequency of self-touch in physical cases (18.8%). This would suggest that, while physical experience is connected to the external elements in our surroundings, emotions are conceived as an internal phenomenon developed and contained within our bodies.
This bond between chest-touching gestures and the emotional meaning of touch certainly is the most interesting result obtained from the gesture analysis. The relevance of this gesture goes beyond its frequent use since it seems to represent the Western conceptualization of hearts as the seat of emotions. Chest-touching gestures could be considered a way of depicting that something or somebody reached the speakers’ heart. The conceptual metaphor that seems to motivate the production of these gestures is the heart is a container for emotions (Kövecses, Reference Kövecses1986; Lakoff & Johnson, Reference Lakoff and Johnson1980) which is at the same time included in the broader metaphor the body is a container for the emotions (Lakoff, Reference Lakoff1987) that can be found in many languages (see Ibarretxe-Antuñano & Valenzuela, Reference Ibarretxe-Antuñano and Valenzuela2021 for an overview of this topic). For example, in English people can brim with excitement and in Spanish people can be full of rage (llenarse de rabia). In a few words, it seems that the emotional meaning of touch is expressed through the embodiment of emotions by means of chest-touching gestures.
Considering that mappings can sometimes vary depending on the speakers’ cultural background, this gesture is not only metaphoric but it is also likely to be culture-dependent. Due to the fact that the language explored in this research is English, it is coherent to find a conceptualization belonging to the Western world. However, as claimed in Ibarretxe-Antuñano and Valenzuela (Reference Ibarretxe-Antuñano and Valenzuela2021), other cultures conceive the seat of emotions to be located in different organs such as the belly as is the case in Kuuk Thaayorre (Gaby, Reference Gaby, Sharifian, Dirven, Yu and Niemeier2008), the liver in Indonesian (Sharifian et al., Reference Sharifian, Dirven, Yu, Niemeier, Sharifian, Dirven, Yu and Niemeier2008), or even different organs as happens in Chinese, where anger is situated in the liver, fear in the kidneys, sadness in the lungs and joy in the heart (Yu, Reference Yu2009). Taking this information into account, we cannot state that chest-touching gestures are associated with the expression of emotions in non-Western cultures since this path has not been researched yet. Hence, the gestures co-occurring with the narration of emotional events in languages such as the aforementioned needs to be explored in order to determine which are the gestures speakers use and whether these also imply the embodiment of emotions.
Another explanation for the propensity to use chest-touching gestures when the verb touch transmits an emotional meaning could be due to speakers seeking to reinforce this less frequent meaning of the verb by adding a more salient multimodal element. Considering the apparent lower frequency of use of the emotional meaning of touch, people appear to draw upon a more noticeable gesture – which does not only involve the hands but also a part of the speaker’s body – to facilitate the recognition of the ‘non-prototypical’ verb sense (i.e., emotional meaning). Furthermore, this explanation does not conflict with the potential use of co-speech gestures as elements helping to disambiguate the meaning of polysemous words. The results of this work suggest that gesture may be a useful supplementary level of analysis in the field of polysemy. Considering that, when there is no access to additional linguistic information, meaning disambiguation seems an impossible goal, the modality of gesture could be valuable to solve those cases occurring in situations with insufficient context or potential ambiguity.
In fact, Ibarretxe-Antuñano (Reference Ibarretxe-Antuñano and Luchjenbroers2006) declared that the emotional meaning of touch is easy to identify when context is provided, but quite challenging to detect when it appears ‘isolated’ since it could have several interpretations. This situation was illustrated in Ibarretxe-Antuñano (Reference Ibarretxe-Antuñano and Luchjenbroers2006) with the sentence John touched Mary (as discussed in the introduction). This is the reason why similar cases are called unpredictable cases of polysemy and their potential meanings argument-driven extensions (see Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano and Luchjenbroers2006, for the detailed explanation of these concepts). Expressed differently, the meaning of this type of sentences cannot be predicted only by the arguments that the verb takes.
4.3. Pronoun frequency
The meanings of touch can also be distinguished by looking at the personal pronouns used with each type of meaning. Focusing on the touch + personal pronouns search, when speakers made a hand gesture related to the physical meaning of touch, they mostly addressed an external referent (79.8%). However, when the hand gestures were associated with the emotional meaning of the verb, speakers referred to themselves in half of the cases. Expressed differently, when speakers convey the physical meaning of touch, they make use of a wider variety of personal pronouns (where it is the most repeated one). The reason why physical cases occur with a diverse range of pronouns is simple: this meaning is more likely to include more elements (i.e., linguistic referents) because the description of external events depends on multiple environmental stimuli rather than on one, as happens when we refer to emotional experiences. Considering that the cases related to the emotional meaning of touch tend to be produced with the personal pronoun me, it appears that speakers do not usually use the verb touch to describe other people’s internal processes. The high proportion of the emotional meaning of touch being used to express our own emotions might occur due to the potential difficulty that entails reading people’s genuine feelings.
On the whole, this correlation between the personal pronoun accompanying touch and the meaning expressed appears to support the aforementioned gesture differences (self-touch vs. other-touch). This means that the wider the variety of pronouns used, the greater the diversity of gesture referents touched by speakers. The only issue concerning the distribution of personal pronouns in our sample of gestures is the fact that sometimes it might be challenging to determine their role; expressed differently, to decide whether chest-touching gestures were motivated by the emotional component of touch or originated as an ‘pronoun emphasizing strategy’. As chest-touching gestures used with the emotional meaning of touch are linked to the pronoun me, it could be argued that speakers touch their chest because they want to make the referent (who/what was touched) stand out. However, this is normally done through pointing gestures without the need of physically touching the referent – in this case, themselves (Cooperrider & Mesh, Reference Cooperrider, Mesh, Morgenstern and Goldin-Meadow2022). Nevertheless, another possibility is that these potential roles of chest-touching gestures might coexist together instead of excluding each other.
4.4. Linguistic markers
The linguistic analysis showed that there is an interaction between the meaning of touch activated and the type of linguistic marker used with the verb. When the co-speech gestures are associated with the physical meaning of touch, speakers often include a negation word while omitting intensifiers. In contrast, when the gestures are related to the emotional meaning of the verb, speakers do not use negation particles and they usually strengthen the meaning expressed by using intensifiers. This interaction can be almost conceived of an inverse relation since when a type of linguistic marker appears with the physical meaning, it is rarely manifested in the emotional meaning – and the other way round.
In general terms, linguistic modifiers interact with each other and can be used jointly since their appearance does not exclude the possibility of using more than one type of modifier within a construction (Carrillo-de-Albornoz & Plaza, Reference Carrillo-de-Albornoz and Plaza2013). In this research, there is also ‘the exception that proves the rule’; that is, one intensifier was found in physical cases and two negative particles in emotional cases. It is worth mentioning that the two negation words found in the emotional cases precede an intensifier and they are used before a comparative structure (e.g., the health care debate has not really touched me as much as it has Kelly). In this sentence, we find a great illustration of the use of negation as a diminisher of the strength of the polarity of the statement. Thus, the actual function of this negation is to emphasize that the speaker’s partner was deeply affected by the health care debate. In a few words, it seems that the negation word in this example is not directly used to ‘cancel out’ the emotional meaning implied by touch, but to be specific about the speakers’ level of affliction in comparison with his partner.
The association of the physical sense of touch with the use of negation might be due to the fact that it is felt as more ‘invasive’ in comparison to the emotional sense. In other words, speakers mostly use negation in order to warn people that they should not attempt to physically touch them or an external element. On the contrary, using negation words while conveying the emotional meaning of touch seems quite futile as individual emotions can be neither controlled nor ‘restricted’ by interlocutors themselves (Fernández Jaén, Reference Fernández Jaén2012, Reference Fernández Jaén2016; Jansegers, Reference Jansegers2017). Also, physical contact could be potentially conceptualized as a binary feature [±physical contact], something either touches you or it does not.
As previously mentioned, the binary conceptualization of the physical meaning of touch does seem to not manifest in the emotional meaning of the verb. Actually, the fact that the emotional sense is associated with the use of intensifiers (e.g., very, really) supports the widely accepted idea that emotions are conceptualized along a gradable scale (Fontaine et al., Reference Fontaine, Scherer and Soriano2013). Thus, it makes sense that intensifiers modify only emotional meaning (rather than physical meaning) because emotions are usually conceived as something which can have different degrees of intensity. Furthermore, in relation to the lack of intensifiers found when touch implies physical contact, it is quite unlikely that speakers would express the intensity of the touch (perception of pressure, temperature, etc.) through modifiers expressing degree rather than using manner adjuncts (e.g., he touched me softly so that he would not scare me). Concerning the degree of emotional involvement that someone might feel in a particular situation, it can also change from person to person and, as a result, it can be ‘graded’. For example, an event might ‘touch’ someone (i.e., they found the situation moving), but that same event might ‘really touch’ someone (i.e., they were deeply emotionally involved). In contrast, this specificity in the level of the action expressed cannot be done with the physical sense of touch.
Argaman (Reference Argaman2009) stated that there is a connection between the intensity of emotions and the lexical modalities that speakers use in their expression of the emotional experience. She concluded that linguistic markers, such as intensifiers, are objective tools for determining people’s emotions since we normally give priority to a correct delivery of the content of the message rather than to a finely determined word choice, which is usually a conscious process. Our findings would confirm that speakers often use intensifiers when sharing their emotions, in our case, in oral productions. All things considered, the correlation between the use of intensifiers and the expression of emotional meaning could potentially apply generally and not only in relation the perception verb touch.
Analyzing the linguistic markers modifying polysemous verbs could be crucial when trying to determine the meaning conveyed. ‘In some cases, the polysemy is due both to one of the words of the sentence, in this case the verb, and to the meaning of the other arguments that the verb takes’ (Ibarretxe-Antuñano, Reference Ibarretxe-Antuñano1999, p. 69). The connections found between linguistic modifiers and the meanings of touch could be much more than mere associations, as they also reveal specific ways in which speakers conceptualize these meanings. Thus, the results in this article would confirm Ibarretxe-Antuñano’s claim (Reference Ibarretxe-Antuñano1999, Reference Ibarretxe-Antuñano and Luchjenbroers2006) that polysemy is not always localized on a single word (touch), but also on other elements of the sentence (in our case, negation and intensifiers) which may contribute to trigger one specific meaning.
5. Conclusion
This research has adopted a multimodal perspective to explore the expression of the physical and emotional meanings of the perception verb touch. Our initial hypothesis has been confirmed: speakers use multimodal information (speech and gesture) to differentiate the meanings of the polysemous verb analyzed. With regards to our first objective, we have seen that while the gestures associated with the physical meaning tend to reach an external referent (other-touch), the gestures associated with the emotional meaning only involve touching the speaker’s body (self-touch). Concerning our second objective, the most frequent co-speech gesture, the chest-touching gesture, was mainly used when touch had an emotional meaning. Its motivation appears to be the conceptual metaphor the heart is A container for emotions (Kövecses, Reference Kövecses1986; Lakoff & Johnson, Reference Lakoff and Johnson1980); therefore, this is the structuring mechanism by which chest-touching gestures embody the emotional meaning of touch. In relation to our third objective, linguistic elements distinguishing both meanings (pronouns, negation and intensifiers), this work showed that the physical meaning of touch is characterized by its use of a wide variety of personal pronouns and of negation words; in contrast, the emotional meaning of the verb touch occurs mostly with the personal pronoun me and it is usually modified by intensifiers. These marked differences reveal two issues: the conceptualizations of each sense and the way polysemy works. While the physical meaning appears to be considered a binary feature, the emotional meaning seems to be conceived along a gradable scale. Thus, this relation between the linguistic context and the meaning of touch corroborates that polysemy can be distributed among several elements of the sentence and not only on the polysemous item itself.
Redirecting the research focus of polysemy studies toward additional communication modalities could certainly contribute to expand our knowledge about this phenomenon and improve future works in this field. The present investigation could generate future experimental studies, which could test whether gestures are indeed elements helping to disambiguate meanings of touch combining the gesture patterns detected in this work with examples of unpredictable polysemy. Apart from this, there are of course questions that we are not able to answer just yet. First, more exploratory research is necessary to firmly establish whether this multimodal dimension can be found in other body-bound metaphors and in different languages. It would also be very useful to check whether the distinction of meanings through gesture is only related to the metaphors motivating the emotional meaning of touch or if this also happens with other figurative meanings, regardless of their emotional component.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/langcog.2024.23.
Data availability statement
All the data collected to conduct this analysis can be found at https://osf.io/v8gp3/.
Funding statement
The research reported here was supported by the 21596/FPI/21 Fundación Séneca, Región de Murcia (Spain), predoctoral fellowship to I.B.M.; by a Fulbright scholarship to I.B.M. (00001/FLB/22, Fundación Séneca, Región de Murcia-Fulbright); by the European Union – NextGenerationEU (Beatriz Galindo award) to D.A.C. and by the Ministerio de Ciencia e Innovación (Spanish State Research Agency) and Feder/UE funds (grant number PID2022-139194NB-I0).
Competing interest
The authors declare none. This manuscript has not been published and is not under consideration for publication elsewhere. Likewise, we have no conflicts of interest to disclose. All the authors have approved this submission and we confirm that the data collection and procedures follow the journal ethical guidelines.
Open access
