1.1 Voice and Face

The first line of research focused on the semantic contribution of intonationFootnote ¹ that can differentiate many situations expressed by the same sentence. For example, 18 different situations can be evoked by modifying the intonation of “Il a une voiture” (He has a car) (Reference CalbrisCalbris, 1973, pp. 104–117). While recording intonations for an experimental auditory recognition test with 90 French subjects, the performer was not able to produce the appropriate intonation without reproducing the physical attitude associated with it. This finding provided an opportunity to observe that the differentiating intonational attitudes constitute “audio-visual units,” both in transmission and reception, that are structured by the same vocal-kinesic dynamics (Reference Calbris and MontredonCalbris & Montredon, 1980). One of the consequences of this test was to demonstrate that phonogestural parallelism actually manifests itself on many levels: (1) in time, (2) in form, (3) in meaning, (4) in function, and (5) in coding (Reference CalbrisCalbris, 1989).

1.2 Face and Gesture

The second type of research investigated the semantic contribution of speechless facial-gestural expressions (Reference Calbris and MontredonCalbris & Montredon, 1986). This involved testing the meaning and the signifying structure of conventional French facial-gestural expressions (emblems), for example, the gesture of twiddling one’s thumbs meaning “idleness.” Thirty-four emblems involving different body segments at different levels of the body were captured on film simultaneously by two cameras; one filmed the whole expression, the gesture and the face, while the second simultaneously filmed just the gesture. For the purposes of the experiment, the 34 expressions were divided into two equivalent lists of items and presented to two equivalent groups of French subjects. Each subject had to match an expression (signifier), partially or completely presented, to its meaning (signified) on the given list of items. For example, the verbal cliché “il fout rien de la journée” (he does nothing all day) associated with the meaning of the thumb-twiddle emblem appeared on the list and not the verbal cliché “il se tourne les pouces” (he twiddles his thumbs) that describes the emblem. The results of the test (85 percent success rate) show that the gesture was sufficient in 83 percent of the cases to identify the signified. The contribution of the face was most often irrelevant, positive if it disambiguated the gesture, and negative if it was superfluous or if its affective connotation seemed to contradict the connotation signified by the gesture (Reference CalbrisCalbris, 1980; Reference Calbris1990, Ch. 1).

To verify the cultural character of these emblems, the semantic test was repeated with Hungarian subjects, with a different native language but the same Western culture (civilization), and with Japanese subjects, with a different native language and culture. The rate of comprehension decreased from 85 percent for the French group to 46.5 percent and 29 percent for the Hungarian group and Japanese group, respectively. It appears that cultural similarities and differences may be more important than linguistic differences in accounting for these results.

The results of this experimental study demonstrated not only the cultural (or conventional) character of gesture but also its analogical (or motivated) character: by systematically searching for analogical links in the foreign subjects’ data, analysis of the errors revealed the motivated character of gesture and, taking into account the numerous links imagined by these subjects while interpreting the same gesture, the possible plural motivation of one single gesture (Reference CalbrisCalbris, 1981; Reference Calbris1990, Ch. 2).

1.3 Gesture on Its Own as a Nonverbal Sign

The third study concerned the semantic contribution of the gesture as a nonverbal sign. Whereas the verbal sign is generally arbitrary (extrinsic), analytical (discontinuous), and explicit (invariant), the nonverbal sign by contrast, whether vocal or gestural, is generally motivated (intrinsic and part of the referent which it signifies), synthetic (continuous, so that any modification implies a parallel modification of the referent), and probabilistic (e.g. Was that a wink or was there dust in your eye?); as such, the nonverbal sign is ideal for expressing what is implicit (Reference Scherer, St. Clair and GillesScherer, 1980). If the verbal sign is conventional and arbitrary, the nonverbal sign is conventional and motivated (Reference FónagyFónagy, 1961), that is, its physical aspect and its meaning are analogous, even though its meaning differs from one culture to another. For example, the finger Ring, as an emblem or a co-speech gesture, is always an analogical sign (Figure 9.1) because it is always based on a link of contiguity or physical resemblance established through analogy between a relevant physical feature of the gesture and physical experience of the world. This analogical link is the source of the gestural semanticity that is tapped to create the specific contextual meaning of a gesture from a range of possibilities offered by its iconic and cultural nature. Hence the finger Ring is an intercultural analogical sign (compare Malta, France, and Japan). Moreover, it is an intracultural analogical sign: in France, as an emblem, it signifies “perfect” and sometimes “zero,” and the facial expression determines whether it means positive or negative appreciation; as a co-speech gesture accompanying a verbal utterance such as “Cela concerne 0,25% de familles” (That concerns 0.25% of families),Footnote ³ it expresses extreme precision, not by depicting a circular form but fingernail pincers. Both cultural and contextual, the gestural sign is always an analogical sign.

Figure 9.1 The gestural sign is always an analogical signFootnote ²

A parallel attenuation on the physical and the semantic levels confirms the analogical nature of the gestural sign. Figure 9.2 shows four typical gestures that express decreasing degrees of threat from left to right: the surface of the body part (hand/index finger) and the dynamics of the movement (segment shaken/simply lifted) decrease correspondingly with the intensity of the notion expressed. We find this parallel attenuation in the physicosemantic expression of quantity as well as in that of opposition (see Reference CalbrisCalbris, 2011, pp. 167–170). Isomorphism is the term that Reference CalbrisCalbris (1990, Reference Calbris2011) uses to describe such parallelism in gestural signs that share a core meaning and present graded expressions of it.

Figure 9.2 A continuous analogical sign

1.4 Distinction between Gestural Sign and Gesture

The semiotic unit constituted by an analogical sign can be common to several gestural units cumulated for more expressivity within the same kinesthetic ensemble, for example, as shown in the simultaneously raised eyebrows, head, and hand expressing “increasing exclamation” in Figure 9.3. The gestural unit defined by the movement of a body segment is more precisely described by means of the various gestural components detailed in Figure 9.4.

Figure 9.3 Identifying gestural units

Figure 9.4 Gestural components

2.1 Physical Classification of Gestures

The most relevant physical criteria determined the hierarchical presentation of a physicosemantic dictionary of all observed gestures referring to the concrete and to the abstract (Reference CalbrisCalbris, 1984, pp. 330–856; see Reference Calbris1990, pp. 211–227 for excerpts) as shown in Table 9.1:

• – According to the active body segment, either mobile (arm(s), hand(s), finger(s)) or not very mobile (head gestures), or the passive object of a gesture (body-focused gestures).

• – According to the movement. A distinction must be made between gestures that draw straight lines or flat surfaces (straight-line gestures) and those that draw curved lines or surfaces (curved gestures) because their respective, relevant secondary components differ.

For straight-line gestures, the most relevant physical features are the directional axis of the movement (up–down, front–back, or right–left) in conjunction with the active body part and the plane in which this is oriented (frontal, sagittal, or horizontal). Furthermore, in the case of a flat hand, the palm, fingertips, or the edge of the hand can be relevant (see “Relevant feature” in Figure 9.7).

For curved gestures, the direction of the circular movement (clockwise/anticlockwise) is of utmost relevance. The active body segment is not really relevant. Curved gestures are therefore classified according to the curved line or surface drawn (e.g. an arc vs. a circle, each possibly repeated with or without displacement) and not according to the active body segment.

2.2 Relations between Gestures and Notions

Polysemy on one side, variation on the other; by comparing gesture variants, the common physical element bearing the analogical link with a common signified is revealed.

2.2.1 Variation

Several gestures, either alternatively or simultaneously (Figure 9.5, right column), can represent one notion. Referring back to Table 9.2 and Figure 9.3, an upward movement of the hand and/or head are gesture variants expressing “increasing exclamation.” Here is another example: One can refer to the recent past by moving the thumb or, alternatively, the head over the shoulder; moving both simultaneously creates a cumulative variant that reinforces the expression (Figure 9.6.1). And to signify “distant past,” the hand and/or head raised high and moved backward refer to a space-time far behind you and no longer just behind you (Figure 9.6.2).

Figure 9.5 Relations between gestures (g/G) and notions

(n/N)

Figure 9.6.1 Recent past

Figure 9.6.2 Distant past

Whereas stylistic variants either amplify or attenuate the degree of expressivity when conveying the same meaning, semantic variants offer the possibility of introducing different shades of meaning when conveying the same basic notion by changing a secondary component.

Variants and motivation.

The four semantic variants of “cutting” are related to the plane in which the cut is made: in the sagittal plane, a “cut in two”; in the frontal plane, a “cut that stops,” and in the horizontal plane, a “total cut” are signified, respectively. In the latter plane, turning the hand so that the palm faces upward to make the transverse cut specifies a “total cut at the base” or “undermining-felling.” Note that the transverse movement is specific to both “total” cuts. But it is the rapid movement of the abruptly stopped cutting edge of the hand expressing the proprioceptive schema of cutting that is common to all the four variants. Each of these variants does not express just one analogical link, but two (Reference CalbrisCalbris, 2003a, pp. 19–46).

2.2.2 Polysemy

One gesture can alternatively represent several notions (Figure 9.5, left column). The best example of polysemous gesture is the transverse movement of the Level Hand(s) in the horizontal plane (Figure 9.7) observed in 131 examples from Source 1; it can represent about 20 different notions depending on the context. This gesture exploits all the relevant elements of the flat hand (fingertips, palm, and edge).

This gesture has not only numerous analogical links, which is why it is said to have plural motivation, but each link is also applicable in numerous domains, giving rise to semantic derivation. For example, directness represented by drawing a straight line from A to B expresses immediacy on the temporal level and an immediate consequence on the logical level, that is, determinism, certainty, or obligation, whereas on the level of moral judgment, directness expresses frankness.

Figure 9.7 The plural motivation of a polysemous gesture

Originating from our motor or perceptual experience, gestures that employ the Fist configuration contain several possible analogical links: A clenched fist (1) is strong, (2) can hold something firmly, and (3) can enclose something. Each of these analogical links may be subject to semantic derivation: strength can be physical (example [1]), psychological (example [2]), or moral (example [3]), and an enclosed object that a fist represents may be real, that is, graspable by the hand, or abstract (example [4]).

1. 1 A pianist is talking about the role of the left hand in Beethoven’s music:

   [left fist] La main gauche, c’est une, c’est une, comme ça, une, une présence d’éner [opens] énergétique.

   [left fist] The left hand, it’s a, it’s a, like that, a, a presence of ener [opens] energetic.

1. 2 [right fist] Je me suis obligée tous les jours

   [right fist] I forced myself every day

1. 3 [both fists] Parce que je pense que on ne peut pas, passée la cinquantaine, ne pas s’occuper des jeunes citoyens.

   [both fists] Because I think that one can’t, once past fifty, not do something for young citizens.

1. 4 A psychologist interviewed at home is talking about educational assessment:

   [left fist lifts up] tellement c’était un [left hand falls on to the sofa] un secret bien gardé

   [left fist lifts up] it was such a [left hand falls on to the sofa] a closely guarded secret

To summarize, the physical polyvalence of the fist determines its polysemy as a sign. The context determines the choice of both the analogical link and the semantic derivation.

Polysign gesture.

In contrast, the polysign gesture simultaneously conveys multiple signs (Figure 9.5, left column) because it contains different analogical links that depend on different relevant physical features associated with the context. Let us take the example of the raised fist that is generally used as a sign of threat. In example (4), it is used as a polysign that provides two simultaneously conveyed pieces of information that are confirmed by the spoken utterance. The analysis of example (4) in Table 9.3 shows that the raised fist contains two physical relevant features: its closed configuration and its upward movement (signifiers) are possible supports of analogical links in accordance with the context. Note that the synthetic gestural information (“so enclosed”) that occurs synchronously with the verbal segment “tellement c’était un …” precedes the confirmatory verbal information “un secret bien gardé.”

Table 9.3 The polysign gesture

Speech	[left fist lifts up] it was such a, a closely guarded secret
Gestural signifiers	Configuration: Fist	Movement: Upward	>	Raised fist
Analogical links	Enclosure	More is up
Gestural signifieds	Enclosed object	Increasing exclamation	>	Very well enclosed
Gestural signs	Gestural sign 1	Gestural sign 2	>	A polysign gesture

Furthermore, more than one sign may be conveyed by just one component, for example, by the type of movement (straight or curved) and by its direction. In example (5), loops, signifying “the course of time,” in combination with their direction toward the speaker, signifying “self-centering,” make the idea of “introspection during action” explicit.

1. 5 Donc on a cette cette double mission de de le faire [the index fingers, facing each other, draw two parallel series of loops going toward the speaker whilst the torso moves backward] et de nous regarder le faisant [smile].

   Therefore we have this this twofold mission to to do it [the index fingers, facing each other, draw two parallel series of loops going toward the speaker whilst the torso moves backward] and to watch ourselves doing it [smile].

If we reconsider the gesture variants that signify “cut,” we can see that each variant, nuanced by the plane of cutting, functions as a polysign: cut + division, cut + obstacle, and cut + totality.

Note that the signifying combination of a polysemous configuration (e.g. the Fist) and a polysemous movement (e.g. forward) is apt to produce a large number of varied polysigns. Forward is polysemous: one moves forward to go toward someone/something/somewhere or to counter an opposing force (two analogical links). Moreover, the former link is subject to semantic derivation: spatial or temporal progression. So, according to the context, a fist moving forward can signify: psychological strength + toward something, hence, the effort required to reach for a goal; psychological strength + against something, hence, the will to attack; or strength + temporal progression, hence, strength and even modernism.

Complex gesture.

Finally, the complex gesture is a particular case of the polysign gesture: it performs the respective figurations of two known gestures by creating a third one that synthesizes them in an original way. For example, in order to simultaneously represent mixing, usually signified by the two hands turning around each other (Figure 9.8.1) + approximation, usually signified by a rotational oscillation of the concave palm facing downward (Figure 9.8.2), a screenwriter (example [6]) and a philosopher (example [7]) perform the same synthesis (i.e. an alternating oscillation of the two concave palms, one behind the other, as if they were interlocked*) to express a “kind of confusion” and an “approximate mixture,” respectively.

1. 6 The screenwriter says:

   *l’espèce de désarroi dans lequel se trouvent les hommes et les femmes maintenant.

   *the sort of disarray (“kind of confusion”) in which men and women find themselves now.

1. 7 Here it is a question of belief:

   Je n’aime pas beaucoup *le judéo-chrétien.

   I don’t like very much *the [term] Judeo-Christian (“this approximate mixture”).

Figure 9.8 A complex polysign gesture

Here are some examples of a complex head gesture: an abrupt lateral lowering of the head*, combining a movement downward and sideways, is often a way of emphasizing something negative, for example, a way of reacting to the exaggeration of others: “C’est maintenant que tu arrives? (You’re coming now?) *Pas trop tôt! (*Not too early!)” or to strongly disagree: “En 74? (in 74?) *Ca m’étonnerait! (*That would surprise me!).” This synthetic movement (downward and sideways) cumulates the motivations corresponding to each of the directions: pressing downward (insistence) and turning away from (negative element).Footnote ⁵

2.2.3 Polysemy and Variation

In order to elucidate the polysemy of a gesture, it is sufficient to research the gesture variants of each signified in order to identify the analogical link inherent to it. For example, in France, the lateral shaking of the head is polysemous: as an emblem, it signifies “negation,” and, as a co-speech gesture, it signifies “totality” or “approximation” (Figure 9.1). However, each of these notions is expressed by a gesture variant with a single movement: the lateral shake repeats the lateral turning away from one side to the other (negation), repeats the transverse movement (totality), or repeatedly sweeps the gap between the two designated boundaries on the right and left of the head (approximation). The polysemy of the lateral shaking of the head is elucidated by the single-movement gesture variant of each notion.

Figure 9.9.1 Polysemy

Figure 9.9.2 Variation

Research and confirmation of the analogical link.

In Source 1, the co-speech gestures that refer to the notion of totality are characterized by:

• – a transverse movement of one or two Level Hand(s);

• – a transverse movement of the head;

• – the lateral shaking of the head.

What is the analogical relation between the transverse movement of the head, hand, or fingers and the notion of totality? Logically, it is a natural reference to the horizon signifying “everything, everywhere” (Figure 9.2), concretely represented by a glance that sweeps across the horizon (in a single or repeated movement of the head) or by the palm that glides across it with a transverse movement (symmetrical or not). Transverse movement, the element common to the gesture variants, is the relevant element that can be linked to the common notion. By scanning the entire perceptible field from left to right, it represents the perceptual schema of totality and becomes the gestural signifier of the notion of totality. This is what we see in the following sequence:

1. 8 Je serai un opposant, mais résolu [thumb and index finger form the Ring configuration (representing “precision,” “rigor”)], j’allais presque dire systématique contre la proportionnelle, et ce [transverse movement, (representing “totality”)] à toutes les élections

   I will be an opponent, but [a] resolute [thumb and index finger form the Ring configuration (representing “precision,” “rigor”)], I was almost going to say [a] systematic [one] against the proportional system, and this [transverse movement, (representing “totality”)] at all the elections

The transverse movement is performed by the Ring configuration because the rigorous attitude of the speaker concerns all elections. The gestural representation of the idea is thus maintained for as long as the idea lasts (cf. the concept of catchment in Reference McNeill, Quek, McCullough, Duncan, Furuyama, Bryll and AnsariMcNeill et al., 2001, p. 11).

Generalizing from this example, one identifies the analogical links in gestural signs by comparing gestures on the paradigmatic (vertical) axis of substitution and on the syntagmatic (horizontal) axis of combination (Figure 9.10).

Figure 9.10 The analogical link

To conclude this section on Calbris’ approach to analyzing the gestural sign, one can say that the elements of the “symbolic Meccano” game (or “symbolic construction kit”) are the analogical links (Reference CalbrisCalbris, 1987, Reference Calbris2008a). Physicosemantic classification reveals that the polysemy of a gesture is not explained by a simple semantic derivation, but that the gesture that represents several notions is apt to contain several analogical links, alternatively or simultaneously “activated” by the context. There is no correlation between the number of analogical links and the number of kinesic entities. Indeed, one observes that one single link is expressed by the same movement of the hand and the head in a facial-gestural ensemble; conversely, one manual configuration may express several links. The relevance is not in the physical element but in the analogical link it contains.Footnote ⁶

3.1 Utterance Functions

As shown in Table 9.4, the three communication channels are associated differently to serve different functions: expressive, appellative,Footnote ⁷ referential (Reference BühlerBühler, 1934), and phatic (Reference Jakobson and SebeokJakobson, 1960). In terms of communication, emblems and co-speech gestures can serve all of these functions. When accompanying speech, gesture also serves utterance functions connected with speech rhythm (a demarcative function) and speech formulation (a predictive function). Gesture thus serves several functions and can simultaneously cumulate them (Reference CalbrisCalbris, 1997).

3.2 Alternation and Synchronicity

Gesture and speech may alternate within an utterance in a form of mixed syntax (cf. Reference Slama-CazacuSlama-Cazacu 1977, pp. 118–122). Regardless of whether it occurs before or after speech, a gesture may function as if it were a word or other unit of linguistic expression, hence, as a component of syntax in the construction of the utterance. In this case, gesture substitutes for speech. The speaker can thus exploit it as an implicit, expressive, synthetic sign, for example, to show diplomacy, increase expressiveness, or make the interlocutor wait for what is to be formulated verbally.

When gesture and speech occur simultaneously, the gestural expression and the co-occurring verbal expression may be semantically related in two different ways: (1) as a coverbal sign, gestural expression augments what is being expressed in words, that is, the respective information units are synchronized, or (2) as a preverbal sign, gestural expression anticipates the verbal expression of an idea, that is, there is a synchronization of information units but an asynchronous distribution of the information they convey.

Co-verbal sign.

A complementary piece of information supplied by a co-speech gesture can:

• – indicate the speaker’s attitude or comment on what s/he is saying in relation to the object of the utterance (Reference CalbrisCalbris, 2011, pp. 250–252);

• – explain something to the interlocutor with a pedagogical aim in mind, for example, giving the visual representation of a spiral quickly helps to make it clear what a spiral staircase is (Reference CalbrisCalbris, 2011, pp. 253–254);

• – disambiguate a polysemous word in the verbal message, for example, the French word “régulier” can mean regular, cyclical, uniform, or in accordance with regulations. Accompanying the phrase “c’est pas régulier” (it’s not …), the circular and repetitive movement of the hand drawing vertical loops specifies that “cyclical” is the intended meaning (Reference CalbrisCalbris, 2011, pp. 254–255).

It also happens that the simultaneous confrontation of gestural and verbal information that reciprocally specify each other results in conventional gestures and locutions acquiring a new, original, contextual meaning. Such individual creativity with socially established metaphorical expressions is demonstrated by example (9), produced by a politician who was in the Opposition at the time of the interview. He is complaining that promises have been made that something will be done but, according to the speaker’s media searches, so far nothing has happened:

1. 9 [the left index finger inclines the ear pinna forward as if to hear better] Et depuis un an, [then the left hand forms the Palm Forward configuration] j’attends les actes: [both hands transformed into the Ring configuration, side by side, symmetrically draw a transverse line] je n’ai rien trouvé.

   [the left index finger inclines the ear pinna forward as if to hear better] And for a year, [then the left hand forms the Palm Forward configuration] I’ve been waiting for actions: [both hands transformed into the Ring configuration, side by side, symmetrically draw a transverse line] I’ve found nothing.

Figure 9.11 details the process of constructing the meaning of the message, in which each of the three gestures that cut the sentence into rhythmic-semantic groups conveys a complementary piece of information. Derived from action, the gesture of placing the index finger behind the ear and pushing the pinna forward, when seen out of context, means something like “to hear better.” In co-occurrence with the phrase “And for a year” its meaning changes to suggest the idea of a watchdog with his ears pricked up and thus to refer to the figurative French locution “tendre l’oreille” (to prick up one’s ears).

Figure 9.11 Interactive construction of meaning by the two types of sign

The Palm Forward gesture that follows, which may be derived from a self-protective reflex, is essentially a defensive stop. In this context, it signifies that the person has been waiting on standby, immobile, for a year.

The last gesture is a polysign: the Ring gesture used to grasp a very small object refers to something minuscule and precise. As discussed above, transverse movement sweeping the horizon can signify “everywhere.” Out of context, the gestural meanings “minuscule” and “everywhere” combined with the verbal context “I’ve found nothing” become, respectively, “not the least little thing” and “nowhere.”

The interaction between the gestural meaning in context and the meaning of the verbal message produce the total verbal-gestural utterance: “And for a year” I’ve been pricking up my ears, immobile “I’ve been waiting for actions: I’ve found absolutely and strictly nothing.” The gestural contribution confirms the implied metaphor of the alert and motionless watchdog. The referential function is performed by the two channels in interaction. The semantic interaction of gesture and speech reported by Kendon (cf. Reference KendonKendon, 2004, pp. 158–175) in situations referring to the concrete is developed here in the abstract domain.

Preverbal sign.

Diverse examples across the corpus provide Calbris with instances of gesture anticipating speech, whereby there is an asynchronous distribution of information across synchronous kinesic units of gesture and prosodic units of speech. Reference CalbrisCalbris (1998) hypothesizes a relation between the gestural anticipation of speech and the type of mental images involved (Table 9.5). The reasoning is based on a possible correlation between the different types of image and the difficulty of encoding them verbally:

• – An original mental image, being initially quite difficult to encode verbally (V), would be formulated gesturally (G) so that its presence in the mind can be maintained concretely for the duration of the verbal formulation. For a fairly uncommon image (i.e. a metaphoric extension or a particular mental image whose verbalization is not automatically available), anticipatory gesturalization would serve as an aid to verbalization (left column).

• – Inversely, in the case of a known, commonly shared mental image transformed into a figurative locution, the initially unsynchronized association between gesturalization and verbalization would be readily transformed into a synchronous ensemble due to its frequency of use (central column).

• – If a speaker-gesturer is not conscious of the analogical link contained in a figurative locution or its gestural representation that s/he is producing, then habit and automaticity would engender synchronous transmission of the analogous bimodal information (right column).

Table 9.5 Relation between the gestural anticipation of speech and the type of mental imagery

G –> V	V|G	V G
Anticipation of gesture for original images	Simultaneity of gesture for common images	Non-consciousness of simultaneous gesture

Gestural information conveyed during speech production often precedes analogous verbal information (example [4] showed a preverbal polysign); it gives an idea of the content to be put into words. As a preverbal sign, it can:

• – summarize the whole content of the utterance that follows and thus function as a gesture-title (Reference CalbrisCalbris, 2011, pp. 257–258);

• – help the speaker to find an appropriate formulation for what s/he wants to say and thus function as aid to verbalization (Reference CalbrisCalbris, 2011, pp. 258–263);

• – create an interplay between tension and relaxation in communicating information for the interlocutor, who is led to guess what will be said in an intellectual game of switching between the two information channels.

Example (10) produced by the former French prime minister Lionel Jospin shows gestural information preceding each piece of verbal information in an astonishing way. Is this to emphasize his viewpoint right from the very start?

1. 10

   [Fingers pointing to the chest] L’euro | [palm opened outward] pour moi | [and moved forward]: c’est un instrument de puissance.
   [Fingers pointing to the chest] The euro | [palm opened outward] for me | [and moved forward]: it’s an instrument of power.
   Gestural information:	For me		it’s		it’s that:
   Verbal information:		The euro,		for me,		it’s an instrument of power.

Let us now consider example (11) showing how gesture can create an interplay between tension and relaxation in the bimodal information that is progressively communicated. It was produced by a technician being interviewed about construction work at the Musée d’Orsay, which he metaphorically refers to as a boat on the banks of the River Seine:

1. 11 Ces micropieux sont ancrés dans le calcaire que l’on retrouve sous le musée pour empêcher le bateau de se soulever avec la montée des crues.

   These micropiles are anchored in the limestone that one finds under the museum in order to prevent the boat from rising with the rise of flood waters.

Stripped of its nouns (micropiles; limestone; museum; boat; flood waters), the message is reduced to a skeleton that gesture articulates: “That anchored in that under that raised by that.” Table 9.6 shows gestures representing: (1) the micropiles, by the thumb and the index finger directed outward like hooks, (2) where they provide anchorage, by lowering the palms facing downward, and (3) rising movement, by raising the palms facing upward. A change of gesture (1, 2, 3) initiates a new rhythmic-semantic group, and the modulated repetition of each gesture segments each group into subgroups of meaning (A, B, C):

Table 9.6 Interplay of tension-relaxation between gestural and verbal information

A	B		C
Ces micropieux These micropiles (1) hooks	sont ancrés are anchored hooked in		dans le calcaire, in the limestone hooked in
que l’on retrouve that one finds (2) under	sous under under		le musée, the museum –
pour empêcher in order to prevent (3) rising	le bateau the boat	de se soulever avec from rising with rising	la montée des crues the rise of flood waters –

A, B, C: rhythmic-semantic subgroups

1, 2, 3: each new type of gesture, accompanied by its contextual meaning

Underlined: gestural information anticipating verbal information

The gestural information conveyed in A anticipates and complements the verbal information conveyed in B and C. For example, Figure 9.12 shows that the visuokinesic information conveyed in (3) is later confirmed by the verbal information. This bimodal structuring of the information flow across the whole sequence creates tension and relaxation as it progresses.

Figure 9.12 Tension-relaxation in the bimodal information progressively communicated

Finally, let us consider example (12) in which gesture is an anticipatory and complementary synthesis of the utterance (Reference CalbrisCalbris, 2011, pp. 303–312). Sitting at his desk, a management specialist is explaining how to design a project in order to achieve an objective:

1. 12 “On part d’un monde | d’hier | dans lequel on entassait les connaissances.”

   “One departs from a world | of yesterday | in which one accumulated knowledge.”

He sums up his previous worldview in a single, rapid, transverse movement to the right along the edge of the desk while saying “of yesterday” (Figure 9.13.2). For ergonomic and semantic reasons, he chooses the transverse axis. This allows him to represent: (1) anteriority on the left and two developments that are intimately connected here: (2) the temporal progression from left to right and (3) the cumulative progression of knowledge. Furthermore, (4) the rapidity of the movement of the back of the hand shifting to the right implies that this world is over, and (5) the gesture is performed along the edge of the table in front of his torso, indicating “here and now”: we are in the present. Suddenly motionless and gazing straight into the camera, he says “in which one accumulated knowledge,” and thus verbalizes the information that has already been gesturalized (Figure 9.13.3). The gesture testifies to the cognitive activity that programs the two-channel communication of the idea. The multireferentiality enabled through the gestural channel is confirmed through the verbal channel (Reference Calbris, Rector, Poggi and TrigoCalbris, 2003b). Figure 9.14 gives a visual synthesis of the cases in which gesture anticipates speech.

Figure 9.13.1 On part d’un monde “One departs from a world”

Figure 9.13.2 d’hier “of yesterday”

Figure 9.13.3 dans lequel on entassait les connaissances “in which one accumulated knowledge”

Figure 9.14 Temporal perception of the gestural-verbal flow

4.1 Gestural Sign and Perceptual Schema

If we consider the ACTION schema “subject-action-object-result,” the result of the action seems to be the intermediary element between the concrete and the abstract expressed by co-speech gesture:

1. 1. A gesture that accompanies the verbal evocation of a concrete action often shows its result, that is, the idea resulting from the action (the written trace and not the typing on the keyboard). The gesture represents an abstraction made out of the concrete – the result of the concrete action. For example, Can you switch the light off? to “reduce” the light explains the concomitant gesture of lowering the palm.

2. 2. On the other hand, abstract notions derived from the perceived results of a cut are accompanied by a gesture that recalls the original action. The gesture reconcretizes the abstract – deduced from the results of the action. For example, a dichotomy occurs …, “a cut in two” explains the abrupt downward movement of the edge of the hand in the sagittal plane. In the same way, if we consider the OPENING schema “resulting openness-clarity and discovery,” the abstract action (clarification or revelation of content) derived from the concrete result is spontaneously reconcretized by an opening gesture, the opening of configurations with one or both hands (Reference CalbrisCalbris, 2003c), as exemplified in Figure 9.16.

The analysis of gesture reveals that the abstract object is conceived in the image of the physical object, perceptually defined by its contour and shape that determine its mode of prehension. One could call it representational deixis. Indeed, while the speaker is using the index finger to indicate* the abstract object uttered by the interlocutor by saying “*that’s it!”, s/he is positioning and giving shape to the abstract object s/he is thinking about. This is conceived as: a distinct entity (encircled, enclosed by the grasping hand); a definite object (framed by the sagittally held palms); a condensed object (constricted in the hand closed in a Pyramid configuration); a precise object (pointed and able to be grasped by the first three fingers joined together); an extremely precise object (to be grasped between the fingernail pincers of the thumb and the index finger). The speaker reifies the abstract object s/he designates (Reference Calbris, Müller and PosnerCalbris, 2004).

Figure 9.16 Opening a hand configuration to explain something

Whatever the type of abstract object the speaker has in mind, s/he opens it at the moment of giving its content verbally (cf. Reference CalbrisCalbris, 2011, pp. 331–342). In Figure 9.16.1, the object in question is spontaneously delimited by the palms because it is mentally defined, as indicated by the concurrent use of the French demonstrative “ces” (these). In Figure 9.16.2, the object in question is considered essential: an essential value which is … hence the choice of the hand closed in the Pyramid configuration to signify a condensation of something. This OPENING schema is also found in Italy: “the grappolo-to-open is associated with the comment … ” (Reference KendonKendon, 2004, p. 234).

4.2 Multiplicity of Perceptual Schemas Gesturally Represented on the Transverse Axis

The variety of gestural signs expressed on the transverse axis alone (Table 9.7) proves how much our mental world is shaped by our perceptual-motor experiences (Reference CalbrisCalbris, 2002a).

The three axes – vertical from bottom to top, sagittal from back to front, and transverse from left to right – point in directions according to our perceptual experiences of the process of physical growth upward, the direction of walking forward, and the direction of writing to the right in the West, respectively. For ergonomic reasons, there is a permutation observed between the axes of growth and progression toward the transverse axis, which cumulates their respective symbolic values that are easier to localize on this axis. In addition, the transverse axis reflects another perceptual schema, that of the symmetry in all living bodies (plant, animal, and human) divided into two halves that are equivalent, opposable, complementary, and essential for balance. It makes it possible to create oppositions by localizing symbolic values on the left and on the right, respectively: the bottom and the top, below and above, the lower and the higher, less and more, anteriority and posteriority, the old and the new, the cause and the consequence. In accordance with the symmetrical development of any living body, the comparison of two evolutionary processes is made by the left hand to the left, and by the right hand to the right. Another schema that impresses itself on the mind is our bipedal locomotion toward a goal, which makes us move forward symmetrically (dichotomous enumeration) along a path from which one deviates only momentarily to take a glimpse to one side (digression), indifferently located on the right or on the left. However, an aside, if conceived as a secondary element, is placed on the left. On this axis divided into two halves, the equivalence of two similar entities disappears when it is a question of otherness. The self takes precedence over others. The notion of self (extendable to the group or country to which one belongs) is naturally (logically) expressed on the right for a right-handed person (Reference CalbrisCalbris, 2003c). Last but not least, this axis is the only one that can represent our maximal field of vision of 180° to signify “totality.”

4.3 Multiplicity of Notions Derived from the Schema of Cutting

The schema of cutting appears to be traced in numerous and various notions: SEPARATION, CUTTING INTO PARTS, DIVISION INTO TWO HALVES, BLOCKING, REFUSAL, NEGATION, END, STOP, DECISION, DETERMINATION, MEASUREMENT, CATEGORIZATION, CATEGORICAL CHARACTER, and INTERRUPTION. The action of making a clean cut involves the characteristics of the tool (1) and of the act (2), as well as of its result (3): first of all the hardness and rigidity of the tool (1), then the force and brevity of the strike (2), the impact of the cutting edge against a material object, and finally, the notching and the separation, the production of pieces, and the irreversible nature of the act (3). The schema of cutting is a visual and proprioceptive percept. The meaning is already in the gesture, in the physical feeling of the gesture performed. The understanding of the gesture is physical. Drawing on perception, the semantic extension is already based on the physical and temporal metonymy supported by the act itself, linking the tool, the action, and its result. The gesture thus serves the metaphorical transfer pertaining to each component: instrument, act, and result (Reference CalbrisCalbris, 2003a).

It is important to note that the permanent gestural reactualization of the various foundational perceptual schemas of abstraction at the core of spoken discourse is the most flagrant proof of the physical anchoring of cognition. The premises of the concrete process of abstraction, observed in young children and called “Mimisme” by Reference JousseJousse (1936), seem to be corroborated by the discovery of mirror neurons in humans (Reference Rizzolatti, Fogassi and GalleseRizzolatti, Fogassi, & Gallese, 2001).

4.4 Summary

The analysis of gestural signs referring to the abstract reveals traces of the perceptual schemas that underlie them: our perceptual experiences are bearers of meaning at the origin of our mental constructions, and gesture is the witness of this mental imagery at work during utterance. If a gesture represents the visible form of an idea (i.e. the sketch of a concept), it is to be expected that the gestural formulation anticipates and probably facilitates the verbal formulation of the concept. The gestural sign’s anticipation of the verbal sign is the logical correlate of the gesture’s semiotic specificity.

The gestural expression of a notion anticipating its verbal expression during the utterance process is for Reference FónagyFónagy (2001, pp. 580–581) just the ultimate extension of the essentially preverbal character of gesture that one recognizes in the ontogenesis of language and which may possibly stem from its phylogenetic source.

5.1 Research Results Related to Each Source

1. Source 1. By comparing the physical and semantic data in the database of co-speech gestures referring to the concrete or to the abstract, it was possible to bring to light: (1) at the physical level, the numerous gesture variants of a notion, be they stylistic or semantic variants, (2) at the notional level, the possibility of several gestural signs within the same gesture, be they signs alternatively revealed by the verbal context (polysemous gesture) or simultaneously cumulated (polysign gesture and complex gesture), (3) the discovery of the gestural sign thanks to the gesture variants of a notion, their common physical element being the signifier linked to the gestural signified, and (4) that as a stylized representation of the concrete and a concrete representation of the abstract, gesture is an intermediary between the concrete and the abstract.

2. Source 2. Facilitated by the software of a prototype videodisc, the meticulous segmentation of the various filmed utterances (first acoustic, then visual by identifying co-occurrent kinesic changes at different levels of the body) and the subsequent identification of verbal and gestural units and the respective information they conveyed enabled their interaction to be studied. An in-depth analysis revealed: (1) complementarity as well as reciprocal disambiguation in the case of concomitant information, (2) the simultaneous multireferentiality of co-speech gesture (polysign gesture), (3) gesture anticipates speech in the spontaneous expression of an original idea to be formulated: In concurrence with speech, the preverbal gesture that visually concretizes an idea maintains its presence in the mind and thus facilitates the verbalization of an idea whose confirmation the interlocutor is waiting for, and (4) gesture is never superfluous: It expresses, animates, explains, synthesizes information, and anticipates speech; it participates and creates participation in the utterance.

3. Source 3. The statistical analysis of certain recurrent gestures performed during the six television interviews of the former prime minister Lionel Jospin made it possible to show how a frequently occurring gesture potentially reveals the unconscious: (1) the split personality of a person who systematically expresses himself with the left hand as the leader of the Left and with the right hand as a private person, (2) his attitude toward his new function is even reflected in his choice of gesture variant for referring to an abstract object; and (3) gesture reveals the mental imagery underlying utterance (Reference CalbrisCalbris, 2003c; for a review, see Reference De ChanayDe Chanay, 2005; for a video presentation in French, see Reference CalbrisCalbris, 2000 [www.canal-u.tv/intervenants/calbris-genevieve-028605403]).

5.2 Application of the Research to Language Teaching

Based on this research, and since 1975, Calbris has produced pedagogical works in collaboration with Jacques Montredon, pedagogue and linguist, to integrate multichannel communication into the teaching of French as a Foreign Language so that students can physically and intellectually assimilate French prosody (Reference Calbris and MontredonCalbris & Montredon, 1975), voco-kinesic expression (Reference Calbris and MontredonCalbris & Montredon, 1980), gestural expression of notions (Reference Calbris and MontredonCalbris & Montredon, 1986), figurative expression of the language (Reference Calbris and MontredonCalbris & Montredon, 1992), and the semantics of verbal connectors in conversational argumentation (Reference CalbrisCalbris, 2002b; Reference Calbris and MontredonCalbris & Montredon, 2011).

5.3 Position in Relation to Other Researchers

Calbris’ (1983) doctoral dissertation “was one of the first studies ever to attempt a systematic analysis of the ways in which [co-speech gesture] can serve as a vehicle for the representation of conceptual meaning” (Kendon in Reference CalbrisCalbris, 2011, p. xvii). Her work is closely related to the pioneering work of Reference Lakoff and JohnsonGeorge Lakoff and Mark Johnson (1980) on metaphors that draw upon our visual and physical experience of our bodies and of the physical world in which we live. She holds that gesture extracts symbolic material from our physical interactions with the world we inhabit. For her, gesture is situated at the interface between mind and matter and reproduces perceptual schemas (cf. embodied schemata and image schema, Reference JohnsonJohnson, 1987) that underlie verbalization. Hence, her research corroborates Johnson’s theory of “bio-functional embodied understanding” (Reference Johnson2015, p. 7) gained through organism–environment interactions.

Adam Kendon acknowledges that Calbris’ analytic approach partly inspired his concept of gesture families (Reference KendonKendon, 2004; cf. Kendon in Reference CalbrisCalbris, 2011, p. xviii), that is, gestures that share the same configuration and orientation but may differ in respect of movement. Nevertheless, Calbris contends that Kendon overlooks the possibility of the simultaneous plural motivation of gesture that she has found in her corpora (Reference CalbrisCalbris, 2011, p. 283).

Calbris was one of the first researchers to demonstrate and hypothesize about the now widespread observation that gesture often anticipates speech in encoding allied information. Her finding that gesture may aid verbalization concurs with David McNeill’s concept of catchment, that is “recognized from recurrences of gesture form features over a stretch of discourse” (Reference McNeill, Quek, McCullough, Duncan, Furuyama, Bryll and AnsariMcNeill et al., 2001, p. 11), by showing that gestural representation can indeed maintain the presence of an idea over a stretch of discourse. Her investigation of this phenomenon takes into account the possible physical, cognitive, and communicative reasons that may underlie it as well as the perspectives of both communication partners. Her semiological theory of gesture thus interconnects cognitive and communication models of language and furthermore accords with Raymond Gibbs’ theory of “the social nature of embodied, contextually embedded cognition” (Reference GibbsGibbs, 2011, p. 81).