2.1 Infusion of Energy

The growth point is so named because it is a distillation of a growth process – an ontogenetic-like process but vastly sped up and made functional in online thinking-for/while-speaking (Reference Slobin, Aske, Beery, Michaelis and FilipSlobin, 1987). It is important to emphasize again that langue does not explain how we speak. It explains speech only when unified with gesture. The gesture brings langue to life. According to this framework, the growth point is the initial pulse of thinking-for/while-speaking, out of which utterance and discourse emerge. Imagery and spoken form are mutually influencing. It is not that gesture is the input to spoken form or spoken form is the input to gesture. The growth point is fundamentally both.

Efforts to separate gesture and linguistic form therefore fail. They are tightly bound – either they remain together or they are jointly interfered with; in either case the speech–gesture bond is unbroken. They are held together by the requirements of idea unit formation, stronger than association or convention: Thought itself is the glue. To think while speaking is to be active in both modes at once. The following are illustrations:

• Delayed auditory feedback – the experience of hearing your own voice played back after a short delay – produces major speech disturbances but does not break speech–gesture synchrony (Reference McNeillMcNeill, 1992).

• Stuttering or disruption of speech and gesture are incompatible. The onset of a gesture inoculates against stuttering and, conversely, once a gesture is ongoing, onset of stuttering stops the gesture immediately (Reference Mayberry, Jaques and McNeillMayberry & Jaques, 2000).

• People blind from birth, who have never seen gestures and have no benefit from experiencing them in others, gesture and do so even to other blind people whom they know are blind (Reference Iverson and Goldin-MeadowIverson & Goldin-Meadow, 1997).

• People born without arms “gesture” as they speak; that is, have the neurological feeling they gesture with full significance (Reference Ramachandran and BlakesleeRamachandran & Blakeslee, 1998).

Figure 18.2 shows a gesture–speech unit taking form during a narration. The speaker had just watched a cartoon and was recounting it to a listener from memory.Footnote ⁴ We had presented the task as story-telling and did not mention gesture. The speaker was describing an event in which one character (Sylvester) attempted to reach another character (Tweety) by climbing inside a drainpipe. To thwart Sylvester, Tweety had dropped a bowling ball into the pipe. The gesture-speech unit consisted of a gesture showing a moving entity (the bowling ball), its path (downward) and the landmark through which it passed (the pipe), synchronized in a single multimodal package with “[/ and it goes down].”Footnote ⁵ The core idea was something like “falling hollowness” – an idea that does not exist outside growth points, according to Reference TalmyTalmy (2000).

Figure 18.2 Speech-synchronized gesture

If we ask (as in the title of Reference McNeillMcNeill, 2016), “why we gesture,” the answer is not that when speech stops we “talk with our hands.” Reference Gullberg, Mattsson and NorrbyGullberg (2013) debunks this ancient idea – “when speech stops, gesture stops too.” Also, it is not that we gesture because we speak. While that may happen, the growth point offers another reason: Gesture is a template for speech and we speak because we gesture. Gesture orchestrates speech (not as a conductor or agent, not in a causal relation, but speech and gesture form a “harmonious unity” based on the gesture and its temporal, kinesic effects. Reference EfronEfron (1941) described a gentleman in an Eastern European neighborhood of New York City complaining that if you laid a hand on his arm (as was common there) he could not speak. It may have been gesture-orchestrated speech the restraining hand prevented.

****

The first property to describe is an imagery–language dialectic. A dialectic adds energy, propels thought and speech forward, and raises langue to the dynamic. It is a style of cognition special to language. Vygotsky described the process: “The relationship of thought to word is not a thing but a process, a movement from thought to word and from word to thought. […] This flow of thought is realized as an internal movement through several planes […] ” (Reference Vygotsky, Hanfmann and Vakar1986, p. 218). A growth point synchronizes a gesture (the “thesis”) with coexpressive speech (the “antithesis”). Coexpressivity means that speech and gesture convey the same meaning in different semiotic modes. The semiotic difference adds energy. The unpacking (the “synthesis”) spreads the energy into speech. The resulting sentence is still langue but is now inhabited dynamically.

The dialectic is not sequential: the thesis, antithesis, synthesis “steps” are not steps but are the logical aspects of the growth point and are all at once. There is an actual time of generation. Imagery and language-forms in a growth point can be activated just partly and parts of them not used. Such fragments do not vanish. They appear in later cycles and may have a long-range effect on the cohesiveness of discourse.

The dialectic, finally, has a built-in stop-order – the speaker’s intuitions of well-formedness tell her she has attained dialectic synthesis. The ending is also a new beginning, the growth point’s generative energy immediately launching the next growth point in a cycle of energy/repose followed by new energy/repose, and this repeating with fresh meanings as long as cohesion remains and the speaker continues speaking.

****

A dialectic can occur because the growth point has a unique semiotic mode: “dual semiosis.” It is a growth point inhabiting meaning both as imagery and as language at the same time, a global/synthetic/non-combinatoric semiotic face-to-face with a coexpressive analytic/segmented/combinatoric semiotic (Table 18.1). Coexpressivity unifies them. Dual semiosis implies that meanings are always cast in contrasting semiotic modes at points of coexpressivity. The term “coexpressivity” and its variants mean that gesture and the speech synchronized with it express the same meaning or idea. This is a basic property of the growth point from which other properties like the dialectic arise. Even if gesture and speech convey the same content, they remain semiotic opposites.

The following apply to Table 18.1’s semiotic contrasts:

• By “global,” I mean that the gesture’s parts (= the hands/fingers/trajectory/space/orientation/movement details) have meanings dependent upon the meaning of the gesture as a whole. Only the gesture’s downward motion has independent meaning. It means downward, but this is all, and it is not enough to generate “falling hollowness.” The parts as they reside in the gesture do not have meanings of their own, and the meaning of the whole is not composed out of the parts: rather, significance flows down, from whole to parts. The linguistic semiotic is the opposite. The meaning of the whole (a sentence or other unit) is composed out of its parts, which then must, and do, have their own meanings.

• By “synthetic,” I mean that meanings are synthesized into one symbolic form (the “falling hollowness” hand) (synthesis was described by Reference Werner and KaplanWerner & Kaplan, 1963). In the companion speech, the meaning is analytic, separated into elements spread over the sentence (“goes” + “down”).

• The third semiotic contrast is combinatoric potential; linguistic forms possess it, gesture imagery does not, and both exist within the growth point. When linguistic forms combine, new syntagmatic values and new units emerge. Even a single word has the property (hence, labeled “potential”). When gestures combine, they add imagery but not syntagmatic value or new gestures to contain them – part of the semiotic difference.

****

Context is the next topic. It is often regarded as a kind of encyclopedia that the speaker “consults,” “guiding” and “constraining” meaning. That is not the conception of context here. For a growth point, context is not passive; it is active and absorbed into the growth point (cf. Reference Goodwin, Duranti, Duranti and GoodwinGoodwin & Duranti, 1992, for a similar proposal). An implication of the absorption is that one meaning is two things. It is (1) a point of differentiation and (2) the immediate context, a field of meaningful equivalents, that it differentiates: “One meaning” is both. The growth point absorbs the context into its very existence. This makes the growth points inherently dynamic. Merely intending or recalling or associating something is not enough. The field of meaningful equivalents must also be included. This conception differs from familiar “one-thing” conceptions such as “signified content,” “association,” and so on that regard meaning without absorbing context.

A growth point “absorbing” the immediate context of speaking activates Vygotsky’s “psychological predicate” (not a grammatical predicate). One of Vygotsky’s examples is a crashing clock (Reference Vygotsky, Hanfmann and Vakar1986, p. 220): There is a crash in the next room – someone asks: “what fell?” (the answer: “the clock”), or: “what happened to the clock?” (“it fell”). Depending on the context – here crystallized in the questions – the newsworthy reply (the psychological predicate) highlights different elements. In forming a growth point, the speaker shapes the context in a certain way to highlight the intended differentiation within it while fulfilling a “two-things” meaning, much as the questioner about the falling clock shaped the context of the replies. This logic applies to the psychological predicate:

• It is newsworthy in its context.

• It implies and, if need be, shapes the context to ensure its newsworthiness and differentiation.

Examples suggest that growth points come in great variety, but as inferred from coexpressive speech/gesture synchrony, a common property runs through them all: psychological predicates are the local maxima of newsworthiness (equally, the point highest in communicative dynamism; Reference FirbasFirbas, 1971).

Figure 18.3 illustrates a different speaker describing Sylvester climbing the pipe on the inside, the event in the cartoon immediately preceding the Figure 18.3 downward event the first speaker described. The psychological predicate is the core idea of rising and interiority in “rising hollowness.” The speaker said, “he goe[s / up / th][rough the pipe] this time,” and with “up through” her hand rose and fingers spread outward to create an interior space. The newsworthiness of “rising hollowness” was the idea of being on the inside. The field of equivalents it differentiated was ways of climbing a pipe. Together, the field of equivalents and the differentiated idea were the growth point – ways of climbing a pipe: on the inside.

Figure 18.3 “Rising hollowness” with “he goes up through the pipe this time.” Computer art by Fey Parrill.

(Reference McNeillMcNeill, 2005, used with permission of University of Chicago Press)

The growth point, as a psychological predicate, means that the mechanism of growth point formation is the differentiation of a point of focus in a context or field of equivalents. A robust phenomenon is that the form and timing of a gesture select just those features that differentiate the psychological predicate in a context that is at least partly the speaker’s own creation (see Reference McNeillMcNeill, 2005, pp. 108–112).

Growth point differentiation is reflected in the very close temporal connection of gesture strokes with peaks of acoustic output in speech, which also highlight newsworthiness (Reference NobeNobe, 1996). Reference ParrillParrill (2008), using a method for cuing psychological predicates on demand, found growth points also arising. Their emergence was automatic. Cuing different psychological predicates evoked different sentence structures, behind which were different growth points which the clue also evoked. A flashing arrow, pointing at Sylvester’s rounded, bowling-ball-shaped bottom, resulted in more ball-subject utterances, compared to an arrow pointing at his head (a cat prompt) – for example, “the ball rolls him down the street,” rather than “he rolls down the street.” The gesture following the ball prompt also included manner – the hand rotating rather than moving in the straight path that was common after the cat prompt – a feature the prompt did not convey and was part of the automatic emergence. This spontaneous manner also suggests a growth point had formed, confirming that growth points and psychological predicates are the same thing.

To see the full genesis of the “up through the pipe,” we trace the psychological predicates leading up to and then within it, as the psychological predicate differentiates its field of equivalents:

1. (1.1) and so he tries to get in the building again

2. (1.2) [he crawls up a pipe]

3. (1.3) and when he gets up to the bird

4. (1.4) the grandma hits him over the head with the umbrella

5. (1.5) and he goes back

6. (1.6) and [he goes up through the pipe] this time

In (1.2), the newsworthy point was climb a drainpipe and the field of equivalents was something like ways of reaching Tweety. The gesture was the hand rising upward with a relaxed flattish palm; it depicted only the upward trajectory. The verbal choice of “crawls up” likewise encoded the upward trajectory and added the manner of movement. That the movement was on the outside of the pipe was the “obvious” and not newsworthy.

By (1.6), the “rising hollowness” example, on the inside differentiated ways of climbing a pipe, which itself was an updating of the (1.2) point of differentiation, climb a drainpipe (creating both a new field of equivalents and a cohesive link back to (1.2)). Like (1.2), it presented a rising hand, but the hand now formed the “rising hollowness” image, with its palm-up cupped shape. The novelty was the fact that the trajectory was on the inside as opposed to the outside, a contrast that the cohesive thread back to (1.2) made meaningful. Newsworthiness was embodied in the extension of the fingers – the newsworthiness adding energy. Speech also received energy in the stress on “through.” The words “this time” likewise indicated newsworthiness and the contrast to (1.2) but over a different route, to be described below.

But why speak of differentiation and fields of equivalents at all? Why isn’t reference enough? This has a straightforward appeal: The speaker remembers the next event (Sylvester climbs the pipe on the inside) and describes it. This would portray the speaking process as incremental and not contrastive.

I can give three arguments why this position does not explain gestures.

• We see the concurrent presence of the field of equivalents in examples like “up through the pipe” – what the gesture shows is not just a reference to going up the inside of the pipe, but a contrast to the previous outside ascent. This contrast was equally part of the gesture’s formation.

• Some gestures are pure contrasts without reference. For example, “[they’re sup]posed to be the good guys” (from a narration of the Hitchcock film “Blackmail”; see Reference McNeillMcNeill, 1992, Reference McNeill2012, Reference McNeill2015), with the hand indexing the left side of space, “[but she] really did kill him,” with the hand indexing the right side – the right-left opposition diagramming (but not illustrating) the concept of virtue-in-appearance vs. crime-in-reality.

• Reference is insufficient to explain the succession of growth points. In the (1.6) “up” example, the growth point included “this time” because its field of equivalents, ways of climbing a pipe, contrasted with the (1.2) field of equivalents, ways of reaching Tweety.

I stick with the overall plan. The sentence arises out of the growth point. The growth point retains its identity and dominates everything that occurs. It is not replaced by linguistic form but is the core around which this form rises. These factors are the invisible details of thinking-for/while-speaking that cannot be ignored. Utilizing the speech–gesture system, it is possible to expose the inner workings of this side of speech and infer the real-time genesis of thought and utterance.

****

Differentiation via psychological predicates is then one way that growth points absorb context. A second is via cohesive threads. We have already seen instances in the links of (1.6) to (1.2). The two modes of absorption convey contrasting new and old relationships to the context: psychological predicates conveying novelty, cohesive threads–connections to earlier growth points. Together, they anchor the growth point in the ever-changing waves of the discourse. Thanks to this cohesion, a growth point includes its current context and a web of earlier contexts to guide its unpacking (and insofar as anticipation is possible, future ones too). It is as if threads reach out to capture a feast of contexts, coherences, presuppositions, memories, and other fields of equivalents. Each moment of speaking, if we look at it in this way, is a synopsis of cohesion. It shapes the growth point, enlarges its world of meaning, becomes part of its field of equivalents and influences its differentiation. A conversation is mutual web-spinning in this fantasy, building a shared world of meaning. It also illustrates how growth points reflect the social exterior.

I have written about the examples in this section before (Reference McNeillMcNeill, 2012, Reference McNeill2016) but the connection to cohesion is new and receives exposition. Adding it deepens our portrayal. Growth points and cohesion work in tandem. Levy and I argued that gestures and referring expressions had similar discourse functions, cooperating in creating new themes and continuing old ones (Reference McNeillLevy & McNeill, 1992, Reference Levy and McNeill2015; Reference McNeill and LevyMcNeill & Levy, 1993), and so provide a basis for the production of the communicatively dynamic part of an utterance, this being the point of differentiation in a psychological predicate.

Broadly speaking, the threads are of two kinds. Some are imagery, called “catchments” – a phenomenon first noted by Kendon in 1972 (he did not use the term “catchment”); the second kind is built from them. The latter is often recycled from earlier gesture–speech dialectics and adds cohesion.

A catchment is when gesture features recur in at least two (not necessarily consecutive) gestures with the same thematic meaning. Catchments are a way of observing a speaker’s grouping patterns. Each catchment is the material carrier of the imagery which motivated it and a context for the growth point.

Convincing proof of the catchment as a psychologically real phenomenon comes from an ingenious test by Reference Furuyama, Sekine, Duncan, Cassell and LevyFuruyama and Sekine (2007). They noticed a systematic avoidance of gestures with a certain spatial content (involving a reversal of a predominant direction). The avoidance occurred precisely where this content, had it been included, would have disrupted an ongoing catchment. The catchment was the actual force and it blocked imagery that was referentially correct but inconsistent with it.

So, to define the catchment:

• A catchment is recognized from recurrences of gesture form features over a stretch of discourse.

• A catchment is a kind of thread of consistent dynamic visuospatial imagery running through the discourse and provides a gesture-based window into discourse cohesion.

• The logic of the catchment is that discourse themes produce gestures with recurring features; these recurrences give rise to the catchment.

• Thus, reasoning in reverse, the catchment offers clues to the cohesive linkages in the text with which it co-occurs.

Figure 18.4 is our example. Table 18.2 lists the catchments of Figure 18.4 and the cohesive threads they launched, as inferred from coexpressive gesture/speech synchrony. The Figure 18.4 Panel 2 growth point is our focus. It launches Catchment 2 and its metaphor thread, and absorbs five others: as I call them, the content, linguistic-pattern, accumulated-growth-points, shared-field-of-equivalents, and conversation/interaction threads, two of which were also part of Catchment 2 and all of which converged on Panel 2, and shaped the growth point and its field of equivalents.

Figure 18.4 Catchment view of the “it down” growth point (Panel 2). Transcription by S. Duncan. From Reference McNeillMcNeill (2016, Figure 4.2). Used with permission of Cambridge University Press

The growth point consisted of the downward thrusting gesture shown, synchronized with the words “it down.” Catchment 2 produced a field of equivalents in which the bowling ball appeared in the role of an antagonist. This set the bowling ball apart from its role in Catchment 3 where the significance was a spatial relationship, and the bowling ball was on a par with Sylvester. Coexpressively, the speaker construed the gesture-speech unit to mean not just releasing the bowling ball but also as a metaphor of conflict: The Good, experienced as the bowling ball going down vs. The Evil, experienced as Sylvester climbing up (outlandish metaphors are to be expected in cartoon narrations). How this came about requires looking at the sentence in a different way, not only as a verbal form but as a process of metaphor construction which the cohesive threads are shaping. The antagonistic-force metaphor, starting in Panel 2, remained until Panel 9.

bowling ball as antagonistic force: down (Panel 2);

: inside Sylvester (Panel 7);

: Sylvester into bowling alley (Panel 8)

: strike (Panel 9).

The spoken “it down” and the downward gesture were the cognitive core of the Panel 2 utterance – the “it” coexpressive with the bowling ball as an antagonistic force and the “down” coexpressive with the significant contrast in the field of equivalents that Catchment 2 embodied (the force heading down).

Other cohesive threads were spin-offs of earlier dialectics. For example, Panel 2 contrasts with the Catchment 1 growth-point accumulation thread. This began in a previous growth point (not illustrated), something like ways of getting at tweety: climb a pipe, which set the stage for the growth point at (1), the (unillustrated) previous psychological predicate (climb a pipe) becoming (1)’s and (2)’s field of equivalents: ways of climbing a pipe: on the inside. The previous growth points and the (1) growth point together thereby formed the Catchment 2’s shared-field-of-equivalents thread that led to Panel 2’s partial repetition of the sentence structure of Panel 1, a “poetic” replication (cf. Reference Jakobson and SebeokJakobson, 1960). The sentences match as closely as possible, given that “drops” is transitive while “going up” is intransitive:

The shared-field-of-equivalents thread launched two more – Catchment 2’s linguistic-pattern and Catchment 3’s content thread (about the drainpipe) – wrapping the discourse together with the thread. Also, the “tries” in Panel 1 sets the stage for the downward force metaphor of Panel 2. The linking of growth points created another thread, the very idea that they are cohesively linked, which I will call the accumulation thread. The continued preparation and prestroke hold of “drops” is now explained: “drops” and the downward stroke were absorbing different threads, “drops” the accumulation thread, linking it to the previous growth point; the stroke the antagonistic-force thread, it launching a new concept. The stroke waited until the accumulation thread had been fulfilled and then launched the downward force.

The sixth thread, conversation/interaction, arose from the speaker’s ongoing interaction with her listener. It organized the interaction to accord with the pragmatic goal that because the listener had to retell the story from what the speaker told her, the narrative had to be accurate and complete. The thread did not reside in specific verbal/gestural expressions but covered the speaker’s pragmatic deployments overall.

****

Sometimes a construction from langue is a Trojan Horse. It sneaks in and threatens the growth point’s field of equivalents and its differentiation. Part of the growth point is the power to override an errant construction. If the growth point does not deploy it, speech immediately stops. Overriding does not erase the construction; the override bars it from orchestrating speech. The growth point at Panel 2 had an override. Overriding was no accident and not a failed attempt. It was part of the multiple cohesions the growth point had absorbed and the override was precise. The Trojan horse construction was the causative, “Ø_(Tweety) drops it down,” which was needed for the growth point but if allowed to orchestrate speech would have tangled the accumulation and metaphor threads. Overriding let the metaphor thread hold back until uttering “drops” had fulfilled the accumulation thread. The growth point then launched the metaphor thread with the gesture stroke (see Box 2 in Figure 18.5). If I say the sentence in citation style, a temporal and aspirated “t” break appears at the syntactic “it”/“down” boundary. Such breaks were absent from how the speaker said it. In her speech, “it down” was a single prosodic package, suggesting it had been orchestrated as a whole.

Figure 18.5 Box 2

A different speaker, in a state of confusion, illustrates what happens when an override does not occur, and the Trojan Horse enters. The confusion was in navigating the sentence and included a midstream change of fields of equivalents. Confusions like this may be typical of failed overrides. The shutdown is at line (2.2).Footnote ⁶

1. 2.1 [and he winds up rolling down the stre][et

1. 2.2 because it th uh well ac][tually what happens is he I you assume that he swallows this bowling b][all

The speaker seems to have had two fields of meaningful equivalents in mind at once, not in a dialectic but as rivals: what happened next vs. what one supposes. The dropped grammatical subjects in line 2.2 – “it,” “th(e)” – belong to what happened next. The field of equivalents shut down and the second field, what one supposes, took over. Had speech continued, it would have been something like “it (or “the bowling ball”) I/you assume goes into him,” with the what one supposes field inserting not simply two words (“I/you assume”) but an entire other-meaning ensemble. Instead, speech stopped. The speaker might have repeated “it/the bowling ball” with “I/you assume” at the start, but this would already have been the what one supposes field. Instead, she kept “you assume he swallows,” and went on unpacking.

****

The beat is a simple up/down or in/out motion linked to speech stress-peaks. Its kinesic simplicity belies its semiotic complexity. Dynamically, beats absorb the cohesive threads embodied in stress peaks. They indicate where the growth point has absorbed the thread. Reference Bosker and PeetersBosker and Peeters (2021) show experimentally “[…] that beat gestures influence the […] perception of lexical stress (e.g. distinguishing OBject from obJECT), and in turn, can influence what vowels listeners hear” (p. 1). A beat at OBject absorbs a thread linking it to an entity; and obJECT one linking it to an action. The “manual McGurk Effect” the authors identify is set up via gesture–speech unity, conjuring a stressed vowel.

In our narrations, four kinds of beats occur:

1. (1) To highlight new content. An example is Panel 2 of Figure 18.4. Another, from a narration of Hitchcock’s Blackmail, is “his gIRLfriend, ALIce, Alice WHIte,” with a beat and prosodic stress accompanying each increment of new information – her functional role, first name, and last name. The beats highlight the increments and add cohesive links to the narrative theme targeting the names.

2. (2) To highlight an anaphor. An example is “the weight came down (with a large downward iconic gesture) and he got clobbered (a beat).” The beat was a miniaturized version of the first gesture, targeting its clause and creating a cohesive link back to the large gesture and its growth point.

3. (3) To highlight a cataphor – the reverse of (2), creating a cohesive link to the future with an obligation to fulfill it. The beat is a miniaturized anticipation of the larger gesture. For example, “so the next thing he does (the beat creating a link to the following) is go in the front door” (narrative with an iconic for motion).

4. (4) To highlight discourse significance. For example, a beat with “instead of” indicating a break from expectation in “the grandmother // instead of Tweety.”

****

Growth points finally can embody both Character and Observer viewpoints. Observer Viewpoint filters everything from the perspective of a detached observer, watching the event as if it occurred on a stage or screen: the hands – the whole character, the space – the space in which the character resides, and the speaker’s own head and body – on the outside, looking in. Character Viewpoint takes the perspective of the participant in the action: the speaker plays the character: her hands – the character’s, her motion – its, and her body – the character in the scene. The viewpoints occur in roughly equal numbers. Once established, a viewpoint is kept intact for more than a single utterance, and often over long stretches. The shifts between them are not random. They set up contrasting story-telling styles: Character, an enactment style; Observer, a picture-painting style. All these qualities, viewpoint, style, energy, differentiation, work together to funnel energy into speech within a viewpoint frame.

A “dual viewpoint” combines the two viewpoints (Parrill, 2009). Dual viewpoints pose contrasts, not alternatives. A unifying theme binds them. In our example (Box 3 in Figure 18.6), the theme was metapragmatic “this is ironic” (implying the theme, not the words). Each viewpoint has its own material carrier within the gesture: Character Viewpoint, grasping Tweety; Observer Viewpoint, tracing Sylvester’s path. Two viewpoints in one gesture embody the idea of contrast. The cartoon showed Sylvester catapulting himself up to Tweety by throwing a weight onto the other end of a kind of seesaw. He grabbed Tweety, fell back down to the ground and landed on the seesaw again. This relaunched the weight at the opposite end of the seesaw. It arced through the air and landed on him (Tweety escaping). The gesture potentiated the idea of irony by pitting Sylvester’s viewpoint (an unfolding triumph) against the observer’s (an upcoming disaster).

Figure 18.6 Box 3

The growth points worked as follows. The initial Character Viewpoint grip-gesture at (1) was coexpressive with “grab,” and the speaker continued with it at (2)~(4) as she added an arced trajectory with Observer Viewpoint. At (2), the contrast between the viewpoints created the irony of Sylvester’s premature self-congratulation. Coexpressivity then shifted up a notch to the metapragmatic and became a field of equivalents. Every step, grabbing Tweety, landing, running off with him,Footnote ⁷ differentiated the field of equivalents.

2.2 Dispensing Energy

The dialectic synthesis dispenses the speaker’s energy into a sentence. The energy covers the sentence, experienced in what Wundt called “simultaneous awareness” (Reference BlumenthalBlumenthal, 1970, p. 22). The speaker also experiences a second awareness, the “sequential,” which can be anywhere – at the start, middle or end, depending on the construction. The two awarenesses are features of the growth point. The simultaneous corresponds to the growth point’s core meaning. The sequential applies to the unpacking, how the energy is disbursed into the construction as it is activated into speech, as the transition from Ergon to Energeia. The gesture acts as a template, orchestrating the flow of vocal energy into its preparation, stoke, and retraction phases. The awarenesses give these gesture phrases more than descriptive interest. They are a record of the energy of the growth point spreading through the sentence out of langue, bringing life to the static construction. The following diagram traces the energy spreading through “he goes [up through the pipe] this time,” parsed in terms of awarenesses and gesture phrase/phases:

How does it all work – and so quickly, that the whole process is over in a second or two? The growth point unpacks itself based on its own meaning. It “summons” a construction and words from langue – finding material compatible with its intentions and cohesive ties. The summons meets the requirement that the construction and words, whatever they may be, do not disrupt the differentiation and field of equivalents. The growth point remains intact while the construction builds up around it. The words and constructions the growth point “summons” are usually just right to unpack it into ratified verbal form. The “rising hollowness” growth point summoned what it needed, a declarative structure and words to complete it. The “it down” growth point summoned a causative construction. A construction has its own units and syntagmatic values, slots for words like “this time” and other constructions, and combinatoric potential (Reference GoldbergGoldberg, 1995). Deployed as a whole, the construction conveys a settled meaning. It also provides the place for the gesture to surface – all this is on the timescale of fluent speech. The information load is high. Hesitations and errors reflect the burden, so it is wrong to say the emergence is flawless, but there is, on the whole, a remarkable speed and accuracy to unpacking.

Like the discovery by Reference Marslen-WilsonMarslen-Wilson (1987), who introduced the concept of a cohort for speech-sound recognition, the summons finds constructions for production by forming and winnowing a cohort of its own. A large cohort can be winnowed quite fast. I use the term, “langue-feelings” for the intuitions that form cohorts. Reference JamesJames (1890) wrote of the feeling of “and” (roughly, more to come) and “but” (contrariness). For “up through the pipe” the langue-feeling is Something Happens. For “it down,” it is Make Something Happen. Feelings are not langue’s systematic features, but a channel into them via one’s own inhabitancies and intentions. Reference Huth, de Heer, Griffiths, Theunissen and GallantHuth, de Heer, Griffiths, Theunissen, & Gallant (2016) mapped regions of semantic-field activity in the brain as subjects listened to extended narrations. The semantic fields could be cohorts that langue-feelings were summoning.

1.1 History

In neuropsychology, gesture research has a long-standing tradition. In 1870, Finkelnburg introduced the concept of “asymbolia” as a new model of aphasia. This model included a wide range of disturbed functions concerning the use of symbols in general, such as musical notation, algebraic, geometrical, and chemical symbols, religious rites, gestures, facial expressions, and pantomime. In contrast, Reference LiepmannLiepmann (1908) argued that deficits in pantomime gestures could not be explained by asymbolia because this disorder often cooccurred with an impairment in imitating gestures. Visuo-motor imitation requires that only the visually perceived movement is reproduced but not that the meaning of the movement is understood. Therefore, deficits in pantomiming should reflect a general disorder in the conceptualization and execution of spatio-temporal movement concepts. Liepmann’s seminal proposition has been the basis for apraxiaFootnote ² models to this day.

Liepmann’s model of apraxia was based on his examination of patients with a left hemisphere damage (LHD) who showed – as is to be expected – a palsy of the right half of the body and often also aphasia, but who also could not perform gestures and actions with the left hand on command, although the left hand was not paretic or ataxic (i.e. there is no lack of strength, coordination, or sensation). In order to follow Liepmann’s thoughts, it is important to know that the left cerebral hemisphere controls the contralateral right half of the body, and vice versa, the right hemisphere the contralateral left half (Figure 20.1a).

Figure 20.1a Neurotypical motor control of the right and left hands

Figure 20.1b Left hemisphere damage: paresis of the contralateral right hand

Figure 20.1c Right hemisphere damage: paresis of the contralateral left hand

Figure 20.1d Callosal disconnection: exclusive contralateral control of the right and left hands

Accordingly, in LHD, the right half of the body is paretic and cannot perform movements (Figure 20.1b), and vice versa, in right hemisphere damage (RHD), the left half of the body (Figure 20.1c). Liepmann inferred from the patients’ symptomatology that the left hemisphere must control movement not only for the contralateral right hand but also for the ipsilateral left hand.

Furthermore, since some of these patients were able to imitate movements with the left hand, Liepmann assumed that, in these patients, the selective process of movement execution (as compared to movement conceptualization) was not disturbed so that they could execute the movements when the movement concept was shown to them. Liepmann localized the process of execution (“innervation”) in the so-called sensomotorium of the motor cortex of each hemisphere. Thus, when we perform a movement, according to Liepmann, first a movement is conceptualized in the left hemisphere or the concept is retrieved from the memory (conceptualization) and then sent to the respective sensomotorium in the left or right hemisphere where it is translated into movement (execution). The movement concept is sent to the left-hemispheric sensomotorium when the movement is to be performed by the right hand and it is sent through the corpus callosum, which connects the two hemispheres, to the right-hemispheric sensomotorium when the movement is to be performed by the left hand (compare Figure 20.1a).

Liepmann also observed that most of the LHD patients were able to use their left hands normally for daily routine movements, for example, drinking from a glass, using a fork. He proposed that first, in the sensomotorium there is a distinct kinetic memory for certain short, automatized stereotypically recurring movements, and that second, the object leads the hand, for example, when using a coffee mill the hand movement is guided by the tool and, therefore, the conceptualization of the movement is facilitated.

The weak point of Liepmann’s two-step hierarchic apraxia model – (1) conceptualization, (2) execution – is that it does not explain several patient cases, for example, people in whom the visuo-motor imitation of a gesture is disturbed more than the performance of the same gesture on verbal command, although only the latter praxis modality requires a conceptualization of the movement (Reference Ochipa, Rothi and HeilmanOchipa, Rothi, & Heilman, 1994). As a consequence, a recent trend in apraxia research is the development of multidimensional models to consider each praxis modality separately and to examine which other cognitive functions are codisturbed, for example, an apraxia in the imitation of finger configuations might cooccur with a deficit in copying complex figures (see below) (Reference Goldenberg, Hartmann and SchlottGoldenberg, Hartmann, & Schlott, 2003). This method enables identification of which cognitive functions the praxis modality is associated with. This paradigm is also followed in this chapter, that is, to explore which cognitive function the generation of a specific gesture type may be associated with.

1.2 Methodology

This section describes specific neuroscientific methods as well as gesture elicitation and analysis procedures employed in neuropsychological gesture research.

1.2.1 Specific Neuroscientific Methods

As outlined above, a focus of neuropsychological gesture research is on the topographical localization of gesture production in the brain – more precisely, on the determination of brain regions that critically contribute to the production of gestures. The traditional method is the study of patients with circumscribed brain damage. In recent decades, technical development has also enabled the use of neuroimaging techniques for the localization of gesture production.

1.2.1.1 Lesion Studies

In neuropsychological gesture research, lesion studies are typically conducted in patients with LHD or RHD and the two patient groups are compared (only rarely do gesture production studies distinguish between different regions within a hemisphere). The specific deficit in gesture production reflects the impaired competence of the damaged hemisphere. Accordingly, the remaining competence in gesture production is likely to reflect the competence of the undamaged hemisphere. Thus, the paradigm behind the lesion studies is if a deficit in gesture production is found, the hemisphere which is damaged is crucial to the function which was found to be lost or impaired in gesture production. A limitation of this method is that it cannot be excluded that remaining undamaged regions in the damaged hemisphere contribute to gesture production, since damage to one hemisphere is rarely complete. However, the direct comparison of LHD and RHD individuals in a study enables the identification of right and left hemisphere contributions to gesture production relatively reliably.

The highest degree of reliability can be achieved by investigating patients with callosal disconnection. In this rare patient group, the corpus callosum, which is the biggest neural fiber connection between the left and right hemispheres, is lesioned,Footnote ³ such that information transfer between the two hemispheres is not possible anymore (Figure 20.1d). As outlined above, the right hand is controlled by the contralateral left hemisphere, and vice versa, the left hand by the contralateral right hemisphere.Footnote ⁴ This neural organization implies that information between the two hemispheres must pass through the corpus callosum, when the right hand executes ipsilateral right hemisphere concepts and vice versa, the left hand ipsilateral left hemisphere concepts. If for example a person with left-hemispheric speech production writes with the ipsilateral left hand, the control runs from the left hemisphere over the corpus callosum to the motor cortex of the right hemisphere and from there to the left hand. In patients with callosal disconnection, the callosal transfer is no longer possible, and thus the left hand can only execute right hemisphere concepts and, vice versa, the right hand left hemisphere concepts (Reference SperryGazzaniga, Bogen, & Sperry, 1967; Reference Lausberg, Kita, Zaidel and PtitoLausberg, Kita, Zaidel, & Ptito, 2003; Reference SperrySperry 1968; Reference Trope, Fishman, Gur, Sussman and GurTrope, Fishman, Gur, Sussman, & Gur, 1987; Reference Volpe, Sidtis, Holtzman, Wilson and GazzanigaVolpe, Sidtis, Holtzman, Wilson, & Gazzaniga, 1982). Therefore, the examination of patients with callosal disconnection reliably enables localization of the production of specific gestures in the left and right hemispheres. Moreover, in these patients the competences of the left and right hemispheres can be directly compared within one individual.

1.2.1.2 Hand Preference Studies

Beginning with Kimura’s seminal studies (Reference Kimura1973a, Reference Kimura1973b) on hand preferences in spontaneous gesture production, neuropsychological gesture research has investigated hand preferences in healthy individuals as an indicator of hemispheric specialization (see detailed review in Reference LausbergLausberg, 2013). As evidenced by behavioral laterality experiments, healthy individuals show a preference for responding with the hand that is contralateral to the hemisphere that performs the task (Reference Zaidel, White, Sakurai, Banks and ChiarelloZaidel, White, Sakurai, & Banks, 1988). As an example, right-handers shift from more right-hand use in verbal tasks, which are processed in the left hemisphere, toward more left-hand use in spatial tasks, which are processed in the right hemisphere (Reference Hampson and KimuraHampson & Kimura, 1984). The limitation of hand preference studies in healthy subjects, however, is that if required, the intact corpus callosum enables each hemisphere to exert control over the ipsilateral hand. Therefore, other factors can overrun the effect of hemispheric specialization for a task on the hand preference and induce the use of the hand that is ipsilateral to the hemisphere that performs the task. In gesture production studies, handedness has to be considered first and foremost. Right-handers as compared to left-handers show a trend toward more right-hand use.Footnote ⁵ Left-handers as compared to right-handers show a trend toward more left-hand use (Reference KimuraKimura, 1973a, Reference Kimura1973b) as well as different gestural roles of the left and right hands (Reference Helmich, Voelk, Coenen, Xu, Reinhardt, Mueller, Schepmann and LausbergHelmich et al., 2021, Reference Helmich, Meyer, Voelk, Coenen, Mueller, Schepmann and Lausberg2022). Furthermore, in one study, in which handedness, however, was not controlled for, males showed a stronger right-hand preference for gestures than females (Reference Saucier and EliasSaucier & Elias, 2001). A semantic purpose, such as when talking about the left or right of two objects, results in the preference for the left or right hand, respectively, in the accompanying gestures (Reference Lausberg and KitaLausberg & Kita, 2003). Likewise, cultural conventions can determine the hand choice, such as when Arrente speakers in Central Australia use the left hand to refer to targets that are on the left and vice versa the right hand for targets that are on the right (Reference Wilkins, de Ruiter, Van Geenhoven and WarnerWilkins & de Ruiter, 1999). Finally, an occupation of the right hand with some other physical activity, such as holding a cup of coffee, can influence the choice of the hand in gesture production.

Thus, if hand preference is used as an indicator of hemispheric specialization in healthy individuals, these factors need to be controlled. Since handedness, in particular, strongly influences the hand choice in gesture production, strictly speaking, only the use of the non-dominant hand can be used as a reliable indicator of hemispheric specialization. As an example, if a right-hander uses the dominant right hand for gesture production, it cannot be reliably distinguished if (s)he uses the right hand because of her/his handedness or because of hemispheric specialization in the production of that specific gesture. If, however, a right-hander spontaneously uses the nondominant left hand, this can be taken as an indicator of hemispheric specialization (if the other factors listed above are ruled out).

1.2.1.3 Tachistoscopic Experiments

In tachistoscopic experiments, visual stimuli are presented randomly in the left and right visual hemifields. The stimuli are presented for a short time, often 150 ms, such that the individual cannot shift the focus of his vision to the stimulus. This has the consequence that the stimuli are first processed in the contralateral visual cortex, that is, stimuli presented in the left visual field are processed in the right visual cortex and vice versa stimuli presented in the right visual field are processed in the left visual cortex (Figure 20.2).

Figure 20.2 Visuo-motor processing of lateralized presented stimuli

The paradigm behind this method in gesture production studies is as follows: If the task requires a fast gestural response, for example, depicting the stimulus in gesture, there is a spontaneous preference to use the contralateral hand (see above). Thus, stimuli presented in the left visual field, which are processed in the right hemisphere, will trigger left-hand gestures. Vice versa, stimuli presented in the right visual field, which are processed in the left hemisphere, will trigger right-hand gestures. The preference for the contralateral hand is only pushed back if the stimulus requires a hemispherically specialized function. Therefore, deviations from the contralateral hand preference are indicative of a hemispheric specialization in the production of that specific gesture.

1.2.1.4 Functional Neuroimaging Studies

In recent decades the spectrum of neuropsychological gesture research has been expanded by neurophysiological and neuroimaging methods. While neurophysiological methods (electroencephalography) which measure the electric activity in the brain, such as event-related potentials, are highly valuable for gesture-perception research, they are less suited for gesture-production research because muscle activity would substantially influence the result. In functional neuroimaging studies, such as functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS), the neural activity during gesture production is localized via an increase of blood flow in a specific region of the brain. Muscle artifacts are controlled by fixating the head in the scanner (fMRI) or by fixating the device at the head (fNIRS). Among functional neuroimaging techniques, fNIRS has the advantage that the display of hand gestures can be performed in a quasi-natural setting as compared to the highly unnatural setting in an MRI scanner, involving lying on one’s back with the head and elbows fixated. However, thus far, fNIRS methodology has focused on block-design paradigms, which require that the same gesture is repeated several times in a row. Thus, the gestures are produced explicitly on command – a condition that differs from the spontaneous implicit gesture production in natural settings (see below).

2.1 Lesion Studies

2.2 Hand Preference Studies

As outlined above, with regard to hemispheric specialization in the production of gestures, in right-handers it is left-hand gestures that are of interest, since in this case handedness can be ruled out as a determining factor. In right-handers, percentages of left-hand gestures in unimanual gestures range between 25 percent and 39 percent (Reference Dalby, Gibson, Grossi and SchneiderDalby et al., 1980; Reference KimuraKimura, 1973a; Reference Lavergne and KimuraLavergne & Kimura, 1987; Reference Sousa-Posa, Rohrberg and MercureSousa-Poza, Rohrberg, & Mercure, 1979; Reference StephensStephens, 1983). As an example, Reference Miller and FranzMiller and Franz (2005) reported for their sample of twelve right-handed individuals a mean proportion of time of 3.15 seconds per minute spent making gestures with the right hand and of 2.32 seconds per minute spent gesturing with the left hand, with great interindividual variability. The question arises as to what causes the production of left-hand gestures in right-handers, in particular since neither handedness nor language lateralization can explain the hand choice in this case.

Some studies report that the topic influences hand preference for gestures. In right-handers, more left-hand use is found in gestures that are accompanied by an emotional facial expression than in gestures without accompanying emotional facial expression (Reference Moscovitch and OldsMoscovitch & Olds, 1982). Reference Kita, de Condappa and MohrKita, Condappa, and Mohr (2007) report a shift toward more left-hand use for those depictive gestures that have a character viewpoint in a metaphor condition. (However, Reference StephensStephens, 1983, reported a stronger right-hand preferences among right-handers for metaphorics as well as for iconics.). Other studies investigate hand preferences for specific gesture types. For batons or beats as compared to other gesture types, some studies show a trend toward more left-hand use (Blonder et al., 1995; Reference McNeillMcNeill, 1992; Reference Sousa-Posa, Rohrberg and MercureSouza-Poza et al., 1979; Reference StephensStephens, 1983).

The few gesture studies on left-handers reflect a high heterogeneity within this group which is linked to differences in language lateralization (Reference KimuraKimura, 1973b) as well as in writing style, that is, left inverted vs. left normal (Reference StephensStephens, 1983). Likewise, in left-handers, hand use for specific gestures appears to be more variable than in right-handers (Reference Helmich, Meyer, Voelk, Coenen, Mueller, Schepmann and LausbergHelmich et al., 2022; Reference MeyerMeyer, 2021).

To summarize, left-hand gestures in right-handed healthy individuals can be considered as an indicator of a right-hemispheric contribution to gesture production. This appears to apply to gestures that are generated in an emotional or metaphorical context as well as to gestures that lend rhythmical emphasis to a verbal statement. Left-handers show a high variability in hand preference for gestures, which reflects that in this group, hemispheric specialization is known to be more variable than in right-handers.

2.3 Neuroimaging Studies

Thus far, there are only a few neuroimaging studies that have investigated gesture production. Some of them examine only the left hemisphere (e.g. Reference Balconi, Crivelli and CortesiBalconi, Crivelli, & Cortesi, 2017), which makes a comparison of the contributions of the right and left hemispheres impossible. All of them investigate gesture production on command; none of them consider spontaneous gesture production.

In functional near-infrared spectroscopy (fNIRS) simple meaningless movements of the right and left hands (here: flexion/extension of the thumb) are accompanied by an increase in cerebral oxygenation in the contralateral hemisphere, that is, for the left thumb in the right hemisphere and vice versa, for the right thumb in the left hemisphere (Reference LausbergHelmich, Rein, Niemann, & Lausberg, 2013). In contrast, pantomime, tool use demonstration, and body-part-as-object (BPO)Footnote ⁷ gestures show a lateralization to the left hemisphere (Reference Helmich, Holle, Rein and LausbergHelmich et al., 2015).

With an event-related fMRI study design, Reference Hermsdörfer, Terlinden, Mühlau, Goldenberg and WohlschlägerHermsdörfer, Terlinden, Mühlau, Goldenberg, and Wohlschlager (2007) showed an increase of activity in the left intraparietal sulcus in pantomime as compared to tool use demonstration. (However, this finding is based on a region of interest analysis of the left inferior frontal and left inferior parietal cortices.) In a fMRI study by Reference Imazu, Sugio, Tanaka and InuiImazu, Sugio, Tanka, and Inui (2007), the contrast of pantomime to tool use demonstration for using chopsticks with the right hand shows significantly greater activity in the left inferior intraparietal lobe. Finally, Reference Lausberg, Kazzer, Heekeren and WartenburgerLausberg, Kazzer, Heekeren, and Wartenburger (2015) investigated tool use pantomime with either hand in response to visual tool presentation and contrasted it with tool use demonstration. The fMRI conjunction analysis of the right and left hands’ executions of tool use pantomime relative to tool use demonstration shows significant activity in the left middle and superior temporal lobe. While all three fMRI studies examined pantomime and tool use demonstration, the latter study was the only one in which the study design was perfectly analogous to the above reported lesion studies, in which differences between pantomime and tool use demonstration were found in LHD and callosal disconnection patients.

To summarize, for the on-command performance of simple meaningless hand gestures there is no hemispheric specialization, while for tool use pantomime there is a left-hemispheric specialization. The contrast of tool use pantomime and tool use demonstration yields an activation in the left middle and superior temporal gyri. Since the only difference between two conditions is that the pantomime condition requires the individual to act with an imaginary tool in hand, the left temporal gyri appear to be specifically involved in integrating the mental image of a tool in the gesture execution (see further discussion below).

3.1 The Impact of Investigating Spontaneously Versus On-Command Produced Gestures on the Study Results

As early as 1907, Liepmann reported a patient who was not able to produce a specific gesture on command, but who readily performed the “same” gesture spontaneously in a natural context (Reference Liepmann and MaasLiepmann & Maas, 1907). Since then, there have been numerous reports of patients with LHD or of patients with callosal disconnection in the left hand (the control of which is deprived of left-hemispheric input) who cannot produce gestures explicitlyFootnote ⁸ upon request, for example, waving goodbye on command, but they can produce the same gesture implicitly without any problem as part of an automated daily routine, for example, waving when actually saying goodbye to somebody. Likewise, patients with LHD or callosal disconnection (left hand) have been reported to spontaneously use a tool correctly in a natural context but to fail in an experimental condition when they were asked to use the same tool on command (e.g. Reference Buxbaum, Schwartz, Coslett and CarewBuxbaum, Schwaryz, Coslett, & Carew et al., 1995; Reference Hermsdörfer, Li, Randerath, Goldenberg and JohannsenHermsdörfer, Terlinden, Mühlau, Goldenberg, & Wohlschlanger, 2012; Reference Lausberg, Göttert, Münßinger, Boegner and MarxLausberg, Göttert, Münßinger, Boegner, & Marx, 1999; Reference Liepmann and MaasLiepmann & Maas, 1907). As an example, a patient with callosal disconnection could even not take something out of his trouser pocket with the left hand when he intended to do so but he could perform the same action when he did not think about it (Reference Lausberg, Göttert, Münßinger, Boegner and MarxLausberg et al., 1999).

As described above, Liepmann proposed that the right hemisphere contains a sensomotorium with a kinetic memory for automatized recurring movements. Later, Reference Rapcsak, Ochipa, Beeson and RubensRapcsak, Ochipa, Beeson, & Rubens (1993, p. 23) suggested that the “praxis system of the right hemisphere is strongly biased toward the ‘concrete’ or context-dependent execution of familiar, well-established routines such as overlearned actions” and intransitive gestures, whereas the left hemisphere controls “‘abstract’ or context-independent performance of transitive movements and in learning of novel movement sequences.” Thus, performing gestures on command appears to require more left-hemispheric competences, while spontaneous gesture production relies more on right-hemispheric competences. Therefore, it is plausible that experimental settings which rely on gesture production on command, such as apraxia tests but also neuroimaging studies, will yield a strong or even exclusive left-hemispheric generation of gestures, while experimental settings in which spontaneous gestures are indirectly elicited, such as in interaction or renarration, will show a stronger right-hemispheric contribution to gesture production.

3.2 The Impact of the Cognitive-Emotional Challenges of the Task on the Study Results

Another methodological factor that strongly influences the study results is the cognitive-emotional challenge of the task. While some tasks require more right-hemispheric functions such as spatial cognition, spatial attention, nonverbal emotional expression including affective prosody, and metaphorical thinking, other tasks address more left-hemispheric functions such as language and praxis. As outlined above, in healthy individuals, the hemisphere that is predominantly engaged in a task influences the hand choice for gesture execution, for example, spatial tasks, which rely on right-hemispheric spatial cognition, induce a shift toward more (contralateral) left-hand use in gesture. It is highly plausible that the right hemisphere not only processes the spatial task and executes the corresponding gesture – as evidenced by the left-hand choice – but also conceptualizes the spatial gesture, which is grounded in spatial cognition. Following this line of thought, the sections below discuss the above findings with regard to gesture production in the right and left hemispheres.

2.1 Child Gesture Indexes Language Knowledge

One way that gesture plays a role in language learning is by indexing when children are ready to make advances in their linguistic development. For example, infants’ own gestures reflect a readiness to learn new words. Babies gesture before they speak, often producing their first pointing gestures between nine and 12 months (Reference Bates, Camaioni and VolterraBates, Camaioni, & Volterra, 1975; Reference Greenfield and SmithGreenfield & Smith, 1976). Importantly, the timing of these earliest pointing gestures indicates and predicts infants’ first words: Items that infants point to between ten and 14 months subsequently begin to show up in their productive vocabularies approximately three months later (Reference Iverson and Goldin-MeadowIverson & Goldin-Meadow, 2005). Early pointing gestures therefore set the stage for language learning, perhaps giving caregivers a signal that the child is ready to receive verbal input about specific items. This relationship has important longer-term implications as well: Children’s gesture use at fourteen months predicts their productive vocabulary three years later (Reference Rowe and Goldin-MeadowRowe & Goldin-Meadow, 2009). In fact, a meta-analysis has found that pointing is both concurrently (r = .52) and longitudinally (over time) (r = .35) related to language development, reinforcing the importance of early gesture in language acquisition (Reference Colonnesi, Stams, Koster and NoomColonnesi, Stams, Koster, & Noom, 2010).

One reason why pointing reflects language learning may be that babies point to items they are interested in, and this interest leads to a desire to learn labels for those specific objects. But if that were the only reason, then any indication of interest on a baby’s part should lead to learning. Instead, there seems to be something special about pointing itself that may be critically tied to language learning (Reference Butterworth and KitaButterworth, 2003). Eighteen-month-olds are more likely to learn labels for objects if they are provided with those labels after they point to the objects than after they reach toward the objects, or look toward the objects (Reference Lucca and WilbournLucca & Wilbourn, 2018). Even though looking, reaching, and pointing are all expressions of interest, receiving a label following pointing is more likely to support word learning than receiving a label following the other two interest behaviors. Infants are also more likely to learn the function of an object if they point to that object than if they reach or look toward the object, suggesting that the effects of pointing on learning may not be restricted to the language domain (Reference Lucca and WilbournLucca & Wilbourn, 2019). Infant pointing may reflect not only an interest in an object, but also a motivation to learn about that object.

Pointing gestures also index semantic development, reflecting infants’ readiness to combine constituents and begin producing two-word utterances. Even after children begin to produce their first words, they continue to gesture, often gesturing together with their speech. Sometimes those gestures provide information that is complementary to speech – for example, when a child points to a cup and says “cup.” But other times the gestures provide information that is supplementary to speech – for example, when a child points to a cup and says “mommy,” indicating mommy’s cup. Children who produce these supplementary combinations begin producing two-word utterances just a few months later (Reference Iverson and Goldin-MeadowIverson & Goldin-Meadow, 2005; Reference Özçalışkan and Goldin-MeadowÖzçalışkan & Goldin-Meadow, 2005). This finding suggests that the ability to combine constituents across two different modalities is a first step in indexing that a child is gearing up to combine constituents all within the verbal modality. Gesture can provide children with a tool to express complex ideas at a time when their language abilities are not up to the task.

As children get a little bit older, gesture can even reflect developing knowledge for linguistic concepts that are abstract, such as number words. Learning how to map the terms “one,” “two,” and “three” to specific exact quantities is harder than learning the meaning of concrete nouns such as “cup” or “chair” because number words can refer to any object or set of objects. Number word acquisition is a drawn-out process that develops gradually over the course of many months and years (Reference CareyCarey, 2009; Reference Sarnecka and CareySarnecka & Carey, 2008; Reference Sarnecka and LeeSarnecka & Lee, 2009; Reference WynnWynn, 1990, Reference Wynn1992). Although children may learn to recite number words as part of a rote counting routine, it is not until they understand that these terms map to specific amounts (i.e. “two” means exactly two, not one and not three) that they have truly learned the number word. Gestures plays a critical role in indexing when children are on the brink of making these difficult conceptual mappings. When asked to indicate the number of objects in a set, children often use both words and gestures in their response (Reference Gibson, Gunderson, Spaepen, Levine and Goldin‐MeadowGibson, Gunderson, Spaepen, Levine, & Goldin‐Meadow, 2019; Reference Gunderson, Spaepen, Gibson, Goldin-Meadow and LevineGunderson, Spaepen, Gibson, Goldin-Meadow, & Levine, 2015). If asked to label a set of two objects, a child who fully understands the meaning of “two” will accurately say “two” while holding up two fingers. But a child who does not yet know “two” may be able to show two before giving the correct verbal response. That is, the child may say “three” to describe two objects but hold up two fingers – a speech–gesture mismatch in which speech and gesture convey different information. When this happens, children are more likely to be accurate in gesture before they are accurate in speech (Reference Gunderson, Spaepen, Gibson, Goldin-Meadow and LevineGunderson et al., 2015). Importantly, children who produce number speech–gesture mismatches of this sort are more likely to benefit from number-word instruction than children who do not produce speech–gesture mismatches (Reference Gibson, Gunderson, Spaepen, Levine and Goldin‐MeadowGibson et al., 2019). In the domain of the number word, gesture can again reflect young children’s readiness to learn.

2.2 Producing Gesture Changes Young Children’s Language Knowledge

We have just seen that gesture can reflect linguistic knowledge, indexing when children are ready to learn concrete nouns or abstract number words, and when they are ready to combine words into sentences. But the gestures that children produce can do more than merely indicate or reflect that a child is ready to make linguistic advances. Producing gesture can play a causal role in the learning process itself.

First, take the example that pointing reflects a readiness to learn new nouns (Reference Iverson and Goldin-MeadowIverson & Goldin-Meadow, 2005). We know that the act of producing gesture plays a causal role in learning because children who have been instructed to point show increases in language outcomes, relative to children who are not instructed to point. In one study, 16-month-old children were either taught words from a picture book (“look at the chair”), taught words while an experimenter pointed at the pictures, or taught words while the experimenter pointed and the child was instructed to point as well (“can you do this?”). After a 7-week intervention, the children who were instructed to point not only increased their pointing in subsequent interactions with their caregivers, relative to the other two groups, but also showed increases in vocabulary (Reference LeBarton, Goldin-Meadow and RaudenbushLeBarton, Goldin-Meadow, & Raudenbush, 2015). This finding indicates that gesture in children can be experimentally increased, and that increases in gesture lead to increases in language outcomes.

Gesture’s ability to causally influence language learning extends beyond infancy and beyond simple nouns. Four-year-old children who were taught to produce gestures (e.g. a gesture illustrating an action on an object) while learning novel verbs (e.g. “this is blicking”) were more likely to learn those verbs and correctly extend them to the action performed on a novel object than children who were not instructed to produce gestures (Reference Wakefield, Hall, James and Goldin‐MeadowWakefield, Hall, James, & Goldin‐Meadow, 2018). Moreover, children who were taught to produce gestures were better at extending the verbs, compared to children who were given experience producing actions directly on the objects. Gesturing while learning a verb can thus play a causal role in helping children generalize that verb to new contexts, and can do so better than concrete action experience.

2.3 Seeing Gesture Changes Young Children’s Language Knowledge

What about learning from other people’s gestures? Infants and young children see their parents and caregivers gesture all the time. What role do the gestures that young learners see play in language learning?

Pointing gestures are intentional referential cues, and the pointing gestures that infants and young children see can help them interpret labeling scenarios as intentional, pedagogical situations. Certainly, gestures are captivating and salient. Even at four to six months old, infants begin to orient to pointing hands (Reference Bertenthal, Boyer and HardingBertenthal, Boyer, & Harding, 2014; Reference Rohlfing, Longo and BertenthalRohlfing, Longo, & Bertenthal, 2012) and, at around twelve months, they understand the communicative intentions of others’ pointing gestures (Reference Behne, Liszkowski, Carpenter and TomaselloBehne, Liszkowski, Carpenter, & Tomasello, 2012; Reference Krehm, Onishi and VouloumanosKrehm, Onishi, & Vouloumanos, 2014; Reference Woodward and GuajardoWoodward & Guajardo, 2002). Not surprisingly, then, starting at around one year, infants show a boost in word learning when taught new words paired with pointing gestures (Reference Booth, McGregor and RohlfingBooth, McGregor, & Rohlfing, 2008; Reference Woodward, Hall and WaxmanWoodward, 2004).

By the time infants are two years old, it is not just pointing gestures that support learning – young children can also use the iconic gestures they see to learn new information. Iconic gestures encode features of the objects or actions that they represent; for example, banging a fist to represent hammering, dragging a finger across the sky to represent the movement of a bird, or touching one’s thumb to fingers in a circular shape to indicate the round shape of a ball. Iconic gestures are relatively rare in spontaneous production in both parents and children (Reference Özçalışkan, Goldin-Meadow, Stam and IshinoÖzçalışkan & Goldin-Meadow, 2011). Nevertheless, experimental studies have shown that iconic gestures can support word learning in early childhood. For example, two-, three-, and four-year olds are able to infer the meaning of novel intransitive verbs – the movement to which the verb refers – if they are presented with co-speech iconic gestures depicting that movement (Reference Goodrich and Hudson KamGoodrich & Hudson Kam, 2009). Language training that includes iconic gesture not only boosts young children’s initial mapping of novel words to referents but also supports greater enrichment, retention, and generalization of those words (Reference Capone and McGregoryCapone & McGregory, 2005; Reference McGregor, Rohlfing, Bean and MarschnerMcGregor, Rohlfing, Bean, & Marschner, 2009).

Gestures are thus well suited to learning words. But gestures can also be useful for learning about things that are hard to describe in words. For example, it is easier to produce gestures explaining how to operate a strange object (like a can opener) than to describe the process in words. Importantly, even two-year olds are able to use gestures of this sort to learn how to operate a novel object. They are able to operate an object after an experimenter demonstrates the action using a gesture (Reference Novack, Goldin-Meadow and WoodwardNovack, Goldin-Meadow, & Woodward, 2015). However, two-year olds are better at learning the function of an object from an action demonstration (specifically a failed-action demonstration that does not show the completed outcome) than from a gesture demonstration. Although children as young as two can appreciate the representational function of gesture, the ability seems to be relatively fragile, making it difficult to learn from iconic gestures at very young ages.

2.4 Summary

Gesture plays an important role in learning for infants and young children. Children’s gesture reflects children’s knowledge and indicates when a child may be getting ready to make a linguistic leap or benefit from language instruction. Infants and young children benefit both from producing gesture and from seeing gesture within the domain of language learning and also in other domains, such as learning about number and object functions. Young learners use, and make use of, pointing gestures quite early in development, and iconic gestures (which have a more complex relation to their referents) a bit later. Gesture thus plays an important role in learning for young children who have not yet developed full language abilities. What happens when language is in full swing? In Section 3, we delve into how gesture supports learning and conceptual change in older children; specifically, how gesture interacts with speech in a developed communication system.

3.1 Child Gesture Indexes Knowledge about Math

To explore how gesture indexes knowledge in older children, we consider how children learn about mathematical equivalence. Mathematical equivalence is the concept that two sides of an equation must equal the same amount, a concept that is surprisingly challenging for eight- to ten-year-old children in the United States, and one that is foundational to more advanced mathematical skills (Reference McNeill, Hornburg, Devlin, Carrazza and McKeeverMcNeill, Hornburg, Devlin, Carrazza, & McKeever, 2019). Equivalence problems that have missing addends, such as 2 + 7 + 8 = __ + 8, present challenges that reveal children’s misconceptions (Reference Perry, Church and Goldin-MeadowPerry, Church, & Goldin-Meadow, 1988). For example, some children incorrectly think that the equals sign is a signal to add up all of the numbers in the problem; in this example, adding 2, 7, 8, and 8 and putting 25 in the blank. Other children choose to ignore the addend on the right-hand side (the rightmost 8 in the example) and add up the numbers to the left of the equal sign, adding 2, 7, and 8 and putting 17 in the blank. If asked to explain why they wrote down an incorrect solution on a missing addend equivalence problem, children have no problem describing these flawed rationales in words. But, importantly for our purposes, children also gesture as they explain their solutions and those gestures can reveal information that differs from the information conveyed in the accompanying speech. It is these gestures, considered in relation to speech, that best predict which children are ready to learn from math instruction, and which children are not.

Sometimes, children’s gestures match what they are saying in speech. For example, a child says, “I added the 2, the 7, the 8, and the 8,” while pointing with her right hand to each of the four addends as she indicates them in speech (an incorrect “add all” strategy that matches in speech and gesture). Other times, children express different information in gesture and speech. For example, a child says the “add all” explanation in speech but uses one hand to sweep under the numbers on one side of the equation, and the other hand to sweep under the numbers on the other side of the equation. Here, the change in hand shape indicates an awareness (albeit implicit) that the equation has two distinct sides, and the parallelism of the sweeps produced under each side indicates an awareness that the two sides should be treated alike (a correct “equalizer” strategy). When children produce different information in speech and in gesture, they have produced a mismatch (Reference Church and Goldin-MeadowChurch & Goldin-Meadow, 1986; Reference Perry, Church and Goldin-MeadowPerry et al., 1988). Children who produce speech–gesture mismatches as they explain their solutions to missing addend equivalence problems are more likely to profit from instruction than children who do not produce mismatches. The gestures that a child produces, taken in relation to speech, can thus offer insight into the child’s conceptual state, signaling that the child is in a transitional state with respect to learning mathematical equivalence (Reference Goldin-Meadow, Alibali and ChurchGoldin-Meadow, Alibali, & Church, 1993).

Why is this the case? One possibility is that gesture and speech provide separate avenues for considering multiple hypotheses, and that concurrent activation of multiple hypotheses is a feature of the transitional state (Reference Alibali and Goldin-MeadowAlibali & Goldin-Meadow, 1993; Reference Goldin-Meadow, Alibali and ChurchGoldin-Meadow et al., 1993). According to this view, when children are in a transitional state, they might not be aware of the strategies they produce in gesture – in other words, these strategies might not yet be explicit and are thus not yet accessible to speech. Instead, these hypotheses emerge implicitly through gesture. Gesture can thus provide an avenue for exploring new ideas that a child might not yet fully understand and is just beginning to entertain.

Although many children spontaneously gesture when asked to explain their reasoning to math problems, children do not always gesture during their explanations. These children may be missing out on opportunities to express implicit knowledge and, perhaps, missing out on opportunities to learn. But it turns out that mismatches can be coaxed out of a learner by encouraging gesture. If an instructor asks a child to gesture while explaining how to solve a math problem, many of the gestures that the child produces not only convey strategies that are not found in the child’s speech, but those strategies are also correct. When the children are later given a math lesson, the mismatching gestures that result from encouraged gesture turn out to predict learning, just as spontaneously produced mismatches do (Reference Broaders, Cook, Mitchell and Goldin-MeadowBroaders, Cook, Mitchell, & Goldin-Meadow, 2007). Encouraging children to gesture thus appears to bring out implicit knowledge and help learners consider novel ideas, which, in turn, facilitate learning.

Although the two-modality system of communication (i.e. language through the spoken modality, gesture through the manual modality) provides an ideal vehicle for two conflicting ideas to emerge, it is not a necessary component. Deaf individuals gesture along with their signs (Reference Emmorey, Messing and CampbellEmmorey, 1999; Reference SandlerSandler, 2009; Wilcox, this volume) and both gesture and sign are produced in the manual modality. Even though this is, for the most part, a one-modality system of communication, deaf children still produce sign-gesture mismatches on missing addend equivalence problems (Reference Goldin-Meadow, Shield, Lenzen, Herzig and PaddenGoldin-Meadow, Shield, Lenzen, Herzig, & Padden, 2012). Importantly, as in their hearing counterparts, deaf children who produce mismatches in sign and gesture are more likely to benefit from math instruction than deaf children who do not produce mismatches. Gesture’s ability to predict readiness-to-learn thus stems not from the fact that two different modalities are involved in the explanation, but more likely from the fact that the two modalities use different representational formats – gesture conveys information gradiently, whereas language (speech or sign) conveys information discretely.

The fact that mismatches index when a child is in a transitional state can be a useful cue for teachers and parents who want to help their students and children learn. If shown videos of children explaining their solutions to mathematical equivalence problems, adults use information in children’s’ gestures to assess their knowledge (Reference Alibali, Flevares and Goldin-MeadowAlibali, Flevares, & Goldin-Meadow, 1997). Adults are thus sensitive to the variability that children display across modalities. Moreover, they can use the information conveyed uniquely in gesture to determine how likely a child is to benefit from instruction. Teachers also adapt their instruction based on their students’ gestures. Teachers provide more variable instruction, and more different types of problem-solving strategies, to children who produce mismatches than to children who do not produce mismatches. Child gesture can thus play a role in getting teachers to adapt their teaching methods in the moment (Reference Goldin-Meadow and SingerGoldin-Meadow & Singer, 2003). The mismatching state, as reflected in the relation between a child’s speech (or sign) and gesture, can be profitably exploited by adults to support child learning.

3.2 Producing Gesture Promotes Learning Math Concepts

Children’s mismatching gestures reflect an internal readiness to learn a particular concept. Can the act of producing a mismatching gesture also play a causal role in the learning process itself? To address this question, Reference Goldin-Meadow, Cook and MitchellGoldin-Meadow, Cook, and Mitchell (2009) essentially turned children into mismatchers by giving them a math lesson and instructing them to produce mismatching speech and gesture strategies. They taught children to say one correct strategy in speech while producing a second, correct (mismatching) strategy in gesture. Children, who were “turned into” mismatchers benefited from instruction significantly more than children who were taught only the single correct strategy in speech. Mismatch between speech and gesture not only reflects readiness to learn a particular concept, but it can even play a causal role in learning that concept.

The causal effects of producing gesture on learning can extend beyond the immediate learning context. Children taught to produce gesture during instruction show continued learning boosts at least four weeks after the lesson (Reference Cook, Mitchell and Goldin-MeadowCook, Mitchell, & Goldin-Meadow, 2008). This lasting learning may be tied to underlying neural pathways that encode gesture production during learning. One study using functional magnetic resonance imaging (fMRI) found that children who were taught to produce gesture during a math lesson later recruited their motor networks when passively solving equivalence problems while lying still in the scanner and solving problems without moving their hands (Reference Wakefield, Congdon, Novack, Goldin-Meadow and JamesWakefield, Congdon, Novack, Goldin-Meadow, & James, 2019). In other words, the experience of learning through gesture influences the way that children learn and changes the way that children later access that learned knowledge. Producing gesture thus has a lasting influence, both on learning and on the brain.

Thus far, we have focused on how producing gesture can support learning. But it is important to remember that gestures do not occur in isolation – they occur together with speech. And although the gesture itself is important, the way that gesture complements, interacts with, and reinforces speech may also play a critical role in the effects gesture has on learning.

One finding that demonstrates the powerful influence of producing gesture with speech comes from a study in which one group of children was taught to produce speech and gesture during a math lesson, and another group was taught to produce speech and action (an action that was similar to the gesture, but involved manipulating objects) (Reference Novack, Congdon, Hemani-Lopez and Goldin-MeadowNovack, Congdon, Hemani-Lopez, & Goldin-Meadow, 2014). In both groups, children were taught the equalizer strategy in speech; they were taught to say, “I want to make one side equal to the other side.” Both groups were also taught the grouping strategy (the idea that the problem can be solved by grouping the non-repeated addends on the left side of the problem and putting the sum in the blank on the right side) in either action or gesture. In the gesture group, the grouping strategy was expressed by making a V-point with the index and middle fingers to the two numbers on the left side that can be added together. In the action group, the strategy was expressed by physically picking up two magnetic number tiles placed over the addends in the problem. Thus, both the action and gesture represented a distinct strategy from the equalizer strategy expressed in speech, and both provided mismatching and supplementary information to the speech strategy.

In this study, children who produced gesture with speech showed greater depth of learning than children who produced action with speech (Reference Novack, Congdon, Hemani-Lopez and Goldin-MeadowNovack et al., 2014). But, importantly, how children understood the speech they were taught to say predicted the extent of their learning. At the end of the study, children’s learning and generalization was measured in a posttest assessment, and they were asked to explain their reasoning to the posttest problems. Children in both conditions tended to repeat the verbal strategy they had learned during the lesson, mimicking the words they were taught and thus expressing the equalizer strategy in speech. But for children who had learned the speech strategy together with gesture, posttest use of equalizer in speech predicted learning and generalization scores – the more they used the verbal strategy, the higher their posttest and generalization scores, indicating that they had truly understood the meaning of the words they were taught. In contrast, for children who had learned the speech strategy together with action, posttest use of equalizer in speech did not predict learning or generalization scores. In other words, children who learned a speech strategy while producing gesture seemed to have internalized the meaning of the rotely produced words, whereas children who learned the same speech strategy while producing action did not. The combined effect of speech and gesture (but not speech and action) thus appears to help learners make sense of speech, helping to tie that verbal information into their broader conceptual understanding.

3.3 Seeing Gesture Promotes Learning Math Concepts

The gestures that learners see can also have a significant and influential impact on learning math concepts. Here again, it is not just gestures in isolation, but rather the relationship between gestures and the accompanying speech, that is critical for learning outcomes. Teachers naturally gesture when explaining concepts, and this is particularly true in math classrooms where gesturing is routinely used to ground and link ideas (Reference Alibali, Nathan, Wolfgram, Church, Jacobs, Johnson Martinez and KnuthAlibali et al., 2014; Reference Alibali and NathanAlibali & Nathan, 2012). For example, teachers asked to explain mathematical equivalence to students spontaneously gesture and convey substantive information in those gestures, which is relevant to the lesson (Reference Goldin-Meadow, Kim and SingerGoldin-Meadow, Kim, & Singer, 1999). Children are more likely to reiterate correct speech when it is accompanied by matching gestures than when it is accompanied by no gesture at all. Gesturing by teachers can thus support speech and, as a result, facilitate student comprehension of a lesson.

Children can also benefit when the information conveyed in gesture does not match the information conveyed in speech. Children are more likely to learn from instruction in which a teacher provides different, but potentially integrable, information in speech and in gesture than when the teacher provides the same information in speech and in gesture (Reference Singer and Goldin-MeadowSinger & Goldin-Meadow, 2005). If a teacher expresses one problem-solving strategy in speech and a second strategy in gesture, children are more likely to learn how to solve the problem than if the teacher expresses both strategies in speech, or if she expresses only one of the strategies in both speech and gesture. Children thus benefit when they are provided with multiple strategies, but only if those strategies are presented across speech and gesture. Getting multiple strategies, all in the verbal domain, appears to be too much information for the child to take in. Gesture provides a method of presenting multiple ideas to children in a format that they can learn from.

If different strategies in speech and gesture support optimal learning, do they have to be presented at the same time? Or can gesture benefit learners even if it is not presented simultaneously with speech? It turns out that the simultaneous timing of speech and gesture is key to its learning benefits. If speech is presented before a gesture strategy, then the learning benefits of gesture instruction are greatly reduced (Reference Congdon, Novack, Brooks, Hemani-Lopez, O’Keefe and Goldin-MeadowCongdon et al., 2017). Children who receive math instruction with simultaneous speech and gesture are more likely to generalize their knowledge, and retain that knowledge, compared to children who receive instruction with speech followed by gesture (or instruction with one speech strategy followed by a second speech strategy). When asked to explain their solutions at the end of the study, children who successfully learned from simultaneous speech and gesture instruction also integrated the speech strategy into their own understanding of the problems. Simultaneously presented speech and gesture seemed to solidify the speech, helping children internalize the meaning of that speech.

Wakefield and colleagues (Reference Wakefield, Novack, Congdon, Franconeri and Goldin-MeadowWakefield, Novack, Congdon, Franconeri, & Goldin-Meadow, 2018) used eye tracking to probe how children visually attend to gesture when it is presented simultaneously with speech. Children saw a teacher explain how to solve missing addend equivalence problems. In a Speech + Gesture condition, the instructor said the verbal equalizer strategy and, at the same time, produced a gestural grouping strategy. In a Speech Alone condition, the instructor provided the same verbal strategy but without moving her hands. As previous literature would lead us to expect, children were more likely to learn how to solve the equivalence problems, as measured by a posttest assessment, after Speech + Gesture instruction than after Speech Alone instruction. Importantly, however, eye-tracking evidence revealed differences in how children in the two conditions attended to instruction, and how their visual attention linked to posttest learning. Children in the Speech + Gesture condition looked at the problem more than children in the Speech Alone condition. However, amount of looking at the problem did not predict children’s learning outcomes. Instead, it was the timing of learners’ looks at the problem that predicted the depth of their learning. Children in the Speech + Gesture condition were more likely to look at the part of the problem that the instructor was mentioning in speech, at the time when the instructor mentioned it, than children in the Speech Alone condition. The gestures helped children follow along with speech. But some children in the Speech Alone condition were able to follow along with speech. Importantly, however, the degree to which children followed along with speech predicted how well they performed on a posttest only in the Speech + Gesture condition, not the Speech Alone condition. In other words, following along with the instructor’s speech was beneficial only when gesture co-occurred with that speech, and presumably helped learners glean information from the speech.

3.4 Summary

Gesture works together with speech to support learning in school-aged children. Gesturing on a task reveals when children are ready to profit from instruction in that task. Producing one’s own gestures can change knowledge, as does seeing other people’s gestures. Here we have focused on gesture’s role in learning within the domain of math. But gesture indexes who is ready to learn in tasks such as Piagetian conservation (Reference Church and Goldin-MeadowChurch & Goldin-Meadow, 1986) and moral reasoning (Reference Beaudoin-Ryan and Goldin-MeadowBeaudoin-Ryan & Goldin-Meadow, 2014); producing gesture helps children learn in domains such as word concepts (Reference Wakefield and JamesWakefield & James, 2015) and second language acquisition (Reference Macedonia, Müller and FriedericiMacedonia, Müller, & Friederici, 2011); and seeing gesture helps children learn about topics such as bilateral symmetry (Reference Valenzeno, Alibali and KlatzkyValenzeno, Alibali, & Klatzky, 2003) and mental rotation (Reference Goldin-Meadow, Levine, Zinchenko, Yip, Hemani and FactorGoldin-Meadow, Levine et al., 2012). The effects of gesture on learning thus extend across learning contexts, suggesting that gesture can play a major role in how children learn.

4.1 Open Questions about How Gesture Indexes Knowledge

The gestures learners spontaneously produce on a task provide insight into when they are ready to profit from instruction in that task. Moreover, encouraging learners to gesture on a task brings out implicit knowledge of the task, which, in turn, makes learners more likely to profit from instruction on the task. As described earlier, Reference Broaders, Cook, Mitchell and Goldin-MeadowBroaders et al. (2007) asked school-aged children to use their hands to explain solutions to their problems, and found that this request increased the number of speech–gesture mismatches children produced, which subsequently led to greater learning outcomes. Similarly, Reference LeBarton, Goldin-Meadow and RaudenbushLeBarton et al. (2015) asked toddlers to point to photos, by saying “Can you put your finger here?” This training manipulation led to an increase in spontaneous pointing gestures with caregivers, which subsequently led to greater vocabulary growth. Explicitly asking learners to use their hands is one way to effectively encourage gesture. But is it the only way?

Reference Cook and Goldin-MeadowCook and Goldin-Meadow (2006) tried to address this question by asking whether children are more likely to gesture if they are asked to gesture (i.e. if they are explicitly told to imitate their teacher’s gestures) or if they just see their teacher gesture during a lesson. A control group was taught by a teacher who did not use gesture. Whether or not they were explicitly told to imitate their teacher’s gesture, children who saw their teacher gesture were more likely to gesture themselves, compared to children who saw instruction without gesture. The mere presence of gesture in a learner’s input (at least, from a teacher) thus has the potential to boost learner gesture production, which can then lead to greater learning gains. Although this study offers some insight into how to boost gesture, it is still not clear whether there are more effective ways to get children to gesture than merely modeling gesture.

An additional question is: What is the best way to increase gesture in a child’s input? Are children more likely to increase their own gesture production if their parents and teachers gesture more, versus if their classmates and siblings gesture more? And how might a child’s relationship with these various social partners matter? That is, would children be more likely to imitate their teacher’s gestures if they liked that teacher, or felt connected to that teacher, compared to if they did not like that teacher? Future work could consider the role that social factors play in encouraging gesture production, and whether those factors can be manipulated to boost gesture in the classroom or at home.

4.2 Open Questions about How Gesture Changes Knowledge

Focusing first on the impact that producing gesture has on learning, we need to ask whether gesture benefits all children, or whether there are certain populations, or children at certain conceptual stages, for whom gesture is particularly helpful. Some studies have found that producing gesture during instruction is differentially beneficial for specific groups of children (Reference Congdon, Kwon and LevineCongdon, Kwon, & Levine, 2018; Reference Wakefield, Foley, Ping, Villarreal, Goldin-Meadow and LevineWakefield, Foley et al., 2019). For example, in a task designed to teach first-grade children about linear measurement, children using a more conceptually advanced (although still incorrect) strategy at the start of the study were more likely to benefit from producing gesture during instruction than children using a more rudimentary (incorrect) strategy (Reference Congdon, Kwon and LevineCongdon et al., 2018). This study raises the question of how individual differences, such as a child’s starting state with respect to a task, affects when a child is most likely to benefit from producing gesture in the task. Future work that adopts an individual-differences approach might be particularly useful for translating gesture findings into classroom settings.

Turning next to the impact that seeing gesture has on learning, we need to explore gesture’s role in classrooms and learning environments as online materials are incorporated into their practices. Teachers now often use online teaching materials in the classroom, and students themselves use internet resources (such as YouTube) to look for instructional videos at home. Future research therefore needs to consider how gesture can be best used in online educational content. Surprisingly, only a minority of online instructional videos include sight of an instructor’s hands, thus giving no opportunity for gesture. One study found that only thirty-two percent of randomly selected YouTube instructional math videos showed hands; the rest presented instructional content without any humans at all (Reference Son, Ramos, DeWolf, Loftus and StiglerSon, Ramos, DeWolf, Loftus, & Stigler, 2018). Of the videos that showed hands, almost all included some sort of gesture, although they were primarily pointing gestures. Research suggests the learning from gesture online has equivalent effects to learning from gesture in person (Reference Koumoutsakis, Church, Alibali, Singer and Ayman-NolleyKoumoutsakis, Church, Alibali, Singer, & Ayman-Nolley, 2016), but more research is needed to best know how to incorporate gesture into online materials. For example, videos can use gesture, but also visual highlighting, animated content, or multiple screens, all to convey multiple ideas at the same time. Are these techniques equally effective? These questions need to be asked by gesture researchers given the changing dynamics of modern learning.