1.3.1.1 Effect of Gestures on Speech Conceptualization

Speech planning and speech fluency are tightly linked. Numerous studies emphasize the supportive role of gestures in accessing lexical items or concepts during speech production (Graham and Heywood, Reference Graham and Heywood1975; Morsella and Krauss, Reference Morsella and Krauss2004; Hostetter et al., Reference Hostetter, Alibali and Kita2007; Hoetjes et al., Reference Hoetjes, Krahmer and Swerts2014; Cravotta et al., Reference Cravotta, Busà and Prieto2019). These studies looked at the effect of gesturing (compared to not being able to gesture) on lexical content, but also on the number of pauses, speech rate, and prosody. The lexical retrieval hypothesis, supported by Krauss (Reference Krauss1998) and Ruiter (Reference Ruiter1998), posits that gestures play a crucial role in lexical access, facilitating access to concepts and words in memory. This theory echoes studies highlighting the neural connections between semantic comprehension of actions and corresponding sensorimotor circuits (Pulvermüller and Fadiga, Reference Pulvermüller and Fadiga2010; Bidet-Ildei et al., Reference Bidet-Ildei, Beauprez and Badets2020). Besides aiding memory retrieval, gestures help organize information for verbal encoding, especially spatio-motoric details (Alibali et al., Reference Alibali, Kita and Young2000; Kita et al., Reference Kita, Alibali and Chu2017). Gestures support the activation of spatio-motoric information and problem-solving in both concrete and abstract thinking, potentially influencing the course of speaking, facilitating novel ideas and affecting discourse orientation. Through this process, gestures influence the packaging of verbal information into clauses. For example, when describing the manner (e.g., rolling) and path (e.g., down) of a motion with two separate gestures, participants tend to produce a two-clause verbal description, while a single gesture leads to a single-clause verbal description (see Kita et al., Reference Kita, Alibali and Chu2017 for references). This role attributed to gestures may have correlates in speech rhythms that require further investigation.

Various hypotheses surround the roles of gestures – whether they serve conceptual or communicative purposes – prompted by the question of whether speech and gestures are distinct or stem from a unified internal representation. Beyond their communicative role, gestures serve pragmatic functions, supporting interpersonal coordination in spoken conversations.

1.3.1.2 Role of Gestures in Conversational Stream

The role of nonverbal communication in dialogue is briefly highlighted here, emphasizing gestures’ contribution to the rhythm of interpersonal verbal exchanges. Gestures serve as visual cues in conversations and convey a variety of information supporting interpersonal convergence and coordination (Holler, Reference Holler2022). They often precede the verbal message or convey additional information without interrupting speech. Specifically, gestures are integral to backchannels and turn-taking processes (Wagner et al., Reference Wagner, Malisz and Kopp2014), which are indispensable for dialogue fluidity and pace. Gestures, and nonverbal communication in general, offer real-time feedback to speakers, aiding in understanding listener processing without disrupting ongoing discourse. This mechanism allows speakers to assess communication success, aiding anticipation of semantic shifts, topic changes, and turn-taking events (Holler and Levinson, Reference Holler and Levinson2019). For instance, Ter Bekke et al. (Reference Ter Bekke, Drijvers and Holler2024) found quicker responses to questions accompanied by iconic gestures. Electroencephalographic studies show that brain responses in dyadic interactions are modulated by the processing of nonverbal cues (Zhang et al., Reference Zhang, Frassinelli, Tuomainen, Skipper and Vigliocco2021). More specifically, beat gestures facilitate semantic processing (Wang and Chu, Reference Wang and Chu2013) and syntactic parsing (Obermeier and Gunter, Reference Obermeier and Gunter2014; Biau et al., Reference Biau, Fromont and Soto‐Faraco2018).

Co-speech gestures significantly influence the timing of spoken communication, affecting thinking, speech production, and conversational tempo. Coordination between speech and co-speech gestures is proposed to be rooted in the conceptual phase, relying on common meaning (McNeill, Reference McNeill1992). Additionally, sensorimotor connections between articulatory and limb control, especially hand and mouth, may underpin coordination between speech and co-speech gestures.

1.3.2.1 Manual and Vocal Development towards Spoken Language

Low-level motor links appear early in ontogenesis and may establish the connection between speech and co-speech gestures (see also Chapter 38). Fetal initiation of facial touching, particularly around the mouth, begins between 12 and 15 weeks of gestation (Gallagher, Reference Gallagher, Bermúdez, Eilan and Marcel1995; Fagard, Reference Fagard2013). At birth, about one-third of hand movements result in hand–mouth contact, with coordination advancing rapidly, especially when the child is hungry (Gallagher, Reference Gallagher, Bermúdez, Eilan and Marcel1995). Rhythmic coupling between limb motion and vocal tract control for vocalization typically begins around six months, serving as a basis for hand-to-mouth coordination development (Iverson and Thelen, Reference Iverson and Thelen1999). In particular, a peak in the frequency of occurrence of rhythmic hand activities, such as rattle shaking, is observed at the onset of canonical babbling (see Iverson and Wozniak, Reference Iverson and Wozniak2007). These concurrent activities also influence each other, with babbling accompanied by hand movements displaying longer syllable lengths than babbling without hand movements (Ejiri and Masataka, Reference Ejiri and Masataka1999). Repetitive entrainments between hand movements and the vocal tract may set the stage for the speech–gesture system’s emergence in the last month of the first year, featuring more controlled gestures such as pointing, followed by spoken words a few weeks later. Gestures play an important role in both lexical and syntactic development and are combined with speech to convey meaning through more and more complex morphosyntax (Iverson and Goldin-Meadow, Reference Iverson and Goldin-Meadow2005; Özçalışkan and Goldin-Meadow, Reference Özçalışkan and Goldin-Meadow2005). Low-level sensorimotor coupling persists in adults, providing insights into the enduring link between co-speech gestures and speech in communication.

1.3.2.2 Speech–Gesture Synchrony: A Somatosensory Phenomenon?

Pointing gestures have been a focal point in the speech–gesture coordination investigation (Levelt et al., Reference Levelt, Richardson and La Heij1985; Rochet-Capellan et al., Reference Rochet-Capellan, Laboissière, Galván and Schwartz2008; Chu and Hagoort, Reference Chu and Hagoort2014). Bidirectional adaptations between speech and pointing gestures have been observed during perturbation of either gestures or speech, preserving their synchronization (Chu and Hagoort, Reference Chu and Hagoort2014). More specifically, Rochet-Capellan et al. (Reference Rochet-Capellan, Laboissière, Galván and Schwartz2008) found that mutual adaptation occurs between speech and pointing gestures, such as to synchronize the lexical stress with the gesture apex and pointing phase. Another type of gestures related to speech prosody are beat gestures. Beat gestures are rhythmic, baton-like gestures that have been shown to modulate prosodic prominence (Krahmer and Swerts, Reference Krahmer and Swerts2007; Pouw et al., Reference Pouw, Harrison and Dixon2020a; see also Section 1.4 of this chapter).

In spontaneous narration, McNeill (Reference McNeill1992) found gestures and speech synchrony to be unaffected by delayed auditory feedback (DAF) but disrupted in scripted situations. These findings align with the Growth Point hypothesis suggesting that synchronization is rooted in the conceptual phase. According to Pouw and Dixon (Reference Pouw and Dixon2019), the coupling between gestures and speech is even more stable under DAF conditions, which suggests that “gesture with its own intrinsic dynamics (and thus different entrainment to DAF) can be utilized to resist DAF in speech production” (p. 27).

Coordination patterns between speech and hand motion have also been found through noncommunicative movements. Studies of spatiotemporal coupling between vocalizations and finger tapping (Kelso et al., Reference Kelso, Tuller, Harris and MacNeilage1983; Parrell et al., Reference Parrell, Goldstein, Lee and Byrd2014; Zelic et al., Reference Zelic, Kim and Davis2015) have found some evidence of hand and mouth being controlled as a single coordinative structure. Their results suggest that the coupling between prosody and hand movements may rely on broad motor control and somatosensory feedback not specific to speech, as it persists during silent speech. Finally, the hand and the mouth seem to be controlled by a common motor command (Gentilucci and Volta, Reference Gentilucci and Volta2008): during the co-occurrence of object picking and vocalizing, the lip aperture and the first formant increase with object size. Further support for this hypothesis comes from neurophysiological findings about Broca’s area, believed to exert control over both the mouth and the hand (Fadiga et al., Reference Fadiga, Craighero and D’Ausilio2009). The sensorimotor connections and common rhythmic patterns between speech and limb movements have been shown to play a key role in speech rehabilitation for patients with Parkinson’s disease and patients with stuttering symptoms (see also Chapter 45).

However, the generalization of synchronization patterns of natural speech with hand movements remains incompletely understood (Parrell et al., Reference Parrell, Goldstein, Lee and Byrd2014). While cognitive processes related to spoken language or motor-level origins for speech–limb coordination are open questions, recent exploration suggests breathing may play a significant role in this coordination.

1.4.1.1 Effect of Exercise on Speech Breathing and Pauses

When speaking during physical exercise, speech breathing must accommodate the heightened oxygenation needs of the body. Unlike speaking at rest, speaking during physical activity involves shorter exhalations, increased airflow during exhaling, and longer, deeper inhalations (Hixon et al., Reference Hixon, Goldman and Mead1973, Reference Hixon, Mead and Goldman1976). Consequently, the syntactic structure of speech undergoes modifications. Baker et al. (Reference Baker, Hipp and Alessio2008) examined the impact of different physical effort intensities in cycling on a reading speech task. They observed an increase in the number of non-syntactic pauses, a reduction in syllable count within breath groups, and an increase in articulation rate. Fuchs et al. (Reference Fuchs, Reichel and Rochet-Capellan2015) found that while engaged in semi-spontaneous speech, individuals exhibited a heightened frequency of breathing when cycling compared to speaking alone. Moreover, this trend became more pronounced with an increased level of biking effort. Trouvain and Truong (Reference Trouvain and Truong2015) investigated reading speech before and after volitional exhaustion on a treadmill. They found that both the duration and the intensity of inbreath segments increase from before to after, and a tendency to observe a higher count of inbreath tokens per second in the post-phase compared to the pre-phase.

1.4.1.2 Effect of Speech on Exercise Breathing

Despite changes in speech organization and in particular pauses induced by physical effort, there is an adaptation in breathing patterns to preserve speech quality. Interestingly, the elevation in ventilation during vocalization is less noticeable, and the respiratory frequency is lower, in comparison to exercising without speaking (Doust and Patrick, Reference Doust and Patrick1981). However, there is a limit to sustaining speech quality when physiological demands reach a certain threshold. In sports studies, the talk test instrumentalized speech during exercise to evaluate and guide exercise training intensity for each individual. The exercise intensity at which individuals initially “experience difficulty speaking comfortably” (utilizing various speech stimuli) serves as a dependable indicator of the ventilatory threshold in healthy individuals (see Foster et al., Reference Foster, Porcari and Ault2018). Recent research indicates that the point at which normal speech can no longer be maintained aligns with the onset of metabolic acidosis during escalating exercise intensity (De Lucca et al., Reference De Lucca, de Oliveira, Foster and Carminatti2021). While sports studies traditionally overlook speech changes induced by physical exertion, recent research in language and cognitive sciences has witnessed a growing interest in investigating these phenomena.

1.4.1.3 Effect of Exercise on Speech Signals

The intricate relationship between speech breathing and physical effort extends to speech acoustics, where parameters change with increased exertion, including a rise in average intensity and fundamental frequency (F0) during biking (Mohler, Reference Mohler1982; Fuchs et al., Reference Fuchs, Reichel and Rochet-Capellan2015; Weston et al., Reference Weston, Fuchs and Rochet-Capellan2020; Serré et al., Reference Serré, Dohen, Fuchs, Gerber and Rochet-Capellan2022) or treadmill tasks (Primov-Fever et al., Reference Primov-Fever, Lidor, Meckel and Amir2013; Trouvain and Truong, Reference Trouvain and Truong2015). Trouvain and Truong proposed that this F0 increase may be attributed to heightened subglottal pressure during physical exertion. Conflicting findings on F0 changes during limb movements and exertion levels highlight significant inter-speaker variability, presenting diverse and sometimes opposing behaviors (Godin and Hansen, Reference Godin and Hansen2008, Reference Godin and Hansen2015; Weston et al., Reference Weston, Fuchs and Rochet-Capellan2020). The relationship between exertion level and F0 in speech is complex. Johannes et al. (Reference Johannes, Wittels and Enne2007) observed that F0 is primarily affected by exercise near exhaustion. These varied outcomes suggest that the impact of physical stress on speech acoustics is neither linear nor constant, emphasizing the need for a detailed examination at smaller scales. Crucial factors include the unique articulation space for each consonant, affecting sensitivity to subglottal pressure fluctuations, and considerable variability in limb movement dynamics influenced by biomechanical mechanisms (Godin and Hansen, Reference Godin and Hansen2011) that may also involve breathing (Pouw et al., Reference Pouw, De Jonge-Hoekstra, Harrison, Paxton and Dixon2020b).

1.4.2.1 A Biomechanical Anchor of Speech–Breathing–Limb Interaction

Pouw et al. (Reference Pouw, Harrison and Dixon2020a, Reference Pouw, De Jonge-Hoekstra, Harrison, Paxton and Dixon2020b) found correlates of arm motions in vocalizations. In particular, when participants imitated a beat gesture in a rhythmic fashion during syllable production or spontaneous speech, the amplitude envelope and F0 increased close to the deceleration peak of the arm movement. The authors introduce the hypothesis of a biomechanical interplay between arm breathing and speech in relation to anatomical constraints between the arms and the breathing system. While moving the arms, thoracic muscles are activated to maintain posture and support limb movements. This activation triggers rib cage displacements, impacting the lungs. When the arms move such as in beat gestures, this induces changes in the respiratory system, leading to an increase in subglottal pressure, and changes in acoustical parameters of co-occurring vocalizations. In line with this idea, Serré et al. (Reference Serré, Dohen, Fuchs, Gerber and Rochet-Capellan2022) observed a correlation between the amplitude envelope of speech acoustic peaks and leg deceleration events during biking, further supporting possible biomechanical interplays involving body movement, breathing, and speech production, not limited to the arms.

Recently, Pouw and Fuchs (Reference Pouw and Fuchs2022) argue that this biomechanical entanglement extends beyond immediate physiological effects, potentially influencing coordinated speech patterns. They propose that this link may have evolutionary significance, contributing to the development of finely tuned vocalizations in species with a diverse coupling of breathing and limb movements, particularly in bipedal species where upper bodies are liberated during locomotion.

However, as much as the biomechanical entanglement of limb, respiratory, and vocal systems can explain the impact of limb movements on speech acoustics, and may contribute to speech and gesture coordination, it can’t account for the entire variability of speech. If speech, breathing, and limb movements obviously share the same physical body substrate, and if motion in one part of this body might spread to others through biomechanical constraints, they also share the same brain as well as sensorimotor and social experiences.

1.4.2.2 The Interplay between Cognitive and Physiological Factors

Distinguishing between the effect of cognitive and physiological factors on speech is not always easy, and hypotheses assuming only one cause often end in a chicken-and-egg loop. For example, in ontogenesis, researchers noted a correlation between the growth of lung capacity and the increasing length of utterances (Bouchet and Lalonde, Reference Boucher and Lalonde2015). However, at the same time, cognitive abilities are also changing. In adulthood, there is no clear evidence that speakers with higher lung capacity are likely to produce longer utterances (Heldner et al., Reference Heldner, Carlsson and Wlodarczak2019). While exploring the involvement of the forebrain in individual tidal breathing patterns, Shea et al. (Reference Shea, Horner, Benchetrit and Guz1990) noted varying cycle shapes among individuals with similar body types and observed diverse body types exhibiting similar cycle shapes. Moreover, motor-respiratory coordination in humans is highly flexible, sensitive to factors such as cognitive effort (Hessler and Amazeen, Reference Hessler and Amazeen2014). New MRC patterns can also be intentionally learned to improve energy consumption during specific physical effort (Hessler and Amazeen, Reference Hessler and Amazeen2014). Furthermore, the modification of speech breathing through learning is evident, as seen in actors and actresses (Master et al., Reference Master, Guzman, Azócar, Muñoz and Bortnem2015). Breathing is also intertwined with cognition, attention processes (Allen et al., Reference Allen, Varga and Heck2023), as well as various behavioral factors related to motor, emotional, physiological, and cognitive processes (Shea, Reference Shea1996; Grassman et al., Reference Grassmann, Vlemincx, Von Leupoldt, Mittelstädt and Van den Bergh2016; Park et al., Reference Park, Barnoud and Trang2020). In this respect, the breathing profiles are markers of ongoing activities of the body and mind as well as speaker specificities and training. The study of the impact of practices such as swimming, free diving, or yoga on breathing during speech and the organization of pauses could bring forth interesting insights regarding the effects of training.

Investigating speech breathing consistency over time and across different limb movements (leg versus arm biking), Serré et al. found that speech breathing cycles are consistent over time and across tasks, despite a significant effect of the limb deceleration on speech acoustics observed in the same dataset (Reference Serré, Dohen, Fuchs, Gerber and Rochet-Capellan2022). The temporal scale of this latter biomechanical aspect is less than a second, while speech breathing cycles take place over three to four seconds on average. The entanglement of these different scales suggests a complex interaction between multilevel rhythms shaping spoken communication (Pouw et al., Reference Pouw, Proksch and Drijvers2021). The emergence of such rhythms might depend on the speaker’s environment and peculiarities such as breathing capacities or spontaneous speech rate, embedded in speech control across individual experience.

1.4.2.3 Breathing as a Bridge Built through Grounded Experiences

Speech, breathing, and limb movements exhibit an interconnected evolution in both phylogenesis and ontogenesis (see also Chapter 7). This coevolution stems from anatomical overlap and connections among the three systems, further supported by the intricate development of peripheral and central nervous systems. This development occurs through concurrent activation in daily communicative experiences from birth. Despite their interdependence, research has predominantly explored speech, breathing, and limb movements separately or in pairs. On the one hand, studies on breathing and limb movements are often driven by sports and physiological approaches, to investigate energy expenditure and expert motor control. On the other hand, fields related to linguistics and cognition focus on the relationship between speech and breathing, neglecting limb movements, with minimal consideration for breathing. As evidenced in this chapter, breathing is a resource intrinsic to speech and limb movements and adapts to speech via utterance planning and to limb movements via afferent connections and chemoreceptors that transmit metabolic needs. Since limb and speech movements frequently co-occur, the brain may have developed specific abilities to control them jointly (Bernstein, Reference Bernstein1967; Latash et al., Reference Latash, Scholz and Schöner2007) and the shape of our evolved behaviors might be strongly determined by physiological factors. Coordinative structures between speech, breathing, and limbs might be learned in a specific way through experiences. Adopting an integrated view of this three-bodies issue might shed light on the complex nature of speech rhythms and their grounding in breathing and body movements.