29.1 Introduction
Interaction Phonology (Wagner et al., Reference Wagner and Niebuhr2013) postulates a process of rhythmic coordination based on entrainment processes that provide the temporal scaffold for higher-order adaptation among interlocutors in critical situations, and hence improves communication. Ten years after the publication of our framework, the time is more than ripe for its first evaluation and a thorough reassessment. To achieve this, I will first give an overview of the general assumptions and motivations underlying Interaction Phonology, and then describe its mechanism as a logistic, attention-guiding component in a model of speech processing in interaction. I will then derive a set of model predictions and evaluate them based on a thorough review of more recent empirical studies. In a last step, I will slightly modify our original model of Interaction Phonology (see Figure 29.1 for an overview of the original model; see Figure 29.2 for the adapted version), and list desiderata for its further testing in the future.
An overview of the processes and structures involved in Interaction Phonology.
The diagram depicts the processes in a listener who entrains to the rhythmic patterns of speech based on the expectations inherent in their language competence. The level of rhythmic-prosodic entrainment can be strengthened in difficult communicative situations. That way, the listener’s attention is guided to higher-order linguistic aspects connected to the rhythmic structures thus enhanced. This attentional process may alter the way that rhythmic-prosodic structures are connected to higher-order linguistic patterns, but also intensify the level of entrainment with an interlocutor. Taken together, these processes are expected to aid mutual understanding, particularly in “difficult” situations. The model relies on a set of modules, some of which are part of the speaker’s grammar. These encompass (1) an entrainment module, (2) an auditory analysis guided by it, which is also linked to (3) motor patterns, which automatically lead to convergence in speech production as an automatic by-product of entrainment, (4) a set of linguistic structures and expectations as part of a speaker’s grammar, which are linked to the levels of entrainment via their corresponding levels of prosodic organization, and (5) a monitoring of communication relevance, which estimates the need for entrainment (informed by the auditory and linguistic analysis) and adjusts the level of entrainment by modulating the coupling strength.
An adapted sketch of Interaction Phonology.
Those parts of Interaction Phonology that have received empirical support are indicated by check marks. Other parts are either commented as optional (auditory-motor mapping and speech adaptation) or have been modified/extended in line with empirical findings. In particular, the language-specific structures and expectations for which we have evidence to guide rhythmic-prosodic entrainment and to be shaped by it currently are restricted to phonetic-phonological ones. It remains unclear whether syntactic or lexical adaptations are connected with entrainment processes likewise.
Figure 29.2 Long description
The diagram shows how incoming speech is processed through rhythmic-prosodic entrainment, phonetic-phonological structures, and linguistic categorization, ultimately leading to motor coordination and the assessment of communicative needs. Stages like rhythmic-prosodic entrainment, guides attention, auditory categorization, phonetic-phonological structures and expectations, entertainment strength guided by communicative needs are indicated with check marks.
29.2 A Sketch of Interaction Phonology
When two or more people communicate, they agree on a shared language system, with the ultimate goal to enable a common understanding with the help of an interactionally grounded, shared symbolic representation. However, assumptions about the shared symbol inventory may differ. For instance, whether you refer to certain vegetables as “potatoes,” “spuds,” “solanum tuberosum,” or “root vegetables” may depend on your individual assessment of the situation, individual preference, spoken variety, or linguistic context. It is likely that speakers will therefore negotiate the conditions of usage of a particular term to clarify reference or to signal mutual cooperativeness and perspective taking in a process called grounding (Clark and Brennan, Reference Clark, Brennan, Resnick, Levine and Teasley1991). During this process, it is not sufficient to agree on a shared inventory of symbols and grammatical constraints (e.g., “English”), because the way that abstract symbols are realized in the speech signal may differ, due to different speaking styles, varieties, registers, or external factors such as cognitive distraction or various types of “noise.” For this reason, sub-symbolic phonetic convergence has been claimed to be closely linked to the phenomenon of symbolic alignment, that is, the tendency of interlocutors to agree on a shared or similar inventory (Pickering and Garrod, Reference Pickering and Garrod2004).
So, agreeing on speaking the same “language” has something in common with two people agreeing on dancing a waltz. While the dance move sequences that qualify as “symbolic” figures of a waltz may be clear to both dancers, the velocity, amplitude, and detail of the pertaining movement trajectories need to be precisely negotiated, helped by an external pacemaker in the form of the rhythm of the accompanying music. In speech-based communication, it is likewise not sufficient to agree on an abstract set of phonemes, lexemes, and syntactic structures. Rather, speakers need to agree on a fine-grained execution of the shared movement patterns within their individual motor systems to achieve pronunciations that are mutually understood, for example, similar to the relative timing of articulators as expressed within articulatory phonology (Browman and Goldstein, Reference Browman and Goldstein1992).
So far, researchers have accumulated plenty of evidence for sub-symbolic coordination processes taking place between interlocutors: Speakers align their pronunciation patterns, speech tempo, and prosody (Bosch et al., Reference Bosch, Oostdijk and Boves2005; Gessinger et al., Reference Gessinger, Raveh, Steiner and Möbius2021; Levitan and Hirschberg, Reference Levitan and Hirschberg2011; Lewandowski and Jilka, Reference Lewandowski and Jilka2019; Pardo, Reference Pardo2006; inter alia), and occasionally even their conversational laughter (Ludusan and Wagner, Reference Ludusan and Wagner2022), but also on higher-order levels of linguistic organization such as lexical choice, syntactic structures, or referential gestures (Bergmann and Kopp, Reference Bergmann and Kopp2012; Brennan and Clark, Reference Brennan and Clark1996; inter alia). However, most studies find a lot of individual variation both in rhythmic-prosodic entrainment and higher-level linguistic alignment. Still, a key assumption of mechanistic accounts of interpersonal alignment (Pickering and Garrod, Reference Pickering and Garrod2004, Reference Pickering and Garrod2007) is that sub-symbolic, rhythmic-prosodic entrainment fosters symbolic alignment, and hence comprehension, on higher levels of grammatic organization.
To this day, speech processing architectures lack a unified account of whether and how any interaction between sub-symbolic and symbolic adaptation is actually achieved. In Wagner et al. (Reference Wagner and Niebuhr2013), we therefore argued for Interaction Phonology as a logistic, attention-guiding component that enables interlocutors to coordinate their articulatory movements on a low-signal level by a process of temporal entrainment. That way, listeners may guide their attention to crucial aspects of phonetic detail (Ghitza and Greenberg, Reference Ghitza and Greenberg2009; Ghitza, Reference Ghitza2012; Giraud and Poeppel, Reference Giraud and Poeppel2012; Chapters 12 and 19) that will permit an easier access to higher-level linguistic information. As a consequence, symbolic alignment should be fostered by temporal coordination in an automatic, bottom-up fashion. We use the term entrainment in a narrow sense (Obleser and Kayser, Reference Obleser and Kayser2019) where it describes a dynamic process of physically coupled oscillatory systems, which adapt their cycles both in period and phase, thereby ultimately achieving a fixed phase relationship. Humans are capable of interpersonal entrainment without an external, isochronous pacemaker, for example, when spontaneously synchronizing their clapping behavior in enthusiastic applause by period doubling (Néda et al., Reference Néda, Ravasz, Vicsek, Brechet and Barabási2000), or when speaking in synchrony (Cummins, Reference Cummins2009). Strikingly, humans have shown to synchronize their brain activities, strengthened by shared engagement and joint activity (Dikker et al., Reference Dikker, Michalareas and Oostrik2021).
In Interaction Phonology, the rhythmic properties of a language play a crucial role in this entrainment process. It has been noted that speech lacks the isochrony or regularity necessary for entrainment (Cummins, Reference Cummins2012). However, it may occasionally show fixed phase relationships or a high degree of regularity, at least in highly formalized speaking styles such as poetry (Wagner, Reference Wagner and Niebuhr2013), which may lend itself to rhythmic entrainment, even though we do not yet understand the exact mechanism behind this. While absolute coordination cannot be meaningfully expected between interlocutors at all times, there is some evidence in favor of entrainment: Across several languages, overlapping speech shows a preference for speakers being in phase with the interlocutor’s syllabic speech stream (Włodarczak et al., Reference Włodarczak, Šimko and Wagner2012). In line with entrainment models of attention (Lakatos et al., Reference Lakatos, Karmos, Mehta, Ulbert and Schroeder2008; Large and Jones, Reference Large and Jones1999), Interaction Phonology postulates that this process helps listeners gain access to language-specific phonological and higher-level properties of the utterance spoken.
Furthermore, we argued that rhythmic entrainment is a necessary prerequisite for the automaticity and swiftness of representational alignment in human interaction. While not excluding the possibility of a reductionist account of the phenomena described, we do not think it necessary for now to subscribe to this idea. Still, we argued for an inter-speaker coordination mechanism as being fundamental not only for speech perception but for communicative interaction, that is, the permanent active attuning to one another. Interaction Phonology can be preliminarily defined as taking care of the coordinative interactive processes that are strongly built on rhythmic-phonological structures.
Interaction Phonology furthermore postulates that there are universal and language-specific structures on which coordination takes place. In particular, it predicts that the rhythmic-prosodic organization of a language constrains the levels of temporal coordination between interlocutors. For a lack of better knowledge, these are assumed to be identical to the language-specific levels of prosodic organization (syllables, feet, prosodic phrases) and their internal metrical organization (Jun, Reference Jun2005). In other words, according to Interaction Phonology, the temporal coordination between interlocutors who speak varieties with a similar rhythmic-prosodic organization should be comparatively easy. However, Interaction Phonology also postulates that the mechanisms of temporal coordination are to some degree universal, based on syllabic structures that are grouped into larger units such as phrases or similar. Even though their regularity, function, and organization within the prosodic hierarchy may differ across languages, there is some space for rhythmic coordination even when interlocutors cannot rely on a large set of common temporal mechanisms that may serve as anchors to higher-level linguistic organization. An example would be an L2 (second-language) listener’s strategic reliance on prosodic universals as well as language-specific prosodic cues as indicators of lexical stress, which often is a useful approach to segment a speech stream into words (Endress and Hauser, Reference Endress and Hauser2010; Ordin and Nespor, Reference Ordin and Nespor2013; Tyler and Cutler, Reference Tyler and Cutler2009). The idea of rhythmic entrainment as a universal coordinative process underlying linguistic organization has received further support by developmental studies that described movement synchronization between neonates and their caregivers (Condon, Reference Condon1974; Jaffe et al., Reference Jaffe, Beebe and Feldstein2001), where a baby’s acquisition may be helped by anchoring into the universal prosodic properties of speech to pave the way for higher-order symbolic alignment (Chapters 35–37, 39, and 41). In fact, neonates are born with an ability to use prosodic strategies independently of segmental properties to identify word boundaries in their early language acquisition process (Fló et al., Reference Fló, Brusini and Macagno2019). An early alignment to the rhythmic-prosodic detail of a caregiver’s movements may therefore be a generally useful strategy in language acquisition. However, as prosodic and phonetic alignment has shown to be to some degree voluntary, situation-specific, and less strong in populations with autism spectrum disorder (Schweitzer and Lewandowski, Reference Schweitzer and Lewandowski2014; Schweitzer et al., Reference Schweitzer, Lewandowski and Duran2017; Wynn et al., Reference Wynn, Borrie and Sellers2018), Interaction Phonology allows for the modulation of underlying entrainment processes. That is, if conversational needs for mutual understanding and grounding are high, it predicts that entrainment can be willingly strengthened, thereby actively supporting mutual comprehension and conversational grounding.
29.3 The Mechanism of Interaction Phonology
Interaction Phonology postulates that the incoming speech signal is subject to a process of rhythmic-prosodic analysis that does the following:
Guides the listener’s attention to the fine phonetic detail of the speech signal that may be of particular relevance for a given language, which coincides with crucial boundaries of higher-level linguistic organization and therefore facilitates their prompt identification. For now, we believe that the levels of entrainment are identical to the levels of organization in the prosodic hierarchies of the different languages. It is possible that this language-specific coordination does not constitute an independent level of a language’s grammar but rather is a by-product of its morphosyntactic or phonological organization.
Is driven by a process of rhythmic entrainment, modulated according to communicative needs such as the overall level of “noise,” and informed by linguistic analyses of the ongoing interaction. Apart from objectively present external noise, this may also relate to the overall level of distraction, or the relevance of successful communication.
Is adaptive with respect to its level of entrainment, or coupling strength; these adaptations are strongly guided by the rhythmic-prosodic patterns of the language chosen to communicate but may also be subject to long-term entrainment between interlocutors, if these (initially) speak different languages or varieties.
Leads to an adaptation in speech production with respect to tempo and rhythmic modulation via perception–production coupling, and hence an improved attention to detail on the listener’s side and representational alignment in (adapted) speech production.
These analyses are organized within various model components, which are described in detail in Table 29.1 and are indicated by their respective numbers in Figure 29.1.
| Model component | Description |
|---|---|
| 1 | Entrainment module, guiding listener’s attention to points in the incoming speech signal that are crucially linked to higher-order linguistic organization |
| 2 | Entrained, or “guided,” auditory analysis of incoming speech input, which interfaces with subsequent linguistic analysis of input |
| 3 | Motor patterns mapped to incoming acoustic analysis, automatically leading to adapted speech output |
| 4 | Set of linguistic structures and expectations as part of a speaker’s grammar, which are linked to the levels of entrainment via their corresponding levels of prosodic organization, and which correspond to attractors in entrainment |
| 5 | A communication relevance monitor, which assesses the situation and ongoing communication (and with this, the need for entrainment), which may adapt the strength of necessary entrainment depending on present noise and the necessity of communicative success |
So far, Interaction Phonology has not yet spelled out connection or interface with existing models of speech production and perception. However, most of these models miss a link between symbolic and sub-symbolic processing, and Interaction Phonology may help improve our understanding of this interface. Given its focus on communication, Interaction Phonology can only be meaningfully integrated with architectures that account for both perception and production.
29.4 Predictions of Interaction Phonology
Here, we spell out a set of testable predictions by Interaction Phonology. The predictions are chosen as they all test crucial aspects of the model. Interaction phonology makes predictions beyond this list, especially with regards to prosodic universals and language-specific constraints. Also, given its current lack of formality and under-specification, it should be clear that this list is currently incomplete and lacks formal rigor.
Prediction 1: Speech rate adaptation should improve speech perception in similar communication settings.
Prediction 2: Entrainment should be visible across the levels of the prosodic hierarchy, in a language-specific fashion.
Prediction 3: The level of entrainment should be situation-specific, and vary within individuals across different situations.
Prediction 4: If rhythmic entrainment occurs, it should automatically result in symbolic alignment.
Prediction 1 falls out of the model, as the model postulates a positive effect of entrainment on speech perception by its guiding the listener’s attention to relevant phonetic detail using the entrainment module (see Figure 29.1, component 1). However, it needs to be taken into account that the model also predicts entrainment for those communicative situations in which perception may be impeded by various types of noise. Therefore, it is crucial for testing Prediction 1 that speech perception and entrainment are measured across similar settings, without added cognitive load or external noise. Speech rate is chosen mostly as a test (in favor of other potential features of rhythmic-prosodic entrainment) as there exists a considerable amount of empirical research on it. Prediction 2 falls out of the assumed link between levels of entrainment and language-specific structures (see Figure 29.1, component 4). That is, Interaction Phonology expects a certain language dependence with respect to the levels of entrainment that mirror the prosodic organization of the involved languages or varieties. Going back to speech rate entrainment, depending on the rhythmic-prosodic structure of the language to be entrained to, speech rate adaptation may concentrate on morae, syllables, prosodic feet, prosodic words, or even phrasal structures. Prediction 3 is derived from Interaction Phonology’s assumption that entrainment is to some degree deliberate and strategically chosen by interlocutors rather than a fully automatized process that will always be enabled (Figure 29.1, component 5). In other words, Interaction Phonology predicts the level of entrainment to a certain speech rate to be stronger in challenging communicative situations. Prediction 4 falls out of the assumed automatic link between sub-symbolic coordination and symbolic alignment (see Figure 29.1, connection between components 1 and 2). Here, the control mechanism that enables entrainment automatically takes into account higher-level similarities. If these two fail to be coupled, this would be a challenge for our control mechanism, and would point to a strongly strategic symbolic alignment that is not necessarily coupled to sub-symbolic, motor-level processes of articulation. In other words, an entrainment to speech rate ought to be also visible in the usage of more similar words, or syntagmatic structures.
29.5 Evaluating Interaction Phonology
Next, it will be determined whether more recent empirical research is in line with the assumptions and predictions of Interaction Phonology, or falsifies (aspects of) it. Where no research results lend themselves to model evaluation, suggestions for future studies will be made in order to better understand Interaction Phonology’s flaws, limitations, as well as strengths. The analysis will concentrate on the four main predictions of Interaction Phonology that have been spelled out above.
29.5.1 Prediction 1: Speech Rate Adaptation Helps Speech Perception
In incremental, online speech perception, listeners need to simultaneously pay attention to several levels of linguistic organization. The ability to do this may be enhanced by the different timescales underlying the spell-out of these levels (phones, syllables, words, phrases), which can be entrained to cortical rhythms working on similar timescales (Ghitza and Greenberg, Reference Ghitza and Greenberg2009; Ghitza, Reference Ghitza2012; Chapter 5). Much work around rhythmic entrainment during perception has concentrated on attentional selection, which ought to focus on crucial parts of the speech signal, for example, the initializations of syllables. There is converging evidence that some form of temporal entrainment indeed helps selectively attending to the incoming speech stream of a particular speaker among several concurrent speakers (Obleser and Kayser, Reference Obleser and Kayser2019). Also, neural entrainment processes have shown to (somewhat) aid speech perception and sentence comprehension (Lamekina and Meyer, Reference Lamekina and Meyer2022; Riecke et al., Reference Riecke, Formisano, Sorger, Başkent and Gaudrain2018; Wilsch et al., Reference Wilsch, Neuling, Obleser and Herrmann2018; Zoefel et al., Reference Zoefel, Archer-Boyd and Davis2018).
However, as speech tempos change dynamically in ongoing speech within the same speaker (Quené, Reference Quené2008), and speech is not isochronous like music (Cummins, Reference Cummins2012), for entrainment to be a successful tool for enhancing speech perception, listeners need to be able to swiftly adapt to these speech tempo changes. Speech rate convergence in production is a phenomenon largely supported by empirical research, appearing in both monological priming tasks (Jungers and Hupp, Reference Jungers and Hupp2009) and conversations (Cohen Priva et al., Reference Cohen Priva, Edelist and Gleason2017; Fuscone et al., Reference Fuscone, Favre and Prevot2021; Schultz et al., Reference Schultz, O’Brien and Philipps2016). For perception, Dilley and Pitt (Reference Dilley and Pitt2010) presented the first evidence for listeners indeed quickly adapting to the speech tempo of an incoming speech signal, leading them to perceptually insert additional syllables/words into a speech stream that was locally produced slowly; for example, “leisure time” was perceived as “leisure or time.” This effect, which they term LRE (lexical rate effect), is restricted to speech processing and does not generalize to tone perception (Pitt et al., Reference Pitt, Szostak and Dilley2016), but can be built up over longer stretches of time, thereby generating the expectations that drive selective attention (Baese-Berk et al., Reference Baese-Berk, Heffner and Dilley2014; Chapter 12). Bosker (Reference Bosker2017) showed in a series of experiments that it is the (isochronous) speech rate prior to a target that creates an anticipatory effect on perception. He interprets this as evidence for an underlying neural entrainment mechanism at play, which is not tied to the speech mode. What is not yet resolved is the question of whether entrainment is restricted to speech processing, or is a general monitoring and adaptation device. The studies reported here that have examined an impact of rhythmic entrainment on speech perception have done so in highly controlled laboratory settings. Thus, it can at least be said that in such contexts, an adaptation to speech tempo can be traced and appears to have a positive impact on speech perception. However, it still remains unclear how entrainment can actually be achieved given the intrinsic non-isochrony present in speech signals. Here, Bosker (Reference Bosker2017), Meyer et al. (Reference Meyer, Sun and Martin2020), and Obleser and Kayser (Reference Obleser and Kayser2019) claim that the – at best – pseudo-rhythmic acoustic properties of speech are sufficient to induce an entrainment mechanism that may lend itself to higher-order synchronicities in more abstract linguistic domains. For the moment, one can only speculate that highly adaptive (neural) oscillators with a fast reset should also be able to achieve a rapid entrainment to dynamically changing rhythms (Inden et al., Reference Inden, Malisz, Wagner and Wachsmuth2012).
29.5.2 Prediction 2: Entrainment Should Be Visible across Language-Specific Levels of the Prosodic Hierarchy
Building on the conspicuous similarities between the multi-timescales of cortical and speech rhythms (Ghitza and Greenberg, Reference Ghitza and Greenberg2009; Ghitza, Reference Ghitza2012), Interaction Phonology postulates that rhythmic entrainment should pertain to various timescales, and these timescales should reflect the rhythmic structure of the language(s) spoken. In particular, this should lead to language-specific rhythmic entrainment, as languages, language varieties, or speaking styles differ with respect to their prosodic organization, and this should be reflected in the long-term abilities of entrainment. For example, languages may differ vastly with respect to the length and complexity of syllable-sized units (Zec, Reference Zec and de Lacy2007). Interaction Phonology now predicts that speakers of languages with a higher degree of phonotactic complexity and variability are either more flexible in entraining to syllable streams or maybe make less use of syllable-level entrainment, as it is more often doomed to fail. Also, languages differ with respect to their higher-order prosodic organization, and may use different patterns of metrical organization (Jun, Reference Jun2005). Such differences should also show in language-selective rhythmic entrainment.
Currently, empirical evidence indeed points towards rhythmic entrainment being active on different timescales: The LRE (see above) has been shown to also apply for syllable-level speaking rate as well as rhythm, indicating a certain degree of higher-order entrainment on the foot or word level, where listeners modulate their perception based on whether they expect a stressed or unstressed syllable (Morrill et al., Reference Morrill, Dilley, McAuley and Pitt2014). Furthermore, the effect has shown to be additive, and listeners are more attentive when several rhythmic boundaries co-occur. However, despite considerable work on entrainment to pulse and higher-order meter in music (see the overview in Fitch, Reference Fitch2013), and despite a long tradition in research to hypothesize about similar processing mechanisms being at play in music and speech processing and organization (e.g., Lehrdahl and Jackendoff, Reference Lerdahl and Jackendoff1983; Wagner, Reference Wagner2008; Chapters 25–28), very little is actually known about language-specific entrainment, and most evidence remains speculative.
Some similarities between music and speech perception can be drawn from finger-tapping studies, a sensorimotor synchronization paradigm that is well established in music rhythm perception research (Repp, Reference Repp2005; Repp and Su, Reference Repp and Su2013). Finger tapping to music rhythms has been shown to help improve music time perception, similar to the perceptual benefits of entraining to speech rhythm (Manning and Schutz, Reference Manning and Schutz2013). In speech perception tasks, finger-tapping duration and intensity have likewise been shown to be sensitive to rhythmic structure linked to linguistic organization such as syllable onsets, lexical stress, sentence stress, or accent (Parrell et al., Reference Parrell, Goldstein, Lee and Byrd2014, Rathcke et al., Reference Rathcke, Lin, Falk and Dalla Bella2021). Another paradigm called speech cycling investigated rhythmic entrainment of repetitive short phrases to external high and low tones, and found cross-linguistic similarities in patterning speech to these external tones for Japanese and English, despite their different syllable structures (Tajima and Port, Reference Tajima, Port, Local, Ogden and Temple2003). While these studies point to a common sensorimotor entrainment mechanism, it should be noted that tapping in real time to an incoming speech signal is extremely difficult to do, and listening to or reproducing repetitive single phrases resembles music rather than speech processing (Anbari et al., Reference Anbari, Włodarczak and Wagner2013). An alternative methodological paradigm, in which listeners tapped a perceived rhythmic structure directly after perceiving an utterance, revealed an ability of listeners to encode language-specific rhythmic-prosodic features in tapping patterns as well, and showed a stronger reliance on acoustic-prosodic features as compared to non-motor prosody perception tasks (Bruggeman et al., Reference Bruggeman, Schade, Włodarczak and Wagner2022; Wagner et al., Reference Wagner, Ćwiek and Samlowski2019). However, it is yet unclear whether the results of tapping are indicative of sensorimotor entrainment proper or simply fall out of a general analysis of linguistic structure, integrating linguistic, acoustic-phonetic, and sensorimotor cues. Similar problems exist with studies on L2 acquisition that show that rhythmic priming has a beneficial effect on learning the target prosody in an L2, as they either rely on multimodal reproduction tasks or use musical (rather than speech) rhythms as priming materials (e.g., Baills and Prieto, Reference Baills and Prieto2021; Wang et al., Reference Wang, Mok and Meng2016) – neither is conclusive as to whether it really is rhythmic entrainment to speech that leads to the positive effects on mastering an L2 prosody. Overall, we have some empirical evidence pointing in the direction that rhythmic entrainment has long-term consequences, leading to long-term rhythmic expectations that result in an improved rhythmic entrainment performance in an L1 as compared to an L2, and that may result in better abilities of learning an L2 prosody in speakers with a high degree of rhythmic training. However, clear-cut evidence for this prediction of Interaction Phonology is still lacking.
29.5.3 Prediction 3: The Level of Entrainment Should Be Situation-Specific, and Vary within Individuals across Different Situations
Interaction Phonology postulates that interlocutors make a deliberate (though not necessarily conscious) choice in whether they employ prosodic entrainment or not, and it is expected that entrainment should be selectively activated in challenging communicative situations, in which the benefits of attention can be exploited best. As the vast majority of studies have been performed in laboratory conditions, often relying on short, highly controlled utterances that show little resemblance with everyday interactions, this has not been investigated in ecologically valid conditions. However, some first approaches do exist.
In a study that looked at rate-dependent adaptive listening in quiet and noisy conditions, Reinisch and Busker (Reference Reinisch and Bosker2022) could show that listeners dynamically adapt at a low level to challenging contexts, and can identify noisy target items more easily when these are preceded by coherently noisy signals. There is also increasing evidence that the selective attention to an individual speaker in a multi-party listening condition decreases entrainment to the ignored voices (e.g., Ding and Simon, Reference Ding and Simon2013; Fuglsang et al., Reference Fuglsang, Dau and Hjortkjær2017). This points towards the level of entrainment being to some degree adjustable according to situation-specific needs.
Several studies investigated the impact of acoustic manipulation (vocoded speech) on the level of entrainment, hypothesizing that vocoding would be detrimental to speech quality and therefore trigger higher entrainment. Peelle and Davis (Reference Peelle and Davis2012) and Peelle et al. (Reference Peelle, Gross and Davis2013) find evidence for neural entrainment to speech being higher when it is vocoded (more difficult to comprehend). While this may point to a direction of selective entrainment in the case of speech that is difficult to process, Baltzell et al. (Reference Baltzell, Srinivasan and Richards2017) showed that vocoded speech preceded by natural speech primes also aided the comprehension of vocoded speech. This is in line with findings on synchronous speech, where synchronization is not influenced by intelligibility but by rhythmic cues (Cummins, Reference Cummins2009).
Rather than manipulating the acoustics of their stimuli, Iverson et al. (Reference Iverson, Song and Bradley2018) and Song and Iverson (Reference Song and Iverson2018) tested the influence of overall intelligibility on neural entrainment by comparing the performance of L1 and L2 listeners when hearing L1 or L2 speech. Their results point to patterns of stronger neural entrainment when listening to the less familiar (L1 for L2 listeners, L2 for L1 listeners) and hence more challenging variety. However, the idea that any challenges to the ongoing communication success lead to an automatic increase in entrainment appears to be overly simplistic: A study by Hjortkjær et al. (Reference Hjortkjær, Märcher‐Rørsted, Fuglsang and Dau2020) indicates that a higher working memory load actively decreases the level of neural entrainment, and also Abel and Babel (Reference Abel and Babel2017) show lower phonetic convergence under high cognitive load. Interestingly, this effect was present both for a more difficult task as well as an increase in acoustic noise that had been tested for not being detrimental to speech intelligibility. These results indicate that entrainment needs cognitive resources by itself, possibly to uphold selective attention.
29.5.4 Prediction 4: If Entrainment Occurs, It Should Automatically Result in Symbolic Alignment between Interlocutors
By now, there is a long research tradition that has demonstrated adaptation between communication partners both on fine phonetic detail such as speech tempo, pause duration, intonation, or segmental articulation as well as more abstract linguistic representations such as the lexicon, syntactic structures, or referential gestures (see Section 29.2). What is unclear is whether low-level coordination on fine phonetic details indeed quasi-automatically triggers an agreement on higher-order linguistic concepts, as predicted by Interaction Phonology, and in line with models of interpersonal adaptation that link production and perception (Pickering and Garrod, Reference Pickering and Garrod2004, Reference Pickering and Garrod2007). However, clear-cut evidence for this idea appears to be difficult to come by, despite the undeniable benefit found for listening entrainment in speech perception (see above). Krivokapić (Reference Krivokapić2013) suggests that speech rate convergence between dialogue partners correlates with their alignment of variety-specific rhythmic patterns (Indian English and American English), indicating a certain automaticity in convergence across low-level and higher-level rhythmic-prosodic organization. Alternatively, this could be explained by falling out of inter-speaker entrainment in speech rate, as duration indicates both speech rate as well as rhythmic organization. One of the few studies that found evidence for a communicative benefit (beyond intelligibility) of speech rate adaptation is Manson et al. (Reference Manson, Bryant, Gervais and Kline2013), who reported an increase in cooperation between interlocutors when they also aligned in speech rate. Similarly, Lubold and Pon-Barry (Reference Lubold and Pon-Barry2014) report an increase in perceived rapport. These results may point to a higher degree of conversational grounding in situations where rhythmic-prosodic entrainment is evident, and may indicate a mechanistic link between low-level speech rate adaptation to higher-order linguistic processing. However, other interpersonal factors such as mutual likeability were not affected by an increase in entrainment (Manson et al., Reference Manson, Bryant, Gervais and Kline2013), which is further evidence that the underlying mechanism may be specialized to linguistic processing.
However, a clear-cut effect of entrainment on symbolic-linguistic alignment appears to be difficult to prove: Weise and Levitan (Reference Weise and Levitan2018) fail to find evidence for a link between acoustic-prosodic and symbolic alignment, while Rahimi et al. (Reference Rahimi, Kumar, Litman, Paletz and Yu2017) suggest that entrainment across different levels of linguistic organization can occur. Generally, most studies report a high degree of individual variation in entrainment, which seems to be at least to some degree driven by personal and interpersonal factors, for example, mutual likeability, perceived attractiveness, gender, as well as the power dynamics between interlocutors (Babel, Reference Babel2012; Michalsky and Schoormann, Reference Michalsky and Schoormann2017; Pardo, Reference Pardo2012; Reichel et al., Reference Reichel, Beňuš and Mády2018; Schweitzer and Lewandowski, Reference Schweitzer and Lewandowski2014; Schweitzer et al., Reference Schweitzer, Lewandowski and Duran2017).
29.6 Conclusion: An Adapted Sketch of Interaction Phonology
For some key predictions of Interaction Phonology, empirical evidence is growing stronger. In particular, we see that rhythmic entrainment does take place in speech tempo adaptations, and has a positive effect on intelligibility. While cross-linguistic studies on entrainment are very rare, there is evidence that it connects to the rhythmic-prosodic structure of individual languages, thereby probably also enhancing higher-level comprehension. Another key assumption of Interaction Phonology is that rhythmic-prosodic entrainment can be adjusted based on situative needs. Here, we indeed saw that listeners do adapt their entrainment to individual voices or increase entrainment in challenging listening situations. However, entrainment is not increased independently of the type of communicative challenge. Contrary to our prediction, working memory load seems to decrease entrainment, indicating that entrainment comes with a certain cognitive load of its own. Here, more research is necessary to better understand which type of situation triggers or decreases its effectiveness. Despite the positive effect entrainment has on intelligibility, when explicitly linking it to higher-level linguistic organization, there appears to be no automaticity in rhythmic-prosodic entrainment and higher-order symbolic alignment between interlocutors. At best, researchers find that this connection is not ruled out. In our account of Interaction Phonology, this connection is therefore removed for the time being, and the link between rhythmic-prosodic entrainment and symbolic-linguistic organization is limited to grammatical aspects of sound structure (phonology). From there, a connection to higher-order linguistic organization can be made as part of listening comprehension, but the connection to further symbolic entrainment needs to be questioned. For now, the results leave a question mark as to the exact nature of the interface between sound-related and higher-order linguistic processing in Interaction Phonology. As for the auditory-motor mapping, which predicts automatic convergence of rhythmic-prosodic patterns in speech-based interaction, it is left as optional in the model, as most data show high individual variation in speakers’ level of converging prosodic patterns, even though it seems to be to some degree automatized for speech tempo. Here, further empirical work is needed to highlight the level of automaticity or deliberate control, and how it covaries with other speaker traits, their level of neurocognitive alignment, or situative factors. Overall, it can be concluded that the model of Interaction Phonology can still be helpful to further inform psycholinguistic models of speech processing, to extend them to models of communicative interaction, and to improve and clarify the interface of symbolic and sub-symbolic processing in the models. Our adapted sketch of Interaction Phonology is illustrated in Figure 29.2.
Summary
Interaction Phonology explains symbolic and sub-symbolic inter-speaker adaptations using the mechanism of rhythmic-prosodic entrainment. Many key assumptions (rhythmic entrainment as an optional process that helps perception and is linked to grammar) are empirically supported. However, the original model was modified: Auditory-motor mapping is optional, entrainment can also be actively decreased under high cognitive load, and the assumed automaticity between entrainment and symbolic alignment is questioned.
Implications
Interaction Phonology provides a testable theoretical framework for evaluating language-specific and language-universal hypotheses related to rhythmic entrainment between interlocutors, and their relationship with higher-order alignment of abstract linguistic representations.
Gains
Interaction Phonology provides a theoretical framework that affords the necessary scaffold for enabling an inter-speaker alignment of phonetic-phonological, and potentially also higher-order, linguistic representations by a mechanism of rhythmic entrainment. Interaction Phonology extends existing speech processing models with an interface between symbolic and sub-symbolic processing, and integrating them into communication models.
Open access