41.1 The Segmentation Problem

A crucial step in language acquisition is to learn to segment the continuous speech stream into possible word candidates. To solve this problem, infants rely on a variety of perceptual and computational mechanisms. During the first year of life, infants can segment speech with the help of phonotactic regularities (Mattys et al., Reference Mattys, Jusczyk, Luce and Morgan1999), lexical constraints (Jusczyk et al., Reference Jusczyk, Cutler and Norris2003), rhythmic structure (Houston et al., Reference Houston, Jusczyk, Kuljpers, Coolen and Cutler2000; Nazzi et al., Reference Nazzi, Iakimova, Bertoncini, Frédonie and Alcantara2006), statistical cues (Saffran et al., Reference Saffran, Newport and Aslin1996), and prosodic cues (Jusczyk et al., Reference Jusczyk, Cutler and Redanz1993, Reference Jusczyk, Houston and Newsome1999; Johnson and Jusczyk, Reference Johnson and Jusczyk2001; Marimon et al., Reference Marimon, Höhle and Langus2022). The prosodic cues that support infants’ speech segmentation are commonly assumed to be language-specific and consist of rhythmic-grouping cues (e.g., Abboub et al., Reference Abboub, Nazzi and Gervain2016), intonational contours (e.g., Shukla et al., Reference Shukla, White and Aslin2011), and stress patterns (e.g., Echols et al., Reference Echols, Crowhurst and Childers1997; Jusczyk et al., Reference Jusczyk, Houston and Newsome1999). However, there is also evidence that more general cues such as transitional probabilities aid early speech segmentation (Thiessen and Saffran, Reference Thiessen and Saffran2003). Understanding how the use of these cues develops and how they interact is a central question in early language development.

Here, we argue that infants’ ability to segment speech, including the ability to exploit all the cues to word boundaries, may be mediated by their ability to build expectations of how the speech signal will unfold. For example, infants exposed to bisyllabic or trisyllabic words in isolation will subsequently only learn words from a continuous speech stream that match the length of the words they first heard in isolation (Lew-Williams and Saffran, Reference Lew-Williams and Saffran2012). Infants exposed to a list of nonwords following a specific stress pattern (either iambic or trochaic) will consequently segment a continuous speech stream following that specific stress pattern (Thiessen and Saffran, Reference Thiessen and Saffran2007). Importantly, infants who are familiarized with words of equal length tend to perform better in a speech segmentation task than infants who are exposed to continuous speech with varied word length (Johnson and Tyler, Reference Johnson and Tyler2010; Mersad and Nazzi, Reference Mersad and Nazzi2012). Also, adult participants listening to passages of sentences that follow a predictable prosodic structure will only segment words defined by statistical regularities if these fall within the prosodic boundaries of the primed sentence structure (Wang et al., Reference Wang, Zevin and Mintz2017, Reference Wang, Zevin, Trueswell and Mintz2020). These studies therefore suggest the ability to use rhythmic expectations in speech, such as syllabic length and stress patterns, to effectively segment and anticipate word boundaries.

41.2 Methodological Considerations

To better understand how infants predict how the speech signal will unfold, we need to move beyond the behavioral methods generally used to investigate speech segmentation abilities in infancy. Typically, speech segmentation experiments first familiarize participants with continuous speech that contains words that follow specific statistical regularities or prominence patterns. Participants are then tested on words that occurred in the familiarization phase, words that violate the regularities of interest, and/or words that did not occur in the familiarization phase. This typically involves measuring infants’ attention in looking-time paradigms such as the head-turn preference procedure (Hirsh-Pasek et al., Reference Hirsh-Pasek, Kemler Nelson and Jusczyk1987) and querying adults whether they remember and can distinguish words from nonwords. However, these experiments rely on different responses in infants and adults that hinder direct comparisons between participants at different ages of development. Furthermore, the familiarization phase that unfolds in a matter of minutes can be taxing to younger infants who have a limited focus, thus limiting the amount of test trials and conditions that can be included in a single experiment. Finally, behavioral methods have also failed to tap into the process of speech perception and segmentation that occurs as the familiarization phase unfolds (but see Gómez et al., Reference Gómez and Mehler2011).

To overcome these limitations, we have explored the use of pupillometry. Under constant lightning conditions, pupils dilate as the result of a variety of perceptual and cognitive processes (Loewenfeld, Reference Loewenfeld1958). In adults, pupil dilation has been linked to a greater cognitive load as well as violations of expectation (Hess and Polt, Reference Hess and Polt1960; Kahneman and Beatty, Reference Kahneman and Beatty1966; Karatekin, Reference Karatekin2007; Jackson and Sirois, Reference Jackson and Sirois2009; Laeng et al., Reference Laeng, Sirois and Gredebäck2012; Fritzsche and Höhle, Reference Fritzsche and Höhle2015; Tromp et al., Reference Tromp, Haagort and Meyer2016; Vogelzang et al., Reference Vogelzang, Van Rijn and Hendriks2016). Similarly, in infants and young children, pupils dilate as the result of surprise and cognitive effort (Karatekin, Reference Karatekin2007; Jackson and Sirois, Reference Jackson and Sirois2009; Gredebäck and Melinder, Reference Gredebäck and Melinder2010; Hepach and Westermann, Reference Hepach and Westermann2013; Hochmann and Papeo, Reference Hochmann and Papeo2014; Langus et al., Reference Langus, Boll-Avetisyan, van Ommen and Nazzi2023) and can provide a more sensitive measure than behavioral methods such as looking time (Jackson and Sirois, Reference Jackson and Sirois2009; Hepach and Westermann, Reference Hepach and Westermann2013, Reference Hepach and Westermann2016). In fact, a growing body of literature suggests that pupillometry can be used to study speech perception throughout infancy into adulthood (Hochmann and Papeo, Reference Hochmann and Papeo2014; Fritzsche and Höhle, Reference Fritzsche and Höhle2015; Tamási et al., Reference Tamási, McKean, Gafos, Fritzsche and Höhle2017; Marimon et al., Reference Marimon, Höhle and Langus2022; Langus et al., Reference Langus, Boll-Avetisyan, van Ommen and Nazzi2023). In speech perception experiments, pupillometry can therefore provide a noninvasive physiological measure that can be used to compare infants and adults directly in the same paradigm using the same stimuli.

Pupillometry may also reveal how listeners perceive and parse continuous speech as the signal unfolds. In particular, changes in pupil size can also entrain to rhythmic auditory stimuli as the auditory stimuli unfold – that is, dilate and constrict in synchrony with acoustic or structural patterns in the auditory stimuli. For example, changes in pupil size can synchronize to repeating musical instrument sounds in adult listeners (Fink et al., Reference Fink, Hurley, Geng and Janata2018). Furthermore, studies with primates show that changes in pupil size can entrain to repeating patterns of tone sequences defined by statistical regularities and that these entrained pupillary responses are comparable to neural entrainment in the auditory cortex (Barczak et al., Reference Barczak, O’Connell and McGinnis2018) (see Chapter 3). These findings are not only remarkable because the relatively slow changes in pupil size spontaneously synchronize to auditory stimuli, but they also suggest that the entrained pupillary responses can reveal how listeners parse continuous auditory stimuli. This is particularly interesting for infant speech segmentation experiments where the stimuli often occur with a predictable regular rhythm (Thiessen and Saffran, Reference Thiessen and Saffran2007; Johnson and Tyler, Reference Johnson and Tyler2010; Mersad and Nazzi, Reference Mersad and Nazzi2012). Measuring pupil size may therefore not only be informative about which words infants and adults remember in speech segmentation experiments, but could also reveal how the process of speech segmentation in the familiarization phase unfolds.

41.3 Methodological Considerations: Pupil Dilation as a Measure for Speech Segmentation

To explore whether pupillary changes can be used to investigate speech segmentation in infants and adults, we carried out a cue-weighting experiment while we measured participants’ pupil size with an eye-tracker. Participants were familiarized with a continuous speech stream in which statistical words signaled by transitional probabilities straddled prosodic word boundaries (see Figure 41.1C). Specifically, in the familiarization stream, every other syllable carried lexical stress that both German-learning infants and German-speaking adults would perceive as word-initial (Höhle, Reference Höhle2002). However, the syllables used to create the familiarization stream were combined in a way that the transitional probabilities between syllables were higher at prosodic word boundaries (between weak-strong syllable pairs) than within prosodic words (i.e., between strong-weak syllable pairs). After the familiarization phase, participants were presented with prosodic words (which followed a lexical stress pattern in the familiarization string, but had low transitional probabilities between the two syllables i.e., 0.2–0.4), statistical words (which had high transitional probabilities between the syllables, i.e., 1.0), and nonwords (which consisted of syllables that never occurred adjacently in the familiarization stream; the transitional probability was 0.0). We measured infants’ and adults’ pupil sizes with an eye-tracker throughout the experiment. Adult participants were additionally asked to indicate with a key press if they had heard the specific test word during the familiarization.

Figure 41.1

Frequency tagging the pupillary response.

(A) The pupillary response during the familiarization phase is transformed from the time domain to the frequency domain through fast Fourier transform (FFT). The FFT decomposes the pupillary response into sine-wave components of different frequencies by determining their amplitude and phase. (B) One of these sinusoidal components will correspond to word frequency (2.08 Hz), that is, pupils dilating and constricting once during the occurrence of each word. (C) To determine whether infants’ pupils entrain to statistical or prosodic words, we will look at the temporal alignment (i.e., phase) of the pupillary response at word frequency. Because the familiarization stream started with a statistical word, if infants’ pupils entrain to statistical words, then the pupillary response at word frequency (2.08 Hz) would be temporally aligned with the familiarization stream onset (solid line, upper side of the panel). In contrast, if infants’ pupils entrain to prosodic words, then the pupillary response at word frequency should be temporally shifted from the onset of the familiarization stream by one syllable that corresponds to half a cycle (π) of the 2.08 Hz response (dashed line, lower side of the panel).

Figure 41.1 Long description

Panel A: A graph of pupillary response over time. The horizontal-axis is time, and the vertical-axis is pupillary size. The graph shows fluctuations in pupil size. Panel B: A graph of frequency over time. It plots five lanes for 1 hertz, 2.08 hertz, 3 hertz, 4 hertz, and 5 hertz. Panel C: Shows the speech signal, acoustic waveform which is aligned with the pupillary response. The top part shows the acoustic waveform with the words, ta De go Bu ta De bu Da and the corresponding pupillary response. The bottom part shows the words, ta De go Bu ta DE bu Da again, but this time aligned with the prosodic aspects of the speech.

This figure is reproduced with permission from Marimon et al. (2022).

Results showed that adults segmented the familiarization stream into prosodic words. Behavioral responses indicated that they had heard prosodic words but not statistical words or nonwords in the familiarization stream. The same results were obtained also from their pupillary response: Their pupils dilated significantly more to nonwords and to statistical words than to prosodic words. The comparison of adult participants’ behavioral and pupillary responses therefore shows that changes in pupil size can reveal recognition of words in speech segmentation experiments also with a spontaneous physiological response that does not require an overt response. In contrast, we failed to find any evidence for consistent segmentation of the familiarization phase with either prosodic or statistical cues in nine-month-old German-learning infants. Infants’ pupillary response at test showed no significant differences between prosodic words, statistical words, and nonwords. This suggests that, as a group, nine-month-olds failed to segment the familiarization stream consistently with either prosodic or statistical cues, replicating previous studies with nine-month-old German-learning infants in a behavioral paradigm (Marimon, Reference Marimon2019).

However, the overall failure of nine-month-old infants as a group to show a preference for prosodic or statistical words does not mean that infants failed to segment the familiarization stream altogether. It is also possible that some infants were segmenting the familiarization stream into prosodic words and others into statistical words. To explore this possibility, we used frequency tagging on infants’ and adults’ pupillary response during the familiarization phase (see Figure 41.1). By decomposing the pupillary response during the whole familiarization phase into different frequency components using a fast Fourier transform, we observed that changes in pupil size at stimulus frequency in adults were temporally aligned with prosodic word onsets. Furthermore, the variability in the phase of the pupillary response was predictive of participants’ performance at test – namely, those participants’ whose pupils were more aligned with prosodic word boundaries during the familiarization were better at recognizing prosodic words in test. In contrast, the temporal alignment of the pupillary response during the familiarization in infants showed considerable variability, with those infants who were more entrained to prosodic words showing better recognition of prosodic words at test and those infants who were more entrained to statistical words showing better recognition of statistical words at test. Pupillary entrainment to temporal regularities in continuous speech is therefore informative about how adults and infants segment words from continuous speech and provides insights about the segmentation process that is difficult to discern with behavioral methods (Marimon et al., Reference Marimon, Höhle and Langus2022).

41.4 Pupils Rapidly Entrain to Auditory Rhythms

The results of the cue-weighting experiments suggest that pupillary entrainment can be informative about how infants and adults parse a continuous speech stream into possible word candidates. However, since the FFT in these analyses considered the familiarization phase as a whole, it did not reveal the dynamics of entrainment as it unfolded during the familiarization phase. This raises the question of whether the pupils can become entrained to rhythmic stimuli fast enough to provide more detailed information about the time course of the perception of rhythmic auditory stimuli. In fact, there is some evidence that nonhuman primates’ pupils can entrain to repetitive auditory patterns within only a few repetitions of the pattern (Barczak et al., Reference Barczak, O’Connell and McGinnis2018). However, because primates in these studies were presented with hundreds of trials, which is impossible with young infants, it remains unknown whether temporally more fine-grained entrained responses could also be observed in younger infants. To answer this question, we reanalyzed data specifically for this chapter from one of our recent speech perception experiments in young infants (Langus et al., Reference Langus, Boll-Avetisyan, van Ommen and Nazzi2023).

This study tested whether German-learning infants (N=31) perceive lexical stress as categorical by presenting infants with short trials consisting of four instances of the nonword “gaba” while we recorded their pupil size with an eye-tracker. The words were chosen from a lexical stress continuum that covaried pitch, duration, and intensity cues ranging from trochaic (i.e., word-initial: strong-weak) to iambic (i.e., word-final: weak-strong). The first three instances of the word in each trial always consisted of the same item from the acoustic continuum and provided the prosodic context that varied across trials. The fourth instance provided the test (trochaic or iambic) that was identical across trials, and it could be the same as the context (i.e., standard trials), be from the opposite category as the context (i.e., between-category trials; it had the same prominence as the standard trials but in the opposite category), or from the same category as the context (i.e., within-category trials; it had the same prominence as the standard trials but the acoustic cues were more prominent). We reasoned that evidence for categorical perception would entail infants’ pupil dilations showing: (a) discrimination of between-category trials from within-category and standard trials, and (b) no discrimination between within-category and standard trials. The analysis of the pupillary response following the fourth word – that is, the test word – showed that only those six-month-old infants who had above-average exposure to various linguistic and/or musical activities at home showed differences in pupil size between rhythmic patterns that span category boundaries. These results reinforce the idea that pupillometry can be used to study linguistic phenomena in very young infants.

For this chapter, we carried out an additional exploratory analysis to investigate whether infants’ changes in pupil size synchronize to the occurrence of the word “gaba” in these short trials, that is, whether and when the pupillary response at stimulus frequency became aligned with the occurrence of the words in the trial. Following the analysis outlined in Barzack et al. (Reference Barczak, O’Connell and McGinnis2018), we bandpass-filtered the pupillary response throughout the trial into a narrow frequency band centered around 1.72Hz (+/− 25%). This frequency corresponded to the frequency at which the words occurred in the trial. To determine whether infants’ pupillary response at stimulus frequency was consistent across trials, we calculated the inter-trial pupillary coherence (ITPC) of the pupillary response at stimulus frequency across all the good trials for a given infant (M = 24.48). The ITC is bounded between 0 and 1, with 1 corresponding to perfect coherence of the pupillary response across trials and 0 corresponding to no inter-trial coherence. We baseline-corrected the ITC by subtracting the average ITC during 500 ms before first-word onset from the ITC throughout the trial.

The results of this exploratory analysis indicate that the phase of the pupillary response at stimulus frequency during the trial showed significant phase clustering from the second repetition of the word in the trial (Figure 41.2). To investigate the evolution of the ITC of the pupillary response during the trial, we also ran a linear regression with the ITC values as the dependent variable and the position of the word in the trial (first to fourth) as a categorical fixed factor. The pupillary response at stimulus frequency did not differ significantly from zero at first-word onset (Intercept: β = .074, SE = .054, t = 1.356, p = .177), and there was no significant increase in ITC by the second-word onset (β = .113, SE = .077, t = 1.468, p = .114). However, a significant increase in coherence when compared to first-word onset was observed at the third- (β =.252, SE = .077, t = 3.279, p < 0.01) and fourth-word (β = 0.374, SE = .077, t = 4.873, p < .0001) onsets (Figure 41.2B and 41.2C). Taken together, the phase of the pupillary response at stimulus frequency shows a significant increase in inter-trial coherence by the third word in the trial. These results suggest that infants’ pupils can rapidly entrain to the occurrence of temporally predictable patterns in auditory stimuli. This indicates that by the second repetition of the word, the pupillary response has entrained to the occurrence of words in the stimuli.

Figure 41.2

Pupillary synchrony to the words in the trial.

(A) The pupillary response at stimulus frequency (1.72 Hz) averaged across trials and infants. The shaded area corresponds to the duration of the words in the trial. Pupil size is negative because of narrow-band filtering. (B) Evolution of the baseline-corrected inter-trial coherence (averaged ITC across infants) of the pupillary response at stimulus frequency during the whole trial. (C) Comparison of ITC at each word onset.

Figure 41.2 Long description

Panel a: A bar and line graph depicts pupil size versus time in seconds. It plots a fluctuating line which originates at (minus 1, 0.00) and terminates at (6, 0.00). The line overlaps the bars labeled baseline first word, second word, third word, and fourth word. Panel b: A line graph of I T C versus time in seconds. It plots a curve which originates at (minus 1, 0.00), rises and peaks at (2.5, 0.35), falls and terminates at (6, 0.08). Panel c: A violin plot of I T C values for each of the four words. The box plots show the distribution of I T C values for each word presentation. The horizontal lines above the box plots indicate statistical significance. The asterisks are marked above the violin plots. The median values of I T C are as follows. Fourth word, 0.27. Third word, 0.25. Second word, 0.18. First word, 0.10. The values are estimated.

41.5 The Role of Rhythm in Speech Perception and Segmentation in Infancy

Rhythm may play an important role in how young infants acquire language as infants are sensitive to rhythmic properties of speech from birth (Jusczyk, Reference Jusczyk1997; Langus et al., Reference Langus, Mehler and Nespor2017; Chapters 11, 35, 36, and 38). Previous studies have shown that the regular occurrence of words of equal length or knowledge of the rhythmic structure of words may mediate young infants’ and adults’ ability to parse continuous speech into possible word candidates (Johnson and Tyler, Reference Johnson and Tyler2010; Mersad and Nazzi, Reference Mersad and Nazzi2012; Wang et al., Reference Wang, Zevin and Mintz2017, Reference Wang, Zevin, Trueswell and Mintz2020). This suggests that temporal regularities in continuous speech may lead listeners to build expectations of when word boundaries are likely to occur. The results outlined in this chapter complement this line of research by showing that adults’ and infants’ changes in pupil size track temporal regularities (i.e., possible word candidates) signaled by statistical or prosodic cues, and that this ability may facilitate the segmentation of continuous speech already during the first year of life. Furthermore, the analysis of six-month-old infants’ pupil size in short trials consisting of only four words further suggests that pupils entrain to auditory stimuli very rapidly, that is, within two repetitions of words. This raises the question of what role rapid entrainment to auditory stimuli that are kept artificially rhythmic may play in language acquisition.

The suprasegmental rhythm of spoken language is contained within intonational contours that correspond to prosodic units with boundaries marked by acoustic cues such as pauses and final lengthening (Nespor and Vogel, Reference Nespor and Vogel1986). This means that rhythm will be most informative about the underlying structure of spoken language within the bounds of a prosodic unit that typically corresponds to a single sentence or a phrase. Infant-directed speech typically uses three–five words per utterance (Saksida et al., Reference Saksida, Langus and Nespor2017), limiting rhythmic units for infants to entrain to. The sentence length (as with the word length; Johnson and Tyler, Reference Johnson and Tyler2010; Mersad and Nazzi, Reference Mersad and Nazzi2012) likely influences rhythmic predictions’ benefit in speech learning. Infants in our analysis entrained by third-word onset, implying that early entrainment allows more learning time as sentences unfold. Rapid entrainment may help infants in extracting structure from relatively short acoustic signals, such as single sentences in spoken language.

However, most studies use highly rhythmic stimuli, unlike naturally occurring speech, which raise questions about the role of entrainment in infants’ real speech perception. It is possible that the natural language temporal variability might hinder entrainment, limiting its applicability from controlled experiments to everyday speech. However, infants seem particularly adept at learning from the rhythmic speech signal that surrounds them (e.g., Kuhl, Reference Kuhl2010; Leong et al., Reference Leong, Kalashnikova, Burnham and Goswami2017; Chapter 35), which includes a variety of rhythmic stimuli – such as nursery rhymes, singing, instrumental music, as well as infant-directed speech – that boost infants’ ability to detect structure from linguistic stimuli (see Suppanen et al., Reference Suppanen, Huotilainen and Ylinen2019; Langus et al., Reference Langus, Boll-Avetisyan, van Ommen and Nazzi2023; see also Chapter 23). This suggests that infants might grasp language effectively from predictably rhythmic stimuli including songs, nursery rhymes, and structured speech (spoken language with relatively regular and temporally predictive rhythmic structure).

41.6 Why Pupils Are Interesting

In contrast to methods commonly used in infant research, the pupillary response can be evoked in a passive listening procedure and does not need an overt behavioral response. Because infants’ pupil size can be measured with an eye-tracker automatically from around 3.5 months of age (Hochmann and Papeo, Reference Hochmann and Papeo2014; Saksida and Langus, Reference Saksida and Langus2024), the pupillary response is one of the few experimental methods that allows for testing speech perception in adults and infants using the same experimental paradigms. While the paradigms used for pupillometry are like those used in electroencephalography (EEG), pupillometry does not require placement of electrodes that can be time-consuming and fastidious for infants. Experiments using pupillometry are, therefore, likely to yield more data and result in less data loss when dealing with young infants. While the analysis of the pupillary response is similar to EEG data, it is computationally less demanding due to lower sampling rate and fewer channels (i.e., only two: the left eye and the right eye). Furthermore, eye-trackers are considerably cheaper than equipment for recording electrophysiological brain responses. As such, pupillometry may be more beginner-friendly in an experimental field that has primarily focused on behavioral paradigms.

Pupillometry is also interesting for measuring rhythm perception. Rhythmic entrainment is characterized by sensorimotor synchronization, that is, spontaneous neural and behavioral synchronization to the rhythmic beat (see Chapter 6). Rhythm perception is therefore often measured in terms of neural entrainment (for an overview: Lakatos et al., Reference Lakatos, Gross and Thut2019; Obleser and Kayser, Reference Obleser and Kayser2019) or by asking participants to tap their finger to the rhythm of auditory stimuli (Repp, Reference Repp2005; Iversen and Patel, Reference Iversen and Patel2008). Evidence for changes in pupil size synchronizing to auditory rhythms are therefore interesting because pupil size is both a correlate of underlying neural activity as well as a spontaneous motor output. For example, studies in primates suggest that neural ensembles in the primary auditory cortex entrain to repeating auditory patterns in a comparable time and manner as changes in pupil size (Barczack et al., Reference Barczak, O’Connell and McGinnis2018). Pupil size could therefore function as a proxy for underlying neural entrainment. However, changes in pupil size are also caused by pupillary muscles in the oculomotor system (Mathôt, Reference Mathôt2018). Similar to spontaneous tapping of the finger to regular rhythm, pupillary entrainment is also evidence for sensorimotor synchronization that results in motor output. While synchronizing changes in pupil size is likely to involve quite different neural pathways than finger tapping, it has the advantage of emerging spontaneously without explicit instruction. This means that pupillary entrainment to auditory rhythms may provide a unique – if not the only – way to study spontaneous sensorimotor synchronization in the laboratory across the lifespan.

Finally, it is also possible that pupil dilation, as the result of cognitive and/or perceptual processing, is physiologically relevant for audiovisual perception. Similar to the effect of varying the size of the aperture in photo cameras, larger pupil size will result in shallower depth of vision and consequently limit the visual detail that is perceived (e.g., a portrait photo with a blurred background: Marcos et al., Reference Marcos, Moreno and Navarro1999; Ebitz and Moore, Reference Ebitz and Moore2019). Because speech is an audiovisual experience where auditory and visual cues are integrated online (McGurk effect; McGurk and MacDonald, Reference McGurk and MacDonald1976; Guellai et al., Reference Guellai, Langus and Nespor2014; Peña et al., Reference Peña, Langus, Gutiérrez, Huepe-Artigas and Nespor2016), pupil size changes elicited by auditory processing may facilitate the visual perception of the speaker’s face by blending out unnecessary visual background information. Further studies could explore whether changes in pupil size due to surprise, cognitive load, or entrainment to auditory stimuli affect depth of field to facilitate visual perception. Since speech is rarely perfectly rhythmic, it is necessary to confirm if pupil changes also entrain to natural speech. While it is uncertain how much pupil size alterations impact audiovisual perception, results suggest that rapid pupil size changes linked to auditory processing may facilitate integration of audiovisual signals. If validated, pupillometry will not just correlate but also significantly contribute to understanding how we perceive and acquire audiovisual information.

41.7 Conclusion

In this chapter we argued that rhythm plays an important role in speech perception and word segmentation by showing results from different studies using pupillometry. The first study showed that both infants’ and adults’ pupils can entrain to a continuous speech stream and that this is informative about their word segmentation abilities. In the second study, a reanalysis of Langus et al. (Reference Langus, Boll-Avetisyan, van Ommen and Nazzi2023), we demonstrated that pupils can rapidly entrain to rhythmic stimuli within only a few repetitions of the words in a trial. Our results show that pupillometry can be a useful tool when investigating speech perception, potentially revealing the dynamics of how continuous speech is parsed into words as the speech signal unfolds. However, the question of whether pupil entrainment can also capture and synchronize to the rhythmic variability of natural speech still remains open, and further research is needed. Furthermore, we suggested that pupillometry can be a great method to complement behavioral responses and to be used to investigate speech perception from early infancy to adulthood. Therefore, it can allow us to gain further knowledge on the underlying perceptual and cognitive mechanisms in word segmentation and speech perception.

41.8 Acknowledgements

This work was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) funding for the Sonderforschungsbereich (Collaborative Research Center) “Limits of Variability in Language” (grant number 317633480, SFB 1287, collaboration between projects C03 and C07), by the European Union’s Horizon 2020 Individual Marie-Curie Fellowship to Alan Langus (grant number 748909; “RHYTHMSYNC”), and the Postdoc Prize of Brandenburg 2022 (Ministerium für Wissenschaft, Forschung und Kultur Brandenburg) to Mireia Marimon.