42.3.2.1.1 Perception of Speech Rhythm by Children with HL

Before the advent of regular newborn hearing screening, one indicator of auditory disability was the atypical speech produced by children with HL. In 1936, educator Charles G. Rawlings noted that the speech of pre-lingually Deaf children was often “arrhythmic,” but that “If the deaf child could be … trained from his first years in school to speak with a normal rhythm even at the cost of perfect articulation, his speech would be far more natural and intelligible to the general public” (Rawlings, Reference Rawlings1936:143). Such differences in production could indicate limited perceptual access to rhythmic properties in speech. For example, researchers report that children with HL (including those fitted with HAs and/or CIs) have poor sensitivity to lexical and syllabic stress (Holt et al., Reference Holt, Demuth and Yuen2016; Kalathottukaren et al., Reference Kalathottukaren, Purdy and Ballard2017; Konadath et al., Reference Konadath, Raveendran and Yeshoda2021; Lyxell et al., Reference Lyxell, Wass and Sahlén2009; Most and Peled, Reference Most and Peled2007; Segal et al., Reference Segal, Houston and Kishon-Rabin2016), which in natural speech are conveyed as variations in intensity, duration, intonation, or a combination of these. This reduction in sensitivity is already apparent when comparing infants (12–33 months old) with CIs to typically hearing infants (Segal et al., Reference Segal, Houston and Kishon-Rabin2016), although the overall pattern of developing a neural response to verbal stress appears to be similar across infants with CIs and those with typical hearing (Vavatzanidis et al., Reference Vavatzanidis, Mürbe, Friederici and Hahne2016). The fine-grained representation of auditory cues, however, are also likely to differ according to hearing device: Children (8–15 years old) who wear HAs have been found to have superior intonation and stress perception in comparison to children with CIs (Most and Peled, Reference Most and Peled2007). Recently, longitudinal experiments have shown that, for children (5–9 years old) with better residual hearing, the combination of HAs and CIs in contralateral ears may lead to improved speech prosody perception outcomes, presumably due to the low-frequency audibility provided by HAs (Davidson et al., Reference Davidson, Geers, Uchanski and Firszt2019). In the same cohort, prosody perception was determined to be uniquely predictive of later achievement in other linguistic areas including reading and phoneme perception (Grantham et al., Reference Grantham, Davidson, Geers and Uchanski2022). Hence, although children with HL tend to encounter difficulties in prosody perception, there is a wide range of ability that likely depends on hearing history, mode of amplification, and many other factors (see Wenrich et al., Reference Wenrich, Davidson and Uchanski2017, for a more in-depth discussion).

42.3.2.1.2 Production of Speech Rhythm by Children with HL

Scientific efforts to quantify the subjectively “arrhythmic” character of speech produced by children with HL generally cohere with Rawlings’ qualitative observation (Reference Rawlings1936). That is, there is consensus for “errors of prosody (e.g., errors of intonation, stress, and/or phrasing)” (Parkhurst and Levitt, Reference Parkhurst and Levitt1978). Children with HL tend to produce atypical patterns of stressed and unstressed syllables (Gold, Reference Gold1980; Lenden and Flipsen, Reference Lenden and Flipsen2007; Murphy et al., Reference Murphy, McGarr and Bell-Berti1990; Osberger and Levitt, Reference Osberger and Levitt1979; Sundström et al., Reference Sundström, Löfkvist, Lyxell and Samuelsson2018) – not only by misapplying stress within multisyllabic words but sometimes also by failing to differentiate or mark any stress at all, resulting in “staccato” speech with roughly equal syllabic weighting (McGarr and Osberger, Reference McGarr and Osberger1978; Osberger and Levitt, Reference Osberger and Levitt1979). Similarly, some analyses suggest that speech by children with HL is disrupted by the unusual placement of pauses (Bochner et al., Reference Bochner, Barefoot and Johnson1987; Gold, Reference Gold1980; McGarr and Osberger, Reference McGarr and Osberger1978; Osberger and Levitt, Reference Osberger and Levitt1979; Rawlings, Reference Rawlings1936). Children (6–10 years old) who use CIs are also less able to correctly imitate syllabic and stress patterns in both pseudo-words (Carter et al., Reference Carter, Dillon and Pisoni2002) and real words (Grandon et al., Reference Grandon, Vilain and Gillis2019), in comparison to typically hearing children.

Presumably, deficits in children with HL’s speech production originate in absent or degraded sensory input during speech perception, including auditory sensorimotor feedback of one’s own voice. Where the link, within-child, between speech prosody perception and production has been formally tested, some studies found a significant association (e.g., Klieve and Jeanes, Reference Klieve and Jeanes2001; Konadath et al., Reference Konadath, Raveendran and Yeshoda2021), although not all (e.g., Kalathottukaren et al., Reference Kalathottukaren, Purdy and Ballard2017). Objective and ratings-based measures of speech prosody production by children with HL have been found to correlate with other aspects of their linguistic development, including vocabulary and word recognition skills (Carter et al., Reference Carter, Dillon and Pisoni2002; Lyxell et al., Reference Lyxell, Wass and Sahlén2009). The reasons for this are unclear, although some possible mechanisms are addressed in the following section. Timing issues during speech communication can emerge as early as infancy: Dyads consisting of a mother and a CI-recipient child (one–two years old, observed three and nine months post-implant) produce longer between-speaker pauses relative to within-speaker pauses, contrary to the pattern associated with typically hearing children (Kondaurova et al., Reference Kondaurova, Smith, Zheng, Reed and Fagan2020). In addition, when compared to typically hearing peers, children with CIs (seven years and older) show reduced speech rate-matching during interactions with a clinician, and continue to demonstrate such delays well into adolescence (Freeman and Pisoni, Reference Freeman and Pisoni2017).

42.3.2.1.3 Auditory Deprivation and the Development of Rhythmic-Sequential Ability

Some authors argue that HL disrupts the development of cognitive faculties important for speech, particularly sequence learning, which is “the ability to encode and represent the order of discrete elements occurring in a sequence” (Conway and Christiansen, Reference Conway and Christiansen2001:539). Sequential learning has been shown to be reduced in children and adults with HL – even when using nonauditory paradigms (Conway et al., Reference Conway, Pisoni, Anaya, Karpicke and Henning2011; Lévesque et al., Reference Lévesque, Théoret and Champoux2014). Why would this be? One theory is that because sound is an inherently temporal signal, its analysis relies on attention, working memory, and the processing of serial order; hence, lack of experience with sound would deprive the developing child of crucial experience with temporal processing more generally (Conway and Christiansen, Reference Conway and Christiansen2005, Reference Conway and Christiansen2009). This coheres with evidence linking sequence learning to perceptual sensitivity for nonverbal auditory rhythms in typically hearing populations (François and Schön, Reference François and Schön2011; MacIntyre et al., Reference MacIntyre, Lo, Cross and Scott2022). Moreover, sequence and rhythm processing appear to share some neural correlates (Bednark et al., Reference Bednark, Campbell and Cunnington2015; Janata and Grafton, Reference Janata and Grafton2003). In the context of speech rhythm, insensitivity to stress and pauses could interfere with sequential processing by, for instance, obscuring syntactic boundaries (e.g., “fruit salad and milk” versus “fruit, salad, and milk”), a cue that is frequently missed by children with HL (Fortunato et al., Reference Fortunato-Tavares, Schwartz, Marton, de Andrade and Houston2018; Kalathottukaren et al., Reference Kalathottukaren, Purdy and Ballard2017). This may also bear some relationship to observed correlations between aberrant rhythm processing and developmental difficulties with grammar and syntax in the typically hearing population (Gordon et al., Reference Gordon, Jacobs, Schuele and McAuley2015; Ladányi et al., Reference Ladányi, Persici, Fiveash, Tillmann and Gordon2020; Lee et al., Reference Lee, Ahn, Holt and Schellenberg2020; Nitin et al., Reference Nitin, Gustavson and Aaron2023; see also Chapters 25 and 39), the latter of which disproportionately affect children with HL (Delage and Tuller, Reference Delage and Tuller2007; Friedmann and Szterman, Reference Friedmann and Szterman2006; Kawar, Reference Kawar2021; Moeller et al., Reference Moeller, Tomblin, Yoshinaga-Itano, Connor and Jerger2007; Nittrouer and Lowenstein, Reference Nittrouer and Lowenstein2021; Szterman and Friedmann, Reference Szterman and Friedmann2014; but see also Briscoe et al., Reference Briscoe, Bishop and Norbury2001).

Critics of the proposed link between HL and sequential learning, however, counter that the “sequence learning deficit in children with CI may be closely tied to the nature of the task” (Torkildsen et al., Reference von Koss Torkildsen, Arciuli, Haukedal and Wie2018:127). For example, some paradigms employing visual stimuli may nonetheless lend themselves to verbal rehearsal, thereby conflating domain-general sequential learning with phonological processing. In a sequential learning task deliberately designed to avoid this, children with CIs did not differ in performance from typically hearing peers (Torkildsen et al., Reference von Koss Torkildsen, Arciuli, Haukedal and Wie2018). This accords with evidence that children with CIs show selective impairments in verbal, compared with spatial, working memory tasks (Davidson et al., Reference Davidson, Geers, Uchanski and Firszt2019). An additional source of ambiguity stems from the challenge of decoupling auditory from linguistic exposure. For instance, children with HL who acquire sign language from birth perform similarly to typically hearing children in tests of sustained attention (Dye and Hauser, Reference Dye and Hauser2014). Early experience with sign language also appears to benefit linguistic development in children with CIs who go on to communicate orally in later life (Lyness et al., Reference Lyness, Woll, Campbell and Cardin2013). Linking back to rhythm, Deaf adults who use sign language can synchronize to a visual rhythmic stimulus as well as hearing adults do to an auditory rhythmic stimulus, with hearing adults performing comparatively poorly in visual-motor rhythmic synchronization tasks (Iversen et al., Reference Iversen, Patel, Nicodemus and Emmorey2015). Hence, active engagement with temporally dynamic patterns, whether visual or auditory, may in fact be key to the development of sequential, and possibly also rhythmic, processing. Further work is needed to disentangle modality-specific from domain-general deficits, as well as rhythm from possibly related cognitive faculties, such as verbal working memory (Saito, Reference Saito2001; Saito and Ishio, Reference Saito and Ishio1998).

42.3.2.1.4 Rhythm-Based Interventions in Speech Therapy

If rhythm processing is not strictly reliant on auditory experience, it may be that difficulties with speech rhythm observed in children with HL are more indicative of the limitations of speech prosody rehabilitation (Hargrove et al., Reference Hargrove, Anderson and Jones2009; Peppé, Reference Peppé2009) than of some innate deficit per se. Specifically, it is unclear whether children “do not know the stress patterns of the language or whether they do not have the articulatory co-ordination that would permit them to produce an intended pattern correctly” (Tye-Murray and Folkins, Reference Tye‐Murray and Folkins1990:2675). Some researchers propose that deficits in speech “flow” experienced by children with HL may be partly due to the “teaching of articulation of individual isolated elements rather than longer, more meaningful units of speech” (Gold, Reference Gold1980:408; see also Howard, Reference Howard2007; Tye-Murray and Woodworth, Reference Tye‐Murray and Woodworth1989). This could also help to explain why coarticulation – how the production of consecutive speech segments contextually affect each other – is reduced in HL, in comparison with typically hearing children (Okalidou and Harris, Reference Okalidou and Harris1999; Waldstein and Baum, Reference Waldstein and Baum1991). Yet, speech rhythm rehabilitation for children with HL may be facilitated by nonauditory sensorimotor experience – for instance, by demonstrating model prosodic patterns via finger tapping (Tye-Murray and Folkins, Reference Tye‐Murray and Folkins1990). Some research groups have explored this idea by using active rhythm training to enhance sensitivity to speech temporal features (see León Méndez et al., Reference del Carmen León Méndez, Fernández García and Daza González2023, and Pesnot Lerousseau et al., Reference Pesnot Lerousseau, Hidalgo and Schön2020, for reviews). For instance, it was found that a short, rhythm-focused training session, rather than the usual speech therapy session, was associated with more rhythmically stable speech production during a subsequent verbal turn-taking task for children with HAs and/or CIs (5–9 years old) (Hidalgo et al., Reference Hidalgo, Falk and Schön2017). The authors further demonstrated using electroencephalography (EEG) that the mismatch negativity, a neural correlate of prediction error, in response to rhythmically irregular trials was modulated by rhythmic training in children with HL (Hidalgo et al., Reference Hidalgo, Pesnot-Lerousseau, Marquis, Roman and Schön2019). These speech-focused studies concur with a similar experiment using musical stimuli, which found that active motor engagement with temporal structure enhanced the ability to identify songs in children with CIs (4–12 years old) (Vongpaisal et al., Reference Vongpaisal, Caruso and Yuan2016). Finally, in another speech rhythm priming study, the authors report that the presentation of a nonverbal rhythm, the structure of which was shared with the following utterance to be produced, improved phonological accuracy in children with various hearing devices (mean age 8.72 years old, SD 2.19; Cason et al., Reference Cason, Hidalgo, Isoard, Roman and Schön2015). It remains to be seen whether the benefits observed in any of these interventions persist over time or are rather limited to the immediate post-intervention period, and the results should be replicated with enhanced control conditions and blinding procedures – including, where possible, blinding of therapists and others administering the rhythm training (Hargrove et al., Reference Hargrove, Anderson and Jones2009; McKay, Reference McKay2021).

42.3.2.1.5 Musical Rhythm Perception and Production by Children with HL

In one study, children with CIs demonstrated poorer musical pitch and timbre perception, but not rhythm perception, when compared to typically hearing children; however, many of the children with CIs (9–13 years old) were involved in external music lessons (Innes-Brown et al., Reference Innes-Brown, Marozeau, Storey and Blamey2013). Another study reports marginally better performance for temporal, rather than pitch-based, recognition of melodies in children with CIs (5–7 years old) (Volkova et al., Reference Volkova, Trehub, Schellenberg, Papsin and Gordon2014). Other authors emphasize that children with HL who wear CIs and/or HAs (5–10 years old) are less accurate than typically hearing children when tapping in synchrony to both simple and complex rhythms, despite similar levels of motor variability when freely tapping (Hidalgo et al., Reference Hidalgo, Zécri and Pesnot-Lerousseau2021). In another study, children with CIs (median age 11.25 years old, IQR 5.58) performed at chance in a nonverbal rhythm discrimination task on average, but individual scores did not correlate with other auditory skills, nonverbal intelligence, nor grades in music class (Stabej et al., Reference Stabej, Smid and Gros2012). Taken together, it appears that temporal cues in nonverbal rhythmic stimuli are at least partially available to children with HL but that aspects of rhythmic processing are nonetheless impeded or delayed in comparison to typically hearing children – in particular, when rhythmic stimuli are more complex or incorporate changes in pitch (Darrow, Reference Darrow1984; Roy et al., Reference Roy, Scattergood-Keepper and Carver2014). Data from EEG reveal a similar pattern of results: When perceiving temporal deviations in musical rhythmic stimuli, children and adolescents with CIs produce a mismatch negativity, but it is reduced in magnitude and appears at a slower latency in comparison with typically hearing children (Petersen et al., Reference Petersen, Weed and Sandmann2015; Torppa et al., Reference Torppa, Salo and Makkonen2012). The heterogeneity of the samples in these studies and the varying levels of complexity and naturalism of the stimuli employed, however, preclude firm conclusions, and I will discuss further evidence regarding nonverbal rhythm perception in adults with HL in the next section (see also Shukor et al., Reference Shukor, Han, Lee and Seo2021, for a systematic review of musical interventions for children with HL).

42.3.2.2.1 Perception of Speech Rhythm by Adults with Post-lingual HL

In an experiment using controlled, isolated stress contrasts, adult CI users did not differ from typically hearing controls when discriminating syllabic stress based on duration cues; however, they did perform significantly worse when stress cues consisted of changes in intonation or intensity (Meister et al., Reference Meister, Landwehr, Pyschny, Wagner and Walger2011). In the same study, the participants then completed a stress identification task with naturally produced stimuli in sentence form – for this more complex and realistic task, the CI users showed significantly poorer discrimination than the controls, and sensitivity to duration did not correlate with performance; however, there was an advantage for CI listeners who could better discriminate stress using a combination of intonation and intensity cues (Meister et al., Reference Meister, Landwehr, Pyschny, Wagner and Walger2011). Another study found that adults with CIs require substantially larger changes in fundamental frequency (F0) to identify pitch accents in multisyllabic words in comparison to adults with typical hearing (Dincer D’Alessandro and Mancini, Reference Dincer D’Alessandro and Mancini2019). Device-related differences in intonation perception are similar to those seen in children, such that adults who use both a CI and an HA have better perception of intonation, syllable stress, and emphatic stress than when using a CI alone (Most et al., Reference Most, Harel, Shpak and Luntz2011). Finally, adults with uncorrected HL (i.e., no CI or HA use) were found to better identify syllabic stress when the target was surrounded by speech envelope-modulated noise, as opposed to in the original, unprocessed sentence or when presented in isolation (Barac-Cikoja and Revoile, Reference Barac-Cikoja and Revoile1996). This study is intriguing in that it suggests a facilitating effect of prosodic context, but a cost of fine-grained phonetic and/or semantic processing, for adults with HL when they explicitly attend to speech rhythm cues.

The effects of specific properties of rhythm in speech have also been studied in the context of HL. For instance, similarly to typically hearing control participants, adults with CIs benefit from predictable patterns of syllabic stress, known in the literature as metrical patterns, during speech in noise perception (Perry and Kwon, Reference Perry and Kwon2015; Spitzer et al., Reference Spitzer, Liss, Spahr, Dorman and Lansford2009). Older adults with and without HL appear to use metrical rhythmic patterns similarly to younger adults when perceiving speech in noise (Woodfield and Akeroyd, Reference Woodfield and Akeroyd2010). There may be a relationship, at least in typically hearing adults, between the ability to perceive speech in noise and sensitivity to metric cues (Borrie et al., Reference Borrie, Baese-Berk, Van Engen and Bent2017). Awareness of pitch-based syllabic stress is associated with greater speech in noise perception among CI and typically hearing listeners (Dincer D’Alessandro and Mancini, Reference Dincer D’Alessandro and Mancini2019). In fact, sensitivity to rhythm more generally has also been linked with speech in noise perception. For instance, nonverbal rhythm – but not pitch or timbre – processing ability significantly predicted speech in noise perception by adults with CIs (Leal et al., Reference Leal, Shin and Laborde2003). Nonverbal rhythm perceptual ability also correlates with superior speech in noise understanding in typically hearing adults (Slater et al., Reference Slater, Kraus and Woodruff Carr2018; Slater and Kraus, Reference Slater and Kraus2016; Yates et al., Reference Yates, Moore, Amitay and Barry2019).

42.3.2.2.2 Production of Speech Rhythm by Adults with Post-lingual HL

Adults who acquire HL later in life may show changes in produced speech rhythm that include excessive lengthening of vowels and abnormal stress patterns (Cowie and Douglas-Cowie, Reference Cowie, Douglas-Cowie, Lutman and Haggard1983; Lane and Webster, Reference Lane and Webster1991; Plant and Hammarberg, Reference Plant and Hammarberg1983). Case studies suggest that, after cochlear implantation, objectively measured and listener-rated speech rhythmic patterns, such as the differentiation between stressed and unstressed syllables, improve (Economou et al., Reference Economou, Tartter, Chute and Hellman1992; Leder et al., Reference Leder, Spitzer and Milner1986; Tartter et al., Reference Tartter, Chute and Hellman1989). The basis of these changes is poorly understood, but CI users with post-lingual HL provide a unique opportunity for researchers to understand the role of auditory feedback in speech production more generally (Perkell, Reference Perkell2012; Ubrig et al., Reference Ubrig, Goffi-Gomez and Weber2011). Although CIs are thought to transmit temporal, in comparison to spectral, speech properties quite well, post-lingual CI listeners still have poorer temporal resolution, as measured via gap detection, than typically hearing listeners (Blankenship et al., Reference Blankenship, Zhang and Keith2016; Cesur and Derinsu, Reference Cesur and Derinsu2020; Duarte et al., Reference Duarte, Gresele and Pinheiro2016) – if better temporal resolution than pre-lingual CI recipients (Wei et al., Reference Wei, Cao, Jin, Chen and Zeng2007). Hence, CI speakers’ variable articulatory strategies may reflect not only the absence of auditory feedback during the pre-implantation period but the temporally degraded nature of the information provided by their device post-implantation.

42.3.2.2.3 Musical Rhythm Perception and Production by Adults with HL

As with the literature concerning children, the evidence for a general deficit in musical rhythm perception in adults with HL is mixed. Some studies report similar performance in rhythm perception across typically hearing and CI-listener groups (e.g., Brockmeier et al., Reference Brockmeier, Fitzgerald and Searle2011; Limb et al., Reference Limb, Molloy, Jiradejvong and Braun2010). A study comparing adults with post-lingual HL who used either HAs or CIs found them to perform similarly in a rhythm discrimination task (Looi et al., Reference Looi, McDermott, McKay and Hickson2008). Another experiment compared CI listeners to typically hearing listeners, although the control group in this case was much younger and more musically trained than the CI group (Kong et al., Reference Kong, Cruz, Jones and Zeng2004). There was no difference in performance on the basis of tempo discrimination but mixed results for CI listeners in rhythmic pattern identification, with particular difficulty associated with complex melodic rhythmic stimuli (Kong et al., Reference Kong, Cruz, Jones and Zeng2004). In another study examining production, adult CI users were observed to synchronize their movements to non-pitched drum rhythms as well as they could to a simpler, recurring tone, as well as a visual timing cue (Phillips-Silver et al., Reference Phillips-Silver, Toiviainen and Gosselin2015). Their sensorimotor timing was poorer than typically hearing controls, however, and they also tended to struggle more with more complex rhythmic stimuli that also incorporated changes in pitch.

It is sometimes assumed that timing information (e.g., the beat in music) is adequately available to most CI listeners (McDermott, Reference McDermott2004); however, an important limitation to the current literature on nonverbal rhythm perception with CIs is that stimuli complexity is rarely parameterized or formally compared within experiments. As a recent review notes, “many of the studies regarding CIs and rhythm perception utilize relatively simple perception tasks, such as basic pattern reproduction or sound stimuli that isolate rhythmic information” (Jiam and Limb, Reference Jiam and Limb2019:26). The few studies that have included both simple as well as more complex stimuli report performance differences across these conditions within CI listeners (Gfeller et al., Reference Gfeller, Woodworth, Robin, Witt and Knutson1997; Kong et al., Reference Kong, Cruz, Jones and Zeng2004; Phillips-Silver et al., Reference Phillips-Silver, Toiviainen and Gosselin2015). For example, adult CI listeners achieved scores similar to typically hearing controls in a simple rhythm perception task, but had significantly poorer performance in a more complex temporal patterning task that the authors suggest may better convey “aspects of musical experience such as phrasing, accent, and complex rhythmic structure” (Gfeller et al., Reference Gfeller, Woodworth, Robin, Witt and Knutson1997:259). Unlike the studies of young CI users mentioned previously, EEG studies investigating musical rhythm perception in adult CI users have not found a robust mismatch negativity to temporal deviants (Sandmann et al., Reference Sandmann, Kegel and Eichele2010; Timm et al., Reference Timm, Vuust and Brattico2014). Given this measure was present in children and adolescents, albeit reduced in comparison to typically hearing participants, the adult data may reflect differences in device advances and age of implantation, musical enjoyment and listening habits, or life experience as a result of changing attitudes towards HL and musical participation. Together, these inconsistencies across studies make it difficult to assess which temporal cues are or aren’t well conveyed by CIs, let alone what commonalities may be found across verbal versus nonverbal rhythm perception. Future work should clarify whether and how musical rhythm perception changes after HL, but if rhythm sensitivity can be trained in adults, it may serve as a specific, operationalizable target for speech-focused interventions as an alternative to general music or dance training, for example.