30.1 Introduction

The scholarly debate around speech rhythm over the last decades has produced a rich literature on its theoretical underpinnings (e.g., Arvaniti, Reference Arvaniti2009; Nolan and Jeon, Reference Nolan and Jeon2014; Gibbon, Reference Gibbon2021), its measurement (using so-called rhythm metrics, for example, Low et al., Reference Low, Grabe and Nolan2000; Deterding, Reference Deterding2001), as well as its applications to a great number of languages and dialects (Szakay, Reference Szakay2006; White and Mattys, Reference White and Mattys2007; Behrman et al., Reference Behrman, Ferguson, Akhund and Moeyaert2019).Footnote ¹

Speech rhythm can be defined as ‘the production, for a listener, of a regular recurrence of waxing and waning prominence profiles across syllable chains over time’ (Kohler, Reference Kohler2009, p. 41). While many scholars of speech rhythm might overall agree with this definition, it is in fact much broader in scope than the actual, practical definition and operationalisation of speech rhythm that many studies have used and continue to use. Most research on speech rhythm focuses on duration; that is, Kohler’s ‘waxing and waning’ is operationalised as the alternation of long and short syllables or vowels. However, Kohler refers to alternation in prominence, which might be realised acoustically by duration, but also by pitch, loudness, and potentially other acoustic features. In Section 30.3, the discussion will return to this point and explore ways of measuring rhythmic alternation involving features other than duration.

Speech rhythm was originally conceived as a suprasegmental phenomenon covering three classes of languages (e.g., Pike, Reference Pike1945; Abercrombie, Reference Abercrombie1967, p. 97):

1. (1) syllable-timed languages, with syllables of equal duration (i.e., isochronous), such as in Spanish (see top panel of Figure 30.1, where syllables are of equal duration and feet vary in duration)

2. (2) stress-timed languages, with feet, that is, a stressed syllable followed by one or more unstressed syllables, of equal duration (i.e., isochronous), such as in British English (see bottom panel of Figure 30.1, where feet are of equal duration and syllables vary in duration)

3. (3) mora-timed languages, with morae (a unit smaller than a syllable but often comprising more than one phoneme) of equal duration (i.e., isochronous), such as in Japanese.

Despite attempts to test this rhythm class hypothesis in terms of speech production (e.g., Dankovičová and Dellwo, Reference Dankovičová and Dellwo2007) and perception (e.g., Ramus et al., Reference Ramus, Dupoux and Mehler2003), there is substantial evidence that it lacks empirical support. In particular, two (interlinked) claims of the rhythm class hypothesis have been criticised: (i) the isochrony of specific prosodic units and (ii) the existence of discrete rhythm classes. Note that these two claims are independent – it is possible for the languages of the world to fall into discrete rhythm classes without isochrony.

Figure 30.1

Idealised syllable-timing and stress-timing.

Idealised syllable-timing involves syllables of equal duration and feet of unequal duration (top), while idealised stress-timing involves syllables of unequal duration and feet of equal duration (bottom).

Figure 30.1 Long description

Syllable Timing: In this pattern, the example feet have 9 syllables, and each is represented by a box of equal width. Out of 9, there are 3 prominent syllables that are indicated in a dark shade. Stress Timing: Here, the feet are of equal duration while the some of the syllables are shorter and others are longer. The legends for prominent syllables and non-prominent syllables are given at the bottom.

Figure from Fuchs (2016, p. 36).

However, evidence from speech production indicates that the claim of isochrony of syllables in syllable-timed languages, feet in stress-timed languages, and morae in mora-timed languages, respectively, is inaccurate (Dauer, Reference Dauer1983). Moreover, the second claim, that is, that there are distinct rhythm classes, also turns out to be problematic. Evidence from speech production indicates that languages do not fit neatly into rhythm classes (Grabe and Low, Reference Grabe, Low, Gussenhoven and Warner2002), but that there are gradual instead of categorical differences in timing between languages. Both adult listeners (White et al., Reference White, Mattys and Wiget2012) and infants (Molnar et al., Reference Molnar, Gervain and Carreiras2014; Gasparini et al., Reference Gasparini, Langus, Tsuji and Boll-Avetisyan2021) are sensitive to temporal information rather than to rhythm class in language discrimination.

This evidence prompted a reconceptualisation of speech rhythm as a gradable phenomenon (see White et al., Reference White, Mattys and Wiget2012), with some languages involving greater variability in the duration of syllables (stress-timed languages) and other languages lesser variability (syllable-timed languages). In addition to or instead of syllables, many studies focused on the nuclei of syllables, that is, vowels, similarly identifying greater variability in duration with stress-timing and lesser variability with syllable-timing. Meanwhile, the concept of mora-timing was all but abandoned.

Nevertheless, the original terminology often shines through, with languages or varieties regularly described as syllable- or stress-timed (as in the contributions to Kortmann and Schneider, Reference Kortmann and Schneider2004). Alternatively, in order to reflect the gradable nature of speech rhythm, other terms can be used, such as ‘more stress-timed’, ‘relatively stress-timed’ or ‘stress-based’, and mutatis mutandis for syllable-timing (Dauer, Reference Dauer1983; Braun and Geiselmann, Reference Braun and Geiselmann2011). Nevertheless, possibly because these newer terms are somewhat unwieldy, the older ones are still in use as a shorthand. Moreover, the expression ‘rhythm classes’ can also still be found (e.g., White and Mattys, Reference White and Mattys2007; Gasparini et al., Reference Gasparini, Langus, Tsuji and Boll-Avetisyan2021), though it is now used more rarely and should be entirely avoided (White et al., Reference White, Mattys and Wiget2012) since it inaccurately invokes clear-cut categories instead of reflecting the above-mentioned gradable nature of speech rhythm.

A considerable body of current research attempting to quantify and compare the speech rhythm of languages and dialects, or involving sociolinguistic variation more generally, relies on this notion of greater or lesser variability in the duration of consecutive syllables or vowels, with lesser variability identified with (relative) syllable-timing and greater variability with stress-timing (thus, in Figure 30.1, syllables would not be identical in duration for syllable-timing but just relatively similar, whereas in stress-timing, syllables vary greatly in duration). Returning to Kohler’s definition above, the basic idea behind this approach is that speech with a tendency towards stress-timing involves great variations in prominence (including duration) between prominent (or stressed) and non-prominent (or unstressed) syllables, while a tendency towards syllable-timing involves small variations between prominent and non-prominent syllables.

Overall, in current practice more studies measure speech rhythm with regard to the durational variability of vowels rather than syllables, while other metrics rely on the durations of consonants or yet other phonetic or phonological units. Technically, this method is more complex than discussed so far, since it is not the durational variability of vowel phonemes that is measured but the durational variability of vocalic intervals, that is, one or more consecutive vowels not interrupted by any consonants, possibly spanning word boundaries.

30.2 Duration-Based Metrics

Duration-based rhythm metrics were initially conceived around the aim of quantifying the notion that there are reduced vowels in unstressed syllables in stress-timed languages, while syllable-timed languages have no or little vowel reduction. In addition, stress-timed languages tend to allow consonant clusters with several consonant phonemes, while syllable-timed languages rarely do (Dauer, Reference Dauer1983, pp. 55–58; Ramus et al., Reference Ramus, Nespor and Mehler1999, p. 270; Schiering, Reference Schiering2007).

The considerable number of duration-based rhythm metrics can be classified along three broad distinctions, that is, (1) whether they rely on the durations of vowels, consonants, or syllables, (2) whether they normalise for variation in speech rate, and (3) how they quantify variability (see Figure 30.2). Thus, a distinction can be made between (1) vocalic, consonantal, and syllabic metrics, (2) rhythm metrics normalised for speech rate or not, and (3) global and local metrics.Footnote ²

Figure 30.2

Taxonomy of common duration-based rhythm metrics.

Duration-based rhythm metrics are classified here according to the segmental unit of measurement (vocalic, consonantal, or syllabic), the operationalisation of variability (local or global), and speech rate normalisation. Theoretically possible but uncommon metrics are shown in brackets.

Figure 30.2 Long description

The rhythm is categorized by phonetic segments, namely, Vocalic, Consonantal and Syllabic. Each segment is further classified into local and global variability. Each variability is further classified into speech rate normalised and not normalised. The terminal ends of the tree list the specific metrics calculated, such as n P V I V, r P V I V, Varco V, delta V, % V, and others.

Figure adapted and expanded from Fuchs (2023a).

Of these criteria, the global/local distinction requires further explanation. Global rhythm metrics compute a measure of variability of duration of all vocalic, consonantal, or syllabic intervals regardless of their position in the utterance. This can be realised by computing the standard deviation, which yields the measures ΔV (read: ‘Delta V’) and ΔC for vocalic and consonantal intervals, respectively (Ramus et al., Reference Ramus, Nespor and Mehler1999). These metrics can in turn be normalised for speech rate by taking the standard deviation, divided by the mean, multiplied by 100, resulting in metrics known as VarcoV, VarcoC, and VarcoS, respectively (also known as coefficients of variation for vocalic, consonantal, and syllable durations, respectively; Dellwo, Reference Dellwo, Karnowski and Szigeti2006; White and Mattys, Reference White and Mattys2007; Rathcke and Smith, Reference Rathcke and Smith2011).

While global rhythm metrics are computed without regard to the temporal order of the vocal, consonantal, or syllabic units, local metrics are based on differences between adjacent pairs of vocalic intervals (for vocalic metrics), consonantal intervals (for consonantal metrics), or syllables (for syllabic metrics). Finally, the mean of all pairwise comparisons is computed. These metrics are commonly referred to as pairwise variability indices (PVI), where an initial lower case ‘n’ indicates the speech-rate-normalised version and ‘r’ the raw or non-normalised version (Low et al., Reference Low, Grabe and Nolan2000). Of the six theoretically possible PVIs, three, i.e. rPVI-C, nPVI-V, and nPVI-S (Gibbon and Gut, Reference Gibbon and Gut2001),Footnote ³ have been applied in research, while the raw vocalic and syllabic (rPVI-V, rPVI-S) and the normalised consonantal index (nPVI-C, shown in brackets in Figure 30.1) are rarely or never used. Moreover, a rhythm metric that goes beyond this taxonomy, but is often used, accounts for the proportion of vocalic durations relative to total utterance duration, known as %V (Ramus et al., Reference Ramus, Nespor and Mehler1999; %V is technically the inverse of %C, and the convention is to refer to %V). In addition to the duration-based rhythm metrics discussed here, there are others that are less widely used; a comprehensive overview can be found in Fuchs (Reference Fuchs2016, pp. 35–52).

The considerable number of duration-based speech rhythm metrics prompts the question of whether they are all equally reliable or, alternatively, if one or a few of them are superior to the rest. Empirical validity tests indicate that nPVI-V, VarcoV, and %V are the most reliable and should therefore be preferred (White and Mattys, Reference White and Mattys2007; Wiget et al., Reference Wiget, White and Schuppler2010; note, however, that these tests did not include syllabic metrics).Footnote ⁴ Additional evidence on the validity of these metrics comes from speech perception and indicates that these speech production metrics appear to at least partially capture the human perception of rhythmicity (Fuchs, Reference Fuchs and Fuchs2023a). Finally, given that nPVI-V and VarcoV both measure variability in vocalic durations and are speech-rate-normalised, while %V accounts for the proportion of vocal durations over total utterance duration, it is recommended that studies on speech rhythm relying on duration-based metrics rely on %V as well as, at a minimum, either nPVI-V or VarcoV. With this choice of metrics, two potentially distinct aspects of rhythmicity are captured.

Current practice in the field does not completely follow this advice. Of all duration-based rhythm metrics, the one probably by far most widely used is nPVI-V. For example, a synthesis of previous research on variation in speech rhythm among varieties of English identified 18 relevant studies on 23 varieties of English applying the nPVI-V to samples of read speech (Fuchs, Reference Fuchs and Fuchs2023b). Taken together, these studies support the widely held assumption that so-called Outer Circle varieties (where English is widely used as a second language and local lingua franca, for example, Indian and Nigerian English) tend to be more syllable-timed than Inner Circle varieties (where English is mainly used as a first language, for exmaple, British, American, and Australian English). A major factor that likely accounts for this result is prosodic transfer from relatively syllable-timed languages widely used in countries where Outer Circle varieties of English are spoken. Moreover, so-called Expanding Circle varieties, where English is mainly used as a foreign language but does not enjoy official status or play a large local role (e.g., in Japan and Germany), also tend to be more syllable-timed than Inner Circle varieties. This tendency may in some cases be explained by prosodic transfer as well. However, other reasons are conceivable, too. These include a possible tendency towards syllable-timing in second-language acquisition, and a selection as well as publication bias in the extant publication record. Selection bias may involve researchers selecting research questions that are likely to confirm assumptions they and the field as a whole would consider to be in keeping with the extant research record. Moreover, publication bias may play a role when researchers engaging in such research are more likely to consider manuscripts worth publishing, and more likely to get manuscripts accepted in research outlets, when the results indicate a non-null finding.

The duration-based metrics discussed here have been criticised in several ways (for a more extensive discussion, see Fuchs, Reference Fuchs2016, pp. 57–69). One objection is that duration is only one of several relevant acoustic correlates of prominence, and a more holistic assessment of rhythm should consider additional acoustic measures (see Section 30.3). Another point of contention focuses on a lack of explanatory power, arguing that rhythm metrics quantify rhythm as a surface phenomenon and are influenced by several phonological parameters simultaneously (Turk and Shattuck-Hufnagel, Reference Turk and Shattuck-Hufnagel2013, p. 109).

Rhythm metrics have also been criticised as empirically inadequate by Arvaniti (Reference Arvaniti2009, Reference Arvaniti2012) because they are influenced by elicitation method and syllable complexity. Arvaniti used sets of sentences that were explicitly designed to elicit a more stress- or more syllable-timed rhythm and showed that this result applies across several languages. However, this approach can in turn be criticised for its circularity. Speech material with specific properties was selected, and the analysis in turn revealed exactly these properties. Moreover, speech rhythm metrics were in fact explicitly designed so as to capture differences in syllable complexity. Finally, Arvaniti’s (Reference Arvaniti2012) results generally go in the expected direction, for example with English and German showing higher nPVI-V values, and lower %V values, than Spanish. In fact, across the three metrics nPVI-V, VarcoV, and %V (shown in previous research to be more reliable than other durational metrics; White and Mattys, Reference White and Mattys2007; Wiget et al., Reference Wiget, White and Schuppler2010) and across five different conditions, the difference between English and Spanish is of a non-negligible size and goes in the expected direction in 12 out of 15 cases. Finally, further empirical evidence supporting duration-based metrics comes from a meta-analysis indicating that language discrimination by infants can be explained by durational rhythm metrics (Gasparini et al., Reference Gasparini, Langus, Tsuji and Boll-Avetisyan2021).

30.3 Acoustic Metrics

While duration-based metrics are widely used in research on speech rhythm, what they actually measure can only partially account for Kohler’s (Reference Kohler2009, p. 41) ‘waxing and waning prominence profiles’ (referred to in Section 30.1). These metrics merely measure variability in duration and neglect any other acoustic correlates of prosodic prominence, the most important of which are intensity and fundamental frequency/f0, alongside their psychoacoustic counterparts loudness and pitch, respectively.

Moreover, range of other proposals do not (directly) rely on a particular acoustic correlate of prosodic prominence but are based on complex transformations of spectral energy or other acoustic information in the speech signal (e.g., Galves et al., Reference Galves, Garcia, Duarte and Galves2002; Tilson and Johnson, Reference Tilsen and Johnson2008; Goswami and Leong, Reference Goswami and Leong2013; Tilsen and Arvaniti, Reference Tilsen and Arvaniti2013; Ravignani and Norton, Reference Ravignani and Norton2017; Gibbon and Li, Reference Gibbon and Li2019; Davis and Jeesun, Reference Davis, Jeesun and Fuchs2023). Some of these approaches to measuring rhythm are difficult to reproduce for other researchers because the code is not publicly available. In the following, the discussion focuses on acoustic rhythm metrics that apply the Varco or PVI concept to acoustic correlates of rhythm beyond duration. The advantage of these approaches is arguably that they present a way of accounting for speech rhythm as a multidimensional acoustic phenomenon (Fuchs, Reference Fuchs2016), realised on multiple acoustic ‘channels’, while at the same time the Varco and PVI indices are mathematically relatively straightforward, and therefore easily interpretable, indices of variability (see Table 30.1 for an overview).

These metrics can be classified according to:

1. (1) the variability index they use (Varco or PVI)

2. (2) the phonological unit from which acoustic information is extracted (vowels or syllables)

3. (3) the acoustic correlate of prominence (f0, intensity, loudness, as well as combinations thereof with duration).

Moreover, variability in the acoustic correlates of prominence can be quantified in different ways:

1. (4a) For intensity and loudness, there are proposals to quantify variability either in their average, or in their peak value, in the syllable or vocalic interval.

Thus, VarcoS(avgInt) captures variability in the average intensity in syllables. Instead of calculating the standard deviation of syllable duration divided by the mean, multiplied by 100, which is the way the regular VarcoS for syllable duration is calculated (see Section 30.2), for VarcoS(avgInt), average intensity over each syllable is entered into this equation. Thus, VarcoS(avgInt) is calculated as the standard deviation of average intensity for each syllable, divided by the mean, multiplied by 100. Following the same approach, an nPVI-S(avgInt) can be calculated by entering average intensity for each syllable into the nPVI equation. Furthermore, instead of average intensity over a syllable or vocalic interval, peak intensity can be used as a measure, for example, in the vocalic PVI for peak intensity nPVI-V(peakInt). Finally, all these metrics can also be applied to loudness as the psychoacoustic correlate of intensity, for example, the vocalic PVI for average loudness nPVI-V(avgLoud)) and for peak loudness (nPVI-V(peakLoud)).

Based on these proposals, researchers can trace rhythmic variability independently for distinct acoustic correlates of prominence. For example, the regular nPVI-V based on durations (nPVI-V(dur) can be used to quantify the degree of variability in durations between vocalic intervals, while nPVI-V(loud) can be used in parallel to quantify the degree of variability in loudness between vocalic intervals.

1. (4b) A further refinement of these measures consists in possible combinations of variability in duration with either variability in intensity or loudness. More specifically, it is conceivable that variability in duration and intensity/loudness are tightly correlated with each other; that is, the longer a vowel is, the louder it is at the same time. In other cases, variability in duration and intensity/loudness might not be tightly linked; that is, there might be particularly long vowels that are not necessarily also louder than vowels of average duration.

There are two proposals to account for the potential linkage between variability in duration and intensity/loudness: applying the nPVI formula to both duration and intensity/loudness simultaneously (e.g., nPVI-V(dur+avgLoud), nPVI-V(dur+peakLoud)) or calculating the so-called Amplitude Integral or Loudness Integral (e.g., nPVI-V(AI), nPVI-V(LI)).

Finally, for f0, the existing proposals are not analogous to the rhythm metrics capturing variability in intensity and loudness discussed so far. Instead, the nPVI-V(f0) proposed by Cumming (Reference Cumming2010, Reference Cumming2011) calculated the mean of the differences between the pitch excursion of successive vocalic intervals. Another proposal, by Fuchs (Reference Fuchs2014a), presents an nPVI for vocalic durations that takes into account that differences in f0 also influence perceived duration. In fact, given two vowels of the same duration but different f0, the vowel with higher f0 will be perceived as longer. The proposed nPVI-V(dur*f0) takes this effect into account and thus offers a psychologically more realistic measure of rhythmic variability in perceived duration.

The rhythm metrics proposed for acoustic correlates of prominence do not cover all possible permutations of the various means of computing variability (Varco and nPVI), acoustic correlate (intensity, loudness, f0), and indices of joint variability in duration and another acoustic correlate. Other combinations, such as a joint index of variability in loudness, f0, and duration, might be useful as well.

The rhythm metrics discussed in this section have the advantage of accounting for multiple acoustic correlates of prosodic prominence, thus presenting a more holistic analysis of speech rhythm as a multidimensional phenomenon. For example, in a study comparing read speech in Singapore English and British English, Low (Reference Low1998, pp. 49, 53) was able to confirm the assumption that Singapore English has a more syllable-timed rhythm than British English. In terms of the variability of vocalic durations (nPVI-V(dur)), the analysis indicated that British English had about 1.63 times the durational variability of Singapore English. By contrast, the variability in the Amplitude Integral as a joint measure of amplitude and duration indicated only 1.15 times the variability.Footnote ⁵

Furthermore, in another study on read speech from Indian English and British English, several rhythm metrics confirmed the assumption that Indian English has a relatively more syllable-timed rhythm than British English (Fuchs, Reference Fuchs2016, pp. 114, 147). However, the magnitude of the differences between the two dialects varied depending on which acoustic correlate of prominence was investigated. Specifically, the PVI for vocalic durations (nPVI-V(dur)) indicated that British English has 1.10 times the rhythmic variability of Indian English in terms of the variability of vocalic durations. By contrast, the difference in rhythmic variability for average loudness (nPVI-V(avgLoud)) was greater – 1.23 times. Finally, the combined index accounting for simultaneous variability in duration and average loudness (nPVI-V(dur+avgLoud)) showed that the rhythmic differences between British and Indian English is in fact even greater, with the former having on average 1.45 times the rhythmic variability of the latter, indicating that duration and loudness may have a compounding effect on perceived rhythmic variability in British English, compared to Indian English.

Finally, turning to the role of f0 in rhythmic variation, an analysis of read speech by Fuchs (Reference Fuchs2014a) found that, in British English, the effect of f0 on perceived duration has a noticeable impact on the degree of rhythmic variability, with a significantly higher nPVI-V(dur*f0) compared to the nPVI-V(dur) that is based on duration only. Importantly, this effect appears to be dialect-specific and was not present for Indian English. In effect, using the nPVI-V(dur*f0) allowed this study to show how the effect of f0 on perceived duration further enhances the differences between relatively more stress-timed British English and relatively more syllable-timed Indian English.

In addition to these metrics, Tilsen and Arvaniti (Reference Tilsen and Arvaniti2013) proposed several rhythm metrics based on ‘empirical mode decomposition of the speech amplitude envelope’, a computationally intensive method from which the authors derive syllabic and supra-syllabic measures. Their approach relies on empirical mode decomposition (Huang et al., Reference Huang, Shen and Long1998) yielding intrinsic mode functions (IMFs), where IMF₁ represents the fastest oscillation in the spectral envelope, IMF₂ the next fastest, and so on. From these, a total of seven power distribution metrics, rate metrics, and rhythm stability metrics are derived.

These studies clearly indicate that a reduction of speech rhythm to the measurement of variability in duration limits the analysis of speech rhythm to just a single acoustic correlate of prominence and may also underestimate the true degree of rhythmic variation between languages and dialects. For a more comprehensive analysis of speech rhythm as a prosodic phenomenon, variability in multiple acoustic correlates of prominence should be taken into account.

30.4 Conclusion: Applying Acoustic Rhythm Metrics

This chapter started out with a reference to Kohler’s (Reference Kohler2009) definition of speech rhythm in terms of ‘waxing and waning prominence profiles across syllable chains over time’. It then presented several duration-based rhythm metrics. While some of these metrics have been widely used to account for variation in speech rhythm between languages, dialects, as well as sociolinguistic variation, they arguably neglect acoustic correlates of prominence other than duration. The discussion then turned to another class of rhythm metrics, which aim to account for rhythmic variation by quantifying variability in other acoustic correlates of prominence, that is, intensity, loudness, and f0, as well as the interaction between these and their interaction with duration. Several examples from the literature illustrate how these acoustic rhythm metrics can be fruitfully applied in order to provide a more holistic analysis of speech rhythm and to capture its multidimensional nature.

In addition to the theoretical desideratum of a more holistic analysis of speech rhythm, studies in this area should also try to adhere to a set of guidelines in order to provide valid and reproducible results. Such studies should try to compare like with like, that is, carefully select speakers and speech material that are constrained and clearly described. This requirement also includes that variation in speech style needs to be accounted for, for instance by using either read or spontaneous speech or by including both, but including speech style as a variable in the statistical analysis. The annotation and segmentation of the speech data requires clear guidelines in order to enhance comparability between and the reproducibility of studies. Further information on these guidelines as well as a guide towards the statistical analysis of speech rhythm data can be found in Fuchs (Reference Fuchs, Wilson and Westphal2023c).

A final point to consider concerns the computation of rhythm metrics. In order to simplify their application in empirical research and to enhance reproducibility, this chapter is accompanied by a Praat script that computes a large number of the duration-based and acoustic metrics presented in this chapter (available online at https://osf.io/79qyg/).