27.1 Introduction

Spoken language unfolds in time, just as music. Consequently, both of these two major communicative sound systems encode structures in temporal relations. However, this ontologically anchored possibility of encoding structural relations is only addressed peripherally and unsystematically in linguistic theory formation. The bulk of work in structural phonology is centered around the abstract systemic values of the spectral qualities (represented as discrete phonemes) and fundamental frequencies (represented as discrete tonal categories) of acoustic events. As we shall see, time appears only occasionally as a distinctive feature of phonemes in structuralist phonology, as an abstract feature of moraic structure in metrical phonology (Liberman and Prince, Reference Liberman and Prince1977; Hayes, Reference Hayes1995), as a feature of pitch accents in intonational phonology, or as a delimitative property of phrase edges in prosodic phonology. Our contribution intends to sketch possible ways to conceive of time as a theoretical primitive of phonological structure, informed not only by linguistics but also by music-theoretical research, where time and the relation between meter and rhythm are fundamental but different concepts for any structural representation of music.

We understand music and language as communicative sound systems that rely on cognitive faculties shared by all human beings (Patel, Reference Patel2008; Rebuschat et al., Reference Rebuschat, Rohrmeier, Hawkins and Cross2012; Arbib, Reference Arbib2013). In a first approximation, it appears fairly safe to say that language organizes sound events primarily to express meaning, while the organizing principles of music are directed primarily to the external form of sound events.Footnote ¹ At a closer look, however, such a clear-cut differentiation is problematic, since music is meaningful in many ways that are observable also in language (Sloboda, Reference Sloboda1998; Koelsch, Reference Koelsch2011; Reich, Reference Reich2011; Schlenker, Reference Schlenker2022). In language, on the other hand, the external form of expression (Saussure’s signifiant) also follows principles that do not contribute to the construction of propositional meaning but to the optimization of itself. In establishing a single and integrative form of representation, we hope to expose more clearly which principles of construction may be claimed to rely on shared cognitive resources and which are particular to language or music.

A caveat is necessary right from the start. Time is a far less stable object to study than our omnipresent watches want to make us believe. The Greek rhetorical tradition knew the difference between chronos and kairos, the latter being conceived as the subjectively perceived time with relation to a meaningful moment. In a similar conception, Husserl (1928) rejects the objective, “vulgar” time and investigates the inneres Zeitbewußtsein (‘consciousness of internal time’). The difference between chronological time and kairotic time may be illustrated with John Cage’s famous piece 4’33 (composed in 1952), in which a pianist or any other musician or group of musicians enter the stage of a concert hall and play during four minutes and 33 seconds nothing. These 4’33 minutes of silence may be perceived infinitely longer than the chronologically same 4’33 minutes in a concert with real music. We believe that the cognitive representation of time in both language and music must be at least partly kairotic, and that is one of the reasons why we face many methodological problems, because kairotic time escapes from approximation with objective instruments (see White and Malisz, Reference White, Malisz, Gussenhoven and Chen2020, for a similar view on time in language).

To understand structural parallels and divergences between linguistic and musical timing, we will have to consider forms and their functions at different levels of organization. In language, this may relate to the construction of morphemes by phonemic features, the metrical construction of feet, or the delimitation of words and phrases. These forms are related to systemic functions such as distinctivity, delimitativity, culminativity – and rhythmicity, to add a fourth to Trubetzkoy’s (Reference Trubetzkoy1939) three core functions in phonology. Many scholars hesitate to accept rhythmicity as part of the linguistic system proper, while others readily posit a “rhythm rule” to project alternating strength to subsequent syllables (Hayes, Reference Hayes1995; Kager, Reference Kager1999). We believe that we need a principled representation of timing in phonological theory, since linguistic rhythm draws core distinctions between languages (see Chapters 32 and 34). In our view, however, not only in music but also in language, rhythm is fundamentally different from metrical projection. Rather, rhythm characterizes how events are placed in time in terms of their onsets and durations, and, in terms of performance, the way speakers or musicians behave towards an abstract metrical grid in time. As rhythm places events onto positions provided by the metrical grid, an acoustic linguistic or musical event may be placed at prominent or less prominent positions at different levels of the metrical grid. In subsymbolic time structure, events may further occur just before, right at, or a little after a prominent position, enabling further ways of subtle expressive differences. In music, creative play with establishing and breaking the metrical grid leads to interesting expressive and perceptual phenomena, for instance, groove (Witek et al., Reference Witek, Clarke, Wallentin, Kringelbach and Vuust2014), while in language, it may serve to encode subtle pragmatic modifications of meaning or just simply correspond to prosodic routines of a given speech community.

Accordingly, this chapter is concerned with prominence and timing and their functions in language and music. First, we review the core characteristics of musical rhythm; second, we turn to review a range of linguistic configurations of time and prominence; third, we propose a new autosegmental-metrical account across music and language; and fourth, we conclude with a discussion of cognitive implications and the identification of some research topics that we hope to clarify in our theoretical perspective.

27.2 Beats and Timing in Music

Time is a central category in music since music unfolds in its temporal order to combine various pitch and other sonic events. The foundation of temporal organization in music is based on an underlying beat structure, which defines the tempo of a musical piece and constitutes a quasi-universal across musical cultures (Patel, Reference Patel2008).Footnote ² The beat defines two interrelated time structures: a symbolic time structure and a subsymbolic time structure (see Figure 27.1, which only focuses on the beat level in both time structures; hierarchical metrical structure is explained in Figure 27.2). The following text describes symbolic time structure first; the subsymbolic structure is characterized in detail at the end of the chapter.

Figure 27.1

The relationship between symbolic time structure (beat time) and subsymbolic time structure (real time).

The relationship between both time structures is characterized by their mapping of the reference beat. In other words, the idealized isochronic beat in beat time is distorted in subsymbolic real time.

Figure 27.2

Two examples of the metrical grid.

The left, two bars from the old folk song “Scarborough Fair,” displays an instance of an isochronic meter (here, 6/8); the right, three bars from the song “Seven Days” by Sting, shows an instance of a non-isochronic meter (5/4); note that the beat level combines 3/8 + 3/8 + 2/8 + 2/8.

At the symbolic level, the beat defines an isochronic unit, and, subsequently, a metrical structure is defined as a grid of prominence patterns based on the isochronic beat unit (Lerdahl and Jackendoff, Reference Lerdahl and Jackendoff1983; London, Reference London2004). The following is a summary characterization of common notions of musical meter, largely following the previous two references (for more information, see Giger, Reference Giger1993, and Patel, Reference Patel2008). In most forms of Western music in the past and present, metrical structures prefer the subdivision of the grid in terms of perfectly regular groups of two or three beats, which continue throughout the entire piece (unless there is a comparably rare case of a change of meter). It is further possible to employ non-isochronic metrical structures (such as 5/8, 7/4, 11/8, 21/8) in which metrical units cannot be fully derived by the prime factors 2 or 3. In such cases, metrical structures are derived as additives from units of 2 or 3 (London, Reference London2004): for instance, 5/8 could be derived as (2+3)/8 or (3+2)/8 or 7/4 as (2+2+3)/4, (2+3+2)/4, (3+2+2)/4. In such cases, the regular 2 or 3 subdivision continues at the other levels below the bar or at the hypermetrical level (beyond the single bar). Finally, given the possibility of non-isochronous meters, regular units could also be subdivided in a non-isochronous way, for example, 9/8 = (2+2+2+3)/8 or 4/4 = (3+3+2)/8. Generally, such non-isochronous metrical structures occur commonly in non-Western music, such as Turkish, Balkan, Arab, or Indian musical traditions. Figure 27.2 displays examples comparing isochronous (6/8) and non-isochronous (5/4) meters.

Furthermore, rhythmic event structures in music are defined in reference to the metrical grid: Rhythmic events may derive from simple integer ratios from the beat unit (e.g., 2, 3, 3/2, 1/2, 1/3, 1/4, 3/4, etc.) and are placed at regular positions on the grid. Since the grid could be potentially unboundedly subdivided by regular subdivision, events never land between grid points at the symbolic level, and simple grid positions are always preferred. Moreover, rhythmic structure of musical events may not occur entirely freely but has been argued to be recursively hierarchically structured (Longuet-Higgins, Reference Longuet-Higgins1979; Rohrmeier, Reference Rohrmeier2020): This is derived from understanding rhythmic structure not in terms of a sequence of event onsets and durations but in terms of time-span subdivisions, insertions, and shifts. For instance, the time span of a half note may be subdivided into two quarter notes, inducing one additional event onset in the middle of the time span. Also, an event at a strong metrical position may be prepared by the insertion of one or more upbeat events at the preceding weak metrical position, leading up to and strengthening the prepared event (this may entail a shortening or elision of the previously preceding event). In the case of a syncopation, an event on a strong metrical position may be shifted to an adjacent weak position (usually by half its duration). Because of its hierarchical substructure, musical rhythm and grouping structure may be conceived of as one converging property (extending Lerdahl and Jackendoff, Reference Lerdahl and Jackendoff1983; see Rohrmeier, Reference Rohrmeier2020).

Figure 27.3 displays an example of metrical and rhythmic structure in music (reproduced from Rohrmeier, Reference Rohrmeier2020; see also Reich and Rohrmeier, Reference Reich, Rohrmeier, Reina and Szczepaniak2014). The staff lines display the melodic line and its rhythm. The grid below the staff line displays the metrical grid, following the convention by Lerdahl and Jackendoff (Reference Lerdahl and Jackendoff1983). In the case of this example, the beat level is the line second from top (quarter-note level, below eighth-note level). Notably, there are points in the grid without note onsets (such as bar 2, beat 2, bar 4, beats 2,3,4 in the upper example). Most rhythmic events in the melody reinforce the grid at the beat or stronger (i.e., graphically lower) levels. This is also reconfirmed by statistical note onset histograms (see Huron, Reference Huron2006). Note that the example also indicates two hypermetrical levels beyond the bar unit. Hypermeter characterizes relations with a longer time span such as strong and weak bars or groups of bars; in the example, for instance, the first events of bars 1 and 3 (and 5) receive even stronger prominence, with bar 1 (and 5) being even stronger. The diagram illustrates the difference between a 3/4 and 4/4 meter. Remarkably, the division into triplets happens only once in the 3/4 grid while all other divisions are duple. Finally, the diagram illustrates that the overarching rhythmic Gestalt of the entire phrase is different in the 3/4 and 4/4 case despite the melodic structure and, particularly, its temporal inter-onset intervals being fully identical. This implies that the entire Gestalt depends on the interrelation of rhythmic structure and the meter. In terms of musical performance, this implies that musicians who would convey one over the other interpretation to the listener would employ various forms of fine-tuned performance parameters to reinforce the sense of meter and metrical prominence: This may include note stress in terms of micro-timing, timbre, or loudness, or highlighting grouping in terms of exaggerated transitions or rests in micro-timing. Further note that metrical ambiguity as in Figure 27.3 is empirically rare in music and hard to construct artificially. In most cases, melodies or musical pieces clearly establish their metrical structure in terms of their rhythm, that is, mainly through note onsets and durations (see also Huron, Reference Huron2006, for further explanation).

Figure 27.3

The same set of onsets and durations placed on two different metrical grids.

The resulting rhythmic Gestalt is different for the two cases.

In terms of music perception and production, one of the main theories of metrical perception understands meter as instantiated in terms of coupled oscillators in the brain, which direct attentional focus to predictive time points in the future (Large and Palmer, Reference Large and Palmer2002; Merchant et al., Reference Merchant, Grahn, Trainor, Rohrmeier and Fitch2015). In this way, the metrical grid is directly linked to models of temporal musical expectancy (see also Huron, Reference Huron2006).

At the subsymbolic level, there are different ways in which the metrical grid may be distorted (and with it the rhythmic structures on top without distorting their simple duration ratios): (i) There may be overarching tempo variations, such as the music speeding up or slowing down (accelerando, ritardando); (ii) there may be timing variations within phrases to accentuate certain parts (rubato, or phrase-final lengthening); (iii) the grid may be systematically distorted, for instance, in the swing ratio in jazz; (iv) events may occur with a subtle anticipation or delay from their expected onset position – this may contribute to aesthetic effects such as event stress or groove; (v) events may occur at imprecise positions compared to the idealized grid because of precision limits of human instrumental or singing performance (even computationally generated music may include such effects as “humanizing”). While tempo variations may affect a musical piece at large, the other effects constitute deviations that are commonly referred to as micro-timing. Micro-timing is usually much smaller than the reference beat level. Micro-timing properties may be highly distinctive of a musical style; for instance, consider the timing properties of a Baroque church organ prelude, which commonly establishes a relatively strict stable beat with rich micro-timing variations to compensate missing dynamic (loudness) variability options, compared with a piano phrase by Chopin, which typically involves free stretching and distortion of the beat in real time (rubato) in order to achieve expressive richness, a bebop solo phrase, which establishes a strict sense of beat, yet a high degree of meter-violating (= obscuring) note onsets (called syncopations), which contribute to establishing groove, and a Cuban salsa phrase, which similarly establishes a systematic conflict between rhythm (note onsets) and metrical structure in terms of high levels of syncopation, which establish its danceability and groove. All of the aspects of subsymbolic time structure in music have in common that they constitute continuous distortions of the grid or the precise timing of events on the grid, while the symbolic relations of rhythmic events and their grid positions remain intact.

27.3 Beats and Timing in Language

While musical structures show rich, playful variations of the relation between symbolic beats in abstract time and sonic events in real time, much of what is perceived as rhythmic in language follows a different teleology: The construction of phonological word forms and their syntactic combination to complex utterances is bound to convey semantics, understood in linguistics as propositions and their modal and pragmatic embedding. However, the ways in which these meanings are encoded by temporal relations and some of the rhythmic preferences of languages, dialects, communities, styles, or individual speakers show clear parallelisms to music that allow for the assumption of shared cognitive resources of these two communicative sound systems.

A good orientation is offered by Dufter (Reference Dufter2003). Building on standard approaches to linguistic rhythm typology (Dauer, Reference Dauer1983; Auer, Reference Auer1993) and metrical phonology (Liberman and Prince, Reference Liberman and Prince1977; Hayes, Reference Hayes1995; see also Chapter 34 for a review of linguistic rhythm typology with illustrations from Romance languages, and Chapter 11 for a critical assessment of rhythmic classes), he assembles a quaternary typology of linguistic rhythm (1) on the grounds of the systematic pairing of prominence and timing to distinctivity:

1. (1) Dufter’s (Reference Dufter2003, 132) four types of linguistic rhythm:

   1. 1. distinctive duration in the lexical phonology – mora-based rhythm

   2. 2. distinctive duration in the sentence phonology – phrase-based rhythm

   3. 3. distinctive prominence (in words and/or sentences) – prominence-based rhythm

   4. 4. no contrasts in the rhythmic contour – alternating rhythm

All four typological possibilities are well documented in the literature and are exemplified below. In the conception of Dufter’s approach, linguistic rhythm is rhythmic mostly in the perception of acoustic events that are shaped by the requirements of distinctivity at different levels of the phonological structure in order to construct meaning. In this conception, only the fourth type corresponds to rhythmicity as a goal in production: If the phonological configuration of a language grammaticalizes neither timing nor prominence for distinctive or delimitative functions, these two major domains of prosody are available for processes that are directed to the rhythmic shape of an utterance itself.

While we readily agree with the major line of this conception, we believe that it requires further elaboration. Firstly, just like it is the case for other linguistic typologies, it is necessary to understand that types do not map directly to particular languages. Rather, languages may feature more than one of the possibilities above and show, for example, both distinctive duration and distinctive prominence (as in German; see Examples (2) and (4) below), alternating prominence besides a system with delimitative duration in sentence phonology, or distinctive segmental length and final lengthening (see Paschen et al., Reference Paschen, Fuchs and Seifart2022), and so on.

Furthermore, not only particular languages but also styles and dialects may show preferences for a certain cluster of form-function pairs. Thus, speakers may superimpose alternating prominence over distinctive prominence in poetic styles and songs, and different dialects of a given language may choose different rhythmic patterns or different degrees of their realizations (see Chapter 34 for rhythmic differences between Spanish and Portuguese that cannot be attributed to different systemic functions of time and prominence).

27.3.3 Moraic Structure: Time as a Property of Syllables

Timing as a property of phonological words is not restricted to distinctive functions. In many languages, timing is also an important feature for the assignment of prominence, and for processes such as compensatory lengthening, reduplication, and truncation. The subsyllabic unit mora (symbolized as μ) accounts both for syllabic complexity and for length in phonological representations, often grouped under the metaphoric label weight. Ladefoged and Johnson (Reference Ladefoged and Johnson2011, 251) take the mora as a unit of timing. In standard approaches to metrical phonology (Hayes, Reference Hayes1989, Reference Hayes1995; Hubbard, Reference Hubbard, Connell and Arvaniti1995), moras dominate segments, and are in turn dominated by syllables (Figure 27.4).

Figure 27.4

Association of segmental timing properties to syllables in moraic structure.

In this representation, length is treated just as a segment in the coda of the syllable: The long vowel [aː] is associated to two moras in (b), while in (c) it is the consonant in the coda that is associated with the second coda. As any other item in symbolic prosodic phonology, moraic structure must be motivated by the observation of processes and rules that take them as domains for their application. Thus, the status of the mora must be specified for each particular language or dialect separately (Hayes, Reference Hayes1989, Reference Hayes1995; Féry, Reference Féry, Féry and Vijver2003; among many others).

27.3.4 Timing of Pitch Accents

Pitch accents are phonologically specified excursions from the phonetic downtrend of fundamental frequency (F0) contours that construct the shape of an intonational tune together with events at the boundaries of these tunes. They are associated with lexical stresses (if the language has lexical stress) in a systematic, meaningful way that specifies the temporal distance of a peak or valley with respect to the stressed syllable. These temporal differences change the pragmatic interpretation according to the specifications of a particular grammar. For example, in German, following Niebuhr (Reference Niebuhr2007) (see also Kohler, Reference Kohler2005), the peaks of tonal events may occur early (the low tone L following the peak H is associated with the stressed syllable: HL* in ToBIFootnote ⁴), medial (the high tone is associated with the stressed syllable: LH*), or late (the low tone is associated with the stressed syllable and the peak is reached after this syllable: L*H) with respect to the stressed syllable, and convey pragmatic meanings such as given, new, and unexpected.Footnote ⁵ In Figure 27.5, we reproduce the three possible tunes of the example eine Malerin from Niebuhr (Reference Niebuhr2007, 176) in a schematic way. The dotted lines delimit the stressed syllable (ma) and the solid line represents the F0 that we perceive as intonation.

Figure 27.5

Early (A), medial (B), and late (C) peaks in German eine Malerin (a painter).

It is the temporal relation between an event and abstract knowledge about a prominent position that specifies the pitch accent in German. Many other, if not most, languages show such temporally specified pitch accents (see Moraes, Reference Moraes2008, for Brazilian Portuguese, and Estebas Vilaplana and Prieto, Reference Estebas Vilaplana, Prieto, Prieto and Roseano2010, for Castilian Spanish, among many others).

27.4 Representation of Time, Tone, and Prominence in an Autosegmental Model

27.4.1 Language

The temporal structures of linguistic forms outlined in Section 27.3 are comparable to the subtle temporal modulations we saw in Section 27.2, if we accept the necessary distinction between meter and rhythm also for language. The meaningful timing of events is possible because of an abstract metrical grid of prominences that are projected by the principle of alternating strength and at the speed entrained by the speech rate of an ongoing conversation. An event may occur early or late or be long only if there is a representation of when and how long it is expected to occur.

This argument relates to an important difference between the five structural timing specifications we presented in Section 27.3. Only the first two of them, segmental length and moraic structure, may be attributed to the phonological form of words. Let us call this level of timing intrinsic time. Final lengthening, relative timing of pitch accents, and iconic timing, however, cannot be derived from the lexicon. They must be associated to a level of time that is extrinsic to words. We believe that it is the very same level of time that forms the horizon for rhythmic patterns and subsymbolic timing in music: beats in time. It is the expectation created by the perceived periodic occurrence of sound events that models the metrical grid as a canvas for rhythmic events to be expected to occur. In music, metrical structure is induced by few note events or percussive events (or, externally, by counting in), while in language, what creates the temporal expectation of future events are the peaks of sonority reached in the nuclei of syllables.Footnote ⁷ Just as in music, speakers infer the speech rate that is expected in a given conversation by entrainment (see Jungers and Hupp, Reference Jungers and Hupp2009, for experimental evidence).

How should we model extrinsic time in phonology? Autosegmental phonology (Goldsmith, Reference Goldsmith1976; Yip, Reference Yip2002; Leben, Reference Leben and Aronoff2013) is a framework that was first developed to account for the independence of tones in tone languages, in which tones are associated to one or many tone-bearing units (TBUs) (moras, syllables, vowels, depending on the language) in a segmental string. In order to be realized as an acoustic event, a tone (T) must be associated with a TBU. If we take syllables (σ) as TBUs for illustration, different phonologies arise out of different association principles. Consider Figure 27.6: In (a) every tone is associated one to one with a particular TBU, in (b) many tones are associated with one TBU, in (c) one tone is associated with many syllables, and in (d) one tone is left without association as a so-called floating tone.

Figure 27.6

Possibilities for the association of tones to syllables.

Thus, the association lines define the tonal grammar by specifying relations across different domains: Tones are seen as an independent layer of structure that is associated to the segmental string.Footnote ⁸

Autosegmental-metrical theory (AM) (Pierrehumbert, Reference Pierrehumbert1980; Ladd, Reference Ladd2008; Arvaniti, Reference Arvaniti, Barnes and Shattuck-Hufnagel2022; Grice, Reference Grice, Barnes and Shattuck-Hufnagel2022) takes the general principles further to intonation and metrical prominence. Pitch accents are associated with stressed syllables and boundary tones are associated with edges of (different instantiations of) phonological phrases. In this line of research, the representation of a segmental, a metrical (often reduced to lexical stress), and a tonal layer have proved to be a successful and elegant model for many linguistic phenomena (e.g., the shapes of pitch accents discussed in Section 27.3.3). It seems to us to be ideal for the modelling of complex relations between different structural domains, and we want to take it further by adding an additional independent layer for extrinsic, kairotic timing relations: beats in time.

Time appears in the analytic practice of AM:Footnote ⁹

1. (i) as a distinctive feature at the segmental level

2. (ii) as a correlate of phonological weight in the moraic structure

3. (iii) in the meaningful association of complex tones to lexical stress.

The third level is not well specified. Most scholars improvise an additional impressionistic diacritic (such as “>” for “late”) that may serve for preliminary differentiation of tonal categories, but it lacks precisely the representation of the extrinsic temporal anchors for the association of tones that we are trying to establish here. There must be a temporal layer that is not a property of word forms but serves as the horizon unto which prosodic events are projected in performance. We suggest to introduce an additional tier that represents these beats in extrinsic time.

Linguistic structure is bound to construct meaning in the first place, and the design of the phonological form itself is a subordinated, secondary function of linguistic phonology. These two different functional realms, however, work on the same substance: tones, time, and prominence. This is also the substance of temporal metrical organization of music, but genuine musical structure has nothing comparable to words. The level of comparison between music and language is to be found in the abstract temporal relations in the metrical grid, rather than in problematic direct analogies (e.g., words = tones, syllables = tones, etc.). In most musical traditions, the melodic, harmonic, and rhythmic structures are primarily organized to trigger aesthetic effects of different kinds, but less for the construction and modification of propositional and pragmatic meanings (Reich, Reference Reich2011). We argue that it is possible to represent both music and language in a unified representation along the main lines of AM, with the difference that in music (i) there are no morphemes and (ii) the inventory of tones is bigger and needs more information than just high (H) and low (L).Footnote ¹⁰

Just for the sake of illustration, we recorded the German utterance Sie fanden eine Lagune in der Wüste (‘They found a lagoon in the desert’) with a reading of unexpectedness and analyzed it in Praat (Boersma and Wennink, Reference Boersma and Weenink2023) to be able to show late peaks (see Figure 27.7). As Kohler (Reference Kohler2005) and Niebuhr (Reference Niebuhr2007) already demonstrated, the peak of the rising tone in Lagune occurs after the stressed syllable. Obviously, there is also a high boundary tone (H-) in the last syllable of the phrase Sie fanden eine Lagune, but the rise that begins with the associated low tone in the stressed syllable clearly reaches its peak also only in this syllable, thus forming a plateau. The following pitch accent is realized with a medial peak: The turning point of F0 occurs within the stressed syllable.

Figure 27.7

Intensity and F0 of an utterance with late peak in German.

Figure 27.7 Long description

The spectrogram shows the frequency in hertz and intensity in decibels of the sound over time. It depicts a solid line for the fundamental frequency, F 0 of the speech and a dotted line for the intensity of the sound. The text below the spectrogram shows the words of the phrase and their phonetic transcription. The row below the text shows the tone of the phrase. The symbols indicating the tones are as follows. L asterisk greater than H, H hyphen, L asterisk H and L.

A full-fledged representation of prosodic domains in autosegmental layers for this example is shown in Figure 27.8.

Figure 27.8

Inventory of prosodic domains association to beats for German eine Lagune in der Wüste.

Figure 27.8 Long description

Syllables are denoted by inverted alphabets. Moras are denoted by the symbol mu. Beats are denoted by x, while feets are denoted by either x or dot. Some segments consist of tones, p boundaries and lexical stress. Tones are denoted by L and H. P boundaries are denoted by the percent symbol. Lexical stress is denoted by the asterisk mark.

The phonological rules that construct the German utterance eine Lagune in der Wüste with the meaning of unexpectedness are the following, starting from the bottom line:

1. (i) The beats in time are entrained by the more or less periodic recurrence of peaks of sonority, the nuclei of syllables in the speech rate of a given conversation. The speech rate experienced in the conversation gives rise to their projection. These beats are not derived from the phonological word forms and are consequently independent from linguistic substance. Thus, they may be associated with iconic lengthening, compensatory lengthening (Hayes, Reference Hayes1989), or processes of catalexis (Jacobs, Reference Jacobs and Mazzola1994).

2. (ii) Feet are constructed by the assignment of alternating strength to the beats. This is the level of metrical grid construction that is in principle identical to the musical metrical grid, with the difference that it is restricted by the association to prosodic domains such as words or phrases.Footnote ¹¹ Moras, then, are temporal representations of the segmental material in syllables and work as hinges between intrinsic and extrinsic time.

3. (iii) Stress is a feature of phonological word forms, projected by a morphological rule or derived by metrical algorithms, depending on the particular language (see van der Hulst, Reference van der Hulst1997, Reference van der Hulst2012; Hyman, Reference Hyman and van der Hulst2014). In German, stress is lexically or morphologically distinctive and must be kept salient for the processing of content. That is the reason why feet must align with stressed syllables and, consequently, some syllables remain metrically unparsed (as the first and the last syllable in Lagune).

4. (iv) The chains of segments construct the morphemes that encode lexical meanings (including their syntactic features).

5. (v) Pragmatics decides on the assignment of phrase boundaries and the form of pitch accents: The partitioning of utterances in background and focus projects phonological phrases delimited by boundary tones and shapes the form of pitch accents,Footnote ¹² specified by their timing with respect to stressed syllables. In Figure 27.8, the pragmatics of the asserted unexpected event is expressed by the late timing of the high tone of the rising pitch accent: Its low tone is associated with the stressed syllable and its corresponding beat in time, while its high turning point is associated only with the beat in time that corresponds to the following syllable. This is an innovative aspect of our representation: Tones may feature an additional association with extrinsic representations of time, just as musical events.Footnote ¹³ This is where we find a clear parallelism to the expressive relation of real-time events to the beat time of the grid in music: Peaks need not occur where the symbolic beats project them but may surface in real time earlier or later. We model these relations by additional associations to beats in time.

27.4.2 Music

If we represent the musical pieces discussed in Section 27.2 and Figure 27.3 in an autosegmental model that specifies independent layers of structure and their association as in Figure 27.9, we find many comparable aspects to prosodic structures in linguistic utterances, but also many differences. In music, the layers of segments and words are absent since there are no words or morphemes. Tones are less directly organized with respect to semantics and pragmatics, but most directly with respect to other musical systems, such as melodic structure, voice-leading, and also harmony (in the case of Western tonal music); a majority of these systems are assumed to be hierarchical (Schenker, Reference Schenker1956; Lerdahl and Jackendoff, Reference Lerdahl and Jackendoff1983; Mukherji, Reference Mukherji2014; Clarke, Reference Clarke2017; Rohrmeier and Pearce, Reference Rohrmeier, Pearce and Bader2018; Finkensiep et al., Reference Finkensiep, Widdess and Rohrmeier2019). Thus, the musical case requires a much more elaborated inventory of tones and systematic pitch relations than just the relative categories High and Low. Notably, since there is no smallest metrical base level in music as the beat can be subdivided indefinitely, there may be levels below the beat level, as the musical score in analysis (Figure 27.10) indicates with the eighth-note level (binary subdivision of the beat).

Figure 27.9

A representation of the last two bars of the example in Figure 27.3, 4/4 version.

Figure 27.9 Long description

The musical notation consists of a treble clef, a time signature of 4 by 4, quarter notes and rests. There are dotted rhythms below the notation. A section of the notation is marked with a rectangular box and is expanded below to show the graphic representation. Below the notation, the graph consists of letters B, C, D, A and x, and parenthesis that correspond to tones or rests, hierarchy of prominence and beat levels.

Figure 27.10

A representation of the last two bars of the example in Figure 27.3, 3/4 version.

Figure 27.10 Long description

The musical notation consists of a treble clef, a time signature of 3 by 4, quarter notes and rests. There are dotted rhythms below the notation. A section of the notation is marked with a rectangular box and is expanded below to show the graphic representation. Below the notation, the graph consists of letters B, C, D, A and x, and parenthesis that correspond to tones or rests, hierarchy of prominence and beat levels.

The phonological rules that construct the musical utterances in Figures 27.9 and 27.10 are the following:

1. (i) A stable beat is entrained in musical performance (snapping, clapping, counting, playing) and defines a first, base (= reference) level of the metrical grid. By convention, beats in time are typically related to the temporal specifications of quarter notes. As the beat may be further subdivided, smaller units such as eighth notes may also be established in the metrical grid, as in the musical examples above. Since it is not central to our considerations, we skipped this level in the autosegmental representation.

2. (ii) The assignment of alternating strength to the beats in time creates a first level of the metrical grid. In a 4/4 or 2/4 meter (Figure 27.9), binary attribution of strength leads to one or two groups of prominence within each bar, which may be assigned additional prominence at higher levels. In a 3/4 system (Figure 27.10), the same tonal events (with the exception of the first B, which is parsed in the preceding bar) are bracketed in a ternary analysis.

3. (iii) Different to stress systems in language, all metrical groups commonly have their strongest stress come first rather than last, that is, they are left-headed (strong is initial), which is a direct consequence of the indefinite cyclic repetition of the metrical grid. Since musical events are not mapped to morphosyntactic categories such as words or phrases, they are in principle unbounded. The same holds for meter in poetry (Lerdahl and Halle, Reference Lerdahl and Halle1993; Lerdahl, Reference Lerdahl2001). Consequently, the decision of regarding a prominent event as initial or final is arbitrary. However, for the phenomenon of preparatory upbeats, partial grids before the prominent first stress are possible (see Lerdahl and Jackendoff, Reference Lerdahl and Jackendoff1983). This may be modelled with a set of offset values for each level of the metrical grid (see Rohrmeier, Reference Rohrmeier2020).

4. (iv) Notes at the beat level, that is, quarter notes and quarter rests, are associated with one beat in time, 1/2 notes and 1/2 rests are associated with two beats in time, and so on for more complex timing values. These associations specify the length of events, just as the intrinsic length of phonemes is represented as a secondary association of the moraic structure to beats in time. Tones may also land between beat positions, depending on their regular subdivision of the beat level.

5. (v) All other levels may reinforce the prominent positions of lower levels, at a regularity of two or three positions, and with a given offset that is smaller than their regularity. For complex non-isochronous meters, additive rules apply, as outlined above (Section 27.2, paragraph 2).

27.5 Conclusions and Perspectives

The unified form of representation for linguistic and musical timing that we introduced in this chapter made some important shared principles of structure building more transparent, but also showed some of the fundamental differences. In music, there is a fundamental distinction between idealized symbolic time structure established by an ideal isochronic beat and a subsymbolic time structure (real time), which governs distortions of the grid for expressive and stylistic purposes; we assume that the same distinction is equally fundamental in language, although it has not yet been postulated in linguistic research. Both music and language specify their temporal structure with respect to both the intrinsic length of individual events (notes or phonemes) and to extrinsic beats in time. Both music and language construct idealized hierarchical patterns of metrical prominence building on beats in time in symbolic time structure. Both music and language organize the temporal relation of real events with respect to the beats in time with which prominent positions are associated (this is further evident in text setting of songs to music: Lerdahl and Halle, Reference Lerdahl and Halle1993; Dell and Halle, Reference Dell, Halle, Aroui and Arleo2009).Footnote ¹⁴ Rhythm defines the placing of events on the metrical grid in both language and music. In music, subtle temporal specifications at the level of real time lead to expressive and potentially stylistic differentiation, while in many languages, the relative real-time timing of pitch events with respect to stress is grammaticalized to convey pragmatic meanings.

Our work also establishes a conceptual ground to establish notions of rhythm and meter in a joint common language, which have different traditions of use in music-theoretical and linguistic research. The word rhythm may be used to denote temporal phenomena as a whole; we refrain from this use and speak about time or temporal structure in music and language. In music, meter and metrical structure refer to an abstract hierarchical grid of time points that correspond at their psychological implementation with cyclical points of heightened psychological attention (London, Reference London2004), which may be reinforced by note onsets or expressive musical parameters. Following others (e.g., Patel, Reference Patel2008), we argue that, in principle, the same metrical structure is established in language and music. If meter establishes a white canvas of potential event positions, rhythm (in the narrow sense) characterizes the placing of events and their duration on the grid. Since music is not bound to words and their meaning, rhythmic structure in music may be extensively more complex than in language in terms of its phenomenological diversity as well as its potential construction principles (see, for example, Giger, Reference Giger1993; Nierhaus, Reference Nierhaus2009; Toussaint, Reference Toussaint2019).

Major differences between language and music arise from the grammaticalization of timing and prominence for the construction of the phonological form of morphemes, words, and phrases that carry semantic and pragmatic meaning. Thus, many of the cues for the perception of temporal structures and metrical patterns are derived in the lexicon and in the morphosyntactic structure that are absent in music. Consequently, the variation of temporal and metrical configurations is more varied and complex in music than in language.

Besides the principal goal to uncover the cognitive architecture of music and language, the establishment and further elaboration of a unified form of representation of timing and prominence across different domains of human behavior may contribute to address many important research questions. The coordination of gestures with linguistic utterances must specify a temporal anchor that is shared by both systems. Furthermore, the projection of linguistic onto musical forms will have to rely on the possibility of the association of both to a common cognitive representation. The same argument holds in principle also for other activities that are coordinated with musical forms, such as dance or even dynamic light installations. If we really want to understand the cognitive architecture of humans, we will need a common model for the representation of general and particular principles of structure building.

27.6 Acknowledgements

The contribution of Martin Rohrmeier has been in part funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (GA No 760081– PMSB). Martin Rohrmeier thanks Mr. Claude Latour for supporting this research through the Latour Chair in Digital Musicology.