2.1. Embodied interaction

The term ‘embodiment’ is used in music cognition and interaction studies to encompass the interconnectedness of the mind, body, language and individual in their social context and inter-corporeal nature (Rohrer Reference Rohrer and Torres2008). Tim Rohrer argues that our human mind has been poured into this human-shaped vessel, and in doing so, our mind is shaped by the body we inhabit (ibid.). The voice is a profoundly embodied instrument, encompassing language and paralanguage, reflecting the unique conditions of each individual’s existence (Cavarero Reference Cavarero2005).

How we perceive music is greatly influenced by our body’s interactions with musical instruments and the ongoing exchange between our mind, body and surroundings (Leman Reference Leman2008, Reference Leman2012; Leman and Maes Reference Leman and Maes2015). Gesture as a sound facilitating movement is crucial for making music – be it a cello, a knob, or a mouse movement (Jensenius et al. Reference Jensenius, Wanderley and Godøy2010) – the motions are bound to structural abstractions in space, forming spatial and emotional narratives (Sessions Reference Sessions1950; Smalley, Reference Smalley2007). Embodied cognition proposes that a musician’s cognitive processes are closely tied to their interaction with their instrument. This collaboration forms a significant part of the music-making environment, influencing the musician’s internal representations and even leaving a trace on the music itself (Leman Reference Leman2012).

The pillars of embodied interaction are play and agency (Kim and Seifert Reference Kim and Seifert2006). Play is an imaginative exploration of tools, themes, and abilities in interaction with devices and topics. Agency is a user’s perception of control in the engagement and their capacity to act and enact choices in activities. In interactive music practices, embodied interaction further considers the symbiotic relationships of ‘human beings, things, technical artefacts, and symbols’ as ‘equal parties’ in collaboration with each other (Kim & Seifert Reference Kim and Seifert2006). Fischman’s mimetic space reinforces this notion, highlighting how interactive tools and gestures dynamically construct shared sonic environments (Fischman Reference Fischman2008). Embodied experiences, including trauma and gestural performance, often defy verbal expression due to the challenge of articulating implicit and experiential knowledge (Höök Reference Höök2018: 236). Immersion in the activity, in these moments, can be facilitated by a tool – in gestural music, the interfaces are the tool.

2.2. Trauma-informed practice as composition

Trauma is an emotional outcome or reaction following an adverse physical or psychological event, which may be a set of isolated, in-the-moment responses or ones that become trapped in the body, replaying the incident when an individual encounters a ‘trigger’ – whether logically or illogically related to the initial catalyst (Kolk Reference Kolk2014). Trauma-informed therapeutic practices originated from feminist movements in the 1970s and 1980s and were merged into medical care structures in the 1990s (Wilson et al. Reference Wilson, Pence and Conradi2013). These therapies are based on ‘safety, trustworthiness, choice, collaboration, and empowerment’ (Fallot and Harris Reference Fallot and Harris2009: 3). The approach shifts focus from determining what is wrong with the individual or their experiences to the need to understand how it impacts their life. While the outward-facing profile of these therapies differs, the core tenets aim to assess the individual’s traumatic experience(s) and environment in relation to each other. This discussion focuses on body-centred movement practices derived from the Somatic Experiencing^® exercise from Levine’s Trauma and Memory (Levine Reference Levine2015).

Trauma affects the voice and body, complicating the artistic performance and composition of the works, as they could feel too uncomfortably personal. One’s voice is often linked to the psyche, and this connection can be damaged through trauma (Kolk Reference Kolk2014; Levine Reference Levine2015; Porges Reference Porges2009). Considering them through a safety-approved pseudo-clinical method offers a barrier to submerging oneself in the content and sound environment. This approach transforms sound into a metaphor for unnameable terrors, echoing Soddell’s (Reference Soddell2019) sentiments. The relationship between the aftermath of trauma and the use of different physical modalities emphasises the potential for agency and empowerment within a trauma-informed composition approach. Extending trauma-informed soundscapes to new therapeutic settings could reveal broader applications in future research, particularly in non-verbal trauma processing and emotional reintegration.

Traumatic experiences can also exhibit spatial patterns in therapeutic re-enactments (Kolk Reference Kolk2014). Spatial music places sonic events around a space to exploit sound localisation for an immersive effect (Tanaka and Gemeinboeck Reference Tanaka and Gemeinboeck2006). Therefore, in my practice, I intertwine voice, technology and gesture into multichannel immersive sound environments to contextualise the landscape of the mind. Integrating therapeutic practices, such as reclaiming boundaries through gestural performance, demonstrates how sound can metaphorically represent healing processes. These methods provide survivors with a non-verbal, empowering medium to explore and process trauma safely.

2.2.1. Somatic experiencing^®

Developed by Peter Levine, this trauma-informed therapy practice uses bodily awareness to titrate trauma responses stored in the body by guiding clients in using internal proprioception to locate tension or pain while recalling distressing events (Levine Reference Levine2015; Payne et al. Reference Payne, Levine and Crane-Godreau2015). The individual processes release, or manually alter, the sensations, discharging psychological damage alongside the physical remnants. A mantra component involves focusing on each body part while clasping it in their hands (Levine Reference Levine2005: 39–42). The individual makes a series of statements while performing simple actions:

1. 1. Stating the body part being touched.

2. 2. That it belongs to them.

3. 3. Ascribing a positive attribute to it.

4. 4. Reiterating that it is part of their body.

Figure 1 contains an excerpt of the following exercise’s use in Crumble. Continuing from the initial instructions, the exercise directs the participant to state and reclaim their boundaries as a physical space while tracing it out with one’s hands (Levine Reference Levine2005: 39–42). Somatic Experiencing^® is generally a slower, gentler treatment than other therapies.Footnote ² However, this specific exercise can be quite confronting to survivors, particularly to those who have endured childhood violence or sexual abuse and who may not connect to any sense of ownership of their bodies (Scheffers et al. Reference Scheffers, Hoek, Bosscher, van Duijn, Schoevers and van Busschbach2017). In Crumble, this exercise provides a point for audience or listener empathy, showing progress towards restoring the mind–body connection. In allowing others into this disordered space through interweaving text layers, this exercise acts as a moment of respite through its dominantly playful character while showing that therapeutic methods need not be recounting and reliving one’s experience, as I have sometimes found with other talk-based therapies. It demonstrates tentative progress towards restoring one’s mind–body connection. This is also relevant to the following section’s related artworks.

Figure 1.

A text excerpt from the Somatic Experiencing® audio layer in Crumble derived from exercises in Healing Trauma.

2.3. Anamnesis

This article discusses an individual’s psychology through the spatial layout of sounds and anamnesis, as described in Jean-Francois Augoyard’s (Reference Augoyard2014) Sonic Experience, a glossary and explication of sonic terms that stress the importance of space and the architectural elements of sound events. I use these disciplines to inform spatial music analysis. Anamnesis is ‘the often involuntary revival of memory caused by listening and the evocative power of sounds’ (ibid.: 21); these memory retrievals are further characterised by their temporal shifts (ibid.: 23). Augoyard’s work examines the psychological applications of sonic events, as humans cannot separate the brain’s interpreting power from the event itself. This is unsurprising, given that the human ear and our sound localisation capabilities evolved over millennia and aided us in many aspects of life.

The spatial configuration can also intervene and separate the past from the present in sonic terms, as can the presence of different languages, accents, and so on. This delineation operates spectromorphologically via distinctions in spectral content, motion typology (e.g., continuous or discontinuous streams) and degrees of source bonding – how strongly a sound is perceived to be linked to a source (Smalley, Reference Smalley2007). For instance, in Crumble, degraded audio textures and radio-quality voices represent historical or distant temporal contexts, while live-spoken text reflects the present. These pre-recorded audio textures exist as a ‘mediatic space’, ‘spaces associated with communications and mass media‘ (Smalley, Reference Smalley2007: 39). Their material is reduced in clarity to represent temporally distant and potentially damaged memories. Their low density and positioning contribute to a weak source-bonding effect, thereby enhancing the sensation of distance and evoking a sense of detachment. In contrast, live-spoken text anchors the listener in the present through clarity and physical presence. Dynamic gesture-linked spatialisation yields higher source bonding between action and sound, making it feel more embodied and immediate. Combining the live voice with the gestural manipulation of vocal fragments within the soundscape enacts a temporal and bodily disconnection. These elements introduce spatial texture, the perception of density, motion and trajectory within a sonic field, which jointly separate and reposition memory and immediacy within the listening field (Smalley, Reference Smalley1997: 117–24).

Spatial movement and speaker placement thus serve as extensions of gestural performance, as the placement of loudspeakers and fluctuating sound movements directly influence the listener’s perception and navigation of the audio environment. This creates an immersive environment that may trigger memories or emotional responses in line with Smalley (Reference Smalley2007) and Ramjil Fischman’s work in mimetic space (Fischman Reference Fischman2008). Smalley highlights how spectromorphological attributes, such as motion, texture and gesture, affect listener immersion and narrative structure as a journey (Smalley, Reference Smalley1997: 113; Reference Smalley2007: 54); in Crumble, it is where sonic positioning can represent fluctuating mental states. This supports Augoyard’s assertion that sound merges perception and memory, forming a powerful tool for understanding psychological fragmentation. By layering fragmented voices and interweaving narratives at different spatial points, the sonic configuration in Crumble mirrors and separates psychological timelines, showing how sound delineates and reconciles temporal shifts in memory without compromising its immersive impact.

Sound colour influences a listener’s emotional response, especially aspects of timbre and filtering. For example, high-pitched or sharp timbres can evoke tension or urgency, while warm, low-frequency tones often create a sense of calm or safety. This can also refer to source-bonded sounds, such as the examples given of thunder or the sea, where the natural association with these sounds triggers emotional responses tied to memory and experience. The listener is the one who attributes this phenomenon rather than being inherent to the sound itself, and it merges ‘sound, perception, and memory’ (Augoyard Reference Augoyard2014: 21). Anamnesis is seated within the individual’s psychology and can be differentiated in discussions of trauma as not being a flashback by not being inherently negative. Augoyard says that sound is rarely at the origin of trauma, but ‘more frequently it is one of the ways to explore and find the initial stimulus’ (ibid.: 22). The effect can also have more generalised triggers for the population, such as rain, thunder, the sea, crackling fires, and so on. Therefore, it is used in film where the music provides the meta-diegetic, or the secondary narrative, for the visual image and dialogue. A further example of this in musicals, film and opera is bis or leitmotivs, which use repetition of melodic fragments and sound colours to call back to one’s memory of specific characters. He regards music as essential in anamnesis because it evokes feelings and memory (ibid.: 24).

3.1. Gestural devices

Wearable gesture technologies, such as datagloves, rings, electroencephalography (EEG) devices and electronically augmented clothing, enable control of sonic information through movement. Technology affords a creatively novel influence on practice that affects aesthetic perception of sound and can impact self-perception in the user, extending one’s peripersonal space – the area around the body where we can directly act (Sykes Reference Sykes2023). This extension, characterised by multisensory integration, allows tools to augment motor capabilities, modifying the body’s spatial metrics (Biggio et al. Reference Biggio, Bisio, Avanzino, Ruggeri and Bove2020). As tools become embodied, they foster a sense of ownership, agency and self-location over the device (ibid.).

These tools add sonic and visual context to the human voice as paralanguage, exploring embodied experiences in gestural music. My compositions focus on the human voice as a central inquiry into the embodied experience. Crumble uses the MiMU dataglove system (MiMU Gloves Ltd 2010) but is adaptable to any gestural interface with inertial measurement units and flexible finger sensors.

Ramjil Fischman (Reference Fischman2008) describes mimetic space as sounds that resemble or evoke real-world sources or actions, creating an intuitive connection for the listener. This aligns with Trevor Wishart’s views on human utterance, where listeners naturally assign meaning to vocal sounds (Wishart Reference Wishart1996). Fischman extends this to gestures, arguing that embedding recognisable or semi-recognisable sounds in a composition evokes narrative and imagery without explicit statement (Fischman Reference Fischman2008). The iterative use of mimetic and abstract sonic elements in Crumble aligns with Fischman’s aural-mimetic continuum, where the ambiguity between real and surreal soundscapes cultivates engagement. Similarly, Smalley’s (Reference Smalley2007) spectromorphological perspectives on spatial form relate to creating performed spaces in Crumble.

Smalley describes utterance spaces as enacted spaces that are voice-driven, human-instigated environments that are intimate and personal and can draw from social or behavioural cues (Smalley, Reference Smalley2007: 38–9). A performed space is inherently ‘gesturally rooted’; in this case, the agent-performer uses an implement to act within and shape that space (ibid.: 41). The performer’s gloved gestures articulate a series of texture motions (ibid.: 117–18), such as streaming, where multiple sonic layers can be perceived distinctly, and flocking, the collective motion of small sound objects moving as a unit, like birds in flight. These gestures function as first-order surrogates, where visible motion closely mirrors the resulting sonic gesture. These motions vary along the ‘continuity–discontinuity continuum’ and may be perceived as evolving patterns or distinct events (Smalley 1997: 117). The performer’s gestures produce sonic trajectories and trails (ibid.: 124) that remap the perceived arena. This foregrounds performer agency through text, spatial placement and touch-responsive gestural and spatial spectromorphologies (Smalley 1997, Reference Smalley2007). Combined with spatialised audio, gestural mediation evokes transmodal perception, enabling audiences to experience tactile and visual associations that mirror the performer’s emotional states.

3.2. Audience feedback

Audience feedback was gathered during post-performance question-and-answer sessions, where attendees could discuss themes, technology, music, gesture-use and other aspects of the works. Rehearsal footage and audio recordings were also informally circulated during composition to collect thoughts on the musical and spatial elements. This iterative process informed the final folio writings and influenced discrete works and mapping approaches.

This feedback highlighted how Crumble’s dissociative elements resonated with listeners, eliciting feelings of unease and disconnection that aligned with the performance’s depiction of dissociation. These insights refined the balance between eerie and intelligible components, ensuring the soundscape remained unsettling yet accessible and emotionally engaging. Such feedback underlined the importance of fostering empathy without overwhelming or alienating listeners, shaping subsequent iterations of the piece.

3.3. The piece

Six pre-recorded interweaving text layers are played through separate loudspeakers (Figure 2). I voiced each text line as a different character based on its narrative content. The scripts reveal layers of thought stemming from sexual trauma, weaving through memory fragments, varying viewpoints and forbidden knowledge, offering a confessional glimpse into the mind’s innermost workings. These layers interweave to create a verbal soundscape, a word-based texture that fills the sound environment. Each character has a corresponding track that live samples the initial voice to fragment the narrative further. I used audio effects to shape the audio according to the role of the voice. For example, a broadcast-style voice has audio degrading effects and EQs that simulate AM-radio quality to intellectually reference educational and newscaster presentations circa the 1970s. For loudspeaker placements, see Figure 2. As in the diagram, S1, S2, and so on will refer to loudspeakers.

Figure 2.

The loudspeaker arrangement and stage plot for the March 2023 performances.

Three neutral-toned voices intellectualise the events, attempting to neutralise the emotional content. The first line is positioned as a loudspeaker announcement broadcasting a trigger warning. It plays through S1, S3 and S5. A corresponding loop of this text plays through S2, S4 and S6. The second voice discusses biopsychological mechanisms for memory and trauma as an academic-style presentation between S4 and S6, closer to S6. Its live gesturally controlled loop is similarly placed clockwise of S6. The third summarises sexual crime statistics in workplaces and universities as a newscaster. It is placed at S3 and slightly towards the centre, while its gestural loop is positioned squarely at S3.

Three emotionally charged lines contrast the neutral tones. One line contains voice recordings taken during a prolonged traumatic experience, which I censored and obscured using audio effects to preserve emotional rawness while protecting personal privacy. This serves as a commentary on the silencing endured by many victims, including myself, through legal and institutional mandates while protecting my right to decide how much to share about my pain (Prasad Reference Prasad2018). Dark humour, a common coping mechanism (Barwick Reference Barwick2012; Jong Reference Jong2018), is injected through audio effects on speech. For example, in the phrase ‘His skill is talking about himself and then gaslighting me’, the treatment of the text sonically insinuates that the loudspeaker becomes a helium balloon, drifting off in the spatial environment. This plays on the morpheme ‘gas’ from gaslighting, simultaneously allowing the voice to escape by drifting away and making light of the horror I feel at the malleability of reality. This audio is positioned slightly clockwise of S2, with its loop closer to S4.

Another line comprises re-enacted excerpts of my journal from that time. The final is derived from Levine’s exercise (Figure 1). This layer introduces an implied physicality in the text, which explicitly examines my body and was recorded while performing the exercise. This text attempts to reconcile, reaffirm and reintegrate body, mind and personal space. This Reconciliation layer is aurally enticing. It draws the listener in through the colourful, intimate and sometimes playful descriptions of body parts and their positive aspects. This voice and its corresponding live gesturally sampled loop are placed in the same position clockwise of S5.

This method of separating the audio creates pockets of sound in the room that can be explored. Owing to the cocktail effect, this is possible in the binaural mix (Augoyard Reference Augoyard2014: 28). However, the physicality of moving through the space incorporates the audience into the choreography as an active participant (Fischman Reference Fischman2008), practises the idea of creating or marking out boundaries, and considers how one takes up space within the context of the performance. Aside from the discussion on boundaries of physical space and ordinary audience–performer separation, self-censorship in the work preserves personal privacy. However, the performance setting of the work considers broader ethical implications, such as informed consent, audience autonomy, and the potential emotional impact. Pre-performance disclaimers, supporting documentation in the programme notes, and post-performance discussions ensured audiences were aware of the nature and intentions of the performance.

Using six loudspeakers in a circular configuration on the horizontal plane was based on logistical and practical considerations. The small room size necessitated a setup that allowed the audience and the performer adequate room to manoeuvre while maintaining an immersive environment. Wall mounting loudspeakers was not feasible in this venue or compatible with the limitations of Ableton’s Surround Panner. While the configuration used for this performance was effective, future iterations could introduce additional spatial information and complexity. For example, Max ICST Ambisonics packages could add height and facilitate more complex loudspeaker arrays, expanding the work’s spatial dimension to align with more sophisticated spatial audio practices.

3.3.1. Choreography, mapping and the live voice

The performer navigates the space, weaving through the audience while delivering a stream-of-consciousness monologue. This monologue is sampled live and deconstructed through granular synthesis. Text is grasped with ‘puppet hand’Footnote ³ and fist gestures (Figure 3), spatialised, and then released by either hand’s [puppet + wrist flick] or [open hand + wrist flick] to imply flinging the thought away.

Figure 3.

Hand positions used to trigger effects: ok, open hand, closed fist and puppet hand.

Each live soliloquy delivered in performance is preserved and woven into subsequent presentations, contributing to layered auditory interferences and intricate patterns of confusion. All automation data for the ghosts of the live voice is also preserved. The oration predominantly discusses the nature of reality, manipulation tactics employed by abusers, statistics and research data surrounding gendered violence, different therapeutic processes, and the transformation of these ruminations into a recursive creation. The piece ends as I sag to the ground, curl into a ball, and place my head in my hands (Figure 4).

Figure 4.

The standing position is followed by the performer slumping to the floor and ending with hands clasped.

4.1. Spatialisation and anamnesis

Spatialisation in Crumble balances disorienting movement with moments of stability, illustrating its dual capacity to challenge and comfort audiences. This functionality highlights its potential as both a therapeutic tool and an innovative artistic medium. By bridging artistic expression and therapeutic methodology, future studies could expand its integration into multidisciplinary practices.

Positioning sounds in Crumble plays a critical role in generating anamnesis. By placing sound sources around the performance space, the composition influences the listener’s perception of sound origin and movement, creating a visceral, immersive experience. This technique enhances the feeling of being surrounded by sound, triggering memories linked to specific spatial and acoustic environments. The use of the real–unreal continuum transitions between recognisable, mimetic sounds and abstract, surreal elements, evoking shifts between conscious memory and dissociative states (Fischman Reference Fischman2008). Precise placement and movement of sounds can act as cues, triggering specific memories or emotional responses. For example, a sudden shift in sound direction or a change in acoustic properties might evoke memories of similar sensory experiences. This aligns with Augoyard’s assertion that sound can merge perception and memory, creating a tool for exploring and understanding past experiences (Augoyard Reference Augoyard2014: 21).

The disorienting arrangement of sounds, combined with overlapping text layers, replicates the fragmentary nature of traumatic memories, evoking anxiety, dissociation and confusion. Crumble immerses listeners in this chaotic soundscape, leveraging utterance (Wishart Reference Wishart1996) and source-bonding sonic material through gesture (Smalley, Reference Smalley2007) to act as mimetic interpretants in shaping narrative meaning (Fischman Reference Fischman2008). For example, gestural mediation disconnects the acoustic and amplified voices, creating an eerie, anxious environment, as noted in informal audience feedback. Figure 5 shows a sequence of actions designed to communicate overwhelming helplessness. The hands are raised with one grasped and twisting; then, the performer turns with a raised fist, which drops fatalistically to their sides.

Figure 5.

The sequence shows exasperation; a frustrated, questioning gesture is followed by lifting one hand to change the audio position, which then drops to the performer’s sides in helplessness. The images follow the sequence in Figure 4.

Dissociation often occurs when a person becomes overwhelmed, and the psyche tries to split from the present to preserve itself (Vesuna et al. Reference Vesuna, Kauvar, Richman, Gore, Oskotsky and Sava-Segal2020). This detachment can manifest as a frozen state or an inability to reconnect with the body or present moment. Over time, repeated ordeals can reinforce these responses. As with all psychological reactions, dissociation varies among individuals and can be categorised into three primary subtypes (Moore and Boland Reference Moore and Boland2018). In Crumble, dissociation manifests as disconnectedness, with the audio becoming commotion. This unintelligibility symbolises stress-induced issues in speech production and comprehension. As noted in informal audience feedback, gestural mediation further amplifies this effect, disconnecting acoustic and amplified voices to create an eerie, anxious soundscape.

Figure 6 shows the rotation spatialisation data captured from a week’s performances in Ableton Live’s Surround Panner plugin (Ableton 2020). The rotation parameter in this setup refers to the angular displacement of the sound source around the listener. The horizontal axis represents time (around 12 minutes total), and the vertical axis represents the spatial positioning from -180 to 180 degrees in a circle. Each horizontal line’s variation indicates the spatial movement of each performance’s live audio track and illustrates how the performer oriented the live monologue in a six-channel-circle surround environment.

Figure 6.

Live voice layers with rotation information automation are displayed in Ableton.

Performance data, such as Live Voice 11 (10 March 2023) exhibits frequent and rapid changes in position, indicating movement, while Live Voice 12 (12 March 2023) shows smoother, less frequent positional changes, suggesting more stable or gradual movement. Live Breath 5 (6 March 2023) has significant oscillations with notable fluctuations. Live Voice 8 (7 March 2023) has a moderate movement level, with less frequent and dramatic changes than Live Breath 5. Tracks such as Live Voice 11 and Live Breath 5 show high levels of dynamism, frequently moving across the entire spatial range.

The movement data shows three activity phases with temporal patterns evident. In the first phase (0–3 mins), tracks generally show high variability, indicating a more fragmented initial soundscape. Some performances throughout the week display a stabilised movement in the second phase (3–9 mins). In contrast, others continue significant movement, suggesting that the monologues and middle section could have been more static due to navigating the audience or unsettled internal states (of the performer). In the third phase (9–12 mins), a trend towards either stabilisation or continued dynamic movement is observed, indicating a concluding segment that maintains intensity or fades to stillness.

The rotation and positioning of sound create a sense of movement and immersion, mirroring the performer’s fluctuating mental states and experiences of chaos. Smalley’s concept of space-form highlights how spatial configurations and sound movement construct immersive environments that reflect narrative and emotional arcs (Smalley, Reference Smalley2007). Similarly, Fischman’s framework of mimetic space demonstrates how transitions between mimetic and abstract sounds evoke shifts between memory and dissociation (Fischman Reference Fischman2008). Directional sound foregrounds specific text sections, drawing the audience’s focus to critical themes such as self-worth and boundaries (in Reconciliation and the live monologue’s speech content).

Beyond immersion and localisation, the spatial typologies enhance the narrative’s psychological depth by differentiating between chaotic and stabilising spaces. For instance, circular configurations reflect dissociative states, while fixed positions represent moments of emotional grounding. Behaviours such as oscillation and rotation of sound sources mirror psychological instability, creating a visceral sense of fragmentation. The interaction between mimetic and abstract elements – such as degraded voices for distant memories versus live voices for immediate realities – accentuates the spectrum of emotional resonance (Fischman Reference Fischman2008; Smalley, Reference Smalley2007). Spatial processes, including real-time gestural control via MiMU gloves, enable performers to control these elements, bridging physical and virtual spaces.

Finally, rotating sound sources enhance the narrative by reflecting the emotional journeys in the texts. Chaotic movements, such as in Diary 1, highlight disorientation, while directed sounds in Technicalities and Reconciliation emphasise the integration or separation of memory types and body parts, enriching the audience’s spatial and emotional experience.

4.2. Semantic information

In Crumble performances, live and pre-recorded texts create an immediate connection to the performer’s lived experiences and broader societal issues. The live voice is fluid and responsive, referencing pre-recorded layers to explore themes of reality, perception and abuse statistics. This real-time interaction adds spontaneity, enabling the performer to engage directly with the audience while tying together thematic threads for a cohesive and thoughtful conclusion. Pre-recorded texts serve as thematic anchors, enveloping the audience in a continuous narrative flow that enhances the performance’s immersive and reflective nature.

The live monologue portrays the ongoing emotional and psychological toll of personal trauma, fostering empathy and highlighting the resilience required to navigate such experiences. Text-based hooks, such as ‘If you haven’t known a life without sexual violence’ immerse the audience in the performer’s emotional landscape. These reflections emphasise self-worth and boundaries in the face of systemic and personal violations, offering moments of empowerment and self-reclamation as affirmations of resilience.

Crumble explores healing through the Reconciliation text, drawn from Levine’s Somatic Experiencing^®. The text varies between light-heartedness and seriousness, with each body part labelled, attributed positively and reassigned to self-control. This is followed by restoring and determining one’s boundaries as a physical space around the body (Figure 7).

Figure 7.

An excerpt of the Reconciliation text segment in Crumble shows a mantra to restore one’s boundaries.

The text critiques institutional failures to address harassment and assault, conveying a sense of abandonment and frustration. Themes of institutional betrayal, supported by informal audience feedback, resonate strongly. For example, statistics revealing worsening conditions in university harassment cases are presented: ‘Considering that the statistics from the last two lots [sic] that happened for sexual harassment in a university setting didn’t actually show any improvement. It showed that things have gotten worse.’

A recurring theme is the attempt to understand and process trauma intellectually. This exploration of the brain’s mechanisms and the effort to ‘hack’ one’s responses reflect a journey towards healing and self-awareness. This balance between emotional narratives and intellectual inquiry underscores the complexity of trauma recovery, as seen in remarks such as, ‘I’ve sought to understand the mechanisms and the reasons for why the brain does everything that it does.’

Rotating sound sources around the listener enhances immersion and mirrors the emotional journey described in the texts. For instance, chaotic movement can reflect the disorientation in the diary excerpt track or the overwhelming nature of the statistics in another. Positioning sound in specific directions guides the listener’s attention and creates a sense of space, perspective and gravitational force (Smalley, Reference Smalley2007: 47). For texts such as the biopsychological audio and the Somatic Experiencing® track, sound positioning emphasises the separation or integration of memory types or body parts, enriching the narrative’s emotional complexity.

4.3. Gestural mediation and spatialisation

Gestural mediation through wearable devices in Crumble enhances anamnesis by allowing performers to animatedly control sound’s spatial and acoustic properties, creating a rich, immersive auditory experience in a performed space (Smalley, Reference Smalley2007). Using MiMU gloves, performers manipulate sound spatialisation and parameters in real-time, transforming gestures into auditory cues that deepen the embodied experience. This interaction fosters agency, aligning with embodied cognition principles, where the mind and body work together to perceive and interpret sensory information (Rohrer Reference Rohrer and Torres2008). The performer’s movements connect the body and narrative, engaging the audience through physical and emotional resonance (Figure 8). Furthermore, these movements directly influence the sound, adding physicality to the auditory experience.

Figure 8.

Changing hand positions to ‘let go’ of a loop or layer of consciousness (left to right).

Gesture data are captured by MiMU wearable devices (MiMU Gloves Ltd 2010) and mapped to various audio parameters such as spatial position, delay effects and visual triggers. Table 1 outlines the specific gestures and their corresponding audio parameters. Real-time manipulation, including yaw and pitch movements, controls sound rotation and focus, creating a shifting environment that reflects the performer’s physical and emotional states. For example, sound rotation around the listener highlights directionality and movement, continuously engaging the audience with an active spatial audio experience.

Table 1.

Gesture routing table for live gestural information in Crumble

Embodied cognition posits that cognitive processes are deeply rooted in the body’s interactions with the world. In Crumble, the performer’s gestures serve as a conduit for expressing and externalising internal emotional and psychological states. The physical act of moving and manipulating sound engages the performer and the audience on a deeper sensory level, nurturing a deeper connection and understanding of the narrative (Blair et al. Reference Blair, Jacobson, Mealey and Salisbury2021; Canazza et al. Reference Canazza, De Poli and Vidolin2022). The gestural mediation ensures that the sound environment evolves, maintaining engagement and emotional depth through spectral expansion and divergence (Smalley, Reference Smalley2007).

Resentment and anger were articulated in Crumble through distortion, effects and text, vocal interpretation (as both a flat and an emotionally charged delivery), and manipulation of the natural voice to serve these purposes. These elements draw from contemporary music tropes and everyday voice use (Cavarero Reference Cavarero2005; Wishart Reference Wishart1996). For example, guitar distortion is used in so-called angry music such as metal and rock. Audio effects on speech amplify the emotional charge, such as the repeated phrase ‘His skills are talking about himself and then gaslighting me’ – processed to sound helium-like, exaggerating the phrase as it drifts through the multi-channel audio environment.

Gestural control using the yaw of each hand rotates the live voice in a full circle through loudspeakers, creating a sense of directionality and movement. During performances (see Figure 6), the rotation values in Surround Panner frequently change, indicating that the sound sources were continuously rotated around the listener and performer to create an active spatial audio experience. The constant changes in rotation keep the audience engaged and intensify the immersive quality of the performance.

Table 1 shows the general spatialisation settings for each track, though some of these move throughout the piece, depending on the content’s expressive function. This spatial diversity shown in the x- and y-axis parameters enriches the auditory experience using 360-degree space around the listener. The varied distribution of sounds in the sound environment and the Focus and Smooth parameters balance the aural landscape, combining pinpoint, directional sounds with broad, enveloping audio. Tracks with lower Smooth values suggest abrupt transitions in their movement, creating a sense of urgency, while higher values indicate smoother transitions, contributing to fluid auditory experiences. This design leverages central positioning, spatial diversity, varied focus, and smooth transitions to create a rich, immersive, balanced auditory experience.

Spatialisation data and gestural control are important in presenting and experiencing the texts in Crumble in order to make the invisible and internal experiences of trauma sonically tangible. Complex emotional states are translated into auditory elements, with anxiety and panic conveyed through abrupt transitions and wide spatial placements, as seen in tracks with lower smooth values and varied x- and y-axis positions (Table 2). For example, ‘I’m not safe’ (Focus: 15.0%, Smooth: 17.3%) uses a narrow focus and moderate smoothness to create a sense of urgency and unease, while ‘I had no one’ (Focus: 66.14, Smooth: 15.7%) employs a broader focus to convey a more encompassing feeling of isolation. Tracks such as ‘Trigger Warning’ (Focus: 100.00, Smooth: 22.8%) use a narrow focus and high smoothness to create a sharp, continuous sound that can represent the persistent nature of traumatic memories. In contrast, Diary Loop (Focus: 25.20, Smooth: 8.66%) uses a wide focus and low smoothness to depict the disjointed and fragmented nature of dissociative states.

Table 2.

Crumble pre-determined audio spatialisation

The rotation data shows fluctuations over the piece’s 12-minute duration (see section 4.1). These variations show intense activity and calmer periods, which work in line with the live monologues, which vary from day to day, depicting the chaotic and fragmented mental states of trauma. The movement of sound around the listener can trigger spatial memory, where certain directions or movements might be associated with specific memories or emotions. Rapid changes in rotation can create a sense of disorientation and confusion, mirroring the mental state of someone experiencing trauma. The focus of sound shifting around the listener can represent the fragmented thoughts and emotions that often accompany traumatic experiences.