1. Introduction: data, instruments and creative agency
Over the past two decades, the integration of sensor data and machine learning into musical systems has produced a growing ecosystem of ‘data-embedded’ instruments, performance environments and installations. These practices are often framed in polarised terms: either as evidence of radical innovation, new forms of intelligence, autonomy and responsiveness or as symptoms of a broader drift towards algorithmic control, surveillance and opacity in culture at large. In both celebratory and critical accounts, however, the analytical tools for describing how data is configured within instruments, bodies and listening situations often remain underdeveloped. Although there is influential work on ethics, interaction and algorithmic sound in electroacoustic and Human–Computer Interaction (HCI)-related literature, mid-level concepts that explicitly connect concrete design decisions to questions of care, agency and power in data-embedded instruments remain comparatively scattered.
This article responds to that gap by proposing a framework for analysing data-embedded instruments that foregrounds both embodiment and ethics and by offering a comparative lens for describing how different systems organise relations between data, performers and audiences. Rather than asking whether data-driven instruments are good or bad in the abstract, the article aims to provide tools for asking what kinds of sensing, mapping and participation particular configurations enable and what kinds of responsibilities and vulnerabilities they create.
1.1. Data-embedded electroacoustic practice
The focus here is on electroacoustic practices in which data is structurally integrated into the behaviour of instruments and performance systems, rather than merely added as an auxiliary control signal. In such settings, biometric signals, gestural input, environmental feedback or affective inference may be captured, processed and mapped in real time to sonic parameters such as pitch, density, spatialisation or timbre (Miranda and Williams Reference Miranda and Williams2015; Duarte et al. Reference Duarte, Cossette and Wanderley2023). Composers and designers act as system architects who specify how bodies, sensors, algorithms and loudspeakers are coupled; performers negotiate feedback loops that may be partially opaque; audiences may become listeners, participants and data subjects simultaneously (Whalley Reference Whalley2015; Danieli et al. Reference Danieli, Witek and Haworth2021).
In this context, the article uses ‘data-embedded instruments’ and ‘data-embedded music’ as heuristic labels rather than as fixed categories. The interest is not in policing the boundary between data-driven and non-data-driven work, but in examining differences in how data is sourced, processed and made audible; how these differences redistribute agency between human and computational actors; and how they intersect with situated questions of care, consent and inclusion. ‘Data’ here refers to quantifiable inputs – biometric, gestural, environmental or affective – that participate in shaping sonic behaviour, whether captured live or drawn from pre-existing datasets.
1.2. An ethical-embodied framework
To analyse these practices, the article develops an ethical-embodied framework. It assumes that data used in musical systems is not neutral: it is produced by specific bodies, within specific histories, and made audible through specific listening situations. The framework organises analysis around three interrelated dimensions:
-
• Data & care: how data is collected, curated and maintained; who is exposed or protected; what forms of labour, extraction and consent underpin data use in musical systems (Whalley Reference Whalley2014a; Wolf and Fiebrink Reference Wolf and Fiebrink2019).
-
• Listening & attention: how instruments script forms of listening and attention – what performers and audiences must learn to attend to, what is rendered stable or variable and what aspects of bodies and environments are backgrounded or ignored (Santaella Reference Santaella2018).
-
• Agency & participation: how roles and responsibilities are distributed among composers, performers, audiences and computational processes; who can intervene, refuse or reshape system behaviour and under what conditions (Wu et al. Reference Wu, Zhang, Bryan-Kinns and Barthet2017; Wolf and Fiebrink Reference Wolf and Fiebrink2019).
Here, ‘ethical’ refers not to abstract codes of conduct but to concrete distributions of care, responsibility and risk around data – who is sensed, who consents, who benefits and who is exposed – while ‘embodied’ refers to encultured and situated forms of perception and action, where listening and performance are shaped by gendered, racialised and institutional conditions as much as by individual technique (Loaiza Reference Loaiza2018; Hayes Reference Hayes2019). These three dimensions operationalise this ethical-embodied orientation, and the framework is offered not as a checklist for ‘ethical’ design, but as a set of lenses for making situated judgements about how specific data-embedded configurations materialise relations of care, listening and agency.
1.3. A comparative lens on data-embedded instruments
Alongside this ethically embodied framework, the article uses a comparative lens for describing the technical and performative configurations of data-embedded instruments. This lens characterises works along four heuristic axes: the types and sources of data they draw on, the modes through which data is integrated with sonic behaviour, the level and structure of interactivity and the roles taken by audiences and other data subjects. Section 3.2 develops these axes in detail and shows how they support comparison across diverse data-embedded practices without presuming a single narrative of technological progress.
1.4. Research questions, contributions and scope
Two questions guide the article’s argument:
-
1. How are data-embedded configurations used to rework understandings and uses of musical instruments in electroacoustic contexts, and what cultural and ethical stakes arise from how data is sourced, mapped and made audible in specific works?
-
2. How can a combined ethical-embodied framework and comparative lens help describe differences between data-embedded practices that are often grouped together under the same technological labels?
The article contributes:
-
1. An ethical-embodied framework articulated through three mid-level dimensions: data & care, listening & attention and agency & participation.
-
2. A comparative lens for analysing data-embedded instruments along four heuristic axes: data type and source, mode of integration, level and structure of interactivity and roles taken by audiences and other data subjects.
-
3. A set of short analytic vignettes that apply this combined framework and lens to selected electroacoustic works, showing how they can surface differences often obscured by generic talk of ‘interactivity’, ‘intelligence’ or ‘autonomy’.
1.5. Methodological approach
The argument combines selective genealogical analysis with comparative close readings of works that foreground embodiment, cultural location and ethical stakes in electroacoustic instrument design. Section 2 traces multiple, culturally situated strands in which sound, bodies and technical systems are entangled through recording, sensing and algorithmic organisation, without proposing a single origin story for data-embedded instruments. Section 3 develops an ethical-embodied framework and a heuristic comparative lens, translating theoretical strands into mid-level dimensions and analytic axes. Section 4 applies these tools to four concise artist-focused vignettes that examine how particular configurations handle data, organise listening and distribute participation. Throughout, the methodological orientation is informed by my practice as a composer working with sensor-based and algorithmic systems, which motivates the focus on concepts that connect concrete design decisions to their cultural and ethical implications.
2. Genealogical and global orientations
The practices I describe as data-embedded do not emerge ex nihilo with contemporary sensors or machine-learning pipelines. The discussion that follows is necessarily selective: I highlight those historical trajectories that most clearly foreground distributed agency, sensing and cultural location, rather than aiming at a comprehensive survey of electroacoustic practice. They extend longer histories in which sound, bodies and environments have been captured, transformed and algorithmically organised as compositional material. Attending to these genealogies is essential for avoiding narratives in which ‘data’ or ‘artificial intelligence (AI)’ appear as autonomous agents that suddenly reconfigure musical practice. Instead, data-embedded instruments can be understood as specific intensifications of tendencies already visible in mid-twentieth-century electroacoustic and experimental work and as situated differently across cultural and institutional contexts (Kane Reference Kane2013; McLaughlin Reference McLaughlin2021).
2.1. Early electroacoustic genealogies
One point of departure is Halim El-Dabh’s Taʿbīr Al-zār (The Expression of Zaar, 1944), constructed from wire recordings made via Cairo’s Middle East Radio and subjected to substantial electronic transformation. Subsequent restoration and scholarship argue that the work should be treated as a fully electronic composition rather than ethnographic documentation and that it destabilises the familiar Paris/Cologne narrative of electroacoustic origins (Puig Reference Puig2019; Redhead Reference Redhead and Young2024). Environmental and ritual sounds are already captured, segmented and recomposed here as material whose provenance is culturally and politically charged.
Four years later in Paris, Pierre Schaeffer’s Cinq Études de Bruits and the 1948 Concert de Bruits articulated another approach in which any recorded sound could serve as compositional material (Kane Reference Kane2013). His emphasis on recording, isolating and cataloguing sound fragments – taken up in later spectromorphological work (Smalley Reference Smalley1997; McAuliffe Reference McAuliffe2017) – foregrounded how techniques of capture and transformation mediate authorship and listening. These early electroacoustic genealogies are not ‘data-driven’ in a contemporary sense, but they show that questions of capture, mediation and control over recorded material have been central to electronic music from its earliest formations.
2.2. Algorithmic, cybernetic and embodied feedback
A second genealogical strand runs through mid-twentieth-century work in which compositional decisions are formalised as explicit procedures and processes. Iannis Xenakis’s stochastic music treats probability distributions, mathematical modelling and large-scale transformations as compositional parameters, foregrounding rule-based organisation and statistical thinking rather than intuitive note-by-note writing; these models operate as one set of constraints among others, not as autonomous data streams that ‘drive’ the music (Xenakis Reference Xenakis1992; Giannakis Reference Giannakis2006; Hahn Reference Hahn2018).
In parallel, experimental practices explored couplings between performers, environments and technical systems. John Cage’s chance operations and use of the I Ching delegate aspects of decision-making to external procedures, while his indeterminate and participatory works foreground contingency and co-agency between performers, media and environment (Cage Reference Cage1961; Cage Reference Cage1965; Cage Reference Cage1977). Alvin Lucier’s Music for Solo Performer (1965) stages amplified alpha-wave electroencephalography (EEG) signals, measurement apparatus, loudspeakers and resonant objects as a distributed performance system, in which apparently ‘direct’ mappings from brain activity to sound depend on choices of equipment, routing and spatial configuration (Straebel and Thoben Reference Straebel and Thoben2014). Pauline Oliveros’s Deep Listening practice and text scores emphasise situated listening, improvisation and collective shaping of sonic space, treating listening and authorship as ongoing social commitments rather than purely technical affordances (Andersen Reference Andersen2022).
What links these examples to contemporary data-embedded instruments is not a shared technology but shared concerns with how agency, attention and control are distributed among people, environments and technical systems. Algorithmic and stochastic logics, feedback structures and participatory practices are treated as compositional and ethical choices that shape responsiveness and co-creativity, rather than as neutral technical features.
2.3. Plural and decolonial trajectories
A genealogical account of data-embedded practice that foregrounds only Euro-American institutions reproduces a narrow view of technological modernity. Recent work emphasises how electroacoustic and digitally mediated practices emerge from and intervene in diverse cultural contexts, often in explicit response to colonial and postcolonial power relations (Lewis Reference Lewis1996; Frimpong et al. Reference Frimpong, Ayaburi and Andoh-Baidoo2024). El-Dabh’s position within Egyptian radio and ritual culture already complicates simple centre–periphery narratives, and later practices likewise embed audio technologies within local epistemologies rather than simply importing Euro-American models (Redhead Reference Redhead and Young2024).
Examples relevant to data-embedded instruments include projects that digitise Indonesian Palompong instruments and use sensor-augmented versions to negotiate between local ensemble practice and global new-music circuits (Gafar et al. Reference Gafar, Sulong and Alfarisi2023); sensor-based ritual compositions engaging Acehnese Meugang traditions under conditions of conflict and reconstruction (Gusmanto and Denada Reference Gusmanto and Denada2023); and hybrid educational tools in Central Europe that use tangible interfaces and embedded electronics to connect vernacular musics, coding and sound art (Bojc and Potočnik Reference Bojc and Potočnik2024). These projects show how electroacoustic techniques are bound up with ritual, education and reconstruction, rather than functioning as placeless technical novelties. In each case, sensing and mapping operate as sites where cultural continuity, authorship and institutional power are actively negotiated.
Anthropological and decolonial frameworks such as acoustemology reinforce this point by treating listening as a situated way of knowing, grounded in specific social and ecological relations (Frimpong et al. Reference Frimpong, Ayaburi and Andoh-Baidoo2024). For data-embedded instruments, this implies that decisions about what and whom to sense, how to map or sonify those signals and which bodies are invited or required to participate are inseparable from questions of cultural and political location.
2.4. From sensing and feedback to machine learning
Against this backdrop, contemporary practices that integrate biosensing, networked data and machine-learning models can be understood as extensions of earlier experiments with recording, algorithmic form and feedback rather than as a clean break. Systems such as gesture-tracking instruments, brain–computer interfaces and affective computing setups attach sensors to bodies and environments in ways that intensify longstanding concerns about surveillance, visibility and control in performance (Duarte et al. Reference Duarte, Cossette and Wanderley2023; Guo et al. Reference Guo, Kang and Herremans2023; Tejada et al. Reference Tejada, Murillo and Mateu-Luján2023). Machine-learning-based instruments add further layers by shifting some compositional decisions into training datasets, model architectures and optimisation procedures (Fiebrink Reference Fiebrink2019; Novelli and Proksch Reference Novelli and Proksch2022).
Recognising these continuities matters because it both counters technological determinism and clarifies why new analytic tools are needed. Data and machine learning do not themselves redefine instruments or creative agency; they are taken up by designers and composers to pursue specific aesthetic, institutional and political projects that build on existing electroacoustic and experimental traditions (McLaughlin Reference McLaughlin2021). At the same time, the combination of embedded sensing, learned mappings and distributed authorship makes it difficult to describe who or what is acting, who is exposed and how listeners are positioned using only the language of traditional instrumentality or notation (Born Reference Born2022).
The ethical-embodied framework and comparative lens developed in the next section respond to this situation by offering ways of analysing data-embedded instruments that neither reduce them to technical novelties nor fold them back into generic stories of ‘interactivity’ or ‘innovation’.
3. The ethical-embodied framework and comparative lens
Building on the ethical-embodied framework outlined above, this section translates theoretical strands from electroacoustic theory, feminist ethics of care, decolonial and Indigenous listening and enactive music cognition into analytical tools for reading data-embedded instruments (Lewis Reference Lewis1996; Loaiza Reference Loaiza2018; Hayes Reference Hayes2019; Borkowski Reference Borkowski2023). These tools operate at two levels: three ethical-embodied dimensions – data & care, listening & attention and agency & participation – and a set of comparative axes that characterise data type and source, modes of integration, interactivity and audience role (Danieli et al. Reference Danieli, Witek and Haworth2021; Duarte et al. Reference Duarte, Cossette and Wanderley2023; Gusmanto and Denada Reference Gusmanto and Denada2023).
Rather than proposing a universal taxonomy, the framework treats both dimensions and axes as heuristic devices that support comparison while remaining accountable to the plural, locally grounded approaches surveyed in Section 2 (Wang et al. Reference Wang, Zhang, Yu, Xu and Deng2021; Jogjaningrum Reference Jogjaningrum2022; Marquez-Borbon Reference Marquez-Borbon2024). The aim is to organise recurring questions to ask of data-embedded instruments – whose bodies and environments are sensed, how listening is organised and how agency is distributed – rather than to impose prescriptive categories or rank practices as more or less ‘ethical’ (Lewis Reference Lewis1996; Hayes Reference Hayes2019).
3.1. Ethical-embodied dimensions
Drawing on feminist ethics of care, decolonial and Indigenous acoustemologies and enactive accounts of music cognition, the ethical-embodied framework evaluates data-embedded instruments along three interrelated dimensions: relations of data & care, modes of listening & attention and distributions of agency & participation (Feld and Brenneis Reference Feld and Brenneis2004; Loaiza Reference Loaiza2018; Hayes Reference Hayes2019). These dimensions build on accounts that treat sound as situated knowledge, listening as an ethical and political practice and musical systems as embedded in networks of interdependence, surveillance and care rather than as neutral technical infrastructures (Waitt et al. Reference Waitt, Ryan and Farbotko2013; Hayes and Loaiza Reference Hayes and Loaiza2022; Borkowski Reference Borkowski2023). Embodiment is understood not simply as physical gesture, but as historically situated couplings of bodies, technologies and environments that shape how musical agency is enacted.
3.1.1. Data & care
Data & care asks whose bodies, environments and cultural practices are being sensed; how biometric, gestural, affective or environmental traces are collected, processed and mapped; and whether these procedures reproduce extractive logics or participate in relationships of care, co-action and cultural accountability (Duarte et al. Reference Duarte, Cossette and Wanderley2023; Tejada et al. Reference Tejada, Murillo and Mateu-Luján2023). Drawing on feminist ethics of care and decolonial data studies, it treats data as embedded in relations of vulnerability, obligation and sovereignty, foregrounding questions of consent, ownership, surveillance and uneven access to technological infrastructures (Jogjaningrum Reference Jogjaningrum2022).
In inclusive instruments for d/Deaf performers, for example, choices about which bodily capacities are privileged, how tactile and visual signals are mapped to sound and how training data are stored and reused become questions of data & care, as designers and performers co-create interfaces that support participation beyond normative sensory configurations (Duarte et al. Reference Duarte, Cossette and Wanderley2023). Audience-responsive installations that capture proximity or movement raise related issues when subtle, often involuntary behaviours are turned into control signals, potentially extending performance data into wider surveillance regimes (Danieli et al. Reference Danieli, Witek and Haworth2021). From decolonial and Indigenous perspectives, data & care also encompass questions of sovereignty and situated knowledge, where projects that embed ritual gestures or land-based practices into sensor and AI-driven systems must negotiate protocols for how recordings circulate and who authorises their use (Lewis Reference Lewis2000; Ribeiro et al. Reference Ribeiro, Kaminski, Lübeck and Boscarioli2019). As an analytical lens, data & care does not assign moral verdicts but provides a vocabulary for describing how systems configure consent, vulnerability, responsibility and benefit around biometric, environmental and cultural data (Hayes Reference Hayes2019).
3.1.2. Listening & attention
Listening & attention focuses on the forms of listening that a system invites: whether it privileges analytic detachment and abstraction or fosters positional and accountable listening that recognises sound as entangled with memory, community and land (Battier Reference Battier2007). Building on deep listening and decolonial listening frameworks, it treats listening as an ethical stance rather than a purely perceptual act. Pauline Oliveros’s deep listening foregrounds social and emotional attunement, offering an ‘ethics of presence’, while Dylan Robinson’s hungry listening critiques colonial modes that extract sound without acknowledging Indigenous sovereignty or protocol (Bell and Oliveros Reference Bell and Oliveros2017; Halstead and Hilder Reference Halstead, Hilder and Lee2024). Enactive accounts emphasise that meaning emerges through ongoing sensorimotor coupling between bodies, interfaces and environments, rather than from internal representations alone (Loaiza Reference Loaiza2018; Hayes and Loaiza Reference Hayes and Loaiza2022).
Data-embedded systems concretise these stakes. In an installation where audience proximity continuously modulates spatialised sound, participants are drawn into a perceptual feedback loop in which their movements alter the sonic field, even as they may not fully register how their behaviours are being tracked (Danieli et al. Reference Danieli, Witek and Haworth2021). Biometric and affective interfaces likewise raise questions about whether listening foregrounds empathy and shared vulnerability or normalises the extraction and display of intimate states (Wexler et al. Reference Wexler, Yip, Lee, Li and Wong2023). As a dimension of the framework, listening & attention functions as a descriptive lens for tracing how systems organise perceptual focus, accountability and positional awareness, rather than as a scale for judging listening practices as inherently virtuous or deficient (Bell and Oliveros Reference Bell and Oliveros2017; Hahn Reference Hahn2018; Hayes Reference Hayes2019).
3.1.3. Agency & participation
Agency & participation addresses how creative agency is shared among composers, performers, audiences and computational processes and how roles are scripted, destabilised or opened across a performance ecosystem (Miranda and Williams Reference Miranda and Williams2015; Whalley Reference Whalley2015). Drawing on posthuman accounts of agency and critical work on algorithmic and improvising systems, it emphasises that computational processes are always entangled with human decisions, social histories and institutional structures rather than acting as neutral partners. Ethical-embodied design treats instruments as relational, adaptive systems in which agency is explicitly distributed across human and non-human components and asks who can influence mappings and system behaviour in practice, how improvisation and co-creation are structured and how claims of technological neutrality may conceal asymmetries of authorship, access and expertise (Lewis Reference Lewis2000; Browne Reference Browne2021; Black and Wilson Reference Black and Wilson2024).
Historical and contemporary examples make these issues explicit. George E. Lewis’s Voyager, conceived as a virtual improvising orchestra, analyses a human improviser in real time and generates responses as an autonomous co-performer, foregrounding that computational systems encode specific aesthetic and social commitments rather than functioning as culture-free partners (Steinbeck Reference Steinbeck2018). Sensor- and machine-learning-based instruments such as Laetitia Sonami’s Spring Spyre or Atau Tanaka’s EMG performances likewise redistribute authorship across bodily effort, sensor infrastructure and signal processing, requiring performers to negotiate noisy, adaptive systems over time (Tanaka and Parkinson Reference Tanaka and Parkinson2016; Fiebrink and Sonami Reference Fiebrink and Sonami2020). Audience-centred systems intensify these dynamics when participants influence sonic behaviour without necessarily being able to reconfigure system logic, blurring the line between co-authorship and being acted upon as data sources, while inclusive instruments for d/Deaf and neurodivergent musicians reconceive agency through co-designed mappings that support different modes of musical presence (Whalley Reference Whalley2015; Danieli et al. Reference Danieli, Witek and Haworth2021; Duarte et al. Reference Duarte, Cossette and Wanderley2023). Within the framework, agency & participation do not rank systems from more to less ‘democratic’ but offer a lens for articulating how specific combinations of sensors, mappings, algorithms and institutional conditions configure who acts, who is acted upon and how shared responsibility is negotiated in data-embedded performance (Keller Reference Keller2000; Brown Reference Brown and Impett2021).
3.2. Comparative axes for data-embedded instruments
Complementing the ethical-embodied dimensions, the comparative framework employs a set of technical axes that characterise how specific instruments and systems configure data, embodiment and interaction. These axes are not treated as exhaustive taxonomic categories but as heuristic descriptors that support comparison across diverse practices without erasing their situated specificity. As such, the axes are not mutually exclusive or orthogonal; they are overlapping perspectives that help articulate how particular configurations of data, embodiment and interaction come to matter in practice. In this paper, ‘algorithmic’ is treated under the category of mode of integration (how processes structure mapping/generation), not as a data type; when sensor streams feed ML processes, we classify by the role that function plays in the work’s mapping and authorship, not by the toolchain per se.
The core axes are:
-
• Data type and source: biometric (e.g., EMG, heart rate, EEG), gestural (motion capture, strain sensors, proximity), environmental (temperature, humidity, positional data) and affective (facial recognition, emotional state via neural networks), including whether the data derive primarily from performers, audiences or environments (Duarte et al. Reference Duarte, Cossette and Wanderley2023; Guo et al. Reference Guo, Kang and Herremans2023; Tejada et al. Reference Tejada, Murillo and Mateu-Luján2023; Wexler et al. Reference Wexler, Yip, Lee, Li and Wong2023).
-
• Mode of integration: the degree and manner in which data streams are structurally bound into the instrument, ranging from embedded sensors and direct mappings to software interfaces and algorithmic mediation, including machine-learning models that generate musical signals from computed features or learned patterns (Moroni et al. Reference Moroni, Manzolli, Zuben and Gudwin2000; Lichtenwalter et al. Reference Lichtenwalter, Lichtenwalter and Chawla2010; Fiebrink and Caramiaux Reference Fiebrink and Caramiaux2016; Lee Reference Lee2023).
-
• Level and structure of interactivity: whether systems are primarily reactive (one-directional responses to live data), adaptive (mappings or strategies change over time), improvisatory (human and computational agents co-create material in real time) or participatory (multiple performers, access users or audiences can directly contribute data or reshape mappings) (Whalley Reference Whalley2014b; Wolf and Fiebrink Reference Wolf and Fiebrink2019; Danieli et al. Reference Danieli, Witek and Haworth2021).
-
• Audience role: whether audiences are positioned as listeners, co-performers or data sources and how far they can influence or reconfigure the system beyond supplying measurable traces (Whalley Reference Whalley2015; Danieli et al. Reference Danieli, Witek and Haworth2021; Gusmanto and Denada Reference Gusmanto and Denada2023; Liu Reference Liu2024).
4. Vignettes: demonstrating the method
The discussion that follows develops the method through four vignettes of artistic practice. Each vignette moves from the work’s context and sonic behaviour to a comparative placement along the axes that organise the analysis (data type and source, mode of integration, level of interactivity, audience role), before interpreting the work through the three ethical-embodied dimensions. Using a consistent frame across cases keeps the analyses comparable while allowing the works’ differences to remain legible.
4.1. Gestural/sensor instruments: Sonami & Tanaka
Laetitia Sonami’s Spring Spyre employs strain sensors and gesture tracking as its primary data sources from the performer’s hand and arm, feeding a reactive yet finely tuned synthesis environment in which small changes in hand pressure reconfigure complex sound fields (Fiebrink and Sonami Reference Fiebrink and Sonami2020). Atau Tanaka’s EMG-based performances use muscle signals as biosignal data, emphasising an adaptive relationship in which performer and system learn to accommodate noise, fatigue and changing corporeal states, redistributing authorship across muscular effort, sensor infrastructure and signal processing.
Data type & source: Sonami – gestural/strain sensing from the performer; Tanaka – EMG biosignals from the performer. Mode of integration: Sonami – continuous parameter mapping; Tanaka – biosignal analysis and mapping that adapt to changing corporeal states. Interactivity: Sonami – primarily reactive micro-gestural control; Tanaka – adaptive co-adjustment between performer and system. Audience role: both centre the performer as the primary agent with audiences as listeners to evolving sound fields (Tanaka Reference Tanaka2016; Fiebrink and Sonami Reference Fiebrink and Sonami2020).
Data & care: Both systems expose intimate bodily traces; EMG especially raises issues of vulnerability and the long-term circulation of recordings beyond the immediate performance context. Listening & attention: Each instrument script close, situated listening – audiences track subtle couplings between micro-gesture or muscular effort and timbral/spatial change. Agency & participation: Sonami configures a tight performer–instrument loop foregrounding long-term bodily attunement, while Tanaka redistributes authorship across body, sensors and processing as performer and system co-adapt (Tanaka Reference Tanaka2016; Fiebrink and Sonami Reference Fiebrink and Sonami2020).
Compared to Sonami’s reactive micro-gestural mapping, Tanaka’s EMG setup shifts more responsibility into managing noisy, fluctuating biosignals, making adaptation and shared authorship central to performance.
4.2. Machine-learning-based vocal system: Herndon/Spawn
Holly Herndon’s PROTO features real-time collaboration with a neural network ‘Spawn’ trained on a multi-voiced vocal corpus, enabling machine-generated vocal textures in performance (Herndon Reference Herndon2019).
Data type & source: vocal traces from human singers and recorded material. Mode of integration: machine-learning (ML) models that generate and transform vocal textures in real time. Sonically, PROTO’s ensemble writing pushes towards tightly clustered, often dissonant harmonies whose ‘mountainous’ nasal timbres are shaped through long-term rehearsal and Max/MSP (Max signal processing) pitch-shifting so that individual lines smear into a dense, electronically inflected choral mass. SampleRNN models trained on this material then generate additional vocal lines whose grainy, sometimes harsh continuations sit alongside the live singers, thickening the texture into a layered human–machine ‘choir’ in which timbral blend and moments of friction become central to the work’s aesthetic impact (Herndon Reference Herndon2019). Interactivity: improvisatory/participatory co-creation between a human ensemble and a neural network. Audience role: audiences listen through layered human/machine voices as singers and data contributors share authorship with the model (Herndon Reference Herndon2019).
Data & care: The use of a choral dataset foregrounds questions of who contributes training data and how those traces are stored and reused as part of system behaviour. Listening & attention: The system scripts attention to perceptual parsing of human/machine vocal layers in real time. Agency & participation: Co-authorship is negotiated among singers and model outputs; instrumentality is reframed as a learning system rather than a fixed tool (Herndon Reference Herndon2019).
Relative to the gestural instruments above, the locus of adaptation shifts from the performer’s biosignals to the dataset and trained model, relocating some labour of care and authorship into data curation and mapping design (Herndon Reference Herndon2019).
4.3. Environmental and audience data: Mapping Meugang
Mapping Meugang draws on the Acehnese Meugang tradition, translating market scenes, cooking activities and communal eating into a staged digital work in which environmental recordings, Rapa’i percussion and audience participation are tightly interwoven. The piece structures Digital Audio Workstation (DAW) edited soundscapes and instrumental parts around Meugang’s seasonal and social rhythms so that sonic materials and performance actions jointly evoke communal memory and shared values of togetherness (Gusmanto and Denada Reference Gusmanto and Denada2023). Structurally, the work unfolds in two broad segments: an opening soundscape in which layered market ambience, clattering dishes, frying oil and shouted greetings gradually thicken around recorded Rapa’i patterns and DAW-based timbres, followed by more overtly musical passages where talempong-with-water, Rapa’i interlocking figures and other traditional idioms articulate shifting rhythmic cycles and dynamic swells. Across this arc, changes in density, spectral brightness and spatial focus track ritual time – for example, moving from dispersed, noisy crowd textures towards more centred, drum-led climaxes as the communal meal begins – so that pacing and mix decisions mirror the transition from preparation to shared celebration (Gusmanto and Denada Reference Gusmanto and Denada2023).
Data type & source: Mapping Meugang works with environmental sound recordings, ritual actions and staged interactions – market and cooking soundscapes, ensemble performance and the invitation for audience members to join the performers in sharing food on stage. Mode of integration: These materials are integrated through compositional structuring, layering, spatialisation and performance cues so that shifts in the soundscape correspond to specific Meugang activities and to changes in the performers’ and audience’s positions and actions. Interactivity: Interactivity is participatory rather than purely reactive: the work depends on live negotiation between musicians, the production team and the audience, and its form is completed through shared rituals of eating and social interaction during the performance. Audience role: Audience members move from being listeners to becoming co-participants in a re-enactment of Meugang, their presence and actions folding into the sonic and social texture of the piece.
Data & care: here, recorded environmental sound and staged actions are treated as culturally meaningful traces of land-based practice and communal history, with the work explicitly framed as preserving and revitalising Meugang’s social values rather than extracting ritual as neutral content. Listening & attention: the piece cultivates listening that follows connections between soundscape, embodied gesture and social interaction: to understand the work is to attend to how market noise, cooking, percussion and shared eating are tied to narratives of togetherness and mutual care. Agency & participation: authorship is distributed across composer, performers, community practices and environmental recordings; Mapping Meugang foregrounds co-creation within an existing ritual context rather than positioning digital tools as autonomous agents.
Read through the ethical-embodied framework; Mapping Meugang demonstrates how environmental and audience-related data can support culturally accountable co-creation, making relations of care, listening and agency explicit rather than implicit in the design of a data-embedded musical system (Gusmanto and Denada Reference Gusmanto and Denada2023).
4.4. Artist case study: data-embedded composition in 19-EDO
To ground the ethical-embodied framework in a concrete compositional practice, this subsection presents a situated case study from my own work with microtonality, machine collaboration and data-embedded instruments. In this context, the instrument is an assemblage of custom Python scripts, musical instrument digital interface (MIDI) hardware, a digital audio workstation and a multichannel loudspeaker setup. Parametric musical data – pitch classes in nineteen equal divisions of the octave (19-EDO), durations, velocities and structural patterns – circulate between code, hardware and diffusion as the primary material with which I compose.
My work with 19-EDO develops from a longer trajectory that includes 12-EDO, 24-EDO and Persian modal systems. Zone 19 (Balighi Reference Balighi2025) extends methods first tested in my earlier album Odyssey in Soundscape but does so within an ongoing, experimental dialogue between my own decision-making and the behaviour of computational tools. Rather than treating 19-EDO as a purely abstract tuning system, I approached it as a practical problem: how to rethink melody, harmony and rhythm when the octave is divided into 19 equal steps and how to make those intervallic resources perceptible in a way that feels musically coherent to me in multichannel space.
Adapting to 19-EDO required me to reassess habits formed within 12-EDO and quarter-tone writing. Familiar tonal expectations did not translate directly; instead, new harmonic and melodic relationships emerged from the specific spacing of the 19 steps. Artificial intelligence and coding entered here as tools for managing the volume and complexity of possible material rather than as sources of autonomous invention. I designed custom Python scripts that generate and transform melodic and harmonic ideas inside the 19-EDO framework under explicit constraints that I define in advance (permitted intervals, registral ranges, and rhythmic densities).
Practically, the workflow centres on generating MIDI sequences from these parameter sets and then iteratively filtering and editing the outputs so that they align with my aesthetic aims. Algorithmic processes are used to explore intervallic patterns and harmonic configurations that would be slow to discover manually, but the selection, modification and placement of this material remain my responsibility. The system proposes options; I accept, reject or reshape them according to how they function within the emerging form. Crucially, the underlying operations of each script are simple, inspectable and bounded by my chosen constraints, which allows me to understand how patterns arise and to revise the code when they drift away from what I hear as musically meaningful.
A further aspect of the project involves connecting these scripts to external MIDI hardware for rapid, real-time auditioning of ideas. By mapping numerical data for pitch, duration and velocity directly to playable instructions, I can move quickly between code and sound, testing how abstract parameter sets translate into the perceptual space of 19-EDO. This loop between scripting, listening and adjustment has become a central part of my compositional method, confirming that many decisions – such as which intervals feel stable, which chordal densities work or how registral spacing interacts with multichannel diffusion – can only be resolved through repeated listening rather than through algorithmic design alone.
Data type & source: internally generated parametric musical data (19-EDO pitch classes, durations, and dynamics) produced by custom Python scripts under composer-defined constraints. Mode of integration: algorithmic mediation, where scripts generate and transform musical material that is auditioned, edited and arranged within a DAW and diffused in a multichannel system. Interactivity: an iterative composer–system feedback loop, in which successive script runs respond to insights gained from listening sessions. Audience role: non-interactive multichannel listening; audiences encounter a fixed work whose textures and spatial behaviours reflect the earlier composer–machine dialogue.
Read through the ethical-embodied framework; the project highlights how data and algorithms can be framed within a tightly bounded, accountable practice. Data & care here concerns the transparency and constraint of algorithmic processes: scripts operate on parameters I define, do not draw on external biometric or community data and are deliberately kept simple enough to be understood and revised. Any risk associated with the data falls primarily on me as the composer, which keeps questions of consent and sovereignty relatively contained but also underlines how easily such tools could operate very differently if fed with other forms of data or deployed in other institutional settings.
Listening & attention is foregrounded by the reliance on iterative audition and spatial experimentation to determine which intervallic densities and timbral configurations render the 19-EDO space perceptually coherent. Composing in this environment involves training my own ear to hear 19-EDO relationships and using timbre and spatialisation to make those relationships legible to listeners who may have no prior experience with the tuning system. Musical meaning emerges not from the tuning system alone but from historically situated interactions between my body, the scripts, the hardware and the acoustics of the multichannel space.
Agency & participation in Zone 19 are co-agentive but not symmetrical. Computational tools propose possibilities within a domain that I delimit; the assemblage of scripts, MIDI hardware and multichannel diffusion functions as an active partner in shaping the material; yet authorship remains anchored in my ongoing critical listening and structural decision-making. In this sense, the project exemplifies an ethical-embodied design in which data-driven systems extend pre-existing compositional concerns – here, microtonality and spatial sound – without erasing aesthetic intent or obscuring the cultural and ethical stakes of algorithmic collaboration.
5. Discussion: what the ethical-embodied framework buys you
5.1. Cross-case patterns
Across the case studies, the ethical-embodied framework helps surface recurring patterns in how data, bodies and instruments are configured that would remain disparate under ad hoc description. By reading each system through the framework’s dimensions and comparative axes, the analysis foregrounds questions of who is sensed, how they are sensed and who can ignore or reshape those data streams (Whalley Reference Whalley2015).
Across cases: (i) gesture-tracking works centralise performer agency through predominantly reactive mappings, with audiences remaining largely observational. (ii) EMG (electromyography)/BCI (brain–computer interfaces) pieces shift data sovereignty and risk toward intimate biosignals; even when interactivity looks similar, consent and care thresholds are higher. (iii) Audience-sensing installations redistribute authorship and evaluation toward collectives; audiences move from observing to co-structuring outcomes. (iv) ML-mediated vocal systems (e.g., PROTO) reallocate key decisions into dataset and model design; co-creativity hinges on who curates, labels and tunes models. Similar sensors and models can therefore support very different arrangements of responsibility and co-agency depending on cultural and institutional embedding, not interface novelty. Read through the framework; the contrasts are legible along data & care (who is sensed and with what protections), listening & attention (what modes are cultivated) and agency & participation (who can intervene, refuse or remap). These contrasts align with the comparative axes and the evaluation criteria in Section 5.2 for assessing design and outcomes across cases.
Across the cases discussed here, a recurring pattern emerges in which biometric, affective and environmental streams are treated not as neutral raw material but as sensitive traces embedded in relations of consent, vulnerability and sovereignty. In biosignal-based performance and inclusive interfaces for d/Deaf and disabled musicians, the framework highlights how access is often tied to questions of exposure – who owns intimate data, how long it circulates and whether it serves extractive spectacle or ongoing support (Duarte-García and Sigal-Sefchovich Reference Duarte-García and Sigal-Sefchovich2019; Wexler et al. Reference Wexler, Yip, Lee, Li and Wong2023). In globally situated works such as Mapping Meugang and drone-based ecological monitoring, the same lens shows how data can function as a vehicle for ritual memory, land-based knowledge and sonic sovereignty, challenging assumptions that sensor infrastructures default to Western design values (Sterne and Razlogova Reference Sterne and Razlogova2021; Gusmanto and Denada Reference Gusmanto and Denada2023).
The listening & attention dimension draws disparate practices into a shared field with feminist and decolonial listening frameworks, showing how instruments and mappings invite hungry, detached or accountable modes of audition (Blackburn Reference Blackburn2011). Audience-responsive installations such as telematic and biometric environments and ritual-inflected systems such as Mapping Meugang all redistribute attention across sound, interface and environment, but the framework lets us distinguish between works that merely capture behavioural traces and those that cultivate deep, positional and culturally specific listening (Gusmanto and Denada Reference Gusmanto and Denada2023). In this view, listening becomes a key site where the ethics of sensing architectures and algorithmic mediation are made audible, rather than a background assumption about ‘engagement’ (Freitas et al. Reference De Freitas, Rousell and Jäger2019).
Finally, tracking agency & participation across cases draws out how creative labour and authorship are distributed among performers, designers, audiences and computational processes. Gesture-tracking instruments, EMG- and BCI-driven systems, audience-sensing installations and ML-mediated works such as PROTO can then be compared not simply as ‘interactive’ or ‘innovative’, but in terms of who can intervene in mappings, who sets evaluative criteria and whose labour remains invisible in dataset curation, annotation and maintenance (Rebelo et al. Reference Rebelo, Renaud and Davis2007; Barrett et al. Reference Barrett, Creech and Zhukov2021). In contrast to ad hoc description, the framework therefore illustrates how similar technical ingredients – sensors, models, spatial sound – may sustain different arrangements of responsibility, co-agency and risk, depending on how they are embedded in social and institutional contexts (Schacher Reference Schacher2022).
5.2. Implications for design and evaluation
If adopted explicitly, the ethical-embodied framework would push design briefs for data-embedded instruments to specify from the outset whose bodies, environments and practices are being sensed, what forms of consent and data sovereignty are in place and how relationships of care will be sustained beyond the moment of performance (Loke Reference Loke2019). Rather than centring novelty of interface or technical complexity, briefs would describe data types and sources, modes of integration and intended levels of interactivity in terms that acknowledge cultural embedding and potential surveillance logics, especially when working with biometric, affective or community-derived data (Açıkyıldız Reference Açıkyıldız2024). This aligns compositional and engineering decisions with a prior recognition that data-embedded instruments are already part of wider infrastructures that organise listening, labour and visibility (Huo Reference Huo2021; Sterne and Razlogova Reference Sterne and Razlogova2021).
Evaluation criteria, in turn, could be further reoriented from purely aesthetic or usability metrics toward the three dimensions of data & care, listening & attention and agency & participation. Assessing a biosignal instrument or audience-responsive installation would involve asking not only whether interaction is compelling, but also whether intimate data are handled as relational and sensitive, whether listening is framed as accountable and situated and whether agency is genuinely shared with performers, audiences and communities rather than concentrated in opaque technical systems (Diac et al. Reference Diac, Damian, Butincu, Knieling and Iliescu2023). For ML-based projects, this implies evaluating the provenance, curation and labelling of training corpora, the visibility of human labour in tuning and maintenance and the extent to which models reproduce or resist hegemonic aesthetics (Šuvaković Reference Šuvaković2019).
Documentation practices would likewise shift, extending beyond patch diagrams and signal flow to include rationales for data choices, consent procedures, mapping strategies and accessibility considerations (Boutard and Guastavino Reference Boutard and Guastavino2012a; Huriye Reference Huriye2023). Ethics-driven prototyping and teaching labs already gesture toward this mode of documentation by asking students and practitioners to record how instruments distribute attention, responsibility and access across institutional and community settings (Tejada et al. Reference Tejada, Murillo and Mateu-Luján2023). Within the proposed framework, such documentation becomes part of the artistic work rather than ancillary paperwork, situating each data-embedded instrument as a concrete arrangement of care, listening and agency that can be examined, revised and held accountable over time (Boutard and Guastavino Reference Boutard and Guastavino2012b; Battier Reference Battier2015). Methodologically, this reframes comparison away from interface novelty toward explicit distributions of care, attention and participation, making differences auditable across cases even when the same sensors/models are in play.
6. Conclusion
This article has developed an ethical-embodied framework for analysing data-embedded instruments and practices in electroacoustic music, showing how, in the selected cases, instruments are treated as responsive mediators within specific sensing/mapping configurations. Taken together, these genealogies, conceptual dimensions and vignettes suggest that the selected data-embedded instruments participate in longer trajectories of electroacoustic practice and make visible specific shifts in how music can be conceived, produced and understood in specific electroacoustic contexts. At the heart of this shift lies a broader theoretical contribution: an ethical-embodied design model that integrates insights from data ethics, feminist and decolonial sound studies and global technological practice, reframing data as an expressive, relational and politically charged material rather than a neutral input or medium. By ‘model’ I refer to the combination of the three dimensions – data & care, listening & attention and agency & participation – with the comparative axes (data type/source, mode of integration, interactivity, audience role) and their application to design, evaluation and ML contexts.
Alongside this, the comparative axes – data type and source, mode of integration, level of interactivity and audience role – have functioned as a consistent grid for situating each case in terms of how data, bodies and audiences intersect, keeping attention on concrete configurations of sensing, mapping and participation rather than on abstract categorisation. The vignettes on gestural and biosignal instruments, ML-based vocal systems and environmental or audience-responsive works show how similar technical ingredients can support very different arrangements of embodiment, care and agency, depending on how they are embedded in specific cultural and institutional contexts (Waters Reference Waters2007). At the same time, the article does not claim to offer a unified taxonomy of data types, formal roles or pedagogical models; earlier sections explicitly warn that such taxonomies risk erasing the very diversity of practices that the framework aims to protect and position the axes as heuristic tools for comparison rather than prescriptive or exhaustive categories (Lewis Reference Lewis1996). The case materials and artist-focused discussion are therefore best read as situated and illustrative, rather than as comprehensive coverage of data-embedded electroacoustic practice.
Looking forward, the article identifies several priorities for future research and practice. First, there is a need to develop robust ethical protocols for biometric, emotional and participatory data in creative systems, addressing consent, transparency and strong protection against misuse as sensing infrastructures increasingly draw on intimate bodily and behavioural traces. Second, interdisciplinary methods such as ethnographic user testing and participatory design can help centre performer and audience experience, bridging critical theory and system design so that technological innovation is matched by inclusion, cultural awareness and reflexivity. Finally, as machine learning becomes more deeply embedded in musical tools, future work must continue to examine how training data, annotation practices and model behaviours encode particular aesthetic and social values and how ethical-embodied frameworks can support more inclusive and reflexive data-driven musical futures by treating sound technologies as systems of meaning shaped by and shaping cultural context.
Acknowledgements
I am grateful to Dr Christopher J. Smith for his careful proofreading of this article. I also thank Dr Hideki Isoda and Dr John Boyle for their invaluable support of my research. Finally, I would like to express my deepest gratitude to my wife, Leila Mirzaei, for her constant support of my research and creative work. All remaining mistakes and interpretations are solely my responsibility.
Open access
