2.1. The origins of harmony in music

Accounts of musical consonance trace back to Pythagoras, who postulated that the consonance of musical intervals is determined by the simplicity of their frequency ratios (Ferguson Reference Ferguson2011). Euler proposed a system grading chord aesthetics based on the least common multiple of the notes (Euler Reference Miranda2006), while harmonicity of specific scales has been quantified using the averaged similarity between each pair of notes and the natural harmonic series (Gill and Purves Reference Gill and Purves2009). Historically, two main schools of thought have shaped our understanding of harmony. The Pythagorean school emphasises temporal features, deriving consonance from wavelength ratios. In contrast, the Helmholtzian school adopts a psychoacoustic lens, emphasising the role of spectral features in auditory harmony. Helmholtz introduced the notion of beating frequencies, denoting the perceived amplitude modulations resulting from the interaction of two closely spaced frequencies, where increasing modulation frequency leads to perceived discordance – at least up to around 20 Hz of beating frequency, beyond which the tones are perceived as two distinct notes (Rasch Reference Rasch1984).

Bridging these viewpoints, Chan et al. (Reference Chan, Dong and Li2019) offered a holistic model of harmony, encapsulating stationary and transitional aspects through inter-harmonic and subharmonic modulations. They demonstrated that such modulations could account for both perceptual tension–resolution and computational harmony statistics, reconciling the Helmholtzian and Pythagorean paradigms. Their contributions pave the way for mathematical frameworks of transitional harmony, aimed at computing tension–resolution trajectories.

Building on harmonic structures, musical tuning systems – dividing octaves into sets of frequency ratios – have evolved to balance harmonicity with practical performance needs. The Pythagorean scale, rooted in the 3/2 ratio (the fifth), expresses harmony through integer ratios in the form of 2^q3^p. Just intonation refined this system by incorporating the prime 5, yielding ratios of the form 2^q3^p5^r, for example simplifying the Pythagorean major third from 81/64 to 5/4. (Fauvel et al. Reference Fauvel, Flood and Wilson2003). However, these tuning systems, dominant until the 18th century, struggled to accommodate key changes, significantly slowing the evolution of new musical genres. Equal temperament resolved this by dividing the octave into 12 equal steps, each corresponding to a frequency ratio of 2^1/12, enabling transposition at the cost of interval accuracy (Barbour Reference Barbour1947).

More recently, computers revolutionised musical tunings by enabling dynamic adjustments for precise intervals without sacrificing transposition (Sethares Reference Sethares1994, Reference Sethares2002). This computational capability has inspired innovative tuning methods, such as the dissonance curve and harmonic tuning, often explored in microtonality and diverse cultural contexts (Werbock Reference Werbock2011; Benetos and Holzapfel Reference Benetos and Holzapfel2015; Ader Reference Ader2020).

Sethares (Reference Sethares2005) connected timbre (i.e., the unique quality of a sound, defined by its harmonics) to tuning. The dissonance curve, rooted in Helmholtz’s beating frequencies concept, maps perceived dissonance across ratios, aligning timbre and tuning. For example, a flute’s harmonic spectrum yields simple ratios – and therefore a corresponding tuning close to just intonation – while a bell’s inharmonic spectrum reflects complex ratios – and therefore inharmonic tuning. This approach allows perceptually dissonant intervals to become consonant when matched to a compatible timbre, making it a valuable tool for deriving adaptive tuning systems from brain signals.

Complementing this timbre-based approach, harmonic (overtone-derived) tunings establish frequency ratios within the unison (1:1) and octave (2:1) range by iteratively dividing harmonic positions, for example, 3:1 or 5:1, by 2 ⁿ , where n represents the number of octaves. This process uncovers simple ratios inherent in classical tunings, as illustrated by the 8th Octave Overtone Tuning, which spans 128 harmonics across 8 octaves (Reinhard Reference Reinhard2011).

These methods, based on spectral morphology and harmonic positions, offer bridges between musical formalism and biological rhythms. Beyond providing tools for constructing EEG-informed tunings, we suggest that these musical formalisms offer a novel lens for characterising the harmonic organisation of brain dynamics.

2.2. Unveiling musical structure in dynamical systems

Before investigating harmonicity in brain dynamics, it is essential to consider prior efforts exploring dynamical systems in music composition. Recent advancements have leveraged the potential of simulated nonlinear dynamics, inspiring novel generative music systems.

Bell and Gabora (Reference Bell and Gabora2016) developed a generative musical algorithm operating ‘at the edge of chaos’, using hierarchical scale-freeFootnote ¹ topologies to produce highly creative outputs. As musical genres often exhibit scale-free behaviours (Levitin et al. Reference Levitin, Chordia and Menon2012), such systems open pathways to deconstruct conventional musical formalisms. Similarly, ecosystem-based generative music (Bown Reference Bown2009) and meta-generative approaches (Dahlstedt Reference Dahlstedt2009), both relying on evolutionary algorithms that simulate adaptation and interaction within multi-agent systems, use statistical properties of these agents’ nonlinear behaviours and ecological dynamics to generate complex, evolving musical outputs.

Project Santiago integrates biologically inspired neuron models with interactive music generation by mapping neural activity, including spikes and firing rates, to musical events such as notes and sound parameters (e.g., pitch, duration, intensity). This real-time system leverages the intrinsic dynamics of individual neurons and their network connectivity to produce complex, evolving sonic structures that maintain a cohesive interaction across temporal and musical scales (Kerlleñevich et al. Reference Kerlleñevich, Riera and Eguia2011). Complementing these algorithmic approaches, Trulla et al. (Reference Trulla, Stefano and Giuliani2018) revealed that the dynamical fingerprints of consonance and dissonance can be recovered directly via Recurrence Quantification AnalysisFootnote ² on continuous glissandi, suggesting a universal, data-driven framework for harmony grounded in dynamical systems theory.

Building on this line of work, we propose advancing algorithmic music systems by integrating real-time biosignal processing, particularly from the human brain. By mirroring the unfolding of harmonic structures in these neural dynamics, we create adaptive musical experiences that evolve in synchrony with human phenomenology (i.e., of auditory perception, at least).

3.1. Neuronal model of consonance

To extract musically relevant features from HABBOs, auditory processing of stimuli offers a meaningful lens through which we can examine the connections between auditory sensory input and the corresponding brain signals. The auditory system, with its finely tuned ability to process harmonic structures in sound, provides a natural example of how external harmonic stimuli are translated into neural patterns. This translation process reveals isomorphisms – or structural similarities – between the organisation of sensory inputs, such as musical tones, and the patterns of neural activity they evoke. By studying these parallels, we can uncover how harmonic relationships in sound are mirrored in the brain’s dynamic responses, providing a foundation for exploring harmonicity within neural processes. To clarify these structural parallels, Figure 2 presents a visual glossary aligning key concepts between neuroscience and music theory.

Figure 2.

Visual Glossary: Neuroscience & Music Parallels. A comparative illustration of the candidate structural isomorphisms between spectral representations of electrophysiological signals and harmonic representations in music theory. The figure aligns key concepts: (1) spectral peaks in the power spectrum correspond to harmonic partials in sound, (2) frequency ratios between neural oscillations mirror musical intervals, (3) the PSD shape (spectral envelope) is analogous to timbre and (4) ratio folding (nested biological oscillations) functions as the mathematical equivalent of octave equivalence.

A neuronal model of consonance highlights the role of the brainstem and primary auditory cortices in processing auditory stimuli (Bidelman and Krishnan Reference Bidelman and Krishnan2009; Lerud et al. Reference Lerud, Almonte, Kim and Large2014). It aligns with evidence of the brain’s sensitivity to musical pitch relationships (Fritz et al. Reference Fritz, Jentschke, Gosselin, Sammler, Peretz, Turner, Friederici and Koelsch2009) and the encoding of consonant and dissonant intervals through phase synchronisation between stimulus and frequency-following response (FFR) – a sustained neural response in the brainstem that reflects the temporal and spectral properties of sound (Bidelman and Krishnan Reference Bidelman and Krishnan2009; Lee et al. Reference Lee, Skoe, Kraus and Ashley2015). Neural synchrony, reflecting the periodicity of musical intervals, is central to auditory processing (Boomsliter and Creel Reference Boomsliter and Creel1961; Lots and Stone Reference Lots and Stone2008) and has been shown to be stronger for consonant intervals compared to dissonant ones (Langner Reference Langner1992; Bones et al. Reference Bones, Hopkins, Krishnan and Plack2014, suggesting that consonance perception may arise from neural synchronisation (Lots and Stone Reference Lots and Stone2008).

A key question remains: does neural encoding of musical intervals merely mirror acoustic properties, or are nonlinear modulatory processes involved? Studies reveal nonlinear FFRs in the brainstem and auditory cortex (Galbraith Reference Galbraith1994; Pandya and Krishnan, Reference Pandya and Krishnan2004; Purcell et al. Reference Purcell, Ross (ß), Picton and Pantev2007; Lee et al. Reference Lee, Skoe, Kraus and Ashley2009), driven by mode-locking,Footnote ³ where periodic stimuli interact with neural oscillations to produce n:m locked cycles (Lerud et al. Reference Lerud, Almonte, Kim and Large2014). This supports the idea that low-order integer frequency ratios underpin consonance as a principle of stability in oscillatory systems (Heffernan and Longtin Reference Heffernan and Longtin2009; Klimesch Reference Klimesch2018). Hence, consonant and dissonant intervals predict brainstem FFRs, with consonant intervals prominently eliciting common subharmonics in FFR, interpreted as a physical manifestation of the missing fundamentalFootnote ⁴ (Lee et al. Reference Lee, Skoe, Kraus and Ashley2015). In contrast, dissonant intervals evoke a higher number of nonlinear resonances, indicating increased neural complexity. Musicians, compared to non-musicians, exhibit greater nonlinearity in interval encoding, reflecting neural plasticity and subjective expertise in musical perception (Lee et al. Reference Lee, Skoe, Kraus and Ashley2009, Reference Lee, Skoe, Kraus and Ashley2015).

These findings highlight the brain’s capacity for nonlinear processing, where neural ‘filling-in’ mechanisms (Bregman Reference Bregman1994; Varela, Reference Varela1999; Cervantes Constantino and Simon, 2017) associated with ambiguous sound perception might shape the tension–resolution patterns characteristic of musical progressions. Collectively, these insights underscore that consonance and dissonance are encoded in the brain through phase synchronisation, mode-locking, and nonlinear dynamics. This highlights a deep connection between musical perception and oscillatory brain dynamics, suggesting that the coordination of frequencies in brain–body oscillations provides a fertile ground for exploring novel musical structures as much as their corresponding experiential dimensions.

3.2. Theories of harmonicity in brain dynamics

Two complementary theories, the Binary Hierarchy Brain–Body Oscillation theory (which proposes HABBOs) and the General Resonance Theory of Consciousness (GRTC), suggest that harmonic relationships in brain dynamics not only underlie auditory perception but also support physiological coordination and brain state transitions with corresponding shifts in lived experience.

The Binary Hierarchy Brain–Body Oscillations theory (Klimesch, Reference Klimesch2018) emphasises the role of phase synchronisation and harmonicity, particularly binary (2:1) frequency ratios, in coordinating physiological systems and cognitive processes across timescales (Klimesch Reference Klimesch2013). This framework draws a compelling parallel to octave-based musical structures, where frequency doubling or halving creates harmonious relationships. Similarly, these binary ratios in neural dynamics minimise spurious synchronisation and enhance communication across distinct frequency bands, supporting efficient integration of brain processes. Klimesch further emphasizes that the architecture is shaped by principles of optimal phase decoupling to reduce interference between frequency domains. Ratios near the golden mean (≈1.618) are proposed to support maximal separation between oscillatory bands. Empirical evidence further supports this theory, showing that EEG harmonic frequency ratios dominate during alert states, while non-harmonic ratios prevail during sleep (Rassi et al. Reference Rassi, Dorffner, Gruber, Schabus and Klimesch2019). Cross-frequency coupling (CFC) techniques,Footnote ⁵ such as phase-amplitude coupling (PAC) and cross-frequency synchrony (CFS), are instrumental in studying how neural frequencies interact to coordinate brain processes at multiple scales (Lisman and Idiart Reference Lisman and Idiart1995; Jensen and Colgin Reference Jensen and Colgin2007; Hyafil et al. Reference Hyafil, Giraud, Fontolan and Gutkin2015). These methods highlight the importance of harmonic relationships in integrating and regulating distributed neural activity, while recent advances have improved the ability to distinguish genuine coupling from spurious effects caused by non-sinusoidal rhythms (Idaji et al. Reference Idaji, Zhang, Stephani, Nolte, Müller, Villringer and Nikulin2022).

Building on these observations of cross-frequency coordination, the GRTC (Hunt et al. Reference Hunt, Schooler and Lane2019) proposes that harmonic interactions among neural oscillators underlie the integration processes that give rise to unified conscious experience. It posits that resonance among coupled oscillators creates nested harmonic structures, optimising energy and information flow for inter- and intra-system coherence (Young et al. Reference Young, Hunt and Ericson2022). GRTC suggests that the harmonic architecture of spontaneous neural activity encodes information about state transitions, with the slowest shared resonance serving as an indicator of subsystem communication (Hunt Reference Hunt2020; Young et al. Reference Young, Hunt and Ericson2022). In this account, networks of coupled oscillators operate near criticality,Footnote ⁶ balancing stability and adaptability (O’Byrne and Jerbi Reference O’Byrne and Jerbi2022), with local resonances potentially driving phase transitions in brain state dynamics. Temporal structures in brain activity, as revealed through rhythmic and harmonic patterns in fMRI studies, provide a foundation for understanding how the brain constructs a world that is simultaneously stable and dynamic, resembling the temporal nature of music (Lloyd Reference Lloyd2020).

HABBO and GRTC converge on the role of harmonic structures as markers of physiological and conscious states. While HABBO emphasises binary harmonic ratios for coordination across scales, GRTC situates these harmonic dynamics within the framework of conscious state transitions. Collectively, these theories offer a robust framework for connecting neural harmonicity to the design of adaptive musical systems. In the following sections, we explore how these theoretical insights inform new sonification techniques and guide the development of real-time feedback systems grounded in harmonic audification.

5.1. Computation of spectral peaks in the framework of harmonic analysis

Traditional methods of frequency analysis in neuroscience categorise brain rhythms into fixed frequency bands (delta, theta, alpha, beta, gamma) based on empirical evidence (Berger, 1929; Teplan Reference Teplan2002; Cohen Reference Cohen2017). While effective, these static boundaries fail to capture dynamic shifts in brain rhythms, such as under altered states of consciousness, where alpha peak frequencies can deviate significantly from the standard 8–12 Hz range (Mierau et al. Reference Mierau, Klimesch and Lefebvre2017).

EMD offers a data-driven alternative by decomposing signals into intrinsic mode functions (IMFs)Footnote ⁷ through a sifting procedureFootnote ⁸ (Rilling et al. Reference Rilling, Flandrin and Goncalves2003). Acting as a dyadic filter bank, EMD isolates components in a log₂ relationship, aligning with octave-based models of brain oscillations (Klimesch Reference Klimesch2018) (see Figure 4). Recent studies using EMD in MEG analyses highlight its precision over traditional time-frequency methods (Skiteva et al. Reference Skiteva, Aleksandr, Vadim, Denis and Boris2016). By computing the Welch transformFootnote ⁹ on each IMF and identifying frequency peaks with maximal power, EMD reveals a detailed spectral hierarchy suited for analysing HABBOs.

Figure 4.

Peaks extraction based on Empirical Mode Decomposition (EMD). Power spectrum density plot of five intrinsic mode functions (IMF; blue) and global signal (pink). The bin with maximum power is selected as a peak and compared to classical frequency bands: Delta (1–3Hz), Theta (3–7Hz), Alpha (7–12Hz), Beta (12–30Hz), Gamma (30–60Hz). Stars (*) beside IMFs in the legend mean that the peak falls within classical frequency band.

We propose a complementary approach, termed harmonic recurrence, which identifies spectral peaks by analysing their recurrence within the harmonic series of the signal. Using the Welch transform, this method extracts all spectral peaks and compares their harmonics across the spectrum. Peaks with the highest recurrence – those whose harmonics overlap with other peaks – are selected, reflecting their embeddedness across spectral scales. This method also quantifies inter-harmonic concordance, capturing alignment between harmonics of distinct peaks (e.g., Figure 5).

Figure 5.

Peaks selection using harmonic recurrence method. Welch transform has been computed on a single time series to derive the power spectrum density. Then, peaks are identified using scipy find_peaks. A pairwise comparison is done to determine if a peak is a harmonic of another. Selected peaks (solid lines) are peaks having the highest number of harmonics (dashed lines) as other peaks of the spectrum. Numbers on dashed lines correspond to the harmonic positions. Hence, the blue peak has its 15^th and 18^th harmonics as other peaks, the yellow peak has its 3^rd and 11^th harmonics as other peaks, while the 11^th harmonic of the yellow peak and the 18^th harmonic of the blue peak coincide.

The harmonic recurrence approach is inspired by the harmonic product spectrum,Footnote ¹⁰ a pitch-tracking method that identifies recurrent peaks across downsampled versions of the same signal (Sripriya and Nagarajan Reference Sripriya and Nagarajan2013). By focusing on harmonic alignment and embeddedness, this method provides a nuanced view of spectral architecture, enabling the identification of key frequencies that resonate across multiple scales. Together, EMD and harmonic recurrence offer robust, data-driven frameworks for analysing HABBOs, laying the groundwork for exploring how brain oscillations reflect physiological rhythms and inform adaptive musical systems.

5.2. Neural tunings as embedding of brain harmonic structures

EEG spectral peaks and their associated frequency ratios reflect both functionally and musically relevant information that can be used to derive tuning systems. We apply two methods to derive tunings from brain spectral peaks and amplitudes.

Building on Sethares’ demonstration of the isomorphism between timbre and tuning (Sethares Reference Sethares2005), dissonance curves are derived from brain spectral peaks and their amplitudes, features that can be understood as the brain’s spectral signature, functionally analogous to musical timbre. Dissonance values are calculated for every pair of peaks, representing how harmonicity varies across frequency ratios. The resulting curve highlights dissonance changes over a range of intervals, allowing identification of local minima that correspond to consonant tunings. Figure 6 illustrates dissonance curves from EEG sensors in the occipital region, with local minima compared to the 12-step equal temperament (12-TET). Notably, shared minima, such as the ratio 9/7 across multiple sensors, indicate converging harmonic structures. These curves reveal tunings reflective of the signal’s timbral structure, serving as a creative tool for musical exploration.

Figure 6.

Dissonance curves from multiple EEG electrodes. Each coloured line corresponds to the dissonance curve of one electrode based on spectral peaks derived using EMD. Grey and red vertical lines represent the dissonance local minima shared between at least two EEG dissonance curves and of 12-TET equal temperament, respectively.

Another approach derives a set of frequency ratios from the harmonic positions identified using the harmonic recurrence method, inspired by the 8th Octave Overtone Tuning (Reinhard Reference Reinhard2011). Harmonic positions are iteratively divided by 2 until ratios fall within the range of unison (1:1) to octave (2:1) (see Equation 1):

(1) $${R_i} = {{{H_i}} \over {{2^n}}}$$

where R is the resulting ratio, H is the harmonic position, and n is the number of divisions required.

Both methods offer opportunities for music creators to explore dynamic tuning systems informed by real-time biological processes.

5.3. Time-resolved and transitional harmony in brain oscillations

Before discussing the broader implications of the methods presented, we introduce sonification methods designed to capture stationary and transitional harmony in brain dynamics. Transitional harmony refers to the dynamic process of resolving perceptual tension between successive harmonic structures, as opposed to stationary harmony, which describes tension within a single chord or harmonic moment (Chan et al. Reference Chan, Dong and Li2019). We first quantify stationary harmony over time in brain signals by deriving spectral chords that represent consecutive moments of harmonicity within a single time series (Figures 7 and 8A). We extract instantaneous frequency (IF)Footnote ¹¹ information by decomposing the signal into IMFs using EMD and then applying the Hilbert transform.Footnote ¹² Harmonicity (e.g., harmonic similarity; (Gill and Purves Reference Gill and Purves2009)) is computed among all pairs of IFs at each timepoint, identifying spectral chords when the harmonicity exceeds a predefined threshold. This approach captures the temporal evolution of harmonic structures within brain dynamics, offering valuable insights for dynamic musical systems.

(2) $$Subharmonic{\mkern 1mu} Tension{\mkern 1mu} \left( {ST} \right) = {1 \over N}\sum\limits_{i = 1}^N {\left( {{{\Delta {t_i}} \over {Su{b_i}}}} \right)} $$

Figure 7.

Identification of spectral chords using time-resolved harmonicity. The top panel represents the instantaneous frequencies (IFs) of each IMF, with dashed lines corresponding to moments of high harmonic similarity between all pairs of IFs. The bottom panel illustrates the corresponding musical notation of the identified spectral chords.

Figure 8.

Transitional (sub)harmony using instantaneous frequencies of intrinsic mode functions. (A) Spectral chords based on harmonic similarity threshold (as in Figure 7). For every point in time, harmonic similarity was computed on each pair of instantaneous frequencies among the five IMFs. When the average harmonic similarity exceeded a value of 20, a spectral chord was identified, corresponding to dashed grey lines. (B) Transitional subharmonic tension representing the level of subharmonic congruence between two successive sets of frequencies. Each set of frequencies corresponds to instantaneous frequencies (IFs) of intrinsic mode functions (IMFs) at a specific moment in time. Congruent subharmonics are identified with maximum distance thresholds set to 25ms (yellow), 50ms (red) and 100ms (blue). Dotted black lines are moments of high harmonic similarity between the peaks of a single set of frequencies (stationary harmony).

To extend this framework, the concept of subharmonic tension is introduced to quantify dissonance or stability across successive sets of spectral peaks derived from IFs, offering a way to quantify transitional harmony in biosignals. The metric, defined in Equation 2, reflects temporal irregularities among subharmonics within an octave span and provides a means to visualise fluctuations in tension–resolution patterns over time (Figure 8B). Notably, periods of high stationary harmonicity often correspond to transitions in tension and resolution, highlighting the interplay between static and dynamic aspects of harmonicity in brain oscillations.

Together, the methods presented in this section provide a robust computational foundation for exploring the harmonic architecture of brain dynamics. By bridging empirical analysis with creative applications, they potentially enable novel pathways for integrating neurophenomenology into adaptive musical systems.

5.4. Biotuner Engine

To make these tools accessible, we created the Biotuner Engine, a user-friendly web application that allow users to upload any oscillatory time-series data (e.g., from EEG and heart rate recordings to audio or mobile sensor data) and automatically derive musical tunings from their harmonic architecture (Figure 9). Outputs include XML musical scores, MIDI tuning files and audio files that sonify the data as evolving chord progressions, enabling both scientific exploration and artistic experimentation. The underlying code is freely available through the Biotuner Python toolbox, which supports advanced customisation and integration into broader research or creative pipelines (Bellemare-Pepin & Jerbi, Reference Bellemare-Pepin and Jerbi2025). Complementing this, the Biotuner Engine provides an immediate, no-code interface for exploring the sonic potential of biological rhythms. To demonstrate the aesthetic outcomes of these processes, we have established the Biotuner Archive, a dedicated companion repository that showcases a growing collection of musical and sonic outputs created by the authors and external collaborators, with the aim of becoming an online community-driven platform of biological music sharing. Both the Biotuner Engine and Biotuner Archive are available at https://biotuner.kairos-hive.org.

Figure 9.

Harmonic analysis of brain signals using Biotuner Engine. This interface enables users to upload time-series data and extract harmonic structures from selected time intervals, visualised in red against the broader EEG waveform in blue. The tuning analysis results panel displays harmonic ratios derived from spectral peaks, including integer-based interval names (e.g., ‘Thirty-fifth Harmonic’), tuning consonance scores and a dissonance matrix illustrating inter-ratio consonance. The matrix enables visual comparison of consonance across ratios based on pairwise harmonicity metrics.

6.1. BCIs and creative extensions

BCIs typically establish communication pathways between brain activity and external devices, translating neural signals into actionable outputs (Zander and Kothe Reference Zander and Kothe2011). Traditionally employed in clinical or performance-enhancement contexts, BCIs often incorporates neurofeedback – real-time information about neural activity – to support self-regulation, emotional modulation, or cognitive enhancement. Broadly, these systems fall into passive and active categories: passive BCIs monitor spontaneous neural activity without user intentionality, whereas active BCIs respond explicitly to user-driven neural modulation.

Extending beyond these paradigms, we propose the concept of a creatice BCI (cBCI). Distinct from traditional BCIs, a cBCI facilitates a generative, reciprocal dialogue between sensory feedback and endogenous neural activity (Tholke et al. Reference Tholke, Bellemare-Pepin, Harel, Lespinasse and Jerbi2024). In this context, cBCIs move beyond passive mappings or control systems towards engines of creative exploration. Sensory feedback loops amplify the coupling between external perception and self-awareness/interoception. In this recursive dynamic, the act of sensing and perceiving becomes an act of creation itself, as the observer’s phenomenological response directly drives the evolution of the generative output. This process aims to dissolve traditional boundaries between subject (the perceiver) and object (the external sensory stimuli), moving towards a more relational understanding of music and creativity in general. As a framework, cBCIs unite real-time neural data streams, adaptive generative algorithms, and participatory phenomenology. This provides a systematic scaffold for art-technology experiments aimed at designing interactive experiences as cyber-ecosystems, defined here as physical and virtual spaces that integrate multimodal sensory signals and feedback loops across biological and technological domains (Friston et al., Reference Friston, Ramstead, Kiefer, Tschantz, Buckley and Albarracin2024; Lewis et al., Reference Lewis, Arista, Pechawis and Kite2018, Reference Lewis, Abdilla, Arista, Baker, Benesiinaabandan and Brown2020; Taylor et al., 2024). Crucially, informed by Indigenous ontologies and perspectives on more-than-human relationality, such as those articulated by (Lewis et al., Reference Lewis, Arista, Pechawis and Kite2018), these ecosystems can be envisioned as more-than-human-controlled environments, becoming spaces where living and computational systems might co-evolve and co-create as reciprocal partners, or even kin. This view fundamentally challenges the idea that music is human-centric, while shifting away from treating technology solely as a tool and aligning with the goal of ubimus of providing open-ended, participatory forms of creative engagement (Mori, Reference de Mori2017). Moreover, such a systematic understanding of music also seeks to provide empirical methodology for dynamically aligning first-person subjective experiences with third-person neuroscientific data (Roy et al., Reference Roy, Petitot, Pachoud, Varela, Petitot, Varela, Pachoud and Roy1999; F. J. Varela et al., Reference Varela, Rosch and Thompson1991), enabling richer phenomenological insight through embodied artistic interaction.

6.2. Implementation of cBCIs

Our proposed implementations of cBCIs leverage harmonic audification, a novel sonification methodology based on harmonic structures extracted from real-time brain dynamics. Rather than imposing predefined mappings or arbitrary associations, harmonic audification directly translates the intrinsic harmonic architecture of brain–body oscillations into adaptive sensory experiences. Concretely, neural spectral peaks and harmonic relationships dynamically generate personalised soundscapes or visual environments.

For instance, cBCIs can transform a musician’s brainwaves into real-time microtonal tunings mapped onto MIDI instruments. This enables musicians to improvise with their biological rhythms, directly interacting with the emergent harmonies of their own neural patterns (see Figure 10). Additionally, well-established neural markers of cognitive and emotional states (e.g. focused attention, meditative state, or creative flow states) could dynamically steer the generative process of neural soundscapes, tailoring auditory outputs to reflect shifts in cognitive or emotional states. By embedding these markers into the feedback loop, the system could adapt its harmonic trajectories to support or enhance desired mental states.

Figure 10.

Schematic representation of a creative brain–computer interface. This diagram illustrates the integration of harmonic analysis and audification, as well as the associated microtonal ‘bio-tunings’, within a creative brain–computer interface (cBCI). Neural data undergo harmonic analysis to extract spectral features from which the system derives adaptive soundscapes through harmonic audification. Alternatively, a MIDI keyboard can be tuned according to the current HABBOs. Passive feedback is represented by the auditory perception of harmonic audification, while active feedback comprises both user-driven interaction via bio-augmented musical improvisation and system-initiated modulation of the harmonic audification.

More broadly, harmonic trajectories may inform personalised sensory experiences by projecting tension–resolution patterns of neural activity into evolving, immersive, and emotionally engaging auditory and visual feedback. In summary, cBCIs offer a platform for emergent forms of expression in which the interplay between harmonic neural processes and sensory feedback stimulates creative trajectories of mental states, inviting speculation about a future where posthuman language could be realised as a form of embodied musical communication between humans and machines (Dueck Reference Dueck2020).