1. INTRODUCTION
This article maps out relationships between performers and their instruments in improvised electronic music. In these practices, musical machines – be they computers, mechanical assemblages or the so-called ‘table full of shit’ (Bowers quoted in Richards Reference Richards2008) act as generators of musical material and sources of unpredictability with which to improvise.
We examine some roles these machines take on, or are imagined as taking on, during improvised performances. These are neither mutually exclusive nor always clearly delineated, and they do not exhaust the range of musical interactions available to us. Nonetheless, they prove useful for exploring dynamics of live and improvised electronic music and new musical instruments. This taxonomy can open us up to the flows of agency, emerging interaction possibilities and ‘more-than-human’ vitality in these technologically mediated musical practices.
Many of our sensibilities in this domain were shaped by the electronic and noise music that we encountered in the late 1990s and early 2000s in our formative years. These practices often involved performers working with diverse ‘ecologies’ of sound-making objects (Bowers Reference Bowers2002: 47), with significant elements of improvisation, but outside of more formally (and academically) conceptualised traditions of improvised music. These practices fall between and across generative music, noise music, live electronics, electronic dance music and free improvisation. However, they all involve some creative relationship with sound-making technologies that differs from traditional instrumental approaches.
The taxonomy we describe here is further informed by our own experiences as practitioners: improvising electronic music with our own musical machines. Our practices, on the surface, may seem more different than they are similar. With Mechanical Techno, Dunning works with sculptural assemblages of records, knitting needles, wires, balls, triggers and sensors that are driven by a turntable, and performances are physical, with the audience seeing the system being built up and musical processes being set in motion. This is used to create music that can be melodic and beat driven, textural and abstract, or somewhere in between. As Dane Law, Parkinson performs with a laptop running a Max patch, a relatively opaque situation for the audience, creating music that moves between plucked, pastoral pieces and glitchy, granular soundscapes through making small, continuous tweaks to the various semi-algorithmic processes he oversees.
Both systems are built around a musical pulse which we can disrupt and manipulate. Over the years, we have sought to develop systems that allow us to work with not only rhythm, repetition and melody, but also variation, disruption, malfunction and failure. Although we also collaborate with others, we are in a sense ‘one man bands’. As such, we have had to make musical machines that allow us to offload elements of the performance to the system so that we are not trying to do much. We each need these offloaded elements to avoid being too repetitive and boring, and we both enjoy being surprised by our machines creating new sounds, rhythms and riffs, giving us something to respond to, or behaving in a completely unanticipated manner. The taxonomy we describe in this article emerges in large part from thinking through the various interactions we have with these machines, and the different ways in which we imagine them, alongside our knowledge and interpretations of the work of other practitioners in the field.
2. ‘PLAYING WITH SOMETHING THAT RUNS’
The particular field of practice that relates to music-making through working with a machine or system which has a degree of autonomy is in some way self-acting or can produce musical sound without ongoing input. We want to explore the specific practice of improvising with such a system: intervening with the machine as it plays. The practice of improvising within loop-based and rhythmic material is discussed at length by Butler (Reference Butler2014) in Playing with Something That Runs: Technology, Improvisation, and Composition in DJ and Laptop Performance. Butler’s definition of the kinds of systems we are grouping together as ‘something that runs’ is helpful in understanding the extent to which there is some autonomy in the system itself. Left without intervention, the system continues to play music. This marks a clear break with ‘traditional’ instruments which in the majority of cases need continuous human effort to sustain sound.
Jensenius (Reference Jensenius2022) calls such systems ‘automatic’ and notes how they disrupt traditional musical categories, a case in point being the question of whether a DJ set is a musical performance. The key concern is that there is some scope for musical choices to be made” ‘Automatic instruments challenge our thinking about traditional musical roles. The users of such instruments can be thought of as perceiver-performers while the creators are a kind of “maker-composer-producer-performer”’ (Jensenius Reference Jensenius2022: 116). While, once started, music will play back regardless, the creative choices during the DJ performance itself come from selection of tracks to play, framing of each within a broader musical collage, consideration of how the constituent parts work together, and so forth. The notion of music playback systems as performable instruments is explored further:
Are you a musician when you turn on music on your mobile phone? According to my definition, yes. A mobile phone with a pair of headphones can be considered a complete musical instrument. And if you start and stop the playback of music and adjust the volume, you are actively taking part in the music-making. You have limited degrees of freedom for controlling the sound, but you are still in control of what’s going on. You can skip songs, change the volume, and modify the sound settings. This is different from standing on stage in front of many people, but the principle is the same: you ‘music.’ Whether you are a performer or perceiver is not about what you do but about the role you take on. (Ibid.: 151)
Within his taxonomy, Jensenius does not have a separate category for ‘playing with something that runs’. Instruments are either automatic or not automatic; ‘a music box, pianola, or LP performance can be thought of as score driven. On the other hand, a performance-driven system relies on the performer to make musical decisions’ (ibid.: 152) The area of interest for this paper is systems which are both automatic and playable, those which sit somewhere in the middle of the spectrum (which, arguably, all music systems do): musical systems which are both ‘automatic’ and ‘performance-driven’.
Discussing initially player-pianos – mechanical acoustic instruments designed to play back whole compositions from punched paper rolls, a precursor to recorded music – Ord-Hume considers the terms self-acting, automatic and mechanical to categorise such machines, finally settling on the latter as it ‘appropriately implies a system of playing by the agency of a mechanism some sort’ (Ord-Hume Reference Ord-Hume1983: 168). While it may be thought that these systems simply play back music without any intervention from the operator, there are degrees of intervention which can affect the musical outcome: ‘In 1891 wavy notation lines were introduced, printed alongside the perforations of the piano roll, to provide a more precise indication of how to lever tempo and lend the pianist a role some claim almost akin to that of a music conductor’ (Satz Reference Satz2010: 77). This aspect of the piece is scored but not automated, so tempo changes are at the operator’s discretion (and skill in varying the tempo correctly). The extent to which any machine or system is self-acting falls on a spectrum between fully programmed playback and fully manual activation. In any case, there is always some player action required, even if it is only the decision of at what precise moment to hit the start button.
One difficulty in defining these music machine systems arises when considering the boundaries of the instrument. Though the performance ecology can be defined as the ‘table full of shit’ (Bowers quoted in Richards Reference Richards2008), that table must sit in a room, be amplified by speakers, and its sonic output be perceived by an audience. Waters’s (Reference Waters2007) term ‘performance ecosystem’ acknowledges these wider elements of the musicking space.
The effects of the broader performance ecosystem on the self-acting playable systems we are examining are neatly illustrated through an analogy described by improviser John Ferguson, referring to Arthur Koestler’s Ghost in the Machine (Koestler Reference Koestler1976):
In examining what we might consider the performance ecology of a driver, Koestler characterises this environmentally situated human/machine interaction as a servo-mechanism, where human agency intervenes within and corrects the performance of an ongoing mechanism. In a step closer to an understanding of these notions in relation to the practice of music, it is useful to divert Koestler’s unpicking of the networks of sensory motor skills involved in driving a car, and consider this in relation to riding a cycle along a narrow and uneven track. The surface of the environment may redirect the flow of the cycle, yet it is possible to remove one’s hands and steer around corners; it is also possible (again with no hands) to navigate relatively large obstacles like speed bumps. A cyclist negotiates velocity via balance and accrued momentum, and may consider themselves either as part of a servo-mechanism, or as a human–machine assemblage. (Ferguson Reference Ferguson2013: 141–2)
While the human–machine assemblage is separate from the environment, it is affected by changes in the terrain and direction of travel. We can contrast the analogy with the performance space: the performer ‘driving’ the machine, with aspects of the environment – room resonances, environmental sound, audience response – as factors which may affect the direction of travel.
3. WHAT IS A MUSICAL MACHINE?
We are writing here about relatively novel instrumental set-ups that we refer to broadly as machines. We trace the origins of these machines to the middle of the twentieth century, though there is every chance that earlier examples exist. These machines differ from traditional acoustic instruments in several ways.
-
they tend to require electricity
-
they tend to be able to continue producing sound after the performer’s input has ceased.
Other qualities that these machines may possess include:
-
the ability to play back recorded or pre-programmed sound or sequences
-
the ability to synthesise sound through analogue, digital and sometimes acoustic means
-
the ability to process or transform sound
-
interfaces (such as buttons, knobs, sliders, touchscreens or sensors) for the performer to interact with them.
Some machines involve software running on computers, whereas others are completely analogue or mechanical. Some are mass manufactured, others hand-made DIY projects. Some use sensors to allow the movements of the human body to control sound, others allow the performer to endlessly reconfigure their architecture to create new sonic affordances. Some involve studio tools such as mixing desks, tape recorders or effects units repurposed as musical instruments. Crucially, these machines afford the composition and performance of live electronic music.
The earliest of these machines may be the turntable, dating back to its adoption as an instrument by composers such as Paul Hindemith (Reference Hindemith1930), Reference OramDaphne Oram (1948-9), John Cage (Reference Cage1939) and numerous others. Other early examples include the tone generators in the studios of the WDR, precursors of the modern synthesiser popularised by Robert Moog, and the tape machines of the GRM. The growth of the consumer electronics market for musical instruments in the past 60 years has given us numerous music-making technologies from keyboards and synthesisers through to guitar pedals, samplers and sequencers, all of which can constitute or form a part of a musical machine. Personal computers are increasingly used as instruments, and a number of distinct fields of practice (such as the NIME and Algorave communities) have been at the forefront of exploring their musical affordances.
Dub mixing has an important place in this tradition, and is an example of the process of performing live, improvised electronic music direct to tape. The producer actively arranges a new mix of a multitrack recording, punching instruments and voices in and out of the mix, adding reverb and delay effects, and tweaking filters and EQ settings. Veal describes the influence of improvisation on King Tubby’s production style: ‘his knowledge of electronic circuitry enabled him to exploit the idiosyncrasies of his equipment in novel and inventive ways. … it seems plausible that his sensitivity to jazz’s labyrinth of split-second creative decisions was reflected in his refashioning of the multitrack mixing board as both an improvisational instrument, as well as pioneering the dub remix as an act of real-time improvisation’ (Veal Reference Veal2007: 117).
4. WHAT DOES A MACHINE DO?
We will now consider the following roles the musical machine can take on during improvised performance. We would like to make clear at the outset that these modes of interaction must not be seen as being mutually exclusive nor clearly delineated. They exist along a continuum with fuzzy boundaries between categories, and we would expect to find multiple different categories co-existing simultaneously in the same machine. The roles, which we explain in further detail below, are:
-
It runs
-
It plays
-
It surprises
-
It evolves
-
It malfunctions
-
It collaborates
-
It learns.
4.1. It runs
Sometimes as performers we just want our machine partners to run, playing back pre-recorded material or sequences of notes to be sent to a synthesiser. This is the default mode of many digital musical instruments, DAWs and other performance technologies. Technics SL-1200s and Pioneer CDJs play back records and digital files respectively with stability and predictability; Ableton Live will let a performer trigger and seamlessly loop sounds. Playback devices including tape recorders, MP3 players and samplers all make regular appearances in the performance set-ups of improvising electronic musicians. Sequencers, drum machines and grooveboxes made by brands such as Roland, Korg and Elektron are all capable of reliably and almost indefinitely playing back looped musical sequences. In this way, the machine is used for offloading elements of the performance. Butler highlights the use of pre-recorded material in live electronic music as the raw material which forms the content of the performance: ‘When electronic dance musicians perform, they are always already working with pre-recorded sound. Stored in the memory of laptop computers and twelve-inch vinyl, like seeds awaiting growth, it is a wealth of musical raw material, the potential of which is unleashed through real-time, improvisational creativity’ (Butler Reference Butler2014: 70).
4.2. It plays
What is the difference between running and playing? Here we propose the term play over run for instances when the machine plays back sound with some element of intervention, unpredictability or randomness: its own machine-personality begins to materialise in the sound. This variation is initiated or introduced by the performer and occurs within defined boundaries.
Ableton Live embeds various features to facilitate randomness and unpredictability, including the ability to set the probability of a MIDI note in a clip being played. Similar levels of configurable randomness within pre-programmed sequences is found in hardware made by music technology company Elektron. Within modular synthesis environments, numerous modules allow for musical sequences to be played back with some degree of controlled random variation, and further variations can be achieved using sample-and-hold and trigger processing modules. In live coding, randomisation can be applied to multiple parameters: TidalCycles includes linguistic terms for different probabilities of an event occurring (including ‘rarely’, ‘sometimes’ and ‘often’). Gregory Taylor’s ‘Step-by-Step: Adventures in Sequencing with Max/MSP’ (Taylor Reference Taylor2018) guides the reader through making a sequencer in the Max visual programming language with probabilistic elements.
The variations in playback could be timbral or textural; changes in timbre can be introduced via acoustically activated sound sources, such as Sarah Angliss’s robotic carillon:
Like any mechanically driven sounding object, the bells of the carillon exhibit a subtle unevenness in timing and volume as they are struck. There are also creaks and movements; slight chaos in spring bounces and sympathetic resonances you’d expect with any instrument with moving parts. These irregularities add an aleatoric charm to the sound, a quality I doubt I could convincingly model in code. (Angliss Reference Angliss, Dean and McLean2018: 323)
4.3. It surprises
Beyond playing, our machines may surprise. A musical machine may not merely play back sequences or pre-recorded material with touches of randomisation, but also generate genuine unpredictability. This can be achieved in numerous ways. For instance, multiple out-of-sync sequencers or low frequency oscillators (LFOs) can give rise to such musical unpredictability, creating patterns that appear to evolve over long durations without obviously repeating states. As Jensenius (Reference Jensenius2022: 91) writes, ‘Such processes are part of the change from “sound makers” to “music makers.” For the performer, such long semiautomatic processes may feel like the instrument plays “itself.”’
Further unpredictable events may occur with the inclusion of sources outside the player’s control. Here the ecosystemic aspect of a musical machine is brought forward, the environment and audience affecting the musical outcome of the work as much as the elements directly controlled by the performer. We find early examples of this in the works of John Cage. The ‘silence’ of 4′33″ (Cage Reference Cage1947) remains outside the composer’s control, as does the incoming sound of radios in Imaginary Landscape No. 4 (March No. 2) (Cage Reference Cage1951). These strategies are also used by improvisers. Brötzmann and Bennink’s Schwarzwaldfahrt (Brötzmann and Bennink 1977) features the sounds of the Black Forest as a third, unpredictable performer; id m theft able’s...l...e...t...t...i...n...g...s... (id m theft able 2022) features instruments and objects left outside to duet with the elements. Electroacoustic improvisation group AMM performed with radios, with Cornelius Cardew noting ‘the use of [radios] as a musical instrument was pioneered by John Cage’ (Cardew Reference Cardew1971: xviii).
We can design unpredictability into our musical machines. Waters (Reference Waters2007: 12) discusses ‘interfaces which do not respond in an entirely consistent and predictable manner’ such as ‘a fader [which] may operate predictably over some parts, but introduce radical discontinuities over others, requiring real-time evaluation and adjustment on the part of the performer’. Similar ‘perverse’ behaviours of an interface are explored by Bowers et al.’s (Reference Bowers, Richards, Shaw, Frize, Freeth and Topley2016) ‘One Knob to Rule Them All’. By designing surprise into them, these instruments eschew traditional instrumental virtuosity and force performers into new, uncertain territories. Mudd et al. (Reference Mudd, Holland and Mulholland2019: 2) explore the value of surprise and unpredictability in digital instruments, noting how ‘unpredictable and unstable aspects are productively engaged with … exploration of the interaction itself becomes an essential part of musicians’ creative practices.’ This resonates with valuing what Zen monk Shunryu Suzuki termed our ‘beginner’s mind’, a perspective which reveals how instrumental expertise (and knowing how a machine will behave) can limit us: ‘In the beginner’s mind there are many possibilities, but in the expert’s mind there are few’ (Suzuki Reference Suzuki2020: 1).
For live coders Rohrhuber and de Campo (Reference Rohrhuber, Campo, Assayag and Gerzso2009: 123), surprises arise by virtue of the occasional opaqueness of musically interactions mediated through code: ‘The distance between what one may expect an algorithm to have as effect and what the algorithm actually brings about is irreducible; this gap is the thread along which conversational sound programming is kept moving.’
Turntablist Maria Chávez actively encourages surprise and unpredictability through the physical material of her sound sources, wherein she stores and transports her records out of their sleeves to allow them to change physically, as described by Thompson (Reference Thompson2017: 67):
The scratches on the surface can cause the record to skip, or allow locked grooves to develop so that the same short segment of recorded sound is repeated. In this context, these affective relations between milieu, medium and sonic content are not seen as inhibitive or degrading, insofar as they result in corruption or loss of information, or prevent ‘normal’ playback. Rather, for Chávez, this processual approach to the material record and the noises that arise ensures that there is always something new to be heard; new sounds, textures and rhythms are generated as the record is ‘damaged’ by the forces of the world.
In surprising us, then, machines fulfil what many musicians find to be a valuable role in improvisation. John Richards (Reference Richards2013: 278) writes about being concerned with ‘finding the moment of discovery to enable constant exploration and maintain a naïve stance. It is a means to create a tabula rasa. Not only does the instrument need to be explored through play, but also the music discovered.’
An instrument that surprises is particularly useful for the solo improviser. Discussing the challenges of solo improvisation, Derek Bailey (Reference Bailey1993: 106) describes as a great loss ‘the unpredictable element usually provided by other players’. The instrument itself can be allowed to speak: ‘The accidental can be exploited through the amount of control exercised over the instrument, from complete – producing exactly what the player dictates – to none at all! – letting the instrument have its say’ (ibid.: 100).
Tom Mudd’s (Reference Mudd2023) survey of no-input mixing desk practitioners further highlights the extent to which improvising musicians value unpredictability in their machines. Mudd interviews a number of practitioners including Yan Jun, who articulates this joy of the unexpected:
Sometimes, sometimes the sounds are so sweet … because you never can expect the sound … this kind of moment comes. It comes: just, it comes. You have to you just play then, it’s a gift from the room, the speakers, the audience, everything. But you have a feeling: it might come, it might happen today. (Yan Jun quoted in ibid.)
4.4. It evolves
A musical machine may be configured in such a way that the starting conditions can be established and the system configured to change or develop over time. Historical precedents include Steve Reich’s phasing pieces and Brian Eno’s generative music such as Music for Airports Volume 1 (Eno Reference Eno1978) which use extended patterns created by musical loops of different lengths or playback speeds. We find something similar in Jem Finer’s Longplayer (Finer Reference Finer2000) installation, which is in the process of playing back a piece that will not repeat until 31 December 2999.
These strategies are employed by those working in live electronic improvisation. Ferguson’s (Reference Ferguson2013: 142) description of a ‘technologically mediated improvisation’ captures this approach: ‘performers are setting processes in motion, probing inertias, and intervening within established trajectories’. Improvising with machines can often be a case of setting some process in motion and then supervising it, intervening when necessary. A comparable role during the process of improvising with sculptural sounding objects in performance-installation is described as the ‘attendee’ (Richards and Shaw Reference Richards and Shaw2022), someone who sets processes in motion, attends to their proper working and halts the processes at the correct moment.
In this sort of system, the performer offloads a degree of control to the machine. However, they may still retain a form of absolute control insofar as they design the process, as Reich (Reference Reich, Cox and Warner2017: 432) describes, ‘Musical processes can give one a direct contact with the impersonal and also a kind of complete control, and one doesn’t always think of the impersonal and complete control as going together. By “a kind” of complete control I mean that by running this material through the process I completely control all that results, but also that I accept all that results without changes.’
Discussing his 2021 sound installation In A Garden, Brian Eno expands the metaphor of composer as gardener:
People tend to imagine that making art is like making architecture – that you have a ‘plan’ or a ‘vision’ in mind before you start and then you set about making it. But my feeling is that making art can be more usefully thought of as being like gardening: you plant a few seeds and then start watching what happens between them, how they come to life and how they interact. It doesn’t mean there’s no plan at all, but that the process of making is a process of you interacting with the object, and letting it set the pace. This approach is sometimes called ‘procedural’. I call it ‘generative’. Just as a garden is different every year, a piece of generative art might also be different each time you see or hear it. The implication of this is that such a work is never really finished – there is never a final state. (Eno Reference Eno2021)
Such generative systems by definition require elements of self-playing audio, and in a live context the player’s input can range from initiating new sound sources (planting, ‘like seeds awaiting growth’, Butler Reference Butler2014: 70) to tweaking frequencies and timbres (pruning). This approach to musicking is fundamentally different from that of an instrumentalist generating each individual sound, and allows the evolution of the system’s characteristics to be forefronted.
Working with algorithms such as Conway’s Game of Life or strange attractors such as the Lorenz attractors give musicians access to evolving musical control data. A further source of this sort of evolving control data can be found in the practice of sonification, wherein data modulates musical parameters or is turned into. Kaffe Matthews (Reference Matthews2012; Matthews and Harrington Reference Matthews and Harrington2012) makes music which draws on the movements of hammerhead sharks in the Galapagos, or (with Laura Harrington) salmon migrating up the River Tyne, while other artists have worked with data including electrical brain activity (Lucier Reference Lucier1965) and ‘geophysical forces of Interior Alaska’ (Adams Reference Adams1994).
4.5. It malfunctions
Sometimes the unreliability of our machine partners may be key to what is musically interesting about them. Bowers and Archer (Reference Bowers and Archer2005) explore the potential for musical interactions with malfunctioning and haphazard instruments. Their ‘infra-instruments’ include the Strandline Guitar, the Victorian Synthesiser and Home Keyboard, Dehoused. These not only have the potential for playing and surprising, but are also deliberately unstable. One of their design principles is to ‘Build an Instrument but Include Obvious Mistakes’, using as an example the infamous Vegetable Orchestra who make the error of ‘selecting fresh vegetables as the material for construction’ (ibid.: 6).
Although it may seem counterintuitive to want to perform with a system that may break mid-performance, there are a number of advantages to the improvising musician. A certain drama can be enjoyed by performer and audience alike when the obvious precarity of the instrument is on display. The ramshackle and precarious nature of the turntable tower in Dunning’s Mechanical Techno project illustrates this drama in action. ‘Observing Dunning in interaction … might also add a certain appealing tension to the concert experience’ (Weissenbrunner Reference Weissenbrunner2017: 242). Moreover, failing devices can provide musical material and structure to the improvising musician. For live coder Shelly Knotts, the possibility of failure ‘introduces a level of indeterminacy, necessitating the following of unforeseen musical paths, constantly rethinking the next move’ (Reference Knotts, Hamilton and Pearson2022: 191).
Not only can a malfunctioning instrument provide fruitful interactions, but it may also be a source of sonic material. Cascone explored the ‘aesthetics of failure’, wherein the sounds of digital technologies malfunctioning becomes the raw material for composers to work with, writing ‘glitches, bugs, application errors, system crashes, clipping, aliasing, distortion, quantization noise, and even the noise floor of computer sound cards are the raw materials composers seek to incorporate into their music’ (Cascone Reference Cascone2000: 13). This is expanded upon by Kelly (Reference Kelly2009) who provides a thorough historical overview and survey of practitioners embracing the broken and normally undesirable as sound sources. Describing Fluxus artist Milan Knížák’s work Destroyed Music (1963–79), Kelly highlights the musical potential in a Frankenstein record made of four quarters from other discs stuck back together: ‘The overall sound is an ever-changing arrangement of music played as the needle scratches, bumps and bounces toward the centre of the disc. The audio is at once confusing and compelling, as unexpected sounds play at strange speeds and in disjunction to what is expected’ (Kelly Reference Kelly2009: 144–5). The glitch and electronic scenes of the 1990s and 2000s featured artists working with malfunction, such as Stefan Betke’s Pole project, which created drum-less dub characterised by a crackling, damaged Waldorf 4-Pole filter, and Oval’s work with scratched CDs.
Some performers have conceptual motivations for favouring musical machines that break. Simon Whetham’s Made to Malfunction and Successive Actions pieces use discarded bits of technology that have reached their ‘planned obsolescence’, ranging from old hard drives to fax machines. With the help of Arduino boards, the devices are ‘programmed to activate erratically, to malfunction’ (Reference WhethamWhetham n.d.). The unpredictable behaviour is exaggerated by the artist’s modifications, and the performance is built around this erratic behaviour and regular failure of the devices.
This tradition of malfunctioning machines draws on a rich tradition which goes back to auto-destructive art and the 1966 Destruction in Art symposium organised by Fluxus artist Gustav Metzger. This itself was indebted to Jean Tinguley’s Homage to New York (Tinguley Reference Tinguley1960), a structure assembled from junkyard finds, including a player piano, designed to be destroyed and resulting in a ‘symphony of percussive noises from the piano and clanging metal’ (MOMA n.d.).
Andy Hamilton (Reference Hamilton, Hamilton and Pearson2022: 11) notes the capacity for creative potential with aspects which might ordinarily be unwanted: ‘imperfection is not just the toleration of errors and imperfections … but a positive aesthetics. … true imperfectionism is a constant striving for new contingencies to respond to’. While in many musical contexts these imperfections originate with the musicians themselves, improvising with machines can facilitate contingencies from outside, introducing obstacles to respond to.
4.6. It collaborates
In some cases, these musical machines may be collaborators, taking on roles akin to band members or improvising partners. For instance, George Lewis’s (Reference Lewis, Everett and Caldwell2003) Voyager is a computer program that can listen to and improvise with one or more human musicians. This approach has also been developed in software by Blackwell et al. (Reference Blackwell, Bown, Young, McCormack and d’Inverno2012: 147) through the concept of ‘live algorithms’, defined as an ‘autonomous machine that interacts with musicians in an improvised setting’. A good overview and discussion of these live algorithms is provided by Bown (Reference Bown2011). With many of these systems, the ultimate end point would seem to be a machine that can imitate an intelligent and conscious human performer in some sort of musical turing test.
Another category of note here is robot musicians. Weinberg and Driscoll (Reference Weinberg and Driscoll2006) developed Haile, a robot percussionist which listens to and analyses human performers in order to accompany them, displaying a ‘robotic musicianship’, while also giving more visual cues to fellow performers through the very physicality of their design – something missing in software. Musical robots date back to at least the mid-nineteenth century, with contemporary examples including Takanishi’s Anthropomorphic Flutist Robot, Jorda’s Electric Guitar Robot and the Modbots. Musical automata in fact date back even further, with the Banu Musa brothers’ flute automaton from around 875 CE also considered to be the first sequencer, and the first ever programmable machine (Long et al. Reference Long, Murphy, Carnegie and Kapur2017: 202–3).
An alternative to imagining the collaborator as an agent we are performing with is the collaborator as a helper who is fleshing out our decisions and working on one part of the music while the performer focuses on another. Laurie Spiegel (Reference Spiegel1986) coined the term ‘intelligent instruments’ to describe novel digital musical instruments which have algorithms embedded within their design to assist performance. An example of this is Spiegel’s Music Mouse (1986). The performer makes high-level decisions and has their broad strokes movements transformed into fluid musical playing by the instrument: ‘It has a variety of options built into it for harmony and melodic patterning, freeing its player to focus on the movement of melodic lines, the shape and density of their elaboration, their electronic “orchestration”, and on the overall form and expressive content of the music itself’ (Spiegel Reference Spiegel1986). Music Mouse can be heard throughout Spiegel’s Reference Spiegel1980 album The Expanding Universe.
There are a number of well-documented instances of genres emerging from collaborations with machines. Prior (Reference Prior2008: 314) notes ‘the history of music bulges with cases that point to the unpredictable, productive and unstable: turntables as DJ instruments, monophonic bassline generators such as the Roland TB303 mis-programmed to beget acid house, telephone bandwidth-saving technologies turned into vocoders’. The music that emerges seems to be equal parts the ingenuity of human performers and the innate but often undiscovered sound of a new technology. As Mark Fell writes, from such a perspective ‘we can redefine technology, not as a tool subservient to creativity or an obstacle to it, but as part of a wider context within which creative activity happens’ (Fell Reference Fell2013). The creative process of working with such technologies is not one of bringing form to inert materials, but of engagement in a creative dialogue, what Ingold (Reference Ingold2010) frames as ‘an improvisatory joining in with formative processes’.
4.7. It learns
At the time of writing, Artificial Intelligence (AI) and Machine Learning have undergone a recent surge in interest, with AI becoming increasingly embedded in musical software. AI tools are used in sound design in synthesis to assist in generating new sounds, for refining settings on plugins, as mixing tools and in instrument design. Music and AI is a vast field that cannot be adequately summarised here, but it is clear that new relationships in technologically mediated musicking may emerge here. Crucial to these systems may well be the ability of the machine to ‘learn’, whether this is to produce creative works that convincingly emulate the style of others, or learn to interact with human performers in compelling ways.
Electronic musicians including Holly Herndon (Reference Herndon2019), Arca (2020) and Lee Gamble (Reference Gamble2023) have released musical compositions that draw upon AI. In the case of Herndon, this is through using Spawn, an AI instrument (and collaborator) that she trains on her voice and other sound sources, which can ‘ mimic, interpret and develop musical ideas, often revealing elements in Herndon’s compositions that she was unaware of’ (Hawthorne Reference Hawthorne2019). The Dadabots (2019) use AI to create continual music streams in the style of Meshuggah and John Coltrane; the AI has ‘learned’ what these artists sound like in order to reproduce them. AI is also used in the development of new musical instruments; Rebecca Fiebrink’s Wekinator software uses neural networks to map sensor and controller data to musical parameters and is used in projects such as Jeff Snyder’s Birl (Snyder Reference Snyder, Bovermann, de Campo, Egermann, Hardjowirogo and Weinzierl2017). Wekinator is a collaborator in the design process and affords a ‘more exploratory, playful, embodied, expressive partnership between human and machine’ (Fiebrink Reference Fiebrink, Bovermann, de Campo, Egermann, Hardjowirogo and Weinzierl2017); in the case of Birl, this gives the digital instrument something of the behaviour of an acoustic instrument and affords extended techniques and alternate fingerings. Fiebrink and Caramiaux (Reference Fiebrink, Caramiaux, Dean and McLean2018) propose that machine learning algorithms be understood as a novel way of interfacing with computers, learning rules from data provided and bypassing the need to write code.
5. DISCUSSION
The taxonomy we have described presents new musical instruments as having the potential for a range of behaviours, from passive playback of sound through to being collaborators, saboteurs, or evolving systems. This is a ‘more-than-human’ world of instruments and assemblages that seemingly posses a vibrant materiality, and a capacity to act and interact, or what Bennett describes as a ‘capacity … not only to impede or block the will and designs of humans but also to act as quasi agents or forces with trajectories, propensities, or tendencies of their own’ (Bennett Reference Bennett2010: viii).
For Bennett, opening oneself up to this vibrant materiality can make us aware of ‘a fuller range of the nonhuman powers circulating around and within human bodies’, a shift in perception that may reshape how society approaches problems ranging from waste to obesity and stem cell research (ibid.: ix). A similar call to openness is found in Abram’s The Spell of the Sensuous: Perception and Language in a More-than-Human World in order to ‘renew our acquaintance with the sensuous world’, address our ‘current estrangement from the animate earth’ and provide conceptual tools to address the ecological crisis (Abram Reference Abram1996: ix–x). Drawing on this, could a mindset which is open to the different roles and potential agencies of musical instruments have the advantage of opening us to the non-human powers circulating around the specific domain of technologically mediated musical practices? Could cultivating an openness to different degrees of non-human agency have the advantage of priming us to find the affordances and hidden-affordances of sounds, instruments and other musical materials?
Our taxonomy, and the work done by thinkers considering the ‘more-than-human’, also offer a starting point for discussions about agency. When we think of the musical machine as collaborator, there is an implication of some degree of agency. Of course, the extent to which our musical machines can be said to possess agency, and thus the capability of making decisions, is somewhat murky, and in non-AI systems this agency is largely metaphorical. Ferguson (Reference Ferguson2013) uses the idea of ‘imagined agency’ when discussing the technologies he improvises with. He argues that this is not just a rhetorical device, but that it is useful for a musician to imagine resistances and relationships in their equipment: ‘imagined agency, as I perceive it, is a useful notion to articulate the ambiguous, and sometimes irrational processes, within a practice that seeks out resistance and agency in a variety of forms, and attempts to interact with it.’ (ibid.: 147). There are other ways of conceiving of this machine agency. Bown et al. (Reference Bown, Eldridge and McCormack2009: 194) use the term ‘Performative agency’ to describe ‘the ability of a software system to influence the outcome of a specific musical performance’. Mudd uses the term ‘entangled agency’ locating agency in the interaction between human and non-human actors (Mudd Reference Mudd2023). Salter reflects on the sound art practice of Bruce Odland and Sam Auinger (known as O+A), and conceptualises agency as a flow, ‘not located in objects or things but situated in practice’ (Salter Reference Salter2015: 40). In our interactions with musical materials and instruments over the course of a performance, we may find agency changing and evolving, moving to and fro across performer, instrument, sound and space. Our taxonomy here provides a model for describing some of these different flows of agency we might find, and the different roles that may be adopted at different times. AI systems that learn and refine their outputs and can be aligned to specific goals may require new ways of talking about and understanding technology and agency.
In attributing agency to inanimate objects there is a risk that our taxonomy, and other new materialist thing-valuing philosophies, are overly anthropomorphic, preposterously projecting human behaviours onto non-human things. However, Bennett argues that such anthropomorphism is potentially useful, serving as the first step in making us aware of the forces and forms in the material world, writing ‘a careful course of anthropomorphization can help reveal … vitality’ in the non-human world (Bennett Reference Bennett2010: 122).
There is sometimes a temptation to evoke a dichotomy between a ‘hylomorphic’ view of creativity, whereby passive and subservient matter is shaped by human will, and a view which gives agency to that matter and sees the creator as following the materials (Ingold Reference Ingold2010). The extremes of both approaches risk a reductionism that silences either materials or human creators. This taxonomy helps conceptualise the way in which our interaction with sonic materials is mediated, encompassing a broad spectrum wherein materials can be shaped by human will or evolving beyond our control. As musicians, we often embody this mass of contradictions. We have clear aesthetic ideas for how we want something to sound, and a specific creative agenda that guides and shapes a project, but we also want to be open to new ideas that emerge throughout the creative process and are suggested by the materials.
The phrase ‘more-than-human’ was coined by Abram who, like many thinkers who may now broadly be considered New Materialists, is in part motivated by environmental concerns in his call for an awareness of the non-human vitality in the world. For Abram, Western civilisation and its technologies are largely responsible for isolating us from an awareness of the vitality of the non-human world, and there is a sense in which technology is irredeemable in this view, as he writes:
the mass-produced artifacts of civilization, from milk cartons to washing machines to computers, draw our senses into a dance that endlessly reiterates itself without variation. To the sensing body these artifacts are, like all phenomena, animate and even alive, but their life is profoundly constrained by the specific ‘functions’ for which they were built. Once our bodies master these functions, the machine-made objects commonly teach our senses nothing further; they are unable to surprise us, and so we must continually acquire new built objects, new technologies, the latest model of this or that if we wish to stimulate ourselves. (Abram 1999: 64)
Others, however, have a more positive view of the more-than-human as it pertains to human made things. Bennett’s ‘thing power’ includes mass produced man-made things – bottle caps and plastic gloves – which form assemblages and exhibit an ‘energetic vitality’. Latour’s actants ‘can literally be anything provided it is granted to be the source of an action’, and regularly includes man made technologies (Latour Reference Latour2017: 179).
Not only can this world of man-made things be said to possess vibrant powers, but in entering into networks and assemblages with these artefacts, our own perception can be extended. ‘Technology thus extends phenomenology’ (Salter Reference Salter2015: 50). Microphones, hydrophones, and other listening technologies enable novel sonic encounters:
The hidden, infinite rhythms of frequencies in the ‘white noise’ of streams could not be made audible ‘without headphones and probes and portable filters to actually hear what is inside that stream,’ Odland explains. Listening is enabled and learned through instruments—through the hydrophones and probes that enable the ear to insert itself into the material expressions of streams, forests, the resonances of cities. (Salter Reference Salter2015: 52–3)
The musical instruments we discuss may also be – or at least in part be constructed from – these mass-produced artefacts, and we would argue that they may still have the capacity to surprise, whether through design, or whether through entering into novel networks and assemblages that may reveal new features and hidden affordances.
A final point of interest for our discussion is that a number of the instruments discussed here are largely ‘unfinished’, in some stage of development, or remain ‘open’ to some degree, and have not stabilised into forms with temporal longevity, as we find with more traditional instruments such as the guitar, piano and even the Moog synthesiser. As such, the taxonomy helps reveal the tensions in meaning, purpose and functionality within musical instruments before these are ironed out and agreed upon. For instance, Pinch and Trocco consider the social construction of the Moog synthesiser, drawing on social construction of technology (SCOT) arguments and methodologies to explore the myriad forces acting upon the design of early synthesizers; writing ‘a new sort of instrument is coming into being and radically different meanings are being given to it’ (Pinch and Trocco Reference Pinch and Trocco1998: 19). Robert Moog’s version of the synthesiser was relatively accessible to musicians, including a chromatic keyboard and with modules whose functions were relatively transparent and understandable, in contrast to the synthesisers of Don Buchla, which did not have keyboards and had modules such as ‘The Source of Uncertainty’. Even the Moog was unintentionally surprising, sonically unstable and prone to ‘malfunction’, drifting in pitch and sounding different from day to day (ibid.: 25). Overlapping notions of running, playing and surprising exist in these technologies. Over time, however, the synthesiser ‘stabilised’ in the popular imagination into a keyboard instrument that can reliably and consistently produce a range of sounds:
From a flexible variety of possible control configurations, the synthesizer eventually stabilised into a keyboard instrument, widely accepted by pop and rock musicians, composers, and creators of original sounds. As the keyboard synthesizer became established, a continuing design effort was made to make it predictable, easy to use in live performance, and to enable it to replicate the sound of conventional instrument. (Ibid.: 28)
Simon Waters, too, writes of instruments as processes, assemblages which serve different roles at different times, reflecting that from a broader historical perspective the non-standard instrument can be seen as more ‘typical’, and that ‘Ephemerality and non-standardness can be regarded as typical of instrument making in many places, and at many times’ (Waters Reference Waters2021: 138). In this context, we hope that a taxonomy of musical interactions such as the one we have offered here can make us aware of the different flows of agency, interaction possibilities and more-than-human vitality that may fleetingly exist in new musical instruments and technologically mediated musical practices.
6. CONCLUSION
We have presented a model for understanding and mapping out some of the different ways in which musicians improvise with technologies, ranging from using the technologies as playback devices, through to sources of surprise and malfunction, and as collaborators and systems that evolve and learn.
These categories are not exhaustive, and there are no doubt many more ways of conceiving of our relationships with new musical instruments. Similarly, these categories are not always discrete, but rather exist on a continuum. There may be overlap between whether something is ‘playing’, ‘surprising’ or ‘malfunctioning’, and it may be doing all three – and more – simultaneously. Despite its inherent limitations, this provisional taxonomy has provided a lens through which to understand some of the myriad roles our instruments can play in a performance. Furthermore, it may help us be open to the vibrant materiality of musical instruments, and recognise them as ever evolving sources of creative dialogue.
One way of making space for such systems to contribute to the musical outcomes of the practice is by acknowledging the positive role that limitations, errors, inconsistencies and obstacles can have in improvised music-making. Where design processes are often goal oriented, aiming to solve particular issues or, in the case of musical instrument design, facilitate smoother and slicker ways of interacting, this taxonomy leaves space for open-ended strategies and potential for embracing mistakes over iterative progression. Instruments do not just translate abstract ideas into concrete sound, they push back and undermine, they help us out and they reveal new sonic possibilities to us.
Open access
