Introduction
This article examines challenges and transformations undergone by practice-based researchers composing digital scores in an intensive international residency. In December 2022 four Monash University Ph.D. researchers, all of them composers, performers and practice-based researchers, took part in a project-oriented residency.Footnote 1 The residency was designed by Monash University professor and composer Cat Hope while she was a resident at the Hamburg Institute for Advanced Study (HIAS) and a partner in the ERC-funded DigiScore project.Footnote 2 The project involved the creation and premiere performance of four new digitally scored works at the Hochschule für Musik und Theater (HfMT) Hamburg. Analysis was done through a series of surveys and interviews of the Australian researchers by DigiScore researchers. This analysis tracked the process of the Australian researchers to gain insights into how the integration of digital scoring within their practice affected their approach and pushed them in new directions. The impact of creating in an intensive transcontinental environment was also determined through this analysis.
A digital score is defined by DigiScore as ‘a communications interface of musical ideas between musicians utilising the creative potential of digital technology’.Footnote 3 The practice and study of digital scores encompasses many areas of computer music research, digital music practices, creative computing and human–computer interaction. Central to the DigiScore project is a study of the transformations in creativity and musicianship afforded by digital and computational technology.Footnote 4 This focus shaped the approach of the DigiScore analysis of the residency, investigating how working with digital technologies affected the processes and outcomes of the practice-based researchers.
The residency aimed to build collaboration between the Melbourne-based Ph.D. candidates and researchers in Hamburg and to expose the group to different approaches and technologies towards the development of new works and the concert presentation. The participants were able to focus on their work, develop relationships and continue or shift their participation in artistic traditions, movements and discourses.Footnote 5 Activities included attending workshops and concerts at HfMT Hamburg, performing a concert of improvised solos at HIAS, workshopping and premiering their new works and performing other digital scores. Their work in the residency fuelled new collaborations and caused disruptions in their typical approaches to composition, resulting in rapid development of technological skills and perspectives on digital notation. Most of the composers had not previously written digital scores and this intensive project allowed them to develop their skills as digital musicians.Footnote 6 The impact of the experience was measured by questionnaires, journals, interviews and personal statements of the composers through the DigiScore research. Before leaving Australia the participating researchers prepared some of the material for their pieces, planning their works around the spaces and technology available in the MultiFunktion Studio at HfMT Hamburg, a black-box theatre where the works would be performed.
Some of the composers engaged technical assistance for their works in Melbourne and in Hamburg, as they were all working with digital processes that were new to them in some way. This technical assistance included programmers helping the researchers realise their ideas with visuals created in Max/MSP, or analysing data with machine learning. It also included assistance with multichannel diffusion of audio, programming lights and working with virtual reality (VR) technology. While this assistance allowed new possibilities for the researchers and their works, it also caused new challenges. The researchers had to communicate their ideas and manage technical collaborators in Melbourne or Hamburg, adding pressure to the already intensive process. The researchers who worked with technical collaborators in Melbourne did not have access to them during the residency itself, meaning that they had to develop their own digital skills to make changes to their pieces, or think conceptually and change the interpretation instructions for their digital scores without altering the actual object. Pieces by the researchers who were working more with technical collaborators and equipment in Hamburg could not be completed and tested before leaving Australia, adding the risk that their piece might not be fully realised in the concert at the end of the residency.
The residency disrupted the usual practices of the Australian researchers by challenging their skill sets, requiring them to compose and perform in a new medium that most had not previously attempted. Each of them integrated new technologies and media into their process while retaining their conceptual and aesthetic values. This proved to be both challenging and rewarding, leading to discoveries about their compositional goals and identities. Being in a different country and institution affected the researchers’ ways of working. Some of them created their pieces around technical equipment and possibilities at HfMT that could not easily be accessed in Melbourne. Others faced challenges because of distance from collaborators who were not part of the residency. For all the participants, the fast-paced intensive nature of the residency meant that quick decisions and changes were necessary. These factors all contributed to changes in the pieces themselves and in the researchers’ digital musicianship skills and compositional approaches.
Participating Researchers and Their New Digital Scores
The four researchers who took part in the residency each composed pieces reflective of their own areas of practice-based research. All of them entered the residency with concepts and aims for their pieces and left with new perspectives on digital scores as well as changes in their musical practices. Here we introduce the research interests and artistic practices of each researcher, an overview of their pieces and how their goals were affected by the residency.
Helen Svoboda is a double bassist, vocalist and composer whose research explores extended techniques and arco harmonics on the double bass to expand the melodic potential of contemporary double bass music. She performed her digital score Wormwood, but it also has the potential to include other instrumentalists. With programming assistance from Australian composer Ciaran Frame, this work was Svoboda's first digital score created with Max/MSP. The residency allowed her to ‘dive deeper into technology as a way of exploring animated graphic notation for the first time’.Footnote 7 She appreciated being able to show a work in progress in front of an audience and found that this let her process become more experimental.
The digital score for Wormwood abstracts patterns in a photo of tree bark into graphic representations of overtone groupings and their integer multiples. As a static background the edited bark photo is visible on the projected screen behind moving animated graphic scapes. The graphic shapes, created in real time through the Max patch, prompt improvisatory moments performed on the double bass.Footnote 8 Working with digital technology gave Helen new ideas for how to express her research. After the concert in Hamburg, Helen planned to build upon Wormwood in the next stages of her Ph.D. research.
A composer and performer on the terpsichora pressure-sensitive floors, Dr Iran Sanadzadeh's work is concerned with relationships between movement and sound (see Figure 1).Footnote 9 Her digital score for the concert, 502 Days of Self, was also programmed in Max/MSP. While Sanadzadeh has experience with Max/MSP, she collaborated with programmer Daniel Pitman in Australia to integrate a machine-learning algorithm into her patch, analysing 502 recordings of her own daily rehearsals on her pressure-sensitive floors. The goal of the machine-learning element was to analyse how her performance practice on her instrument has developed throughout changes in her life and environment and to create an animated graphic score that allowed an ensemble to sonify those changes:
The piece aimed to use recordings of my developing instrumental practice on the interactive pressure-sensitive floors to understand how it is that I have formed my habits of movement and gestural vocabulary, sonifying them using other instruments with existing gestural vocabularies.Footnote 10
Figure 1: Iran Sanadzadeh on the terpsichora pressure-sensitive floors. © Alexandra Davies.
Performed by five musicians, Sanadzadeh's digital score showed each performer coloured polygons that graphically represented different types of sound and movement, reflecting her performance habits and how they changed throughout different periods of her practice.Footnote 11 Developing and rehearsing her piece during the residency changed her understanding of her own musical aesthetic and how to communicate that in open scores, particularly when collaborating with new performers and without extensive rehearsal time.
Chloë Sobek is a composer–performer whose practice revolves around performance on the Renaissance violone and electronic music. Her piece Immanence is a digital score in VR and was her first experiment with VR technology. Integrating her research and imagery from paintings by her brother, Julian Aubrey Smith, Sobek's digital score explored post-anthropocentric ideas in art and technology.Footnote 12 Sobek's original idea was to create an interactive VR work in which the audience could move through virtual environments and interact with objects, triggering sonic events. She found that she was limited by what she was able to create using Blender and challenged by time restraints and computer processing power, so for the residency she created a static VR environment that served as a graphic score for solo performance.
At the concert in Hamburg, Sobek performed Immanence herself, wearing a VR headset and responding to the visual cues in the virtual space as a graphic score. A video version of the work was projected for the audience live, and after the performance viewers were invited to experience the VR environment. Sobek considered this the first iteration of the work and stated that she would need to learn a game engine such as Unity or Unreal to realise her original concept. While in Hamburg she attended a session with HfMT's Dr Konstantina Orlandatou on Unreal, but there was not enough time to learn this program thoroughly and rebuild her piece. Discovering the possibilities for future iterations of her piece was an important part of Sobek's experience in the residency; she said that ‘the creation of this score was a transformative experience in terms of thinking about how the score could operate in a more interactive, immersive VR environment’.Footnote 13
Jaslyn Robertson is an experimental multimedia composer who works with both classical musicians and improvisers as well as performing electronic music. Her current practice-based research is focused on ideas of censorship in music. Jaslyn had composed digital scores before this residency, mostly as animated graphic scores. She wanted to take this opportunity as a challenge to think more openly about what a digital score could be and to create a score that was not presented on paper or a screen. She decided to try light as a score, working with Max/MSP to program DMX lights to be activated randomly within the structure of her piece.
Her piece Shadow Aria is a provocation about erasure, asking the audience to consider silenced voices within the history of music.Footnote 14 The light score was interpreted by four musicians at the HfMT concert (see Figure 2). Spotlights were randomised and performers were instructed to respond by freezing when spotlit. A spatialised electronic tape track in 24.2 diffusion supplemented the live performance, with the musicians instructed to respond to the movement of sound around them. Jaslyn's experience of the residency transformed her idea of what a score could be, and she developed new ways of composing for improvising musicians. She developed new technical skills by working with HfMT's Dr Jacob Sello on programming lights through Max/MSP and learned the difficulties and limitations of this technology. From discussion with the performers after the concert she realised how much the spatialisation of the tape track affected each performer's experience and that this kind of work needs additional development and rehearsal time.
Figure 2: Aaron Wyatt and Helen Svoboda rehearsing Jaslyn Robertson's Shadow Aria at HfMT Hamburg, December 2022.the
Analytical Approach
DigiScore's analysis of the intensive residency in Hamburg centred around four areas of research: materials, flow, digital musicianship and transformational experiences and impact. Focusing on these indicators elucidated the transformative and disruptive experiences of the participants in each area.
In addition to supporting the four practice-based researchers, the DigiScore project collected research data throughout the lifecycle of the project to provide critical insights into the benefits and challenges of digital score creation and realisation in a collaborative residency environment. The complete data set encompasses reflective journals, semi-structured interviews, intention statements and legacy questionnaires.Footnote 15 The questioning process was undertaken by the DigiScore project researchers, guided by a theoretical framework that built upon that presented in Craig Vear's The Digital Score.Footnote 16 The questions focused on four categories:
• The materials of the digital score – the connections to the materials that form the parts of the digital score (sounds, images, game worlds). Also, how the musicians formed relationships with the active materials, such as pre-recorded melodies, machine intelligence, creative media, evoked music worlds or the other musicians.
• Flow in the moment of performing – what journeys were the musicians taken on, how involved in the music did they become.
• Digital musicianship – what skills, knowledge and approaches did the musicians use to facilitate a creative engagement with the piece.
• Transformative experiences and impact – for example, did the score communicate innovative music ideas, new music experiences, novel compositional approaches, new performance opportunities, music-making engagements or broader accessibility/inclusivity for musicians? Has this experience changed their outlook on music-making in general? Will the impact of this experience carry over to future projects?
Digital Musicianship
The data provided valuable insights into digital score creativity in addition to critical insights into changes in participants’ digital musicianship as the result of their new experiences. For the analytical research the DigiScore project evaluated the participants’ digital musicianship within the context of the HfMT residency from the point of view of ‘a person's ability to perceive, understand and create sonic experiences’,Footnote 17 and was an expansion of ideas presented in Hugill's The Digital Musician.Footnote 18 Specifically, DigiScore examined digital musicianship in terms of skills, contexts, cultures and literacy, musical identity and creative practice and perception and awareness of digital music.
During their residency, the participants developed many new digital skills, building on their previous backgrounds and the contexts in which they were working as composers and performers, and were able to implement elements of their creative practice in the making of the digital scores. For example, Sobek used her skills in editing and recording with Reaper DAW as a springboard to learning Blender, the application in which she created assets for her VR piece. Moreover, she employed her expertise in Reaper to record a soundscape featuring field recordings and her instrument, enhancing the immersive ambience of the digital score. Robertson used her skills in Max/MSP and spatialisation to learn how to program DMX lights through Max, adapting both previous skills in audio spatialisation and new skills using DMX to the performance space in Hamburg. Svoboda had little previous experience with Max/MSP, but by collaborating with a programmer she was able to animate a graphic notation for double bass overtones that she had been developing previously. Sanadzadeh learned new digital skills before the residency, such as which types of algorithms work effectively for data categorisation and their transference to visualisation in Max/MSP. She already had a good knowledge of Max/MSP and learning to use machine learning with this software extended her skill set.
The change in context for the participants played a major role, changing their perspective on music-making by working with digital scores in an unfamiliar setting. In particular, the shift from working with technical partners in Australia to a residency in Hamburg for projects like Wormwood and 502 Days of Self had a significant impact, requiring the composers to navigate technical challenges and adjust their compositions independently, without the immediate support of their collaborators. This experience led to a deeper reflection on the nature of collaboration and the extent to which relinquishing some control is often necessary for the sake of a work's development, providing valuable insights and highlighting the challenges of using new technologies in unfamiliar performance spaces with different set-ups. The positive outcome was the acquisition of new skills and a greater sense of independence in managing the technological aspects of their digital scores.
All the digital scores presented were accessible to performers and the audience. None of the pieces necessitated advanced technical digital music skills from the performers, except for a basic understanding of interpreting animated digital scores and the capacity to improvise. Wormwood and Immanence were accompanied by projected video for the audience, while Shadow Aria utilised lighting as both a score for the performers and a dramatic display for the audience. The participants also had the advantage of working with Decibel New Music Ensemble, an Australian ensemble who were touring during that period and are renowned for their expertise in animated notation.Footnote 19 Some performers, however, had to reframe their approach to interpreting the digital scores. While 502 Days of Self was an animated score easily interpretable by the expert group, it had some special instructions from the composer that performers had to observe. For example, the performers had to follow the intentionality of their sonic gestures rather than following changes to the shapes. One of the instructions, added during the process of workshopping and rehearsing the piece, was that players could stay silent through two or three changes if it helped them to start or finish their gestures more smoothly. Without changing the score itself, changes to the interpretation instructions enabled a resulting sound that better reflected the musical concept. Creating a digital score is a technologically demanding process that can become separate from the resulting performance; the residency environment allowed the composers who had created scores for other musicians to see and hear how they would be interpreted and to make adjustments before the performance.
Challenges and Changes during the Residency
Through the examination of the data DigiScore collected, it became evident that each Australian researcher experienced challenges that ultimately enhanced their practice. Developments occurred in their digital musicianship and their connections to the materials of making and performing digital scores. Each researcher reported unexpected results that altered the way they thought about technology in their practice and led to reflections on how they would change their compositional process in the future. Such moments of surprise, whether disruptive or creatively useful in the moment, give practice-based researchers the opportunity to reflect and make changes in their process.Footnote 20 The transformative experiences varied: some researchers were profoundly changed by the residency; others found small ways of integrating their new skills and understandings into their future practice-based research.
Iran Sanadzadeh's creation of 502 Days of Self, using machine-learning analysis from recordings of her performance practice to form an animated graphic score, changed her perspective on how to translate her conceptual and aesthetic ideas into an open score. She was affected by working with a technical collaborator, by cultural experiences in Hamburg and by the process of working with musicians in an intensive environment. Sanadzadeh had communicated the concept for her piece with the programmer but felt that the patch he sent her looked quite different from what she had imagined; her limited understanding of machine learning prior to this project left her unsure about what the results would be.
In the end she was happy with how machine learning contributed to her piece and was surprised that the challenges she faced were less about technology and more about the musicians’ interpretation of her digital score. In rehearsals the graphic shapes and the directions for the performers to interpret them did not result in the sounds that Sanadzadeh wanted from her piece, as ‘there is too much and too little in the score simultaneously’.Footnote 21 Because she was away from the programmer with whom she had worked and had little time to change the Max patch, Sanadzadeh had to change the directions for the performers instead of the digital score itself. To make the piece sound more in line with her own soft and sparse performance aesthetic she added directions that would prioritise this soundworld. This caused her to examine the relationship between movement and sound, encouraging performers to explore their gestural vocabulary. She felt that the residency transformed her compositional practice: ‘This week's been really intense and really short. It's going to give me so much to keep going with and so many things to develop that I'm really excited to keep developing.’Footnote 22
Sanadzadeh was affected by cultural experiences as an audience member and tourist in Hamburg, as were many of the practice-based researchers. Being exposed to new possibilities for performance and research and growing connections with the other researchers and Decibel members were important parts of the residency. The researchers were inspired by a concert they attended with Berlin-based pianist–researcher Magda Mayas in which practice-based research was presented as a performance–lecture.Footnote 23 Informal conversations with the other residency participants and HfMT and HIAS staff also affected the participants. Sanadzadeh recalled a conversation about practice with new instruments that took place while wandering through a German Christmas market.Footnote 24 The transcontinental context of the residency meant that the researchers were exposed to musicians and academics from Europe who could inspire their future research, and leisure time between scheduled residency activities allowed them to process and discuss their new thoughts and experiences with each other.
Helen Svoboda's visualisation for the double bass in Wormwood encouraged spontaneity and challenged her improvisational habits, leading to the discovery of new sound combinations. The graphics that appeared in her digital score were randomised, with the goal of changing her usual way of improvising, but although this was her plan, she still found herself confronted by this system when rehearsing and performing the piece:
I think in that way it was surprising because I didn't really know what to expect in terms of the shape categories and there were times when I was just really thinking, oh, I wish a pink one would appear. And I really wish that would happen. And it didn't. And I just had to sit with that, which is I guess going with that, breaking away from my usual habits and being OK with, you know, remaining in a space for a long time.Footnote 25
Having conceptualised her piece without fully knowing what to expect from the programmer, the result challenged her usual practice more than she had expected. The residency broadened her perspective on the benefits of animating digital scores and digital technology. She plans to expand Wormwood into a longer piece to further elaborate these ideas and provide greater opportunities for experimentation.
In Shadow Aria Jaslyn Robertson created a performance environment that facilitated sensitive responses to spatialised audio and light. She faced intense challenges because she was working with technology with which she was familiar but not yet confident. While she had previous experience with multichannel spatialisation, she had not anticipated how difficult it would be to follow her learned process in a new environment. Technical and logistical issues meant that the concert venue was not set up to test her diffusion until two days before the concert, leaving her worried that her piece might fail. She taught herself how to implement DMX signals through Max/MSP before arriving in Hamburg and significantly developed these skills with Dr Jacob Sello's assistance at HfMT. However, she was not aware that the hardware and software she was using were particularly unstable and prone to crashing even when carefully programmed. For the concert Jaslyn's set-up worked well, but she felt that she could have improved the performance with more time to fine-tune the details of the diffusion, performance directions and structure of the piece. Making a digital score inspired her to think creatively about new possibilities for collaborations with improvisers, and she gained an understanding of how to manage time and run rehearsals when working with new forms of technology.
Chloë Sobek's experience making Immanence allowed her to expand her sonic practice into visuality, creating a graphic score within the VR environment. The facilities and technology available to her at HfMT inspired her work: she knew that the school had several VR headsets, and this allowed her to plan for the audience to experience her VR work. However, because of a lack of time and resources to learn the programs required to realise her original idea, she changed the direction of her piece before arriving in Hamburg. After returning to Melbourne, she had concerns about the difficulty of creating VR works while still maintaining her practice as a musician and composer. While she saw that making a digital score was beneficial to her relationship with music composition, she also questioned how deeply she could engage with VR without collaborating with media artists who work in this field.
Each of the researchers integrated their own performance practices into their compositions. Sobek and Svoboda performed their own works, drawing heavily on techniques that are central to their individual performance styles. Sanadzadeh and Robertson did not perform in their own works but referenced their performance practices: Sanadzadeh by using her rehearsal recordings as a data source in the algorithm and Robertson by including recordings of herself on synthesisers in the spatialised tape track. While each work had a different theme, ranging from environmental to personal to political, they all drew on concepts and musical languages that sit firmly within each composer's practice and research. This resulted in challenges and experimentation as each researcher dealt with the different processes and outcomes that digital notation brought to their work.
Conclusion
The changes experienced by each of the participants continued to contribute to their practices throughout the following year. Both Sanadzadeh and Robertson presented their compositions in new iterations at the International Conference of Technologies for Music Notation and Representation (TENOR) in 2023. Svoboda performed her work at the Australian Jazz and Improvisation Research Network (AJIRN) conference in 2023. Most of the pieces were re-performed in a concert at Monash University in 2023, bringing these works developed in Germany back to their home in Melbourne. Their choice to continue developing their pieces, and particularly to restage them in conference concerts, shows the connection they feel between these pieces and their research.
During the project in Hamburg each participant developed or strengthened collaborations which have led to new projects. Sanadzadeh and Sobek performed and presented together at the International Conference for New Interfaces for Musical Expression (NIME) in 2023. They have continued to perform as a duo after experimenting with ways of playing together in Hamburg. Robertson returned to HfMT Hamburg to undertake fieldwork for her Ph.D. and continue developing her skills in multimedia composition. The residency affected not only the development of new digital scores but also the way the participants see their musical practice and their confidence to explore new approaches in the future.
Each researcher was inspired by the opportunity to create a new digital score and used the prospect of developing their works at HfMT as a chance to experiment with technology that was new to them in some way. They learnt new digital music skills and developed confidence in their abilities as digital musicians. Svoboda experimented with Max/MSP, Sobek with VR, Sanadzadeh with machine learning and Robertson with controlling DMX lights through Max/MSP. They all found benefits and challenges in these new digital processes but ultimately left with a better understanding of technological skills that they wanted to develop more, or where it would be useful to collaborate with skilled technicians. A joint decision to frame the concert at the end of the residency as a work in progress allowed each researcher more freedom to experiment.
The intensive environment of the transcontinental residency was a key factor in the effectiveness of the project, supporting rapid developments in skills, knowledge of digital scores and associated processes, and collaborative relationships between the participants. Being in a different continent, some of them away from their technical assistants and all of them working in a high-pressure environment, disrupted the usual flow of their composing process. They acquired new skills and a heightened sense of autonomy in handling the technological elements of their digital scores and experimented with new ways of composing that required them to question the impact of these new approaches. This in turn changed their understanding of themselves as composers, performers, and researchers, evident in their continued outcomes.
Acknowledgements
The Digital Score project (DigiScore) is funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (Grant agreement No. ERC-2020-COG – 101002086). This project was possible thanks to the support of the Hamburg Institute of Advanced Study, Decibel New Music Ensemble, the Hochschule für Musik und Theater Hamburg and the Sir Zelman Cowen School of Music at Monash University.
Open access
