1. INTRODUCTION
The prevalence of digital production technologies has seen the emergence of a ‘digital fatigue’ in sound-based art resulting in some artists longing for a ‘physical connection to the materials of their work’ (Kelly Reference Kelly2018). Citing a media archaeological perspective, Caleb Kelly (Reference Kelly2018) argues that such an emergence ‘coincides with a return to various philosophies of materialism’ with ‘works that, through the sounding of their materials, deliver a series of outcomes that call on us to think about where these materials have come from and where they will end up’. Ethan Rose (Reference Rose2013) critiques the use of technology’s transformative qualities, arguing that they can disassociate sound from its source. He argues that such technological developments (both mechanical and electronic) have, in some cases, removed the performer from the performance. This has been most evident in the use of recording technology and the transformative qualities of the studio that disassociate sound from its source in creating an ‘idealized form of absolute sound’ (Rose Reference Rose2013: 66). Rose’s (Reference Rose2013: 65) categorisation of the object-based sound installation – which works with physical artefacts, rather than sound alone – has been defined as the type of installation that engages ‘an audience by actuating a visibly present object’ with the intention for these types of installations to ‘translate the transformative powers of the digital production technologies into visibly embodied articulations’. He sees the emergence of object-based sound installations as an effort by sound artists to expose and foreground the sounding object as a significant part of their works.
Along with the prevalence of digital production technologies in sound art, a culture of expression using technology has been developed in data sonification for manipulating and presenting data. Such an expressive use of technology through the development of systems, processes and tools manipulates data flows and synthesises sounds to create an ‘impressionistic wash of sound’ as a tangible experience of the sonification of data (Gresham-Lancaster Reference Gresham-Lancaster2012: 208). However, abstracting the sound from the source in this way can bias our understanding of our relationship with the sound, especially when we consider the context in which the sounds originate, the place in which we hear them, and how our experiences are mediated by technology (Harris Reference Harris2013). Concerned with the disingenuous arbitrariness of sounds assigned to data, Norie Neumark (Reference Neumark, Cobussen, Meelberg and Truax2016: 383) questions sonification’s disregard for the ‘machinic utterings and mutterings and matterings’ as a method for expressing an underlying data source. She argues that sonification’s over-aestheticisation of sound is, in a move to make data ‘sensible and meaningful’, a denaturing of sound and is just one of the many ways that data can be translated (Neumark Reference Neumark, Cobussen, Meelberg and Truax2016). To this, Neumark (Reference Neumark2017: 98) suggests an approach that, by bringing back the voice of the ‘machinic and technology … we can listen to machines voicing themselves’.
The fields of auditory display and media archaeology are not without their tensions and contestations regarding means and methods. Arguably, the use of data sonification as a technique of media archaeological inquiry, presented as sound-based art, has not been widely employed. Similarly, the material representation of obsolete media has been under-utilised in data sonification to allow the artefact’s voice to be heard. It is not the intent of this article to cover the range of possibilities for media archaeological sound art. Even while focusing on activities of tactile engagement with media artefacts as an approach to the creation of sound art, the field is increasingly diverse and, at times, difficult to distinguish between what is explicitly intended as media archaeology and those works that exhibit characteristics of such approaches. However, it is within this diversity of possibilities that the authors present a framework for the application of sonification techniques beyond the direct translation of waveforms into sound (audification) for sound installations that foreground the materiality of the physical object, explicitly informed by a media archaeological approach to historical media representations.
This article examines the use of data sonification techniques employing obsolete media in sound-based art through the lens of media archaeology. This is a way to extend media archaeology’s material approach as a method of expression utilising the sonic artefacts of the visibly present object. Following a brief introduction to media archaeology, the artistic use of data sonification is briefly discussed. Next, the material perspective of media archaeology to facilitate the creative shaping of a narrative between obsolete and contemporary media through sound is discussed. Following this, a framework for facilitating the organisation of sound is presented. The application of this framework allows the artwork to utilise the voice of the obsolete artefact, or elements of it, thus anthropomorphising what may be considered disembodied or immaterial informational technologies and processes. As such, it is employed as part of an approach to explore new methods of inquiry into the obsolete object’s engagement and relationship with current media technologies through sound-based art with exemplar sound-based artworks developed by the authors presented to give concrete form to the framework and the ideas established in this article.
2. MEDIA ARCHAEOLOGY: A MATERIAL PERSPECTIVE
Media archaeology emerged in the latter stages of the twentieth century as a reaction to the progressive trajectories of historical media technology narrative. As an approach to the critique of contemporary media culture and history, media archaeology encompasses a range of different approaches and attitudes to technology. To anthropomorphise the term, it is a bricoleur, borrowing and re-using available materials as necessary to connect seemingly unrelated forms of media. While criticised for its lack of a common objective, having no one fixed method affords it the opportunity to borrow methods of inquiry from other disciplines and apply them in creative ways.
In the intervening decades since the emergence of the term, media archaeology’s tendrils have touched an increasingly diverse range of media. Despite this reach and the increasing number of works published under the term ‘media archaeology’, answering the question of what media archaeology is remains somewhat elusive. There continues to be disagreement about whether it is an approach, a model, a project, an exercise, a perspective, or a discipline and, for some, it has remained an undefinable research approach with inconsistent features (Strauven Reference Strauven, Hediger, Noordegraaf, Le Maitre and Saba2013; Keidl Reference Keidl2017). Indeed, Michael Goddard (Reference Goddard2018: 22) contends that these debates, contestations and evasions render any ‘stable delimitation of the field at the very least problematic’.
While some see that the ‘methodological repertoire of media archaeology has been geared to discourse analysis’, media archaeology allows for the exploration of the material possibilities of obsolete media through an engagement with the physical artefact (Fickers and van den Oever Reference Fickers and van den Oever2019: 53). A plea for new directions in media archaeological research issued by Andreas Fickers and Annie van den Oever (Reference Fickers and van den Oever2014) specifically called for an experimental approach to media archaeology over discursive enterprises through the historical re-enactment of past media technologies. For artists, understanding media’s sociocultural and technical conditions of existence and physically engaging with the artefact can provide new opportunities for re-presenting media in unfamiliar contexts and ways through which novel connections can thus emerge between past and present media. In this way, a material approach can be a way to critically engage with obsolete media and its relationship with contemporary technologies in the creation of media archaeology-informed sound-based art. The next section presents an overview of sonification in an artistic context through the application of musical elements.
3. DATA SONIFICATION: SCIENCE OR ART?
Successive attempts have been made to narrow down the field of sonification to scientific approaches. Thomas Hermann has argued that while music can be listened to for different layers of interpretation, sonification is expected to have a precise connection to the underlying data (Grond and Hermann Reference Grond and Hermann2012). His proposed definition sought to narrow the field. His single phrase definition states:
Sonification is the data-dependent generation of sound, if the transformation is systematic, objective and reproducible, so that it can be used as scientific method. (Hermann Reference Hermannn.d.)
Aligned with this, Carla Scaletti (Reference Scaletti, Dean and McLean2018: 19) argues that while they share technologies and techniques, a distinction between data sonification and ‘data-based music, data-based sound art or simply music’ is the difference in intent. She describes the purpose of data sonification as the discovery of something new about the original phenomenon that produced the data, while using the terms ‘data-based music’, ‘data-based sound art’ or simply music to describe the ‘use of data as a component of artistic expression’ (Scaletti Reference Scaletti, Dean and McLean2018: 19). However, Scaletti’s use of music to differentiate data sonification from other forms of data-driven works appears reductive of the types of works that share this distinction. Instead, the term ‘sound-based music’ opens the ‘boundaries of music to all forms of sound organisation’ (Landy Reference Landy2016). Alexandra Supper (Reference Supper2011) argues that limiting the field to a scientific method reduces subjective intervention to a ‘mechanistic objectivity’ and effectively denies the existence of ‘artistic sonification’. Stephen Barrass and Paul Vickers (Reference Barrass and Vickers2011: 165) contend that an aesthetic approach to sonification will allow its practical reconfiguration ‘from an instrument solely for scientific enquiry into a mass medium for an audience with expectations of a functional and aesthetically satisfying experience’. The artistic applications of sonification can engage with data to express a range of global themes including pollution, weather patterns, tsunamis, earthquakes, satellite tracking, population movement and urban environments to name a few. The aesthetic potential of sonification as an artistic medium has been developed by many sound artists. Some examples include, but certainly are not limited to, works by Julian Clauss, Chris Chafe, Paul DeMarinis, Dugal McKinnon, Dmitry Morozov, Andrea Polli, Nick Ryan and Mo Zareei (Figure 1). Indeed, Chris Chafe (Reference Chafen.d.) prefers to classify his work as data musification.
Figure 1. Left: NOISE SQUARE (2015) by Mo Zareei. Image courtesy of the artist. The work is described as the ‘physical sonification of cellular automata through a mechatronic sound-sculpture’ in the online documentation (Zareei n.d.). Right: Ground Noise (2019) by Julian Clauss. Image used with permission. The installation presents an active sound topography by extracting characteristic datasets from a space by playing its ground surface like a vinyl record (Clauss Reference Claussn.d.).
3.1. Balancing the musification of data with meaning
While the result of sonification processes may not necessarily be attractive to listen to, Vickers (Reference Vickers, Cobussen, Meelberg and Truax2016) suggests incorporating musical elements as one way to engage and hold audience interest. Such an approach has been described as musification, defined as the representation of data through the musical interpretation of source processes or a course of events in the data ‘designed to go beyond direct sonification and include elements of tonality and the use of modal scales to create musical auralizations’ (Edlund Reference Edlundn.d.; Coop Reference Coop2016: 177).Footnote 1 The adaptation of data sets as series of patterns to capitalise on the ear’s ability to detect anomalies that might become noticeable in a continuous stream of data is an aesthetic strategy for the musification of data. Such an approach incorporates features that represent more traditional elements of music such as melody, harmony and rhythm (Coop Reference Coop2016). An argument for this musification is to make the underlying data more palatable and more familiar within cultural contexts. As such, music, its systems for the organisation of sound, and its aesthetic values have influenced sonification designers (Dubus and Bresin Reference Dubus and Bresin2013). As music is an important part of everyday life for a large number of people, framing data sonification within musically stylistic forms can be one method of highlighting or framing familiarity and difference in data patterns (Gresham-Lancaster Reference Gresham-Lancaster2012). Barrass (Reference Barrass2012: 149) notes that musification can have ‘narrative qualities that can convey … affective musical dimensions of valence and arousal’ as a basis for affective sonification. In this respect, Varni et al. (Reference Varni, Dubus, Oksanen, Volpe, Fabiani and Bresin2012: 159) suggest that using music material should be allowed in sonification, provided that the main goal is ‘to optimise efficiency of information communication’ and not ‘to be pleasant to hear or to arouse particular feelings for the participants’.
If data is over-musified, its essence may be displaced by music and by doing so, the significance of sounds are reduced. In this process, the goal of communicating information is subverted by the pursuit of aesthetic interest. Just as listening to the unvarnished output of a data set can be intrusive and tiresome for some people, the overuse of musical information also risks presenting a sound that is beyond the perceptual abilities and immediate grasp of a non-musical audience, because the sound and data source cannot be connected (Gresham-Lancaster Reference Gresham-Lancaster2012). To this end, it has been suggested that, while embracing elements of aesthetic representation, giving an audience a ‘sense’ of the underlying data, which is less about hard facts and ‘more how it might serve as a stimulant for curiosity’, may be an appropriate compromise (Neuhoff Reference Neuhoff2019: 328). An example of data musification, as Chafe calls it, is Shawn Graham and Jaime Simons’s experiment with image sonification, Listening to Dura Europos (2021). The work sonifies archaeological images using a standard musical scale allowing an audience to ‘listen to its digitized data’ thus establishing ‘a new kind of historical hermeneutics of visual sources’ (Graham and Simons Reference Graham and Simons2021).
To this point, the use of melodic and harmonic material to create continuous or chromatic scales that represent data points may produce harmonically consonant or dissonant sequences. Similar concepts apply to rhythmic alignment, symmetries in phrases, or timbral balance. Scot Gresham-Lancaster (Reference Gresham-Lancaster2012: 208) contends that the real interest of a sonification-based work is ‘manifest in the discovery of unpredictable and emergent sound qualities’ resulting from sonically interesting and abstract interrelationships between the source data and the sonic output of a given process or set of processes. Conversely, Grond and Hermann (Reference Grond and Hermann2012) note that sometimes it is not the sound itself that catches our attention, but the technological apparatus that produces or projects it. They further state that the apparatus producing or projecting the sound is an attractive point of access to the work and creates a strong context that influences how we perceive the sound. Exposing and foregrounding the sound-producing object as a key audiovisual and aesthetic element can provide such an experiential connection.
4. SOUNDING DATA: A MEDIA ARCHAEOLOGICAL PERSPECTIVE
If the auditory display community disagrees on aesthetic principles and the musification of sound for expressing data relationships, perhaps it is time to look beyond that community. Media archaeology, as a form of spatiotemporal conversation, can create an understanding of physical media that carries the past into the present through contemporary renderings of media (Hertz and Parikka Reference Hertz and Parikka2012). Introducing sonification techniques as a method of interrogation within media archaeology provides additional insights to historical relationships through sound. Paraphrasing Gresham-Lancaster, such relationships between the past and present may lead to the discovery of non-linear engagements between obsolete and current technologies expressed by such a sonic relationship. Michael Goddard (Reference Goddard2015: 1762) contends that a key value of media archaeology is ‘its insistence on the materiality, and material ecologies of media objects, systems and processes, contrary to the still lingering tendency to view informational technologies and processes in disembodied and immaterial terms’. From this perspective, data sonification can be utilised as a technique for transforming and aiding the organisation of sound expressed through the artefact itself.
If sonification approaches are broadly categorised as event-based, model-based or continuous (as audification), little evidence exists in media archaeology of the use of data sonification techniques beyond what Walker and Nees (Reference Walker and Nees2011) describe as an audification technique. This is simply because the outcomes are usually the direct translation of waveforms into sound with minor processing for the signal to become audible. This includes ‘all sound recordings’, the ‘vibrational data of mechanical waves’ and ‘physical processes outside the mechanical domain’ – for example, electromagnetic waves – (Dombois and Eckel Reference Dombois, Eckel, Hermann, Hunt and Neuhoff2011: 302). The time axis manipulation of sound recording data is noted as the simplest form of intentional audification (Ernst Reference Ernst2013). Shintaro Miyazaki (Reference Miyazaki2012a) adds that audification is ‘mainly oriented towards real world signals and is thus connected to the timing of the physics and hardware of the medium to be inquired’, considering sonification ‘mainly a symbolico-logical and more software intensive approach to understanding media’. To this, Wolfgang Ernst (Reference Ernst2016) differentiates between a functional sonification and epistemological sonification, functional in the sense that sonification follows certain principles to aid the understanding of the data – for example, a higher pitch for increasing data values, a lower pitch for decreasing values. Epistemological sonification refers to the ‘subliminal message behind the apparent musical content’. One example of such an audification process is Detektor (2011) (Figure 2), a collaborative work between Miyazaki and Martin Howse. Cultivating a close reading of and, through audification, a close listening to the technical workings of computers and their networks on the level of physical signals, Detektor is a device that demodulates otherwise inaudible electromagnetic signals and rhythms into audible sound (Miyazaki Reference Miyazaki2012b).
Figure 2. Detektor (2011) by Shintaro Miyazaki and Martin Howse.
Through a series of recordings, Detektor sonically and rhythmically exposes the infospheres that surround us as a critical inquiry into the ‘epistemic situation of contemporary urban life in the age of ubiquitous information networks and devices’ (Miyazaki Reference Miyazaki2013: 514). However, the application of higher-level sonification techniques such as event-based and model-based data representations is equally applicable as a media archaeological approach to reinterpreting historic relationships between different forms of media by utilising their specific qualities to interrogate media history. Applying such higher-level sonification techniques can facilitate an approach to inquiry through a creative exploration of, and change in, the expression of data to generate new interpretations between seemingly unrelated forms of media. A turn to media archaeology in the presentation of sonified data can also stimulate a rethinking of the performance of archival research. Listening to the archive in this way may produce unexpected sonic utterances of the past through new forms of articulation (Ernst Reference Ernst2016).
5. AN APPROACH FOR MEDIA ARCHAeOLOGICAL SONIFICATION
Explicitly representing data through a physically present artefact is a strategy that acknowledges the representation of sound as more than just itself. Such an approach can be used to specifically connect sound and source through historic cultural and technical relationships. In representing relationships between obsolete and contemporary media, the sound-producing capability of the artefacts under inquiry may be limited in relation to their melodic and harmonic qualities. Single parameters of sound or timbral homogeneity as a reductive process can be adopted as an aesthetic strategy in negating the over-musification of data. By adopting this strategy, a method of facilitating the organisation of sound is imperative. The development of a framework for the use of sonification techniques that is adaptable for use across a range of works can facilitate new perspectives for the representation of information within media archaeological inquiry. In the absence of strong melodic and harmonic material that may occur through the physical use of an artefact’s sonic byproducts (in other words, in the absence of musical pitch), a sensitivity to, as well as changes in, repetition and rhythm can be exploited to indicate changes in the state or value of sonified data (Neuhoff Reference Neuhoff2011). As an aesthetic strategy employing sonification, rhythm as repetition allows us to establish patterns of similarity and difference. Elizabeth Margulis (Reference Margulis2014) argues that musical repetition can be ‘involving – like a call and response’, inviting listeners ‘to participate in the phenomena’ by tracing out a musical path and then representing it for the listener to follow. The rhythmic form and familiarity of events through repetition produces a pleasure in the listening experience as an audience becomes cognitively aware of new elements or patterns as they emerge. If, from a musical perspective, repetition exposes temporal interrelationships then such repetition and familiarity of rhythmic events can give a sense of form and structure of the listening experience to sonified data voiced by the visibly present object itself.
The framework presented in Figure 3 has been developed as a basis for facilitating the organisation of sound within the context of media archaeologically informed sound-based works. This framework has been established to provide a more structured or practical approach as an aesthetic method of media archaeological representation in relation to works conceived, designed and constructed in this context. What has evolved is a framework that can be adapted and extended to examine relationships between past and present media utilising a range of such mechanisms and supporting its application within media archaeological research and sound-based art.
Figure 3. High-level media archaeological sonification framework.
The overarching approach here is based on media archaeological methods of inquiry. Therefore, the archive can refer to the multiplicity of information sources able to be referenced. In this context, interrelationships, both cultural and technical, are developed between different media. The framework uses the sender–message–receiver communication model as a useful heuristic for its representation. The sonification process transforms the data into a format that can be utilised to manipulate the sound-producing object. In the exemplar that follows, parameter mapping is the key sonification technique employed. Following the mapping, rhythmic elements are included to organise the data output and the expressivity of the sounding object. The alternative representation of data as a media archaeological method of inquiry is used to realise different outcomes unique in a work’s diversity of inquiry and intent. However, the insertion of data sonification techniques can be considered a defamiliarising agent of one form of communication, refamiliarised by way of another medium, sound. As a form of remediation, such a process can bring to light new dimensions of digital archives through the deformance of visual representation, expanding the range of sensory comprehension (Graham and Simons Reference Graham and Simons2021).Footnote 2
5.1. Parameter mapping as a sonification technique
Parameter mapping is, arguably, the ‘simplest way to map data to sound’ and the ‘most established and widely used sonification technique’ (Dubus and Bresin Reference Dubus and Bresin2013: 4). It maps data attribute values and changes in one or more of those values to trigger or effect a change or changes in the sonic event. Practitioners argue that parameter mapping, as an event-based process, should not be completely arbitrary or overly complex as the perceptual interactions of an abundance of varying parameters may obscure data relations and confuse the listener (Worrall Reference Worrall and Dean2009: 323; Dubus and Bresin Reference Dubus and Bresin2013). Neuhoff (Reference Neuhoff2019: 328) states that learning from past sonification research efforts should focus on ‘perceptual space where audition performs well, and individual differences are smallest’. He suggests abandoning simple parameter mapping dimensions such as pitch and loudness. Instead, leveraging ‘audition’s temporal advantage’ while exploiting spatialisation as an audiovisual aesthetic approach may prove more constructive (Neuhoff Reference Neuhoff2019: 329). Given the example work’s limited sonic palette, a reductive approach to parameter mapping may have the advantage of eliciting the most interesting results from small changes in the data employed by it.
Click::TWEET, presented in the following section, has been conceived, designed and constructed in relation to the framework. The general approach has been to extract information from a visual form of media, transform it and re-present it aurally. Accordingly, the use of parameter mapping as a data sonification technique to facilitate the organisation of sound is a key element of the work.
5.2. Click::TWEET: a material media sonification
Using an approach grounded in media archaeology, Click::TWEET (Figure 4) has been developed in conjunction with the use of the framework. The work uses six telegraph keys as the primary sound objects. The work questions whether social media is entirely a twenty-first century phenomenon by considering the electric telegraph as an early form of social media. In doing so, the sound-based artwork reinterprets historic relationships and connects the telegraph and Twitter as historically related media within a broader history of social media. With sound no longer manipulated and rendered solely through digital audio software, Click::TWEET expresses itself using the sound of the telegraph key’s mechanism through indexical relationships and the re-presentation of Twitter data. Playing the messages in this way re-presents, what some consider, one noisy medium (the sociocommunication of social media) through another (acoustic key clicks) to create a presence of telegraphic communication within the contemporary realm of social media. Beyond the exploration of the telegraph key as the sound-producing object and its specific technical characteristics and the telegraph’s signaling properties, the work considers some common sociocultural aspects during the ideation and creation of the work. As such, placing the telegraph key in a new artistic context – making the familiar strange – alongside Twitter as social media creates an awareness of characteristics manifest in earlier media that maintain a presence in contemporary media.
Figure 4. Click::TWEET.
Contextualising Twitter within a general history of communication media reveals similarities with, and departures from, the electric telegraph. Arguably, both mediums bought an immediacy and brevity to communication, compressed space and time and brought the private into the public. As such, the telegraph provided a significant advance in the global reach and immediacy of communication, an advancement amplified with social media platforms such as Twitter. In a similar manner to criticism of the telegraph that it would bring the downfall of traditional forms of communication, Twitter has been criticised for potentially threatening longer length forms of electronic communication. Each medium has exerted an influence that has transformed relationships, facilitated new cultural networks, and mediated and changed people’s everyday experiences. Interacting with these media has impacted the written word through the use of abbreviated forms of language but also led to the rise of expressive forms of writing by adopting the modalities of telegraphic language in poetry and utilising Twitter’s brevity and conciseness communication as a form of micro-poetry. As such, both the telegraph and Twitter’s economies of resource have been utilised as creative aesthetics.
Morse code is a text-based communication medium. As such, the rich content such as emojis, video and some text characters able to be included in a tweet cannot be represented by Morse code. Therefore each message’s content is reduced to its base Morse elements by removing symbols that are not able to be represented in the Morse system. If both Twitter and telegraphic communication have received criticism for their brevity threatening longer forms of communication and a general dumbing down of society through the ‘impoverishment of grammar, vocabulary, spelling and so on’, Click::TWEET’s need to dumb down Twitter’s messages can be perceived as being situated within the moral decline of communication of which both technologies have been accused (Murthy, Reference Murthy2013: 48).
Click::TWEET uses six telegraph keys as the primary sound objects. As International Morse code uses six elements to represent its extended character set, Click::TWEET’s use of six telegraph keys is directly related to the maximum number of Morse elements used to construct these character representations. The ability to spatialise the Morse elements across the set of keys is used as a compositional method to interpret and enhance the patterns and rhythms within the encoded message. The temporal manipulation of the data is based on Morse code’s signalling properties. The tempo for the message replay is based on Morse code’s word method for determining a standard transmission speed. Based on this method, 125bpm (60ms) and 150bpm (50ms) are used as the dot duration time parameter and mapped accordingly to each Morse element (see Table 1).
Table 1. Morse elements and determination of measure
Once stripped of unsupported content, each Twitter message is reduced to individual characters that are mapped to their Morse encoded equivalent representation. This encoding is then separated into individual symbols of dots and dashes. For asynchronous message playback each symbol representing the Morse encoded character is mapped to a physical telegraph key. The audiovisual spatialisation of the message replay draws further attention to the rhythmic patterns inherent in the Morse code through the sonic and physical nuances of each key. Synchronous message playback maps individual messages to each key. In this way new patterns, such as syncopated rhythms, may emerge as keys engage and disengage and messages end at different times. Each message, in either form, is experienced as a rhythmic performance.
Through a series of rhythmic compositions, Click::TWEET leverages audition’s temporal advantage through the repetition of interwoven patterns of sound and/or the disruption of these, either naturally or by intervention, or through an audience’s perception of a periodic pulse or ‘beat’. The use of such rhythmic methods as compositional elements has been contextualised within Henri Lefebvre’s characterisations of rhythmanalysis. Presented as eurhythmia (rhythms of equality), polyrhythmia (rhythms of diversity) and arrhythmia (rhythms of disturbance), these forms are a way of exploring the sounds and rhythms inherent within the data utilised in each work. Facilitated by employing parameter mapping as a data sonification technique, each compositional form provides a method of exposing the rhythms of machinic processes and, following Neumark, allows an audience to listen to machines voicing themselves. An example of the framework applied to Click::TWEET is shown in Figure 5.
Figure 5. Application of the framework for Click::TWEET.
By sonifying Twitter messages as Morse encoded transmissions, a machine-like prosody inherent in Morse code’s signalling logic can be heard in the rhythms of the replayed messages. Click::TWEET, as a creative work, can be considered a return to more traditional modes of transmission as a form of ‘sonic orality’ where information was transmitted through the medium of sound.Footnote 3
Click::TWEET can be considered a return to the material representation of media through the physical re-presentation of Twitter messages transported as invisible digital media through the physical materiality of the telegraph key. As such, by re-engaging the material analogue world in tandem with the digital it can be considered a return to the tactility of pre-digital media.
Utilising media archaeology as an creative approach and utilising the material presence of, and foregrounding, the sound-producing object to explicitly connect sound and source allows Click::TWEET to be expressed through the ‘voice’ of the artefact. Representing relationships between the visual domains of the original sources and the acoustic domains expressed through the obsolete artefacts, each composition gives a voice to what would more generally be considered immaterial or disembodied objects. The use of this method becomes evident when listening to the machine-like prosody of Twitter messages replayed through the telegraph keys. This approach to data sonification as a media archaeological reinterpretation of historical relationships is a way to unlock the past by utilising the specific sonic qualities of the source materials. In this way, data, as an abstraction of reality, is given concrete form through its sonification in Click::TWEET.
With a constrained sonic palette, Click::TWEET utilises the sound of the telegraph key along with the electronic sound of a sine tone associated with telegraphic signal transmission and the temporal properties associated with Morse code’s signaling elements. The work iterates the artefacts and sound as a further audiovisual strategy to enhance the use of the limited sound parameters.
6. DISCUSSION
This article has introduced a novel symbiosis of data sonification with media archaeologically informed sound-based art. This approach brings related artefacts taken from different moments in time into dialogue with each other and this article has presented a framework for employing data sonification techniques as a media archaeological approach. These works have been presented as a method of validating the application of such a framework to develop sound-based artworks within a media archaeological context. In these works, sound is no longer abstracted from its source(s), then manipulated and rendered solely through digital audio software as an arbitrary interpretation of data relationships (as it is in electronic music practices, for example). Through a process of transformation, the work’s source information is defamiliarised then refamiliarised as a series of sound events. The transformation creates a deformance as an interpretative reading of the past in the present to expose relationships between the obsolete and contemporary forms of media (Graham Reference Graham2016). The work expresses itself in its own voice through indexical relationships where an interpretative sign or series of signs are used to refer to a symbol or object. For example, — (dash, dash, dash) in Morse code refers to the letter O. By reducing the message content through such translations, the messages are listened to at an indexical level rather than for semantic meaning through historically related artefacts drawn from forms of communication media. While representing different media relationships, the compositions share a common approach where information is extracted as data, then transformed and re-presented in another medium. Through this process, a strong interlinking principle within these is that sound events are mediated through data sonification techniques.
This interdisciplinary combination of data sonification and media archaeology provides a new perspective for the representation of information. Employing data sonification techniques to facilitate the organisation of sound, Click::TWEET explores the musicality of numeric sequences in space and time through the rhythmic and spatialised patterns created by the absence and presence of data. Doing so offers new combinatory representations of the past through data sonification, its transcoding, transposition and transference from one medium to another. This approach can be one method of transporting data between artefacts that are distant in time and/or place, thus extending our spatio-temporal perception of relationships and recontextualising historical meanings (Sinclair Reference Sinclair2012). Using data as an input control for some sound-producing mechanism is a way of making aspects of a data set or system perceptible, thus allowing the examination of, and the drawing of inferences from, the artefact to gain an insight into the subject of the work. Click::TWEET shows that sonically engaging results can be obtained by creating abstract relationships between source data and the sounding object.
As an art method, defamiliarising obsolete and contemporary media relationships through media archaeology approaches combined with data sonification techniques provides a new perspective for the listener’s engagement with, and interpretation of, sound by hearing the present through the past. By interfacing the appropriate data source with a form of obsolescent media, a sonically interesting interpretation of data can be achieved. Connecting the past to the present in this way extends the temporal boundary from which new phenomena may emerge when listening to the sounds of these works.
Debates about whether data sonification is science or art will continue. Where the practitioner sits on the science/art continuum will depend on their discipline or their intention for using data sonification. However, the interdisciplinary combination of sonification and media archaeology can form an harmonious association between such divergent ways of thinking. This association, expressed through the physical artefact and listened to as consonant and dissonant points of difference, can open new creative potential and ideas by attentively listening to what each voice has to say.
Open access
