2. CONCEPTUAL FRAMEWORK

The pedagogical approaches presented in this article are underpinned by the social model of disability. According to this model, DHH is viewed as an interaction ‘between impairment and the surrounding social world, rather than being an individual medical problem’ (Emens Reference Emens2012: 214). Restrictions faced by individuals with DHH in their social interactions are a result of institutional forms of exclusion and cultural attitudes deeply rooted in the society, which fails to provide equitable social and structural support meeting their specific needs (Terzi Reference Terzi2004). In contrast to the dominant medical model, which considers disability as a problem to be remedied to conform with normative standards (Paley Reference Paley2002), the social model perceives disability as a societal issue rather than solely an individual problem (Cameron Reference Cameron2009). The social model is also different from the cultural model of disability, with the latter advocating for a ‘similar and different’ view that recognises and celebrates disability as part of our lives (Devlieger, Rusch and Pfeiffer Reference Devlieger, Rusch, Pfeiffer, Devlieger, Rusch and Pfeiffer2003). Apart from these models of disability, there are scholarly efforts and perspectives from DHH communities advocating that deafness should not be considered a form of disability (Lane Reference Lane2002; Harvey Reference Harvey2013). A spectrum of alternative terms, including ‘marginal’, ‘bicultural’ and ‘handicapped’, are used to describe the identity of individuals with DHH (Stumer, Hickson and Worrall Reference Stumer, Hickson and Worrall1996; Chapman and Dammeyer Reference Chapman and Dammeyer2017).

Historically informed performance practice in music necessitates physical capabilities and sensorimotor skills to manipulate musical instruments, which can pose challenges for individuals with certain forms of functional diversity (Howe Reference Howe, Howe, Jensen-Moulton, Lerner and Straus2016). Moreover, established instrumental designs and rigid school music curricula have limited flexibility to accommodate specific needs (Challis Reference Challis, Norman and Kirakowski2018; Rizzo Reference Rizzo2022), further exacerbating the inequality in music and audio education for DHH learners. As a result, these learners may lack the ability to conform to socially constructed aesthetic values in certain musical cultures and contexts (Lubet and Hofmann Reference Lubet and Hofmann2006).

Meeting the needs of DHH students in the music classroom or teaching studio may require substantial changes in both the social and the physical environment (Siebers Reference Siebers2008). In addition to modifying the physical space, it is essential to address the social dynamics between students and teachers to counteract the perpetuation of socially constructed injustices. Lubet (Reference Lubet2009) argues that music teachers should actively engage in political actions against systemic ableism in various educational contexts. Bell (Reference Bell2017) suggests exploring alternative approaches in the design of music teaching activities. Rather than adhering strictly to historically informed instrumental practices, music education can be reimagined to incorporate digital musical instruments (DMIs) and hacking activities (Bell Reference Bell2015; Bell, Bonin, Pethrick, Antwi-Nsiah and Matterson Reference Bell, Bonin, Pethrick, Antwi-Nsiah and Matterson2020), which offer increased accessibility for all learners. This sentiment is echoed by Landy (Reference Landy2012), who advocates for music education to embrace and celebrate the diversity of musical expressions.

3. RELATED WORKS

Music instruction for individuals with different levels and types of hearing loss continues to face practical challenges, largely due to misconceptions surrounding the ability of individuals with DHH to hear and appreciate music (Darrow Reference Darrow1985). Earlier studies have highlighted that instrumental instructors often exhibited hesitancy in actively recruiting DHH students, primarily stemming from a lack of familiarity with the musical capabilities of deaf learners (Darrow and Gfeller Reference Darrow and Gfeller1991), or concerns that these musicians may have a negative impact on ensemble performance quality (Sheldon Reference Sheldon1997). Music educators have often perceived individuals with hearing loss, alongside other behaviourally or emotionally disadvantaged students, as the most challenging and exceptional populations within mainstream classroom music education (Gfeller, Darrow and Hedden Reference Gfeller, Darrow and Hedden1990).

Darrow (Reference Darrow1993) discovered that individuals with hearing loss derive enjoyment from singing or signing songs, listening to music, and engaging in movement and dance with music, similar to their hearing counterparts. She suggested that individuals with hearing loss often exhibit a stronger sense of rhythm than pitch-related abilities, and are more proficient in discriminating lower frequency ranges compared with higher ranges (Darrow Reference Darrow2007). While DHH students perceive and communicate music differently from others, they can still find pleasure in music (Chen-Hafteck and Schraer-Joiner Reference Chen-Hafteck and Schraer-Joiner2011; Vaisberg, Martindale, Folkeard and Benedict Reference Vaisberg, Martindale, Folkeard and Benedict2019). Research has shown that individuals with DHH are more capable of recognising rhythm and tone duration than pitch and melody, while their ability to differentiate between timbres is generally diminished (McDermott Reference McDermott2004; Drennan and Rubinstein Reference Drennan and Rubinstein2008; Looi, McDermott, McKay and Hickson Reference Looi, McDermott, McKay and Hickson2008). Vaisberg, Beaulac, Glista, Macpherson and Scollie (Reference Vaisberg, Beaulac, Glista, Macpherson and Scollie2021) investigated the preferred frequency-gain shaping for speech and music perception in individuals with cochlear implants. Their study found that low-frequency gain was significantly increased relative to the prescription for speech and music stimuli, and that gain adjustments varied among listeners. The findings also revealed that music preferences were driven by changes in perceived fullness and sharpness, while speech preferences were driven by changes in perceived intelligibility and loudness. The study suggested that prescribed amplification to optimise speech intelligibility and alternative amplification for music perception should be recommended for most listeners. This implies that a prescribed and personalised audio workstation could be a viable approach to facilitating the learning process of DHH students in music and audio production.

Current practices often incorporate other sensory inputs, such as visual and tactile cues, to address the challenges faced by individuals with DHH (Trivedi et al. Reference Trivedi, Alqasemi and Dubey2019; Deja, Torre, Lee, Ciriaco and Eroles Reference Deja, Torre, Lee, Ciriaco and Eroles2020; Hopkins, Maté-Cid, Fulford, Seiffert and Ginsborg Reference Hopkins, Maté-Cid, Fulford, Seiffert and Ginsborg2021). These approaches have shown effectiveness in musicianship training across different levels of hearing loss. However, the competencies required for music producers and sound designers extend beyond the conventional musicianship framework and encompass skills such as design thinking, computer literacy and studio practices (Alsop and Berry Reference Alsop and Berry2009). This indicates that teaching and learning approaches in music production and sound design can differ significantly from those in general music education (Hug and Kemper Reference Hug and Kemper2014). While some existing practices in music education can be adopted, the strategies proposed in this article focus on sound processing and audio editing within DAWs and other related hardware and software to enhance the learning and workflow of music production and sound design.

4. SOUND VISUALISATION

Sound visualisation has proven to be a valuable assistive tool for audio recording and editing, particularly in optimising volume and timbre (Lima, Dos Santos, and Meiguins Reference Lima, Dos Santos and Meiguins2021). It also serves as a useful tool to enhance situational awareness for people with DHH (Azar, Saleh, and Al-Alaoui Reference Azar, Saleh and Al-Alaoui2015), compensating for the absent of or reduced auditory perception. To facilitate the workflow of DHH audio engineers, guidance can be prepared by other teammates providing information about the desired levels, relative distances between sources and the microphone, reflective surfaces and background noise sources prior to recording. This allows a DHH audio engineer to choose appropriate recording devices and estimate input levels and gain without relying on real-time auditory monitoring. Background noise levels can be visually displayed on a meter to ensure they remain below the desired sound sources. If audio metering is not available in the hardware device being used, there are numerous assistive mobile apps that can serve as substitutes. Many of these apps are free and can be connected to the audio source via line or auxiliary (Aux) audio output, with display format configurations typically available without much effort. A test recording is always recommended, so that recorded output can be visually assessed to identify any issues such as a high noise floor and excessive reverberation. Impulsive sounds are recommended for testing purposes, particularly indoors, to identify and minimize any undesired eigentones using equalisation or downward expansion techniques during post-production processing and sound-effect editing. Visual cues can also assist in identifying suitable edit points and confirm the source, intensity and nature of interactions depicted in filmed content. When using an external microphone, the best setup can be achieved by maintaining a hand’s distance between the microphone and the sound source. This ensures off-axis delivery and helps prevent plosives, which can be tested by pronouncing the letter ‘P’ in front of the microphone.

In different stages of production, various types of sound visualisation can provide valuable information. The peak programme meter (PPM) is commonly used to measure volume levels and identify instances of clipping. Waveforms, displayed in the sound editing software, illustrate the dynamic range of audio, aiding in determining the appropriate amount of compression to apply. Spectrograms, although less commonly used in everyday audio engineering, can be helpful in identifying unwanted sounds and enhancing specific resonances.

Visual matching of content can also be employed to ensure consistency with other takes or reference audio files. Some software offers visual analysis of audio clips, providing measurements such as root mean square (RMS) and peak and loudness unit full scale (LUFS). This information is useful for ensuring consistent volume levels among different sources, particularly when working with voices of different genders and age groups, as the spectral content may vary significantly, making it challenging to hear certain voices in comparison.

5. HAPTIC FEEDBACK

Haptic feedback refers to the use of mechanical vibrations to provide alternative sensory feedback. An early example of this approach can be found in the rumoured story of Beethoven, who modified his piano by removing its legs to allow for the perception of sound through tactile vibrations felt at floor level (Wallace Reference Wallace2018). Technological advancements nowadays have made haptic feedback more approachable, and numerous affordable haptic devices have been made available that can translate sound into vibrations. These devices can be particularly useful in sound editing and mixing, as they enhance the perception of timing and facilitate the differentiation and similarity of signals in a temporal manner akin to sound itself (Beattie, Frier, Georgiou, Long and Ablart Reference Beattie, Frier, Georgiou, Long and Ablart2020). The field of haptic technology is continuously evolving, and the use of ultrasound for haptics has opened up new possibilities, including the potential for depth perception (Morales, Marzo, Freeman, Frier and Georgiou Reference Morales, Marzo, Freeman, Frier and Georgiou2021).

In addition to haptic devices, haptic feedback can also be achieved through the vibrations of loudspeaker cones. Touching the loudspeakers is not uncommon in the audio production process, which allows DHH audio engineers to perceive the relative timing and intensity of different signals, particularly for the extended bass content. Furthermore, with the growing trend of science, technology, engineering, art and mathematics (STEAM) and hacking, individuals with DHH have the opportunity to develop their own do-it-yourself (DIY) haptic feedback devices (Andreotti and Frans Reference Andreotti and Frans2019; Bell et al. Reference Bell, Bonin, Pethrick, Antwi-Nsiah and Matterson2020). This DIY approach empowers individuals to explore and customise haptic solutions according to their specific needs and preferences.

6. NON-LINEAR EDITING

Non-linear editing involves the manipulation and arrangement of audio clips or tracks in a non-sequential or non-linear manner, providing flexibility in editing and manipulating audio recordings beyond on-site and real-time production processes. This approach allows for the integration of assistive tools to facilitate the workflow of audio engineers and sound designers. Such tools include context-aware editing platforms for text-based speech (Morrison, Rencker, Jin, Bryan, Caceres and Pardo Reference Morrison, Rencker, Jin, Bryan, Caceres and Pardo2021), algorithms for sound source recognition (Yang Reference Yang2021), and neural networking-based plug-ins for automatic compression (Singh, Bromham, Sheng and Fazekas Reference Singh, Bromham, Sheng and Fazekas2021). However, these assistive tools may not fully capture the underlying emotions conveyed in the text or the applied sound effects in the soundtracks. To address this issue, editors can utilise mirrors to confirm the mouth shapes of voice actors and interpret the emotional intent of the dialogue through lip reading. In cases where non-linear editing involves voiceover recordings with separate sound and motion, editors can keep track of word start and stop times and sentence completion to facilitate editing and ensure natural speech pacing. Additionally, visual references can be employed to verify the timing of audio effects. By capturing video simultaneously with sound-effect recording, the audiovisual material can be imported into the video editing software, allowing for concurrent viewing of visual content alongside sound. The visual reference provides valuable information for the sound designer regarding the timing and source of sound effects, which can be hidden in the final export.

While many of the aforementioned approaches are applicable to individuals with different types of DHH, there are specific strategies that can enhance the workflow for those who are hard of hearing but not completely deaf. These individuals may have significant hearing loss but can still be able to understand speech through auditory processes and/or special adaptations (Heward Reference Heward2006). In non-linear and non-destructive editing, for instance, temporary pitch shifting can be employed to lower the audio content down into an audible range for a sound editor with upper frequency hearing loss. Once the editing process is complete, the audio content can then be restored to its original frequency ranges. Destructive editing procedures can be logged in the pitch-shifted copy and then applied to the original audio content to avoid degradation.

7. AUTOMATED TRANSCRIPTION

After the non-linear editing process, the clarity of the dialogue can be reaffirmed through automated transcription. Any discrepancies between the script and the transcribed text can then be identified for further editing. However, cultural and environmental factors can impact the accuracy of transcription. For example, the English dialogue may sound correct locally in a non-English-speaking country but may not be accurately transcribed in a particular transcription platform. Additionally, the choices for non-English languages may be limited in those transcription services. This is coupled with other issues such as the differences in speech and writing, and the impact of background sounds on the accuracy of transcription.

Many online video sharing and social media platforms, as well as certain video editing software, offer the feature of automatic captioning for user-uploaded videos. Numerous studies in the existing literature have explored the application and effectiveness of automatic captioning in areas such as language learning and deaf culture (e.g. Smith, Crocker and Allman Reference Smith, Crocker and Allman2017; Perez Reference Perez2022). DHH sound editors can utilise auto-captioning services to identify poorly pronounced dialogue and take follow-up actions. This process is particularly valuable in editing dialogue in animation and games, where retakes of the voice-overs are common.

In addition to speech-to-text captioning, automated transcription is also available for certain musical materials. Established techniques in music information retrieval (MIR) enable accurate transcription of recorded music into organised and structured musical data, such as MIDI and XML files (Benetos, Dixon, Duan and Ewert Reference Benetos, Dixon, Duan and Ewert2019). These technologies have the capability to separate music clips into distinct components such as instrumental tracks (Kumar, Biswas and Roy Reference Kumar, Biswas, Roy, Johri, Verma and Paul2020), extract melodies sourced from different timbres (Hernandez-Olivan, Zay Pinilla, Hernandez-Lopez and Beltran Reference Hernandez-Olivan, Zay Pinilla, Hernandez-Lopez and Beltran2021), and detect the genre of a particular piece of music (Estolas, Malimban, Nicasio, Rivera, Pablo and Takahashi Reference Estolas, Malimban, Nicasio, Rivera, Pablo and Takahashi2020). Many of these applications are commercially available as standalone software or DAW plug-ins (Benetos et al. Reference Benetos, Dixon, Duan and Ewert2019). Parameters of these musical elements can then be identified through visual or haptic feedback, enabling DHH sound engineers to edit music based on their technical knowledge. In some cases, it may even be possible to make changes to the musical content of a recorded performance. Operating systems, such as those from Apple and Microsoft, are gradually incorporating sound analysis and peripheral functions into their integrated development environments (IDEs), which will lead to the availability of more accessible technologies beyond their current applications primarily in the industrial and healthcare sectors.

8. DIGITAL SIGNAL PROCESSING

Noise reduction can pose challenges as the sensitivity to noise varies among individuals with different levels of hearing loss (Heinonen-Guzejev et al. Reference Heinonen-Guzejev, Jauhiainen, Vuorinen, Viljanen and Rantanen2011). DHH audio engineers and sound designers can employ various techniques to address this issue. They can record sample audio clips of background ambience beforehand and compare them to the recording of the actual take to identify the noise floor and spectral balance of typical sound sources through sound visualisation tools. Compression and downward expansion can be applied to optimise levels and reduce noise with visual reference to waveform displays. Similarly, sound equalisation can be optimised by comparing spectrograms. De-essing can be achieved using a similar approach to the aforementioned noise reduction techniques. Pronunciation of sibilant letters such as ‘F’, ‘X’ and ‘S’, as well as other soft consonants, can lead to a significant increase in levels that require correction. These peaks can be identified through waveform displays and by identifying candidate words using subtitles.

While mastering is often outsourced to specialists, smaller studios and projects may involve audio engineers handling it themselves. In this case, DHH audio engineers may face challenges, as their hearing conditions can affect their precision in critical listening. However, advancements in audio technology have made digital mastering more accessible and affordable compared with analogue mastering. The inclusion of digital signal processing has led to the development of sophisticated algorithms and software tools that automate various mastering processes (Birtchnell Reference Birtchnell2018), including parametric equalisation and compression/expansion. These parameters can be optimised by reviewing and benchmarking the final mix with reference material such as broadcasted programmes and commercially available music products, using spectrograms, waveforms displays and metering.

9. IMPLICATIONS FOR EDUCATORS

In the previous section, we proposed various approaches to facilitate the learning process of DHH students in audio production and sound design, including sound visualisation, haptic feedback, automated transcription, tactics in non-linear editing and digital signal processing. These approaches are not overly demanding in terms of technical skills or resources and can be effectively integrated into different educational contexts and curricula. While there have been efforts to promote EDI in the fields of electroacoustic music and sonic art, it is important to also consider the specific needs of DHH students within the constraints of the mainstream music curriculum in school learning environments. The increasing integration of technology in school music education presents opportunities for the design and implementation of more accessible and inclusive pedagogical approaches (Ruthmann and Manite Reference Ruthmann and Manite2018), as proposed in this article.

Additionally, it is beneficial to select music that incorporates varied rhythms and tone durations, as these elements are particularly accessible to DHH students (McDermott Reference McDermott2004; Drennan and Rubinstein Reference Drennan and Rubinstein2008; Looi et al. Reference Looi, McDermott, McKay and Hickson2008), which can help enhance inclusiveness in the school music teaching and learning environment. If feasible, music with more pronounced drum and bass elements can be options for students with upper frequency hearing loss. The sequence of skills taught in the lesson content can be optimised to better accommodate DHH students. For example, techniques such as pitch shifting and band filtering can be taught prior to other DAW skills, which facilitates a smoother learning progression for those students and nurtures their aesthetic awareness and creativity (Daniel Reference Daniel2020).

In higher education contexts, such as undergraduate and postgraduate degree programmes in sound design and music production, alternative teaching and learning arrangements can be arranged for students with different levels of hearing loss. These include the development of self-directed learning resources that incorporate the aforementioned approaches, tailored practical tasks and assessment methods for DHH students. Detailed rubrics can be created in consultation with students and programme leaders, and advice can be sought from the university’s equality and diversity officers, if necessary. A successful example can be found in the Accessible Podcasting programme ratified by the Canadian Hard of Hearing Association and offered by Seneca Polytechnic,Footnote ¹ where the second author serves as an advisory committee member.

In addition to the pedagogical approaches outlined in this article, the successful integration of DHH students into music and audio education can be further facilitated through teacher training and professional development in special needs education (Wolf and Younie Reference Wolf and Younie2019). It is essential to provide instructional support in the classroom environment and ensure that music objectives are clearly defined, with appropriate resources available to support DHH students (Darrow and Gfeller Reference Darrow and Gfeller1991). The development of an inclusive audio education and music curriculum is crucial to enhancing the effectiveness and appropriateness of learning content for students with hearing loss and other special learning needs, thus promoting a broader integration of EDI (Gouge Reference Gouge1990). These aspects are vital in creating an inclusive and supportive learning environment that caters to the diverse needs of all students, including those with hearing loss.

10. CONCLUDING REMARKS

This article contributes to the current body of knowledge by challenging established norms in mainstream music and audio education, presenting pedagogical approaches for inclusive teaching and learning practices. These approaches aim to make music and audio education more accessible and inclusive for DHH students without requiring significant additional resources. By modifying teaching practices, DHH students can learn to become sound engineers, producing audio and musical content for their hearing counterparts, who form the majority of consumers in the music industry and media sector. They can also create content that is meaningful to their community, a contribution that may not be achieved by hearing sound engineers. Moreover, some of these approaches can benefit other students by providing alternative perspectives and ensuring the quality of their work. This fosters mutual understanding between students with and without hearing loss, thereby promoting a more inclusive sonic environment.

In the broader context of electroacoustic music studies, the proposed pedagogical approaches have the potential to empower DHH individuals to advance their sonic practices and promote the wider integration of EDI. This integration should encompass the inclusion and representation of marginalised groups across all dimensions of identity, including individuals with DHH who may have often been left out of the equation when it comes to music and sound (Gouge Reference Gouge1990). By embracing EDI, the fields of electroacoustic music and sonic art can flourish as diverse and vibrant spaces that value and celebrate the contributions of all individuals, ensuring that functional diversity is not overlooked in the realm of music and sound.