1. Introduction
A common technique for eliciting auditory verbal hallucinations (AVH) is to present people with a masking sound and suggest to them the presence of a target sound under the masker. The most famous demonstration of this (using music rather than plain speech) is the White Christmas effect in which participants are played stochastic (‘white’) noise and told to indicate when they hear a snippet of the Bing Crosby song ‘White Christmas’ underneath the noise (hence the name of the effect). Many participants report hearing the song, despite there being nothing in the audio signal but stochastic noise (Barber & Calverley, Reference Barber and Calverley1964). Several replications have demonstrated the effect with speech as well (e.g., Feelgood & Rantzen, Reference Feelgood and Rantzen1994; Fernyhough et al., Reference Fernyhough, Bland, Meins and Coltheart2007; Hartley et al., Reference Hartley, Bucci and Morrison2017). Several modulators of the effect have been explored, including caffeine (Crowe et al., Reference Crowe, Barot, Caldow, D’Aspromonte, Dell’Orso, Di Clemente, Hanson, Kellett, Makhlota, McIvor, McKenzie, Norman, Thiru, Twyerould and Sapega2011), fantasy proneness (Merckelbach & van de Ven, Reference Merckelbach and van de Ven2001), hypnosis (Barber & Calverley, Reference Barber and Calverley1964), cognitive suppression (Rassin & Heiden, Reference Rassin and van der Heiden2006) and stress (Hoskin et al., Reference Hoskin, Hunter and Woodruff2014). The White Christmas effect is structurally similar to phoneme restoration (discussed below) in which people hear a deleted speech sound as present when the sound’s empty position is masked. Based on this similarity, and the dependence of phoneme restoration on masking quality, the current experiments explore whether similar masking relationships modulate the White Christmas effect.
AVH have been induced using methods different from, but resembling, the White Christmas effect. For example, B.F. Skinner presented participants with a recording of overlapping indistinct vowel sounds, leading to reports of hearing words (discussed in Deutsch, Reference Deutsch2019). Young et al. (Reference Young, Bentall, Slade and Dewey1987) asked participants to simply imagine sounds (and visuals). Many participants reported having vivid sensory experiences, though none reported believing the experiences to be genuine percepts and so this is not clearly a demonstration of hallucination. In a signal-detection set-up very similar to the White-Christmas methodology, Frost et al. (Reference Frost, Repp and Katz1988) played recordings of words masked with noise modified to have the same amplitude profile as the words. This masked audio-stimulus was presented alongside text of the word, leading participants to report hearing the word more vividly. In another approach, Feelgood and Rantzen (Reference Feelgood and Rantzen1994) were able to induce AVH in non-clinicalFootnote 1 participants by having them listen to spliced together backward recordings of a male voice. There is also the similar verbal transformation effect in which simply playing the same word or phrase on a loop leads a hearer, after sufficient repetition, to start hearing the loop as containing different words (Reisberg, Reference Reisberg1989; Warren & Gregory, Reference Warren and Gregory1958). Diana Deutsch (Reference Deutsch2003) has produced a version of this illusion, the methods for which are available for download at http://philomel.com/phantom_words/pages.php?i=1010. In this version, a two-word (or two-syllable) sequence is played from speakers positioned to the left and right of the listener, with a slight temporal offset. The listener will typically begin to hear words other than those being played.
1.1. Auditory verbal hallucinations
AVH, or hearing voices, is most commonly associated with schizophrenia, and is indeed one of the key diagnostic symptoms of the disease (Frith, Reference Frith1992). However, hearing voices is not limited to people with schizophrenia, and is, perhaps surprisingly, a common experience in the general population, with approximately 9.6% of people with no known psychological issues reporting the experience (van Slobbe-Maijer, Reference van Slobbe-Maijer2019).Footnote 2 The Diagnostic and statistical manual of mental disorders (2013, 23) provides a definition of hallucination:
Hallucinations are perception-like experiences that occur without an external stimulus. They are vivid and clear, with the full force and impact of normal perceptions, and not under voluntary control.
To account for the fact that many hallucinations are more likely to occur in the presence of certain sensory stimuli, Slade and Bentall (Reference Slade and Bentall1988, 23), have proposed a broader definition:
Any percept-like experience which (a) occurs in the absence of an appropriate stimulus, (b) has the full force or impact of the corresponding actual (real) perception and (c) is not amenable to direct and voluntary control by the experiencer.
This definition would include the sounds heard in the White Christmas effect as hallucinations, since, while there is an auditory stimulus inducing the experience, there is not an appropriate auditory stimulus. Under this definition, however, the distinction between hallucination and illusion becomes blurred. An illusion is usually considered a non-veridical interpretation of a sensory stimulus, rather than a perception in the absence of a sensory stimulus (e.g., Hoffman, Reference Hoffman, Blom and Sommer2012). In the White-Christmas effect, participants listen for sounds buried under a masking sound, and so the effect could be interpreted more along the lines of an illusion than a hallucination (i.e., misinterpreting components of the masking sound as being speech or music). Under this interpretation, the White-Christmas effect is an illusion very similar to the phoneme restoration effect, in which people hear a speech sound that has been artificially removed from a recording and replaced with a masking sound, such as a cough (Samuel, Reference Samuel1996; Warren, Reference Warren1970). People generally do not notice that a phoneme has been replaced and, when asked, are usually unable to determine at what point in the word the masking sound occurred. It is generally assumed that the lexical and phonological context in which the missing phoneme is situated provides an expectation of hearing the particular phoneme and that this expectation allows the missing sound to be heard under the masking sound (Srinivasan & Wang, Reference Srinivasan and Wang2005). There is no sharp boundary between illusion and hallucination, however, as no sensory experience happens without a context. Even in a sensory deprivation situation there is background sensory noise internal to the body itself that provides potential for sensory misinterpretation. We are not particularly concerned about whether the White Christmas effect should be categorized on one side or the other of the hallucination/illusion divide; however, since the term ‘hallucination’ has generally been used in the White Christmas literature, we will retain that label here for the sake of consistency.
There are two primary theories about the mechanism of AVH. One argues that the voices are the result of a misattribution of an internal mental event, such as a memory or a person’s own inner speech (e.g., Bentall et al., Reference Bentall, Baker and Havers1991; Brookwell et al., Reference Brookwell, Bentall and Varese2013; Frith, Reference Frith1992). Hugdahl has proposed a variation on this model which assumes that the mental event is perceptual and explains misattribution as the result of a failure of inhibitory control (Reference Hugdahl2009). The other approach suggests that voice hearing is the result of a signal detection bias in which the hearer’s own expectations lead to the misinterpretation of an external signal (e.g., Alganami et al., Reference Alganami, Varese, Wagstaff and Bentall2017; Bentall & Slade, Reference Bentall and Slade1985). This explanation of AVH, as with the White Christmas effect, blurs the line between hallucination and illusion as the ‘hallucination’ is being caused by misinterpretation of an external sensory signal (rather than in the absence of a sensory signal). These two potential mechanisms for explaining AVH are not mutually exclusive, as the biasing top-down expectation in the signal-detection viewpoint is often assumed to be an internal mental event (Moseley et al., Reference Moseley, Smailes, Ellison and Fernyhough2016). This literature is reviewed in Toh et al. (Reference Toh, Moseley and Fernyhough2022).
The signal-detection theory of AVH would seem to be an obvious candidate for explaining the White Christmas effect, given that the White Christmas effect relies on biasing the interpretation of a masking sound to hear a (missing) signal. However, it is plausible that it is a mental event (such as a memory of a word or the word being spoken in inner speech) that provides the top-down expectation that influences processing of the masking sound in the White Christmas effect. Given this, both theories on the mechanism of AVH may be relevant to the White Christmas effect. It is possible that there are in fact multiple mechanisms that trigger voice hearing and thus different types of voice-hearing experience. This is suggested by the fact that the phenomenology of voice hearing is quite diverse – some people hear a single voice, while others hear multiple voices; some people hear the voice as located in external space, while others hear voices that are experienced internally (Toh et al., Reference Toh, Thomas, Robertson and Rossell2020).
The White Christmas effect induced in the experiments reported below is with people not suffering from any known mental illness (which is how the White Christmas effect is normally used). The question remains as to whether the voice hearing of non-clinical populations can be informative about clinical populations. As reviewed in Toh et al. (Reference Toh, Moseley and Fernyhough2022), the discontinuous model argues that the voices associated with psychosis have a different etiology from those in non-clinical populations, while the continuum hypothesis argues that these experiences are essentially the same and differ simply in degree. The current experiment does not address this debate. However, if the continuum hypothesis proves to be correct, then research on the White Christmas effect in non-clinical populations will obviously be relevant to research on hallucinations related to psychosis.
1.2. Phoneme restoration
If viewed as being the result of signal detection bias, the White Christmas effect is very similar to the phoneme restoration effect in which people hear a speech sound that has been artificially removed from a recording and replaced with a masking sound, often stochastic noise (Samuel, Reference Samuel1996; Warren, Reference Warren1970). People generally do not notice that a speech sound has been replaced, and when asked, are usually unable to determine at what point in the word the masking sound occurred. It is generally assumed that the lexical and phonological context in which the missing phoneme is situated provide an expectation of hearing the particular phoneme and that this expectation allows the missing sound to be heard under the masking sound (e.g., Bashford & Warren, Reference Bashford and Warren1987; Srinivasan & Wang, Reference Srinivasan and Wang2005; Trout & Poser, Reference Trout and Poser1990). This is a case of expectation causing noise (or other masking sounds) to be heard as speech and so is clearly similar to the White Christmas effect.
Given that the phoneme restoration effect is modulated by masking relations, the experiments reported here examine how masking relations may also modulate the White Christmas effect. The probability of inducing phoneme restoration is dependent on the masking potential rule:
The necessary condition for the auditory continuity illusion is that the acoustic (spectral, temporal, and spatial) characteristics of the interrupting sound must be sufficient to mask the interrupted sound if the two sounds were presented simultaneously. (Kashino, Reference Kashino2006, 319–320)
As elaborated by many researchers (e.g., Bashford et al., Reference Bashford, Riener and Warren1992; Bashford & Warren, Reference Bashford and Warren1987; Warren et al., Reference Warren, Obusek and Ackroff1972), this principle seems to be of broad application and in its broader form is often referred to as ‘auditory induction’. For example, in a non-speech parallel to phoneme restoration, an intermittent tone is heard as continuous if the silent gaps in the tone are filled with noise that would be sufficient to mask the tone (Bregman, Reference Bregman1990).
Despite the similarity between phoneme restoration and the White Christmas effect there has been little research on how the White Christmas effect is modulated by masking relations. The current experiments address this gap by examining two types of masking sound (multi-talker babble versus spectrally matched speech-shaped noise) and two types of phonetic expectation (expecting to hear a ‘normal’ modal voice versus expecting to hear a whispered voice). Experiment One examines the ‘masking’ side of the White Christmas effect, looking at how babble versus stochastic noise modulates the effect. Experiment Two examines these two maskers in the context of an expectation of an easily masked kind of speech (whisper). Of course, as there is no actual signal in the White Christmas effect, there is no actual masking either, there is only potential masking.
2. Experiment one
This experiment explores the effect of (potential) masking on the White Christmas effect. The prediction is that there will be more AVH for masking from babble than for masking from stochastic noise. This prediction is based on the similarity between the White Christmas effect and phoneme restoration. Given this similarity, it is plausible that similar principles will govern how the quality of the masker influences the likelihood of the effect.
When it comes to the White Christmas effect, there is no actual speech to be masked, and so the masking potential rule cannot be applied directly as it can in phoneme restoration where the surrounding speech provides a reference point for masking requirements of the deleted sounds of the speech signal. However, if the same principle is at work, then the White Christmas effect should still be influenced by the characteristics of the masker. Specifically, a higher rate of White Christmas effect should occur when the masker is multi-talker babble than when the masker is speech-shaped stochastic noise, as babble is generally the better masker of speech (Simpson & Cooke, Reference Simpson and Cooke2005).
The primary reason that babble is a more effective masker is that it has fragments of harmonic content. This harmonic content is a particularly effective masker of the harmonic components of speech (the voiced components) (Alm et al., Reference Alm, Behne, Wang and Eg2009; Popham et al., Reference Popham, Boebinger, Ellis, Kawahara and McDermott2018; Steinmetzger & Rosen, Reference Steinmetzger and Rosen2015). This is seen in phoneme restoration, where voiced consonants are less likely to be restored than voiceless consonants when the masking used is stochastic noise (Kim & Davis, Reference Kim, Davis, Vroomen, Swerts and Krahmer2007; Trout & Poser, Reference Trout and Poser1990). It should be pointed out that the greater masking of babble over stochastic noise does not hold when the number of masking speakers in the babble is small enough that there are ‘gaps’ in the masker allowing the perceiver to ‘glimpse’ the target (Darwin, Reference Darwin2008). It should also be pointed out that periodicity in the masker can actually reduce masking when there is a single masking voice, that is, when the task is to distinguish a target voice from a single other voice (de Cheveigné et al., Reference de Cheveigné, McAdams, Laroche and Rosenberg1995). In this situation, the perceptual system can use the periodicity of the masker to segregate it from the periodicity of the target (as long as they are sufficiently different in fundamental frequency).
In addition to this harmonic content on harmonic content masking, there are other factors that make babble a more effective masker of speech than stochastic noise. Simpson and Cooke (Reference Simpson and Cooke2005) point out that the auditory system is particularly sensitive to sound onsets, and multi-talker babble contains many more speech onsets than speech-shaped noise, and further, that real speech (as contained in babble) more naturally causes the auditory system to pay attention to it than does noise. These factors are related to informational masking rather than the energetic masking that is typically considered in phoneme restoration and other forms of auditory induction. This issue is taken up in the General Discussion below.
To test the dependence of the White Christmas effect on masker type, Experiment One induced the White Christmas effect using multi-talker babble and spectrally matched speech-shaped stochastic noise. As a preview of results, there was a higher rate of AVH for the multi-talker babble in comparison to the spectrally matched speech-shaped stochastic noise.
2.1. Methods
2.1.1. Participants
There were 42 participants recruited from United Arab Emirates and Qatar University. All were native Arabic-speaking and female.Footnote 3 All participants reported no known hearing problems. The average age was 19.76 years (SD = 1.74). Participants were paid or given course credit for their participation.
2.1.2. Ethics approval
Both experiments reported in this paper were run with the approval of the respective university ethics boards; Ethics approval numbers: United Arab Emirates University ERSC_2023_2569 and Qatar University QU-IRB 990-E18.
2.1.3. Procedure
Participants sat in front of a computer screen wearing headphones (Extreme Isolation 30). In each trial, the participant was presented with a written target word. Given the demography of participants, all visual and audio stimuli were presented in Arabic.
The participants were told to listen for the target word during the masker that was to follow. Any time they heard the full word (not just a part of it), they were to hit the spacebar on the computer keyboard. They were told that the word could appear multiple times during a trial, so they could hit the spacebar more than once per trial. They were also warned that the word may not occur at all during the trial (or throughout the experiment), so it was fine not to hit the spacebar at all for any given trial, or indeed over the entire duration of the experiment.
When ready to start a trial the participant pressed the computer’s ‘enter’ key at which point either multi-talker babble or spectrally-matched speech-shaped stochastic noise would play for 35 seconds (the sound started at silence and rose to approximately 80 dB over the first 2 seconds, remained at that intensity for 31 seconds and then ramped back down to silence over the last 2 seconds). So that participants would have something to look at while listening to either of the two forms of masking sound, a 100-dot random dot kinematogram was presented on the screen for the duration of the masking sound. The kinematogram was added after initial piloting because several participants complained that staring at a blank screen for the 35 seconds of the trial made it difficult to remain on task.
The experiment was run by research assistants who were familiar with the White Christmas effect but were not aware of the predicted effects. They were instructed to assure participants that many people never hear the specified sound and so not to feel that they had to hit the button (that it was fine never to hit the button throughout the experiment).
A schematic outline of the experimental set-up is shown in Figure 1, and a schematic outline of a single trial is shown in Figure 2.
Schematic of experiment set-up.
Schematic of a single trial.
There were 14 such trials over the course of the experiment. The entire experiment, including instructions and initial familiarization and practice, lasted about 20 minutes. The dependent measure was the average number of reported instances of hearing the target word during a trial (AVH per trial). For each participant, this was measured simply as the average number of times that participant hit spacebar on each trial.
2.1.4. Materials
Masking sounds The auditory stimuli for this experiment were two types of masking sound: Multi-talker babble and spectrally matched speech-shaped stochastic noise. The multi-talker babble was created by downloading audio files from the Qatari Arabic Corpus, available at: http://www.isle.illinois.edu/dialect/data.shtml.
This corpus consists of interviews from the Al-Jazeera network (207 minutes of relatively formal Arabic from a variety of Arabic dialects, recorded in Qatar) and 110 minutes of ‘Sabah El-Doha’ an interview and news broadcast with several Arabic dialects used, recorded in Qatar using relatively informal Arabic. Twenty-one of these files (or parts of files) were selected that had no music or other non-speech sounds. These selections were all normalized to the same intensity. The waveforms were cut at crossing (‘zero’) points to avoid pops where waveforms were joined together. These sounds were then overlapped to create a multi-talker babble of approximately 30 voices. While individual words could not be distinguished in this babble, as an extra precaution the transcripts of the recordings used were searched for the experimental target-words and none of these words occurred in the sections used.
The spectrally matched speech-shaped stochastic noise was created by Praat script (Boersma & Weenink, Reference Boersma and Weenink2001), which measured the long-term average spectrum and the intensity profile of the multi-talker babble and applied this to stochastic noise of equal duration. The sounds in the two masker conditions thus had equivalent average spectra and identical intensity profiles.
Words Two lists of seven Arabic words (14 words altogether) were created as the target words that participants would try to identify in the noise. This is one word for each of the seven trials in each masker condition. The words in these lists were selected to use similar sounds (roughly equal numbers of nasals, sibilants and stops between the lists). This phonological balancing of the lists was an extra precaution to minimize variability in how easy these words were to hallucinate within each run of the experiment. The words were all chosen to be concrete, easily imageable bisyallbic nouns. For a given participant, each masker condition was assigned to a particular list, but the assignment of list to masker was switched for half of the participants, so the word-to-masker assignment was counterbalanced over the full run of the experiment. The lists of words are given in Table 1.
Word lists for the two conditions. The Arabic is presented in Romanized form
2.2. Results
In debriefing, participants were asked if they had any ideas about the purpose of the experiment and whether they had realized that there were no recordings of the relevant words being played under the masker. No participant reported realizing that they were hallucinating the words that they reported hearing (and many expressed significant surprise when informed of this).
The dependent measure was the number of times a participant heard the target word in a given trial (AVH per trial) as determined by the number of spacebar hits per trial.
All statistical analyses were conducted with the R statistical software package (R Core Team, 2014). A generalized linear mixed model with a negative binomial distribution was fit using the glmer function in the lme4 package (Bates et al., Reference Bates, Mächler, Bolker and Walker2015). The fixed effect was Masker type and the random effects structure specified random intercepts for Participant and random slopes for Masker and Trial number. This showed a significant effect of Masker type (p
$ < $
0.0001), with more AVH induced by babble than by stochastic noise. These results are shown (along with the results for Experiment Two) in Figure 3.
Results of experiments one and two combined. 95% Confidence Intervals are shown; individual data-points are also shown, jittered for visibility.
The analysis of the effect of Masker is taken up further in the combined results section below, which includes the results for Experiment Two.
All data collected (for both experiments) are available at the Open Science Framework: https://osf.io/hbkqy/.
2.3. Discussion
These results show that hallucinations of speech are quite easy to induce (in line with previous work on the White Christmas effect) and that multi-talker babble is a more effective inducer of AVH than spectrally matched speech-shaped stochastic noise. Further discussion of the effect of Masker type is taken up in the general discussion.
3. Experiment two
Experiment Two replicates the masking effect explored in Experiment One, but with a change in the expected ‘signal’ side – testing whether an expectation of more easily masked speech (whisper) will show the same masking relations. Experiment Two also examines, by comparing results against Experiment One, whether the expectation of more-easily masked speech will lead to a higher rate of AVH. Comparison between expectation of whisper versus modal voice was done ‘between-subjects’ because it was felt that participants would find it difficult to switch their expectation of modal versus whispered speech – they would likely forget which kind of expectation they should have in a given block, thus muddling conditions. To avoid this problem, each participant was given only one type of speech to expect: modal voice (Experiment One) or whisper (Experiment Two).
The primary contributor to whisper’s greater susceptibility to masking is the fact that whispered speech is simply inherently less intense than modal speech (Legou et al., Reference Legou, Ghio, Ch and Giovanni2010; Schwartz, Reference Schwartz1970; Traunmüller & Eriksson, Reference Traunmüller and Eriksson2000). This results in whispered speech showing lower rates of intelligibility (Tartter, Reference Tartter1991).
When it comes to masking relationships, whisper is not just more easily masked because it is quieter but also because it lacks harmonic structure. Whispered speech is generated by creating a constriction in the larynx, likely involving the false vocal folds, but with suppression of vocal fold vibration (Hardcastle et al., Reference Hardcastle, Laver and Gibbon2012; Tsunoda et al., Reference Tsunoda, Niimi and Hirose1996). This laryngeal constriction creates turbulent airflow which excites resonances in the vocal tract above, but as there is no vocal fold vibration, there is no periodicity in whisper. In modal speech, the periodic source means that the component frequencies of the speech sound are integer multiples of the fundamental frequency (termed harmonicity); whisper, however, lacks this harmonic relation and so is inharmonic (Popham et al., Reference Popham, Boebinger, Ellis, Kawahara and McDermott2018).
Harmonicity plays a central role in allowing perceivers to separate a target sound from background (Bregman, Reference Bregman1990; Nittrouer & Tarr, Reference Nittrouer and Tarr2011): ‘a common harmonic series helps to group together the different formant regions that make up a voiced sound such as a vowel’ (Darwin, Reference Darwin2008, p.1016). This explains why periodic sounds are more resistant to masking from stochastic noise (Alm et al., Reference Alm, Behne, Wang and Eg2009; Popham et al., Reference Popham, Boebinger, Ellis, Kawahara and McDermott2018). This is presumably part of the reason for the lower rates of AVH for the speech-shaped stochastic noise masker in comparison to the multi-talker babble masker in Experiment One.
Given that whisper lacks harmonicity it should not show the same resistance to masking from stochastic noise that was shown in Experiment One for modal voice. We thus predicted that there should be an interaction of whisper with masking sound, whereby the difference in AVH between babble and spectrally matched speech-shaped stochastic noise shown in Experiment One should be reduced in Experiment Two.
A connection between whispered speech and AVH is highlighted by the fact that the voice hallucinations experienced by non-clinical populations typically sound to the hearer ‘quiet (or like a whisper)’ (Toh et al., Reference Toh, Thomas, Robertson and Rossell2020). This, of course, fits well with the signal-detection account of AVH, though, as discussed above, there may be several mechanisms by which AVH are caused.
3.1. Methods
The design of Experiment Two was identical to Experiment One except that participants were told that the words they were instructed to listen for would be whispered.
3.1.1. Participants
There were 42 new participants recruited at United Arab Emirates and Qatar University (none of the participants were the same as for Experiment One). All participants were native Arabic-speaking and female. All participants reported no known hearing problems. The average age was 20.34 years (SD = 3.38). Participants were paid or given course credit for their participation.
3.2. Results
In debriefing participants were asked if they had any ideas about the purpose of the experiment and whether they had realized that there were no recordings of the relevant words being played under the masker. No participant reported realizing that they were hallucinating the words that they reported hearing (and again, many reported surprise that there were no actual recordings of words).
As with Experiment One, a generalized linear mixed model with a negative binomial distribution was fit. The dependent measure was the average number of AVH per trial (as measured by the number of times each participant hit the spacebar per trial). The fixed effect was Masker type and the random effects structure specified random intercepts for Participant and random slopes for Masker type and Trial number. This showed a significant effect of Masker type (p
$ < $
0.0001), with more AVH induced by babble than by stochastic noise.
The effect of Masker type and Voice expectation type are reported in the combined results section below and a graph of these results (along with those from Experiment One) is shown in Figure 3.
3.3. Combined results of experiments one and two
To explore the effect of Voice expectation type, Masker type and their interaction, a generalized linear mixed model with a negative binomial distribution over the combined data for Experiment One and Two was fit. The dependent measure was the average number of AVH per trial. The fixed effects were Masker type and Voice expectation type and their interaction. The random effects specified random intercepts for Participant and random slopes for Masker type and Trial number. The results of this analysis are shown in Table 2. This found an effect of Masker type (p
$ < $
0.0001), and a slight non-significant trend for Voice expectation type (p = 0.213), with more AVH being reported for whisper in comparison to modal voice. Contrary to prediction, there was little evidence of an interaction of Masker type with Voice expectation type (p = 0.322). The results of both experiments are shown in Figure 3.
Summary of GLM results
3.4. Discussion
These results replicate the effect of Masker type on the White Christmas effect found in Experiment One by demonstrating the same effect with an expectation of whispered speech. That AVH would be easier to induce when the expectation was of hearing whispered speech makes sense from a signal-detection perspective as whispered speech is easier to mask. However, while there was a slight trend toward a stronger effect for the whispered voice in Experiment Two in comparison to the modal voice in Experiment One, this did not reach significance. Furthermore, the predicted interaction of Voice expectation type (modal versus whispered) with Masker type (multi-talker babble versus spectrally matched speech-shaped stochastic noise) was not found.
4. General discussion
These results show that speech hallucinations are influenced by the type of masking sound and perhaps by phonetic expectation. With multi-talker babble masking, the rates of AVH are, as predicted, higher. Similarly, when the participant is told to expect whisper, AVH rates trend higher (non-significantly in this data). Multi-talker babble is a more effective masker than stochastic noise, and whisper is more easily masked than modal voice, and so these results support the view that the masking potential rule applies to the White Christmas effect. When the masker is sufficient to mask an expected word, listeners are more likely to hallucinate the expected word. These results mirror findings on phoneme restoration, which also follows the masking potential rule (e.g., Bashford et al., Reference Bashford, Riener and Warren1992; Bashford & Warren, Reference Bashford and Warren1987; Warren et al., Reference Warren, Obusek and Ackroff1972).
A consistent issue with the White Christmas effect is that participants may feel pressured, no matter how clear the instructions are, to indicate they heard the target word, even when they did not. In the experiments reported here, this pressure may (as with other experiments on the White Christmas effect) inflate the total number of ‘hits’. However, it does not plausibly explain the rate of the effect differing across conditions.
Given the greater masking of voiced speech (in comparison to whisper) by babble than by stochastic noise, we predicted that there should be an interaction between Masker type (multi-talker babble versus spectrally-matched speech-shaped stochastic noise) and Voice expectation type (modal versus whisper). Specifically we predicted that there should be a smaller difference between masking conditions for whisper than for modal speech. This interaction was not significant.
There are differences between modal and whispered speech other than periodicity. For example, whispered consonants are typically longer (Schwartz, Reference Schwartz1972; Tartter, Reference Tartter1989; Xu et al., Reference Xu, Shao, Ding and Wang2022), formants of vowels tend to be higher in frequency (Kallail & Emanuel, Reference Kallail and Emanuel1984a, Reference Kallail and Emanuelb) and the vowel space for whispered vowels may shrink in comparison to modal speech (Houle & Levi, Reference Houle and Levi2020). Of particular importance to masking, the spectral tilt of whisper emphasizes the higher frequency end of the spectrum and the intensity of vowels reduces disproportionately to consonants in whispered speech, altering the consonant-vowel intensity ratio, which plays a role in intelligibility under masking (Freyman et al., Reference Freyman, Nerbonne and Cote1991, Reference Freyman, Griffin and Oxenham2012). Thus, perhaps differences between modal and whispered speech other than voicing explain the lack of significant interaction between Voice expectation and Masker type in this data set.
Another possibility is that the likelihood of AVH is not perfectly aligned with the ‘masking potential rule’ – perhaps the auditory system’s top-down expectations of whether a masker would mask a particular speech sound are not entirely accurate. If this is the case, then probing the ability of various qualities of masker to induce illusions such as the White Christmas effect or phoneme restoration would serve as a useful way to probe differences between the actual masking potential of types of masker and the auditory system’s predictions of masking from these various types of sound.
One way in which the ‘masking potential rule’ may be insufficient to explain the White Christmas Effect and possibly other continuation effects such as phoneme restoration, is that it relies on energetic masking. Energetic masking refers to the decrease in audibility of a sound when there is a competing sound in the same frequency band (Fletcher, Reference Fletcher1940; Pollack, Reference Pollack1975; Srinivasan & Wang, Reference Srinivasan and Wang2008). In contrast, informational masking is the “perceptual degradation caused by the listener’s inability to segregate a target signal from interference” (Srinivasan & Wang, Reference Srinivasan and Wang2008, 3213). This form of masking may also play a role in continuity illusions. There is no actual signal to be masked (either energetically or informationally) in the White Christmas or phoneme restoration effects, but the ‘masking potential rule’ assumes that the perceptual system can determine whether a particular masker would have energetically masked a (non-existent) sound. It is possible that informational masking is also taken into account by the perceptual system – perhaps the system estimates how easy it would be to segregate a target sound against a given background and so would consider it more likely, via biasing in a signal-detection sense, that a sound occurred in a high information-masking context.
The concept of informational masking is complicated by the fact that the term is used variably by different authors and is something of a ‘suitcase word’ (Watson, Reference Watson2005), lumping together all causes of masking that cannot be explained by mechanisms in the auditory periphery (which are instead considered to cause energetic masking). The list of mechanisms potentially contributing to informational masking includes, among others, similarity between masker and signal, stimulus/masker uncertainty and distraction. These factors are not mutually exclusive and may interact (as when stimulus uncertainty causes distraction). When multi-talker babble with a small number of speakers is used as a masker, informational masking is high because many of these mechanisms are at play – there is essentially a cocktail party problem (Cherry, Reference Cherry1953) of segregating the target from the background. As the number of speakers in the babble increases, individual distractor voices are harder to pick out and so informational masking is reduced and energetic masking dominates (Leibold & Buss, Reference Leibold and Buss2013). Some researchers would argue that when speech is no longer intelligible in multi-talker babble, the masking should no longer be labeled informational masking. While energetic masking does dominate informational masking when there are a high number of speakers in multi-talker babble, the problem of similarity between masker and target remains (to a degree) as can the problem of stimulus uncertainty. These would not normally be considered energetic masking. Whether this should be labeled information masking is a debate to which these experiments do not make a contribution. If information masking (in this restricted target-masker similarity sense) does play a role in the White Christmas effect, then that would predict that AVH should be more common for expecting whispered speech in the stochastic-noise condition (in comparison to the babble condition); and more common when expecting modal speech in the babble condition (in comparison to the stochastic-noise condition). Both of these situations involve more similarity between (expected) target and masking sound – greater similarity between target and masker should lead to more informational masking (Brungart et al., Reference Brungart, Simpson, Ericson and Scott2001; Kidd et al., Reference Kidd, Mason, Richards, Gallun, Durlach, Fay, Popper, Yost, Popper and Fay2008). An experiment conducted by Buss et al. (Reference Buss, Miller and Leibold2022) followed a similar logic, arguing that whispered speech should be easier to detect against a voiced masker and voiced speech should be easier to detect against a whispered masker. Their argument was based on the assumption that greater dissimilarity between target and masker should reduce informational masking. If this form of informational masking played a role in the experiments reported above, it would push the pattern of AVH in the opposite direction to the interaction that we predicted and so may explain the absence of the predicted interaction between Voice expectation and Masker type.
The results of these experiments are consistent with a signal-detection bias interpretation of AVH – keeping in mind that this signal-detection bias is provided by top-down expectations which could come from a memory or from mental imagery.
A link between mental imagery and hallucination was suggested by Ulric Neisser (Reference Neisser1976, Reference Neisser1978), who argued that perceptual expectations (which he called schemata) are normally used to shape perception but can be triggered, in the absence of external stimulation, to constitute imagery. Recent experiments have demonstrated an overlap between imagery and hallucination, showing that there is a tendency to hear both imagined speech and AVH induced through white noise as occurring in the right ear. Prete et al. (Reference Prete, Marzoli, Brancucci and Tommasi2016, Reference Prete, D’Anselmo, Brancucci and Tommasi2018) in line with the well-established ‘Right Ear Advantage’ (REA) for the perception of external speech (Broadbent, Reference Broadbent1954; Hugdahl, Reference Hugdahl2011). This REA for imagined speech sounds was recently replicated in an rTMS study (Prete et al., Reference Prete, Rollo, Palumbo, Ceccato, Mammarella, Di Domenico, Capotosto and Tommasi2024). However, a study investigating REA among voice-hearers with schizophrenia did not find such lateralization for either real or imagined speech (Altamura et al., Reference Altamura, Prete, Elia, Angelini, Padalino, Bellomo, Tommasi and Fairfield2020). The 2018 study by Prete et al. showed an interesting interaction with intensity, as participants were more likely to experience an REA for syllables presented at a lower intensity in comparison to a masking noise (though this was not a White Christmas effect, as there was a syllable spoken in the noise).
Variations of the idea of a common source for imagery and hallucination have been put forward by several authors, (e.g., Clark, Reference Clark2013; Grush, Reference Grush2004). In a recent variant of this idea, sensory predictions generated by the motor system (known as forward models) have been argued to play a role in both perception and mental imagery. Forward models have a primary use in motor control and in preventing confusion between self-caused and externally caused sensations (Aliu et al., Reference Aliu, Houde and Nagarajan2009), but it has been argued that forward models can be run ‘offline’ (as in Neisser’s schemata discussed above) to generate speech imagery which, in turn, has been shown to influence the perception of external speech (Scott, Reference Scott2016; Scott et al., Reference Scott, Yeung, Gick and Werker2013). Malfunction of this forward model system has been proposed as the origin of voice hearing in schizophrenia (Frith, Reference Frith1992). This shaping of ambiguous speech by expectations generated by the motor system can even be caused by the static position of speech articulators (Yeung & Scott, Reference Yeung and Scott2021). Similarly, Sato et al. (Reference Sato, Schwartz, Abry, Cathiard and Loevenbruck2006) have found a link between articulation and the verbal transformation effect, which, as discussed above, is very similar to the White Christmas effect. A useful summary and synthesis of the role of prediction mechanisms in perception, imagery and hallucination is presented in Clark (Reference Clark2013).
The role of predictive mechanisms in perception, imagery and hallucination has more recently been cast in Bayesian terms (e.g., Cassidy et al., Reference Cassidy, Balsam, Weinstein, Rosengard, Slifstein, Daw, Abi-Dargham and Horga2018; Fletcher & Frith, Reference Fletcher and Frith2009; Powers et al., Reference Powers, Mathys and Corlett2017). The brain is argued to use a Bayesian-like weighting of incoming sensory stimulation and prior probabilities to achieve the most plausible percept despite noisy and underdetermined sensory input. Thus, perception is dependent on both top-down (expectation) and bottom-up (sensory stimulation) factors. Under this framework, we would predict that both the details of what is expected and the details of the incoming sensory stimulus to play a role in the likelihood of a hallucinatory experience, as supported by the current experiments.
An alternative to a masking explanation of why babble induces higher rates of AVH is that babble more strongly engages the speech centres of the brain and thus makes the experience of hearing speech more likely. This would be similar to the proposed attentional explanation of the REA, by which simply engaging the left hemisphere by drawing a person’s attention to the right side of space is sufficient to boost language processing (Kinsbourne, Reference Kinsbourne1970).
4.1. Limitations and future research
The results of these experiments suggest the possible involvement of informational masking, but were not specifically designed to address this involvement. Future research will use variations in babble masking, with a smaller number of talkers in the babble, to explore this possibility.
The current experiments did not take a baseline measure of participants’ proneness to hallucination or require disclosure of any mental health conditions. This was for reasons of both privacy and experiment brevity. It would be interesting to explore, in future research, whether the masking relations demonstrated in the experiments above are influenced by participant proneness to hallucination. As part of this, there was no objective measure of participant hearing threshold. It is possible that proneness to hallucination is influenced by overall hearing acuity and so future research will include such a measure.
It is likely that semantic and word-frequency effects play a role in the probability of hearing a particular word in the White Christmas effect, but the experiments reported here concentrate on how the auditory aspects of potential masking modulate verbal hallucinations. While the experiments reported above do not directly address any issues specific to the Arabic language, future research is planned which will use this White Christmas methodology to explore the special morphology of Arabic – examining whether it is easier to induce this effect for words that fit the standard triliteral consonant morphological structure of Arabic in comparison to equally common words which do not have this structure. Through collaborations with other institutions, we also hope to address the sex-balance issue: We were not able to collect a sex-balanced sample because the departments from which participants were recruited only have female students. This limits the generalizability of our results, something we hope to address in future research.
5. Conclusions
These experiments show that masking relations modulate the White Christmas effect, in line with predictions based on the phoneme restoration literature. Masking from multi-talker babble led to a higher rate of verbal hallucinations than masking from spectrally matched speech-shaped stochastic noise. Similarly, there was a trend toward a higher rate of verbal hallucinations when participants expected to hear a more easily masked sound (whisper) than for ‘normal’ modal speech. Contrary to prediction, no significant interaction was found between these factors.
Acknowledgments
Participants were run by research assistants Maryam Moustafa Gadalla Aref, Fatima Boush and Rofida Babiker Hamid Ibrahim. Preliminary results from a subset of this data were presented at the conference of the Canadian Acoustics Association (Scott, Reference Scott2022).
Funding statement
This research was funded by a Qatar University grant QUUG-CAS-ELL-17_18–7, and publication was made possible through the generous sponsorship of the publication fee by United Arab Emirates University.
Open access
