Introduction
In speech communication, listeners experience considerable variation in the acoustic characteristics of speech, including speech that is produced with an unfamiliar accent and speech that contains disfluencies. One source of acoustic variation in speech that can impact speech perception is the speakers’ native language background. Numerous previous studies have demonstrated that a non-native accent influences the way native listeners process speech (e.g., Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014a; Bosker & Reinisch, Reference Bosker and Reinisch2015; Hanulíková et al., Reference Hanulíková, van Alphen, Goch and Weber2012; Munro & Derwing, Reference Munro and Derwing1998, Reference Munro and Derwing2001, Reference Munro and Derwing2006; Pinget et al., Reference Pinget, Bosker, Quené and De Jong2014). For example, for native listeners, non-native speech is often less intelligible (e.g., Bent & Bradlow, Reference Bent and Bradlow2003; Ferguson et al., Reference Ferguson, Jongman, Sereno and Keum2010; Munro & Derwing, Reference Munro and Derwing1995a) and is processed more slowly (e.g., Floccia et al., Reference Floccia, Butler, Goslin and Ellis2009; Munro & Derwing, Reference Munro and Derwing1995b) than native speech. Furthermore, the information conveyed in non-native speech can be perceived to be less credible or less reliable than the same information conveyed in native speech (Foucart et al., Reference Foucart, Santamaría-García and Hartsuiker2019; Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010; Livingston et al., Reference Livingston, Schilpzand and Erez2017; cf: De Meo et al., Reference De Meo, Vitale, Pettorino and Martin2011; Souza & Markman, Reference Souza and Markman2013; Stocker, Reference Stocker2017).
In addition to speakers’ native language background, disfluencies, which are present in both native and non-native speech (Goldman-Eisler, Reference Goldman-Eisler1968; Lennon, Reference Lennon1990), also impact the perception of speech (e.g., Bosker et al., Reference Bosker, Pinget, Quené, Sanders and De Jong2013; Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014a; Cucchiarini et al., Reference Cucchiarini, Strik and Boves2002; Kormos & Dénes, Reference Kormos and Dénes2004; MacGregor et al., Reference MacGregor, Corley and Donaldson2010; Rossiter, Reference Rossiter2009). Particularly, mid-utterance disruptions, such as silent or filled pauses, have consequences for listeners’ perception. For example, listeners are quicker to respond to a target word in an utterance when the target word is preceded by a silent pause or filled pause compared to when it is not, suggesting that certain forms of pausing facilitate listeners’ processing (Brennan & Schober, Reference Brennan and Schober2001; Corley & Hartsuiker, Reference Corley and Hartsuiker2011; Fox Tree, Reference Fox Tree2001). Other studies also show that pauses can impact listeners’ holistic perception of the speech or the speaker, in terms of how eloquent the speech sounds, or how anxious or honest the speaker is (Christenfeld, Reference Christenfeld1995; Fox Tree, Reference Fox Tree2002).
While studies have shown that both a non-native accent and disfluencies impact speech perception (e.g., Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014b), mixed patterns have been found in how these factors together impact native listeners’ processing of native and non-native speech. Specifically, on one hand, manipulations of the characteristics of silent pauses, in terms of their number, duration (Bosker et al. Reference Bosker, Quené, Sanders and de Jong2014b), location, and distribution (Kahng, Reference Kahng2018), showed similar effects on perceived fluency of native speech and non-native speech. That is for both native and non-native speech, Dutch utterances with more pauses or longer pauses were rated as less fluent than utterances with less pauses or shorter pauses by native speakers (Bosker et al. Reference Bosker, Quené, Sanders and de Jong2014b). Further, for both native and non-native speech, English sentences that contain pauses placed within a clause were rated as less fluent by native speakers than sentences that contain pauses placed between clauses (Kahng, Reference Kahng2018). While these studies have shown that pauses impact perception of native and non-native speech in a similar manner, other work suggests that pauses are perceived differently across the two types of speech. For example, when listening to native speech, the filler word “um” triggered prediction of low-frequency referents for native listeners, but this effect was not found during perception of non-native speech (Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014a), suggesting that disfluency impacted native listeners’ perception differently depending on whether the speech was native or non-native speech.
One way to understand these mixed patterns may be to take listeners’ approach to speech processing into consideration. For example, the above studies have all examined the effect of pauses in native and non-native speech in a single perception task (e.g., fluency rating task: Kahng, Reference Kahng2018; visual world paradigm: Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014a; comprehension task: Hanulíková et al., Reference Hanulíková, van Alphen, Goch and Weber2012). That is, the previous studies did not use the same set of stimuli across different perception tasks. However, it is possible that perceptual consequences of certain acoustic characteristics (e.g., pausing patterns, features of a non-native accent) generally differ depending on how listeners are encouraged to approach the listening activity (e.g., they may differ depending on the task). Literature on speech perception more broadly suggests that listening is an active process, in which task-specific demands impact the way listeners process auditory information (e.g., Fritz et al., Reference Fritz, Elhilali, David and Shamma2007; Heald & Nusbaum, Reference Heald and Nusbaum2014). As such, it seems likely that different tasks could encourage listeners to process the same auditory stimuli in different manners depending on the goal of the task. For example, listeners’ behavioral and neural activation patterns differ for the same auditory stimuli when they are encouraged to pay attention to different aspects of the auditory stimuli (e.g., Brechmann & Scheich, Reference Brechmann and Scheich2005; Hugdahl et al., Reference Hugdahl, Thomsen, Ersland, Rimol and Niemi2003; Pallier et al., Reference Pallier, Sebastiangalles, Felguera, Christophe and Mehler1993). Similarly, for the perception of disfluency, Christenfeld (Reference Christenfeld1995) demonstrated that listeners were more likely to detect filled pauses in spontaneous speech when they were instructed to focus on the style of the speech (e.g., does the speaker use the language well?) as compared to the content of the speech (e.g., does the argument make logical sense?); such differences in the instructions also impacted subjective perception (e.g., speech with filled pauses was perceived to be more eloquent when listeners were focused on the content of the speech than the style of the speech). Thus, as mentioned earlier, since many of the previous studies have examined the influence of disfluency on perception in the context of a single task, it is possible that the mixed results are due to different task demands.
Furthermore, studies examining native listeners’ perception of native and non-native speech also suggest that the extent to which listeners process native versus non-native speech similarly depends on what part of the speech listeners are encouraged to pay attention to. For example, as discussed above, listeners processed disfluencies in native and non-native speech in a similar manner when they were evaluating the form or style of the speech (e.g., in a fluency rating task: Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014b; Kahng, Reference Kahng2018). However, listeners processed native and non-native speech differently when they were required to process the content of the speech, such as rating credibility of the information (Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010) and answering comprehension questions (Hanulíková et al., Reference Hanulíková, van Alphen, Goch and Weber2012; Lev-Ari & Keysar, Reference Lev-Ari and Keysar2012). Given these studies, it is possible that listeners process certain acoustic characteristics—those that give rise to a non-native accent and disfluencies (in both native and non-native speech)—differently depending on how they are encouraged to approach the listening activity. However, it is unknown to what extent the listening focus of the task impacts the way listeners evaluate native and non-native speech with variable disfluency patterns.
Current study
The aim of the current study is to examine how the perception of silent pauses is affected by differing task demands in native and non-native speech. In order to do so, we ask how various characteristics of pauses impact perception of native and non-native speech in different tasks. Specifically, we examine the perceptual consequences of silent pauses, particularly, the effect of the presence of a pause, and location of a pause (e.g., within-clause location vs. between-clause location), as these characteristics have been shown to impact perception of native and non-native speech (Bosker et al. Reference Bosker, Quené, Sanders and de Jong2014b, Kahng, Reference Kahng2018). We examine the effects of task demands using two tasks: in a task where listeners are encouraged to focus on the content of the speech by evaluating the credibility of the trivia statements (e.g., Polar bears can swim more than 60 miles without a rest) and in a task where listeners are encouraged to focus on the style/property of the speech by evaluating the fluency of the same trivia statements. Crucially, we use the same materials across tasks, allowing us to draw direct comparisons. It is possible that silent pauses might affect the perception of native and non-native speech differently depending on the task. For example, silent pauses might affect fluency ratings in a similar manner (e.g., speech with more pauses may be rated as less fluent than speech with fewer pauses for both native and non-native speakers), but might affect credibility ratings in a different manner (e.g., speech with pauses may be rated as less credible than speech without pauses for native speakers, but this might not be the case for non-native speakers).
In addition to the relationship between task demands and disfluencies, we ask whether there is a relationship between fluency and credibility ratings. For example, it is possible that speech that is rated as more fluent might also be rated as more credible, and we examine whether there is a correlation between fluency and credibility ratings for either native or non-native speakers. If there is a difference in the correlation between fluency and credibility ratings for native versus non-native speech (i.e., if the correlation between fluency and credibility ratings for native is different from those of non-native speech), then that would suggest that the way we process speech depends on whether the speech is native or non-native.
Method
Materials
Materials were 24 English trivia statements from Lev-Ari and Keysar (Reference Lev-Ari and Keysar2010). Of the 24 statements, 12 were true (e.g., Polar bears can swim more than 60 miles without a rest) and 12 were false statements (e.g., The koala is the only known animal that never gets sick). The true and false statements were of comparable length; the average number of words were 9.25 for true and 8.58 for false statements.
Two native Japanese learners of English (20 and 23 years old) and two native English speakers (20 and 21 years old) recorded these statements. All speakers identified themselves as female and reported no history of speech or hearing impairment. The native Japanese speakers were international students in an intensive English program, who were studying English before entering an American university as matriculated students. They reported their TOEFL ITP scores to be 493 and 498, which are identified to be B1 level in the Common European Framework of Reference for Languages (CEFR)Footnote 1 . All speakers were recorded in a sound-attenuated booth using a Blue Yeti USB microphone. The statements were written on a sheet of paper, and the reading was self-paced. Recording was done on a single channel at a sampling rate of 44,100 Hz (16 bit) using the Praat speech analysis software package (Boersma & Weenink, Reference Boersma and Weenink2001). Speakers were given time to familiarize themselves with the list of statements in order to prevent disfluencies and mispronunciation when reading. The speakers were not told which statements were true or false. After the recording, speakers completed a language background questionnaire.
In order to examine whether the presence of a pause and the location of a pause influence listeners’ perception, for each of the 24 statements produced by each speaker, we created three pause conditions: no pause, between-clause pause, and within-clause pause items (following Kahng, Reference Kahng2018). To create no pause items, all the silent pauses in the speech samples longer than 100 ms (Idemaru et al., Reference Idemaru, Wei and Gubbins2019; Trofimovich & Baker, Reference Trofimovich and Baker2006) were cut from the sound files using Praat. These “no pause” items were used to create between-clause and within-clause pause items. Between-clause and within-clause pause items were created by adding a pause of 600 ms either in a between-clause or within-clause location (Kahng, Reference Kahng2018). Following Foster et al. (Reference Foster, Tonkyn and Wigglesworth2000), a clause was defined to be a unit that consists minimally of a finite or non-finite verb and at least one other clause element, such as subject, object, complement, or adverbial. For between-clause items, a pause was added either between a subject and verb of the sentence or before an adverbial phrase (e.g., Some crocodiles [Pause] may eat other crocodiles; Polar bears can swim more than 60 miles [Pause] without a rest). For within-clause pause items, a pause was added within a noun phrase (e.g., Some crocodiles may eat other [Pause] crocodiles; Polar [Pause] bears can swim more than 60 miles without a rest). This resulted in 288 unique items (24 statements × 4 speakers × 3 pause conditions). The stimuli were root mean square (RMS) normalized to an approximately equal amplitude level across stimuli.
Participants
Participants were 277 native English listeners (123 females, 154 males; mean age = 37.2 years), recruited using Amazon Mechanical Turk. Of the 277 participants, 148 participants completed the credibility-rating task and 129 participants completed the fluency-rating taskFootnote 2 . None of the listeners reported a history of speech or hearing impairment. All participants resided in the United States and self-reported to be native speakers of American English. None of the participants reported experience with Japanese on the language background questionnaire.
Procedure
The experiment was conducted online using a Qualtrics (www.qualtrics.com) link provided to participants via Amazon Mechanical Turk. Each participant completed either the fluency- or credibility-rating taskFootnote 3 . Before participants began the trials, they were given clarification information and a series of tasks based on the procedure used in Lev-Ari and Keysar (Reference Lev-Ari and Keysar2010). Specifically, in order to help participants understand that speakers’ background (e.g., native language status) is irrelevant to the credibility of the statements, they were told that the speakers were not expressing their own knowledge but only reading aloud statements that were provided (following Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010). They were also asked to read aloud three trivia statements (different from those in the test trials), so that they would better understand that the speakers (that the participants would listen to later in the listening task) were just reciting statements provided by the experimenter. They were then informed of the truth value of each statement that they read aloud (True or False), so that they would understand that the speakers had learned the truth value only after reciting the sentence. Then, they were told that they would listen to statements recorded by past participants.
In the credibility-rating task, participants listened to each statement and were asked to evaluate the truthfulness of the statement, using a 6-point Likert scale (1: Definitely false, 6: Definitely true). Next to the scale there were two boxes labeled “I know the answer” and “I didn’t understand what was said” (following Hanzlíková & Skarnitzl, Reference Hanzlíková and Skarnitzl2017); these responses were later excluded from the analysis in order to ensure that participants’ prior knowledge or the intelligibility of the speech did not affect the credibility ratings (e.g., to exclude the possibility that participants rated a statement to be “definitely false” because they did not understand what the speaker said).
In the fluency-rating task, participants listened to each statement and were asked to evaluate how “easily and smoothly” the speech is delivered, using a 6-point Likert scale (1: Extremely disfluent, 6: Extremely fluent). They were also asked to not evaluate the speech based on overall language proficiency of the speaker.
In both credibility- and fluency-rating tasks, participants could listen to the sentence only once, but could take as much time as needed to answer. Prior to the test trials, they completed two practice trials with speakers and sentences that were different from the 24 test trials. In the test trials, each participant listened to all 24 statements. They listened to 8 no pause items, 8 between-clause pause items, and 8 within-clause pause items. In each of the three pause conditions (i.e., 8 items), 4 items (2 true and 2 false statements) were produced by native speakers, and the other 4 items (2 true and 2 false statements) were produced by non-native speakers. Each listener heard all the speakers (i.e., two native and two non-native speakers). The presentation order of the 24 statements (pause conditions and speakers) was randomized for each listener. The combinations of the statement, speaker, and pause conditions were counterbalanced across listeners. The experiment took 10–12 minutes to complete.
Analysis
There was a total of 6,143 data points analyzed: 3,047 data pointsFootnote 4 for the credibility-rating task and 3,096 data points for the fluency-rating task. In order to analyze whether the effect of speakers’ native status (native vs. non-native English speakers) and the effect of pause conditions (no pause, between-clause pause, within-clause pause) on listeners’ ratings differed depending on the task (credibility- vs. fluency-rating), we implemented a Bayesian logistic ordinal regression modelFootnote 5 with uninformative priors, using R package brms (Bürkner, Reference Bürkner2017), with the rating as the dependent variable. The fixed effects were Task (credibility vs. fluency), Statement type (false vs. true), Speaker group (native vs. non-native), Pause (no pause, between-clause pause, within-clause pause), Speaking RateFootnote 6 , and the interaction among these factors. Sum coding was used for the model. The intercepts represent the threshold or cutoff value to move from one rating to the subsequent rating (i.e., the first intercept corresponds to the threshold to go from a rating of 1 to a rating of 2, the second intercept corresponds to the threshold to go from a rating of 2 to a rating of 3, and so on and so forth. See Barreda and Silbert (Reference Barreda and Silbert2023) for a more in-depth explanation of thresholds in ordinal regression models). For example, if a threshold to move from a rating of 1 to a rating of 2 was −0.3, then the model would predict a rating of 1 for any latent variable value below that number. The coefficient value represents the estimated increase of rating in log-odds for a 1-unit change in the variable: a positive coefficient value corresponds to a positive effect of the variable on rating. For binary level factors (in our case, Task), in order to get the estimate of the other task (fluency rating task), we simply reverse the sign on the estimate for Task1. That is, if the coefficient value is positive, then the coefficient value for the fluency rating is negative, and vice versa if the coefficient value is positive. For variables with 3 levels (In our case, the variables with three levels are the Pause variable and all the interactions with Pause), we recover the third level by taking the negative sum of the estimates for the other 2 levels. In other words, Pause3 (within-clause pause) = −(Pause1 + Pause2), which correspond to no pauses and between-clause pauses respectively.
The random effects structure included random intercepts for speaker, listener, and item. The random effects structure also included by-speaker random slopes for Task, Pause, and Statement type, by-item slopes for Speaker group, Pause, and Task, and by-listener slopes for Pause, Statement type, and Speaker group.
In this paper, we report 95% credible intervals for the effects of interest as well as the effects that were found to be credible. Bayesian analyses do not force us to interpret the results in terms of significance, but we can interpret an effect to be meaningful if the 95% credible interval does not contain zero. Additionally, in some cases, we also report the percentage of posterior samples greater than zero. This is useful in cases where the credible interval crosses zero because it allows us to determine specifically how confident we can be in the direction of the effect.
Results
We present the results in two sections. First, we present the results of a Bayesian cumulative ordinal mixed-effects regression model with a logit link, examining the effect of task and pause characteristics on perception of native and non-native speech. Next, we present the results of a correlation analysis, examining the relationship between fluency and credibility ratings and whether it differs for perception of native and non-native speech.
The effect of task and pause characteristics on perception of native and non-native speech
Figure 1 shows the credibility ratings and fluency ratings for the statements produced by native and non-native speakers in the three pause conditions (no pause, between-clause pause, within-clause pause). In this figure, the false and true statements are collapsed as the rating patterns were similar across the two statement types. The figure suggests that while silent pauses affected fluency ratings, they did not affect credibility ratings. Additionally, sentences with pauses within clauses were rated as less fluent than sentences with pauses between clauses, and both were rated as less fluent than sentences with no pauses. For the credibility rating task, patterns appear similar for responses to native and non-native speakers, while for the fluency rating task, native speakers appear to be rated as more fluent than non-native speakers.
Credibility (1: Definitely false, 6: Definitely true) and fluency (1: Extremely disfluent, 6: Extremely fluent) ratings for statements produced by native and non-native speakers, presented in three pause conditions (no pause, between-clause pause, and within-clause pause). The error bars represent 95% confidence intervals.
The results of the regression model revealed a main effect of Speaker group (native vs. non-native, β = 0.826, CI-2.5% = 0.04, CI-97.5% = 1.54) and an interaction effect between Speaker group and task (β = −1.19, CI-2.5% = −1.87, CI-97.5% = −0.43). Note that the credible intervals for both of these effects do not cross zero, suggesting we can be confident in the direction of the effect. This indicates that while the ratings generally were higher for the statements spoken by native speakers than those spoken by non-native speakers, this effect was larger in the fluency task than in the credibility task. To follow-up on this interaction effect, we examined the effect of speaker group in both tasks. We found a meaningful effect of speaker group for fluency ratings (β = 2.02, CI-2.5% = 0.91, CI-97.5% = 3.03), but not for credibility ratings (β = −0.36, CI-2.5% = −1.39, CI-97.5% = 0.68). This suggests that while listeners perceived statements spoken by native speakers to be more fluent than those spoken by non-native speakers, they did not perceive native speakers’ statements to be more credible than non-native speakers’ statements.
The credibility intervals for the main effect of no-pause, between-pause, and within-pause all contained zero, though it is worth noting that the within-clause pause condition’s credible interval is mostly negative (β = −0.43, CI-2.5% = −1.10, CI-97.5% = 0.23), suggesting that sentences with pauses within clauses receive slightly lower ratings than the other two pause conditions. We also examined the effects of pauses to determine whether there was a difference in the ratings between the no pause condition and the within-clause pause condition. We found a small positive effect for this, but note that the credible interval crosses zero (β = 0.65, CI-2.5% = −0.35, CI-97.5% = 1.65). While the credible interval crosses zero, we examine the posterior samples to determine what percentage of the sampled coefficients were greater than or less than zero and found that 90% of the posterior samples were greater than zero.Footnote 7 This suggests that in general, regardless of task (credibility or fluency), sentences without pauses received a higher rating (i.e., perceived to be more credible and more fluent) than sentences with pauses within a clause. We also examined whether the simple effects for each pause condition varied by task (i.e., we examined the difference between the effect of each pause condition in each task). We found that for the no pause condition in the fluency task, there was a small effect (β = 0.68, CI-2.5% = −0.21, CI-97.5% = 1.57). While the credible interval crosses zero, over 93% of the posterior samples were greater than zero. Compared to this, the effect of no pause in the credibility task was negligible (β = −0.25, CI-2.5% = −1.17, CI-97.5% = 0.68), and the credible interval is almost centered around zero. This suggests that speech with no pauses is rated higher in the fluency task, but not in the credibility task. Further, we also found that for the within-clause condition in the fluency task, there was a decrease in perceived fluency (β = −0.77, CI-2.5% = –1.71, CI-97.5% = 0.21). While the credible interval for this effect also crosses zero, over 94% of the posterior samples were less than zero. Contrastively, the effect of within-clause pauses in the credibility rating task is also rather negligible (β = −0.10, CI-2.5% = −1.15, CI-97.5% = 0.94). This suggests that the effect of pauses within-clauses was greater in the fluency rating task than the credibility rating task. Finally, to confirm if there was a meaningful difference between the no pause and within-clause pause condition in the fluency task, we examined the difference between the two estimates and found a small difference (β = 1.45, CI-2.5% = −0.24, CI-97.5% = 3.10). While the credible interval crosses zero, over 95.5% of the posterior samples were greater than zero. On the other hand, the difference between these two pause conditions in the credibility task was rather negligible (β = −0.14, CI-2.5% = −1.91, CI-97.5% = 1.63). In other words, while sentences with no pauses are rated as more fluent than sentences with pauses within-clauses, sentences with no pauses are not rated as more credible than sentences with pauses within-clauses. Thus, the effect of pauses is different across tasks. The full model results are included in the appendix section, and the code that we used for the ordinal regression model as well as the simple effects calculations are all included in the supplemental materials.
The above results demonstrate that native listeners’ perception of native and non-native speech was impacted by the task (credibility-rating vs. fluency-rating) as well as by the characteristics of a pause in speech (no pause, between-clause pause, within-clause pause). Specifically, the effect of task was clear, such that speakers’ native language status (native vs. non-native) influenced fluency ratings but not credibility ratings. Further, there is some support that pause characteristics influenced ratings differently depending on the task (though the credible interval for this did cross zero). That is, the presence of a pause (no oause vs. pause) impacted fluency ratings more readily than credibility ratings, for perception of both native and non-native speech. These results suggest that particular acoustic characteristics of speech, originating from speakers’ native language background and pause characteristics, had different perceptual consequences depending on how listeners were encouraged to evaluate the speech in different tasks.
The relationship between credibility- and fluency-ratings for native and non-native speech
In order to better understand the nature of task demands when evaluating native and non-native speech, we examined whether fluency and credibility ratings correlated with one another. Figure 2 shows the relationship between the credibility ratings (x-axis) and fluency ratings (y-axis) for native (left panel) and non-native speech (right panel). Each data point represents a mean rating for an item for a speaker; here, pause conditions (i.e., no pause, between-clause pause, within-clause pause) are collapsed as the observed patterns were similar across all conditions. The solid lines represent the best-fitting linear regression lines. A Pearson correlation test was conducted, and the correlation between the two types of ratings was significant for native speakers’ speech (r = .35, t(46) = 2.51, p = .016), indicating that items that were perceived to be more credible were also perceived to be more fluent. However, the correlation was not significant for non-native speakers’ speech (r = .11, t(46) = −.73, p = .47). These results suggest that there was a relationship between how native listeners evaluated perceived credibility of the statements (meaning-focused evaluation) and fluency of the same statements (form-focused evaluation) produced by native speakers, but not for those produced by non-native speakers.
Scatterplots of correlation between credibility ratings (x-axis) and fluency ratings (y-axis) for native speakers’ speech (left panel) and non-native speakers’ speech (right panel). Pause conditions (no pause, between-clause pause, within-clause pause) are collapsed. The solid lines are best-fitting linear regression lines.
Discussion
The present study examined how characteristics of silent pauses (i.e., presence vs. absence of a pause and pause location) in native and non-native speech impact native listeners’ subjective ratings of these types of speech in different tasks, namely, a credibility rating task and a fluency rating task. The current results suggest that native listeners’ processing of native and non-native speech with silent pauses is impacted by task demands. The findings have implications for our understanding of how disfluencies in speech have different perceptual consequences depending on how listeners approach the listening task.
The effects of native language status and pauses on perception in different tasks
The current results demonstrated that the effects of speakers’ native language status and pauses on listeners’ perception were different across tasks (fluency vs. credibility ratings). Specifically, while silent pauses affected the fluency ratings of both native and non-native speech, we found no meaningful effect of this for the credibility ratings. Since different tasks were designed to draw attention to different parts of the speech, it is likely that the fluency task may have highlighted the form of the speech, while the credibility task may have highlighted the meaning of the speech. Indeed, this would help to explain why the location of pauses impacted fluency ratings of native and non-native speech, but not credibility ratings. We address this possibility in more depth later on in this section.
The present study replicated findings from previous studies that native speech is perceived to be more fluent than non-native speech by native listeners (e.g., Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014b; Kahng, Reference Kahng2018). However, it contrasted with previous findings that the same statements produced by non-native speakers sound less credible than those produced by native speakers (Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010). That is, the present study is one among several studies that fails to find a relationship between speakers’ non-native status and their reduced perceived credibility as compared to native speakers’ (e.g., De Meo et al., Reference De Meo, Vitale, Pettorino and Martin2011; Souza & Markman, Reference Souza and Markman2013; Stocker, Reference Stocker2017). One difference between the present study and Lev-Ari and Keysar (Reference Lev-Ari and Keysar2010) which may have influenced our contradictory results is the speaker population: the speakers in our study were different from the speakers in their study, which may have influenced the results. For example, since we did not explicitly control for overall accentedness and intelligibility of the speakers (though Lev-Ari and Keysar (Reference Lev-Ari and Keysar2010) also do not explicitly control for these either), it is possible that the speakers in their study were more accented than speakers in our study. Additionally, the speakers in Lev-Ari and Keysar (Reference Lev-Ari and Keysar2010) came from different native language backgrounds, which may have further contributed to potentially increased processing costs for the listeners in their study than ours, where the non-native speakers were of the same native language background. Despite this difference, however, there are many studies that have failed to replicate the effect of the non-native speaker status on perceived credibility (e.g., De Meo et al., Reference De Meo, Vitale, Pettorino and Martin2011; Souza & Markman, Reference Souza and Markman2013; Stocker, Reference Stocker2017), suggesting that the relationship between a speaker’s non-native speaker status and their perceived credibility may not be as straightforward as it seems.
Another possible explanation for our results is that credibility may no longer be influenced by a non-native accent once an intelligibility threshold is met. In the current credibility task, in addition to the rating scale, listeners were also given the option of choosing “I didn’t understand what was said,” meaning that the stimuli that were given numeric credibility ratings were likely intelligible enough for listeners to evaluate the credibility of the meaning of the statements (though note that this option was also included in Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010). As shown in previous studies (e.g., Munro & Derwing, Reference Munro and Derwing1995a), more strongly non-native accented speech is not necessarily less intelligible to listeners. Thus, it is possible that acoustic characteristics of the speech that do not impede listeners’ understanding of the meaning of the speech (e.g., mild non-native accent, disfluencies) do not significantly impact perceived credibility. Additionally, we do not find evidence that the effect of pauses was different across the speakers’ native language status. This is interesting because previous research has demonstrated that filled pauses (e.g., “um”) affect the perception of native speech differently than non-native speech (Bosker et al., Reference Bosker, Quené, Sanders and de Jong2014a). Specifically, as mentioned earlier, Bosker et al. (Reference Bosker, Quené, Sanders and de Jong2014a) demonstrated that listeners predict a low frequency word to follow “um” when listening to native speech, but not when listening to non-native speech. One possible explanation for the different results is that perhaps the processing consequences of filled pauses (e.g., “um”) is different from silent pauses. Indeed, previous research has provided evidence that filled and silent pauses have different functions (e.g., Rose, Reference Rose2019; Swerts, Reference Swerts1998). An interesting avenue for future research then might be to examine the differences between the perception of silent and filled pauses across different tasks.
Lastly, as discussed throughout this paper, differences in task demands also influenced the way pause characteristics impacted the perception of native and non-native speech. Specifically, while the location of pauses (i.e., within vs. between-clauses vs. no pauses) impacted the fluency ratings of native and non-native speech, this was not true for credibility ratings. That is, the location of the pauses had little effect on the perceived credibility of the statements. In other words, when listeners were focusing on the form of the speech (i.e., in the fluency task), the disfluencies had a large effect on the ratings, but when listeners were focusing on the content of the speech (i.e., the credibility task), the disfluencies had almost no meaningful effect on the ratings. One possible explanation for this result is that the differences in the task demands draw the listener’s attention to different parts of the speech; fluency rating tasks encourage participants to pay attention to the form of the speech, while credibility rating results encourage participants to pay attention to the meaning of the speech. Thus, we suggest that the differences in the effects of pauses on fluency ratings versus credibility ratings is due to the differences in the task demands. The current results provide support for the claim that the perception of speech is influenced by the task that people are carrying out (e.g., Christenfeld, Reference Christenfeld1995), and further extends such results to perception of native and non-native speech. We explore the relationship between different task demands further in the section below.
Relationship between fluency and credibility ratings
The present study also examined the relationship between fluency and credibility ratings, and whether this relationship differed depending on the native versus non-native status of the speakers. The results indicate that while there was a relationship between fluency and credibility for native speech (i.e., native speech that was rated as more fluent was also rated as more credible), there was no such relationship for the perception of non-native speech.
One possible explanation for this result is that disfluencies in native speech may be more surprising than disfluencies in non-native speech. Listeners’ experience with speech may drive their expectations of upcoming speech (e.g., Hanulíková et al., Reference Hanulíková, van Alphen, Goch and Weber2012). Thus, speech that violates expectations (i.e., the expectation that native speech should be relatively free from disfluencies) may draw attention to the form of the speech even when the task itself does not. This might explain why less fluent speech is rated as less credible for native speech; the results above indicate that silent pauses in native speech have a larger effect on fluency ratings than silent pauses in non-native speech, suggesting that disfluencies in native speech are more noticeable. If disfluencies are more noticeable in native speech than non-native speech, it is possible that this is the reason that disfluent native speech is rated as less credible than fluent native speech, while disfluent non-native speech is not rated as less credible than fluent non-native speech. Further, it is possible that the reason that disfluent non-native speech is not rated as less credible is simply because listeners are not as attentive to the silent pauses (since disfluencies in non-native speech are not as surprising as those in native speech). In other words, the present results may indicate that even when the task (i.e., the credibility task) does not draw attention to the form of the speech, speech characteristics that are surprising enough may call attention to its form. Future studies could examine whether the number of disfluencies impacts credibility ratings in native speech. If the number of disfluencies directly affects credibility ratings in the perception of native speech, this would provide further evidence that speech with more salient disfluencies is rated as less credible.
Conclusion
Together, these results revealed that pauses and speakers’ native language status impact listeners’ perception when they are encouraged to pay attention to the form of the utterance (via fluency rating task), but much less so when they were encouraged to listen for the meaning of the utterance (via credibility rating task). These results suggest that listeners may perceive disfluencies in speech differently depending on whether those variations are relevant to the listening task at hand, adding support to the body of research demonstrating that listening is a goal-oriented, attentionally guided behavior (e.g., Chirstenfeld, Reference Christenfeld1995; Fritz et al., Reference Fritz, Elhilali, David and Shamma2007; Heald & Nusbaum, Reference Heald and Nusbaum2014; Hugdahl et al., Reference Hugdahl, Thomsen, Ersland, Rimol and Niemi2003). Additionally, our results suggest that listeners’ expectations of upcoming speech may further play a role in the effects of attention on processing: listeners confronted with surprising patterns of speech (i.e., speech that is not in line with their expectations) may focus more on the form of that speech, even when performing a task that does not draw attention to the form of the speech. However, it is an open question as to what extent form-oriented evaluation and meaning-oriented evaluation are related to one another.
In summary, the present study contributes to the current body of literature by demonstrating task effects on the processing of silent pauses in native and non-native speech. Specifically, we demonstrate that the perception of non-native and native speech differs depending on whether the listener is attending to the form or the meaning of the speech, which is directly affected by the task they are performing. We also demonstrate the need for future studies exploring the relationship among different types of holistic perception.
Acknowledgments
The authors would like to thank Dr. Santiago Barreda and Dr. Phillipe Rast for their insightful discussion and advice on the statistical analyses and the reporting of the results in this paper.
Appendix
Full model result
Open access
