Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-23T12:20:07.623Z Has data issue: false hasContentIssue false
  • English
  • Français

DECONSTRUCTING COMPREHENSIBILITY

Identifying the Linguistic Influences on Listeners’ L2 Comprehensibility Ratings

Published online by Cambridge University Press:  15 August 2012

Talia Isaacs  and
Pavel Trofimovich
Talia Isaacs*
Affiliation:
University of Bristol
Pavel Trofimovich
Affiliation:
Concordia University
*
*Correspondence regarding this work should be addressed to: Talia Isaacs, Graduate School of Education, University of Bristol, 35 Berkeley Square, Clifton, Bristol, United Kingdom BS8 1JA; e-mail: talia.isaacs@bristol.ac.uk.
Get access Rights & Permissions [Opens in a new window]

Abstract

Comprehensibility, a major concept in second language (L2) pronunciation research that denotes listeners’ perceptions of how easily they understand L2 speech, is central to interlocutors’ communicative success in real-world contexts. Although comprehensibility has been modeled in several L2 oral proficiency scales—for example, the Test of English as a Foreign Language (TOEFL) or the International English Language Testing System (IELTS)—shortcomings of existing scales (e.g., vague descriptors) reflect limited empirical evidence as to which linguistic aspects influence listeners’ judgments of L2 comprehensibility at different ability levels. To address this gap, a mixed-methods approach was used in the present study to gain a deeper understanding of the linguistic aspects underlying listeners’ L2 comprehensibility ratings. First, speech samples of 40 native French learners of English were analyzed using 19 quantitative speech measures, including segmental, suprasegmental, fluency, lexical, grammatical, and discourse-level variables. These measures were then correlated with 60 native English listeners’ scalar judgments of the speakers’ comprehensibility. Next, three English as a second language (ESL) teachers provided introspective reports on the linguistic aspects of speech that they attended to when judging L2 comprehensibility. Following data triangulation, five speech measures were identified that clearly distinguished between L2 learners at different comprehensibility levels. Lexical richness and fluency measures differentiated between low-level learners; grammatical and discourse-level measures differentiated between high-level learners; and word stress errors discriminated between learners of all levels.

Type
ARTICLES
Information
Studies in Second Language Acquisition , Volume 34 , Issue 3 , September 2012 , pp. 475 - 505
Copyright
Copyright © Cambridge University Press 2012

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Anderson-Hsieh, J., Johnson, R., & Koehler, K. (1992). The relationship between native speaker judgments of nonnative pronunciation and deviance in segmentals, prosody, and syllable structure. Language Learning, 42, 529–555.CrossRefGoogle Scholar
Barkaoui, K. (2008). Effects of scoring method and rater experience on ESL essay rating processes and outcomes. Unpublished doctoral dissertation, University of Toronto, Canada.Google Scholar
Boersma, P., & Weenink, D. (2010). Praat: Doing phonetics by computer (Version 5.1.29) [Computer program]. Retrieved March 10, 2010, fromwww.praat.org.Google Scholar
Brace, N., Kemp, R., & Snelgar, R. (2006). SPSS for psychologists: A guide to data analysis using SPSS for Windows (3rd ed.). Mahwah, NJ: Erlbaum.Google Scholar
Breiner-Sanders, K., Lowe, P., Miles, J., & Swender, E. (2000). ACTFL proficiency guidelines—Speaking revised 1999. Foreign Language Annals, 33, 13–18.CrossRefGoogle Scholar
British Council, IDP: IELTS Australia, & UCLES. (n.d.). IELTS speaking band descriptors (public version). Retrieved April 8, 2011, fromwww.ielts.org/PDF/UOBDs_SpeakingFinal.pdf.Google Scholar
Brown, A., Iwashita, N., & McNamara, T. (2005). An examination of rater orientations and test-taker performance on English for academic purposes speaking tasks. TOEFL Monograph, 29. Princeton, NJ: Educational Testing Service.Google Scholar
Chalhoub-Deville, M. (1995). A contextualized approach to describing oral language proficiency. Language Learning, 45, 251–281.CrossRefGoogle Scholar
Cobb, T. (2000). The complete lexical tutor. Retrieved April 17, 2012, fromhttp://www.lextutor.ca.Google Scholar
Cohen, L., Manion, L., & Morrison, K. R. B. (2000). Research methods in education (5th ed.). London: Routledge.Google Scholar
Colby-Kelly, C., & Turner, C. E. (2007). AFL research in the L2 classroom and evidence of usefulness: Taking formative assessment to the next level. Canadian Modern Language Review, 64, 9–37.CrossRefGoogle Scholar
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. New York: Cambridge University Press.Google Scholar
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
Derwing, T. M., & Munro, M. J. (1997). Accent, intelligibility, and comprehensibility: Evidence from four L1s. Studies in Second Language Acquisition, 19, 1–16.CrossRefGoogle Scholar
Derwing, T. M., & Munro, M. J. (2009). Putting accent in its place: Rethinking obstacles to communication. Language Teaching, 42, 1–15.CrossRefGoogle Scholar
Derwing, T. M., Munro, M. J., & Thomson, R. I. (2008). A longitudinal study of ESL learners’ fluency and comprehensibility development. Applied Linguistics, 29, 359–380.CrossRefGoogle Scholar
Derwing, T. M., & Rossiter, M. J. (2003). The effects of pronunciation instruction on the accuracy, fluency, and complexity of L2 accented speech. Applied Language Learning, 13, 1–17.Google Scholar
Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54, 665–679.CrossRefGoogle Scholar
Deterding, D. (2001). The measurement of rhythm: A comparison of Singapore and British English. Journal of Phonetics, 29, 217–230.CrossRefGoogle Scholar
de Villiers, P. A. (2004). Assessing pragmatic skills in elicited production. Seminars in Speech and Language, 25, 57–72.CrossRefGoogle ScholarPubMed
Educational Testing Service. (2005). TOEFL iBT tips: How to prepare for the next generation TOEFL test and communicate with confidence. Princeton, NJ: Author.Google Scholar
Fayer, J. M., & Krasinski, E. (1987). Native and nonnative judgments of intelligibility and irritation. Language Learning, 37, 313–326.CrossRefGoogle Scholar
Field, J. (2005). Intelligibility and the listener: The role of lexical stress. TESOL Quarterly, 39, 399–423.CrossRefGoogle Scholar
Foote, J. A., Holtby, A., & Derwing, T. M. (2011). 2010 survey of pronunciation teaching in adult ESL programs in Canada. TESL Canada Journal, 29, 1–22.CrossRefGoogle Scholar
Foster, P., & Skehan, P. (1996). The influence of planning and task type on second language performance. Studies in Second Language Acquisition, 18, 299–323.CrossRefGoogle Scholar
Fulcher, G. (1996). Does thick description lead to smart tests? A data-based approach to rating scale construction. Language Testing, 13, 208–238.CrossRefGoogle Scholar
Gaies, S. J. (1980). T-unit analysis in second language research: Applications, problems and limitations. TESOL Quarterly, 14, 53–60.CrossRefGoogle Scholar
Gilbert, J. B. (2010). Pronunciation as orphan: What can be done? Speak Out! 43, 3–7.Google Scholar
Goldwater, S., Jurafsky, D., & Manning, C. D. (2010). Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates. Speech Communication, 52, 181–200.CrossRefGoogle Scholar
Hahn, L. D. (2004). Primary stress and intelligibility: Research to motivate the teaching of suprasegmentals. TESOL Quarterly, 38, 201–233.CrossRefGoogle Scholar
Hamp-Lyons, L. (1991). Scoring procedures for ESL contexts. In Hamp-Lyons, L. (Ed.), Assessing second language writing in academic contexts (pp. 241–276). Westport, CT: Ablex.Google Scholar
Harding, L. (in press). Pronunciation assessment. In Chapelle, C. A. (Ed.), The encyclopedia of applied linguistics. Hoboken, NJ: Wiley-Blackwell.Google Scholar
Isaacs, T. (2009). Integrating form and meaning in L2 pronunciation instruction. TESL Canada Journal, 26, 1–12.CrossRefGoogle Scholar
Isaacs, T. (in press). Assessing pronunciation. In Kunnan, A. J. (Ed.), The companion to language assessment. Hoboken, NJ: Wiley-Blackwell.Google Scholar
Isaacs, T., & Thomson, R. I. (2009, March). Judgments of L2 comprehensibility, accentedness, and fluency: The listeners’ perspective. Paper presented at the Language Testing Research Colloquium, Denver, CO.Google Scholar
Isaacs, T., & Thomson, R. I. (in press). Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly.Google Scholar
Isaacs, T., & Trofimovich, P. (2011). Phonological memory, attention control, and musical ability: Effects of individual differences on rater judgments of second language speech. Applied Psycholinguistics, 32, 113–140.CrossRefGoogle Scholar
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S. (2008). Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29, 24–49.CrossRefGoogle Scholar
Jenkins, J. (2000). The phonology of English as an international language. Oxford: Oxford University Press.Google Scholar
Jenkins, J. (2002). A sociolinguistically based, empirically researched pronunciation syllabus for English as an international language. Applied Linguistics, 23, 83–103.CrossRefGoogle Scholar
Kang, O., Rubin, D., & Pickering, L. (2010). Suprasegmental measures of accentedness and judgments of language learner proficiency in oral English. Modern Language Journal, 94, 554–566.CrossRefGoogle Scholar
Kennedy, S. (2009). L2 proficiency: Measuring the intelligibility of words and extended speech. In Benati, A. G. (Ed.), Issues in second language proficiency (pp. 132–146). New York: Continuum.Google Scholar
Kennedy, S., & Trofimovich, P. (2010). Language awareness and second language pronunciation: A classroom study. Language Awareness, 19, 171–185.CrossRefGoogle Scholar
Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307–322.CrossRefGoogle Scholar
Lennon, P. (1990). Investigating fluency in EFL: A quantitative approach. Language Learning, 40, 387–417.CrossRefGoogle Scholar
Levelt, W. J. M. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.Google Scholar
Levis, J. M. (2005). Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 39, 369–377.CrossRefGoogle Scholar
Levis, J. M. (2006). Pronunciation and the assessment of spoken language. In Hughes, R. (Ed.), Spoken English, TESOL and applied linguistics: Challenges for theory and practice (pp. 245–270). New York: Palgrave Macmillan.CrossRefGoogle Scholar
Martin, J. R., & Rose, D. (2003). Working with discourse: Meaning beyond the clause. New York: Continuum.Google Scholar
Morley, J. (1994). A multidimensional curriculum design for speech-pronunciation instruction. In Morley, J. (Ed.), Pronunciation pedagogy and theory (pp. 64–91). Alexandria, VA: TESOL.Google Scholar
Munro, M. J. (2008). Foreign accent and speech intelligibility. In Hansen Edwards, J. & Zampini, M. (Eds.), Phonology and second language acquisition (pp. 193–218). Amsterdam: Benjamins.CrossRefGoogle Scholar
Munro, M. J., & Derwing, T. M. (1999). Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning, 49, 285–310.CrossRefGoogle Scholar
Munro, M. J., & Derwing, T. M. (2006). The functional load principle in ESL pronunciation instruction: An exploratory study. System, 34, 520–531.CrossRefGoogle Scholar
Munro, M. J., & Derwing, T. M. (2011). The foundations of accent and intelligibility in pronunciation research. Language Teaching, 44, 316–327.CrossRefGoogle Scholar
North, B. (2000). The development of a common framework scale of language proficiency. Bern: Peter Lang.CrossRefGoogle Scholar
Peperkamp, S., & Dupoux, E. (2002). A typological study of stress “deafness.” In Gussenhoven, C. & Warner, N. (Eds.), Laboratory phonology 7 (pp. 203–236). Berlin: Mouton de Gruyter.CrossRefGoogle Scholar
Pickering, L. (2001). The role of tone choice in improving ITA communication in the classroom. TESOL Quarterly, 35, 233–255.CrossRefGoogle Scholar
Riggenbach, H. (1991). Toward an understanding of fluency: A microanalysis of non-native speaker conversations. Discourse Processes, 14, 423–441.CrossRefGoogle Scholar
Rozeboom, W. W. (1960). The fallacy of the null-hypothesis significance test. Psychological Bulletin, 57, 416–428.CrossRefGoogle ScholarPubMed
Saito, K., & Lyster, R. (2011). Effects of form-focused instruction and corrective feedback on L2 pronunciation development of /ɹ/ by Japanese learners of English. Language Learning, 62. Retrieved April 17, 2012, fromhttp://onlinelibrary.wiley.com/doi/10.1111/j.1467-9922.2011.00639.x/abstract;jsessionid=490BD498483DC416780FB0D418D12EE2.d04t04?userIsAuthenticated=false&deniedAccessCustomisedMessage=.Google Scholar
Segalowitz, N. (2010). The cognitive bases of second language fluency. London: Routledge.CrossRefGoogle Scholar
Skehan, P., & Foster, P. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning, 49, 93–120.CrossRefGoogle Scholar
Stein, N. L., & Glenn, C. G. (1979). An analysis of story comprehension in elementary school children. In Freedle, R. (Ed.), New directions in discursive processing (pp. 53–120). Westport, CT: Ablex.Google Scholar
Swan, M. (1997). The influence of the mother tongue on second language vocabulary acquisition and use. In Schmitt, N. & McCarthy, M. (Eds.), Vocabulary: Description, acquisition and pedagogy (pp. 156–180). New York: Cambridge University Press.Google Scholar
Swan, M., & Smith, B. (Eds.). (2001). Learner English: A teacher’s guide to interference. New York: Cambridge University Press.CrossRefGoogle Scholar
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.Google Scholar
Thomson, R. I., & Isaacs, T. (2010, June). Variation in L2 oral performance: An examination of task type, topic, and speaker involvement. Paper presented at the annual conference of the Canadian Association of Applied Linguistics, Montreal, QC.Google Scholar
Trofimovich, P., Gatbonton, E., & Segalowitz, N. (2007). A dynamic look at L2 phonological learning: Seeking processing explanations for implicational phenomena. Studies in Second Language Acquisition, 29, 407–448.CrossRefGoogle Scholar
University of Cambridge Local Examinations Syndicate (UCLES). (2008). Certificate of Proficiency in English: Handbook for teachers. Cambridge: Author.Google Scholar
Upshur, J. A., & Turner, C. E. (1999). Systemic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16, 82–111.CrossRefGoogle Scholar
Varonis, E. M., & Gass, S. (1982). The comprehensibility of non-native speech. Studies in Second Language Acquisition, 4, 114–136.CrossRefGoogle Scholar
Wennerstrom, A. (2001). The music of everyday speech: Prosody and discourse analysis. Oxford: Oxford University Press.CrossRefGoogle Scholar
Zielinski, B. W. (2008). The listener: No longer the silent partner in reduced intelligibility. System, 36, 69–84.CrossRefGoogle Scholar