Skip to main content
×
×
Home

Flawed self-assessment: Investigating self- and other-perception of second language speech*

  • PAVEL TROFIMOVICH (a1), TALIA ISAACS (a2), SARA KENNEDY (a1), KAZUYA SAITO (a3) and DUSTIN CROWTHER (a4)...
Abstract

This study targeted the relationship between self- and other-assessment of accentedness and comprehensibility in second language (L2) speech, extending prior social and cognitive research documenting weak or non-existing links between people's self-assessment and objective measures of performance. Results of two experiments (N = 134) revealed mostly inaccurate self-assessment: speakers at the low end of the accentedness and comprehensibility scales overestimated their performance; speakers at the high end of each scale underestimated it. For both accent and comprehensibility, discrepancies in self- versus other-assessment were associated with listener-rated measures of phonological accuracy and temporal fluency but not with listener-rated measures of lexical appropriateness and richness, grammatical accuracy and complexity, or discourse structure. Findings suggest that inaccurate self-assessment is linked to the inherent complexity of L2 perception and production as cognitive skills and point to several ways of helping L2 speakers align or calibrate their self-assessment with their actual performance.

Copyright
Corresponding author
Address for correspondence: Pavel Trofimovich, Concordia University, Department of Education, 1445 de Maisonneuve Blvd., West Montreal, Quebec, Canada H3G 1M8 pavel.trofimovich@concordia.ca
Footnotes
Hide All
*

This study was funded by grants from the Social Sciences and Humanities Research Council of Canada (SSHRC) and Fonds de recherche sur la société et la culture (FRQSC) awarded to the first, second, and third authors, a Marie Curie Career Integration Grant (European Commission) awarded to the second author, and by the Grant-in-Aid for Scientific Research in Japan (No. 26770202) awarded to the fourth author. We are grateful to participants, Ze Shan Yao for his technical help, and to anonymous BLC reviewers for their helpful input and feedback on the content of this manuscript.

Footnotes
References
Hide All
Alderson, C. J. (2005). Diagnosing foreign language proficiency: The interface between learning and assessment. London: Continuum.
Ammar, A., & Spada, N. (2006). One size fits all? Recasts, prompts, and L2 learning. Studies in Second Language Acquisition, 28, 543574. doi:10.1017/S0272263106060268.
Anderson-Hsieh, J., Johnson, R., & Koehler, K. (1992). The relationship between native speaker judgments of nonnative pronunciation and deviance in segmentals, prosody, and syllable structure. Language Learning, 42, 529555. doi:10.1111/j.1467-1770.1992.tb01043.x
Bachman, L. F., & Palmer, A. S. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6, 1429. doi:10.1177/026553228900600104
Blanche, P. & Merino, B. J. (1989). Self assessment of foreign-language skills: Implications for teachers and researchers. Language Learning, 39, 313340. doi:10.1111/j.1467-1770.1989.tb00595.x
Brantmeier, C., Vanderplank, R., & Strube, M. (2012). What about me? Individual self-assessment by skill and level of language instruction. System, 40, 144160. doi:10.1016/j.system.2012.01.003
Brown, P., & Levinson, S. (1987). Politeness: Some universals in language usage. Cambridge, UK: Cambridge University Press.
Burson, K. A., Larrick, R. P., & Klayman, J. (2006). Skilled or unskilled, but still unaware of it: How perceptions of difficulty drive miscalibration in relative comparisons. Journal of Personality and Social Psychology, 90, 6077. doi:10.1037/0022-3514.90.1.60
Carter, T. J., & Dunning, D. (2008). Faulty self-assessment: Why evaluating one's own competence is an intrinsically difficult task. Social and Personality Psychology Compass, 2, 346360. doi:10.1111/j.1751-9004.2007.00031.x
Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge certificate exams, IELTS, and TOEFL. System, 28, 523539. doi:10.1016/S0346-251X(00)00036-1
Critcher, C. R., & Dunning, D. (2009). How chronic self-views influence (and mislead) self-assessments of task performance: Self-views shape bottom-up experiences with the task. Journal of Personality and Social Psychology, 97, 931945. doi:10.1037/a0017452
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2014). Assessing lexical proficiency using analytic ratings: A case for collocation accuracy. Applied Linguistics. doi:10.1093/applin/amt056
Crowther, D., Trofimovich, P., Saito, K., & Isaacs, T. (2014). Second language comprehensibility revisited: Investigating the effects of learner background. TESOL Quarterly. Published online 28 October 2014. doi:10.1002/tesq.203
Davidson, F., & Henning, G. (1985). A self-rating scale of English difficulty: Rasch scalar analysis of items and rating categories. Language Testing, 2, 164179. doi:10.1177/026553228500200205
de Saint Léger, D. (2009). Self-assessment of speaking skills and participation in a foreign language class. Foreign Language Annals, 42, 158178. doi:10.1111/j.1944-9720.2009.01013.x
Derwing, T. M., & Munro, M. J. (2009). Putting accent in its place: Rethinking obstacles to communication. Language Teaching, 42, 476490. doi:10.1017/S026144480800551X
Derwing, T. M., & Rossiter, M. J. (2002). ESL learners’ perceptions of their pronunciation needs and strategies. System, 30, 155166. doi:10.1016/S0346-251X(02)00012-X
Derwing, T. M., Rossiter, M. J., Munro, M. J., & Thomson, R. I. (2004). Second language fluency: Judgments on different tasks. Language Learning, 54, 655679. doi:10.1111/j.1467-9922.2004.00282.x
Dlaska, A., & Krekeler, C. (2008). Self-assessment of pronunciation. System, 36, 506516. doi:10.1016/j.system.2008.03.003
Dunning, D. (2005). Self-insight: Roadblocks and detours on the path to knowing thyself. New York: Psychology Press.
Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5, 69106. doi:10.1111/j.1529-1006.2004.00018.x
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12, 8387. doi:10.1111/1467-8721.01235
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989). Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability. Journal of Personality and Social Psychology, 57, 10821090. doi:10.1037/0022-3514.57.6.1082
Eckman, F. (2004). From phonemic differences to constraint rankings: Research on second language phonology. Studies in Second Language Acquisition, 26, 513549. doi:10.1017/S027226310404001X
Ehrlinger, J., & Dunning, D. (2003). How chronic self-views influence (and potentially mislead) estimates of performance. Journal of Personality and Social Psychology, 84, 517. doi:10.1037/0022-3514.84.1.5
Ehrlinger, J., Johnson, K., Banner, M., Dunning, D., & Kruger, J. (2008). Why the unskilled are unaware: Further exploration of (absent) self-insight among the incompetent. Organizational Behavior and Human Decision Processes, 105, 98121. doi:10.1016/j.obhdp.2007.05.002
Eid, M. & Diener, E. (2001). Norms for experiencing emotions in different cultures: Inter- and intranational differences. Journal of Personality and Social Psychology, 81, 869885. doi:10.1037/0022-3514.81.5.869
Falchikov, N., & Boud, D. (1989). Student self-assessment in higher education: A meta-analysis. Review of Education Research, 59, 395430. doi:10.3102/00346543059004395
Fay, A. J., Jordan, A. H., & Ehrlinger, J. (2012). How social norms promote misleading social feedback and inaccurate self-assessment. Social and Personality Psychology Compass, 6, 206216. doi:10.1111/j.1751-9004.2011.00420.x
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Thousand Oaks, CA: Sage.
Flege, J. (2003). Assessing constraints on second-language segmental production and perception. In Meyer, A. & Schiller, N. (Eds.), Phonetics and phonology in language comprehension and production: Differences and similarities (pp. 319355). Berlin: Mouton de Gruyter.
Flege, J. E., Munro, M. J., & MacKay, I. R. A. (1995). Effects of age of second-language learning on the production of English consonants. Speech Communication, 16, 126. doi:10.1016/0167-6393(94)00044-B
Foote, J. A., Trofimovich, P., Collins, L., & Soler Urzúa, F. (2013). Pronunciation teaching practices in communicative ESL classes. The Language Learning Journal. Published online 16 April 2013. doi:10.1080/09571736.2013.784345
Fussell, S. R., & Krauss, R. M. (1992). Coordination of knowledge in communication: Effects of speakers’ assumptions about what others know. Journal of Personality and Social Psychology, 62, 378391. doi:10.1037/0022-3514.62.3.378
Glover, P. (2011). Using CEFR level descriptors to raise university students’ awareness of their speaking skills. Language Awareness, 20, 121133. doi:10.1080/09658416.2011.555556
Guenther, C. L., & Alicke, M. D. (2010). Deconstructing the better-than-average effect. Journal of Personality and Social Psychology, 99, 755770. doi:10.1037/a0020959
Hacker, D. J., Bol, L., Horgan, D., & Rakow, E. (2000). Test prediction and performance in a classroom context. Journal of Educational Psychology, 92, 160170. doi:10.1037/0022-0663.92.1.160
Harris, M. M., & Schaubroek, J. (1988). A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personnel Psychology, 41, 4362. doi:10.1111/j.1744-6570.1998.tb00631.x
Hayes, A. F., & Dunning, D. (1997). Construal processes and trait ambiguity: Implications for self-peer agreement in personality judgment. Journal of Personality and Social Psychology, 72, 664677. doi:10.1037/0022-3514.72.3.664
Heilenman, L. K. (1990). Self-assessment of second language ability: The role of response effects. Language Testing, 7, 174201. doi:10.1177/02655322900070004
Heine, S. J., Kitayama, S., & Lehman, D. R. (2001). Cultural differences in self-evaluation: Japanese readily accept negative self-relevant information. Journal of Cross-Cultural Psychology, 32, 434443. doi:10.1177/0022022101032004004
Hodges, B., Regehr, G., & Martin, D. (2001). Difficulties in recognizing one's own incompetence: Novice physicians who are unskilled and unaware of it. Academic Medicine, 76, S87S89.
Hutcheson, G. D., & Sofroniou, N. (1999). The multivariate social scientist. London: Sage.
Isaacs, T., & Trofimovich, P. (2011). International students at Canadian universities: Validating a pedagogically-oriented pronunciation scale. Unpublished corpus of second language speech.
Isaacs, T., & Trofimovich, P. (2012). Deconstructing comprehensibility: Identifying the linguistic influences on listeners’ L2 comprehensibility ratings. Studies in Second Language Acquisition, 34, 475505. doi:10.1017/S0272263112000150
Janssen van Dieten, A.-M. (1989). The development of a test of Dutch as a second language: The validity of self-assessment by inexperienced subjects. Language Testing, 6, 3046. doi:10.1177/026553228900600105
Jun, S.-A. (2005). Prosodic typology. In Jun, S.-A. (Ed.), Prosodic typology: The phonology of intonation and phrasing (pp. 430458). Oxford: Oxford University Press.
King, C. (1994). One language, two scripts: The Hindi movement in nineteenth century North India. Delhi: Oxford University Press.
Klin, C. M., Guzman, A. E., & Levine, W. H. (1997). Knowing that you don't know: Metamemory and discourse processing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 13781393. doi:10.1037/0278-7393.23.6.1378
Kruger, J., & Dunning, D. (1999). Unskilled or unaware of it: Difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77, 11211134. doi:10.1037/0022-3514.77.6.1121
Kruger, J., & Mueller, R. A. (2002). Unskilled, unaware, or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. Journal of Personality and Social Psychology, 82, 180188. doi:10.1037/0022-3514.82.2.180
Larson-Hall, J. (2010). A guide to doing statistics in second language research using SPSS. New York: Routledge.
Levis, J. M. (2005). Changing contexts and shifting paradigms in pronunciation teaching. TESOL Quarterly, 39, 369377. doi:10.2307/3588485
Lichtenstein, S., & Fischoff, B. (1977). Do those who know more also know more about how much they know? Organization Behavior and Human Performance, 20, 159183. doi:10.1016/0030-5073(77)90001-0
Little, D. (2005). The Common European Framework and the European Language Portfolio: Involving learners and their judgements in the assessment process. Language Testing, 22, 321336. doi:10.1191/0265532205lt311oa
Long, M. (1996). The role of the linguistic environment in second language acquisition. In Ritchie, W. C. & Bhatia, T. K. (Eds.), Handbook of second language acquisition (pp. 413468). New York: Academic Press.
Long, M. H. (1991). Focus on form: A design feature in language teaching methodology. In de Bot, K., Coste, D., Ginsberg, R., & Kramsch, C. (Eds.), Foreign language research in cross-cultural perspective (pp. 3952). Amsterdam: John Benjamins.
Lyster, R. (2001). Negotiation of form, recasts, and explicit correction in relation to error types and learner repair in immersion classrooms. Language Learning, 51, 265301. doi:10.1111/j.1467-1770.2001.tb00019.x
Mabe, P. A., & West, S. G. (1982). Validity of self-evaluation of ability: A review and meta-analysis. Journal of Applied Psychology, 67, 280296. doi:10.1037/0021-9010.67.3.280
Mackey, A. & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research synthesis. In Mackey, A. (Ed.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407452). Oxford: Oxford University Press.
Mattern, K. D., Burrus, J., & Shaw, E. (2010). When both skilled and unskilled are unaware: Consequences for academic performance. Self and Identity, 9, 129141. doi:10.1080/15298860802618963
Moyer, A. (2013). Foreign accent: The phenomenon of non-native speech. Cambridge: Cambridge University Press.
Ochs, E. (1979). Transcription as theory. In Orchs, E. & Schieffelin, B. B. (Eds.), Developmental pragmatics (pp. 4372). New York: Academic Press.
Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 15, 120. doi:10.1177/026553229801500101
Saito, K. (2013). Communicative focus on second language phonetic form: Teaching Japanese learners to perceive and produce English/ɹ/without explicit instruction. Applied Psycholinguistics. Published online 28 April 2013. doi:10.1017/S0142716413000271
Saito, K., Trofimovich, P., & Isaacs, T. (forthcoming). Using listener judgements to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and generalization study. Applied Linguistics.
Schmidt, R. (2001). Attention. In Robinson, P. (Ed.), Cognition and second language instruction (pp. 332). Cambridge: Cambridge University Press.
Segalowitz, N. (2010). Cognitive bases of second language fluency. New York: Routledge.
Shackle, C. (2001). Speakers of South Asian languages. In Swan, M. & Smith, B. (Eds.), Learner English: A teacher's guide to interference and other problems (pp. 310324). Cambridge: Cambridge University Press.
Simons, D. J. (2013). Unskilled and optimistic: Overconfident predictions despite calibrated knowledge of relative skill. Psychonomic Bulletin & Review, 20, 601607.
Sinkavich, F. J. (1995). Performance and metamemory: Do students know what they don't know? Instructional Psychology, 22, 7787.
Smiljanic, R., & Bradlow, A. R. (2009). Speaking and hearing clearly: Talker and listener factors in speaking style changes. Linguistics and Language Compass, 3, 236264. doi:10.1111/j.1749-818X.2008.00112.x
Stemler, S.E., & Tsai, J. (2008). Best practices in estimating interrater reliability. In Osborne, J. (Ed.): Best practices in quantitative methods (pp. 2949). Thousand Oaks, CA: Sage.
Tajima, K., Port, R., & Dalby, J. (1997). Effects of temporal correction on intelligibility of foreign-accented English. Journal of Phonetics, 25, 124. doi:10.1006/jpho.1996.0031
Tesser, A., & Rosen, S. (1975). The reluctance to transmit bad news. In Berkowitz, L. (Ed.), Advances in experimental social psychology (pp. 193232). New York: Academic Press.
Tokumoto, M., & Shibata, M. (2011). Asian varieties of English: Attitudes towards pronunciation. World Englishes, 30, 392408. doi:10.1111/j.1467-971X.2011.01710.x
Trofimovich, P., & Isaacs, T. (2012). Disentangling accent from comprehensibility. Bilingualism: Language and Cognition, 15, 905916. doi:10.1017/S1366728912000168
Upshur, J. (1975). Objective evaluation of oral proficiency in the ESOL classroom. In Palmer, L. & Spolsky, B. (Eds.), Papers on language testing (pp. 19671974). Washington: TESOL.
Weigle, S. C. (2010). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing, 27, 335353. doi:10.1177/0265532210364406
Yule, G., Damico, J., & Hoffman, P. (1987). Learners in transition: Evidence from the interaction of accuracy and self-monitoring skill in a listening task. Language Learning, 37, 511521. doi:10.1111/j.1467-1770.1987.tb00582.x
Yule, G., Hoffman, P., & Damico, J. (1987). Paying attention to pronunciation: The role of self-monitoring in perception. TESOL Quarterly, 21, 765768. doi:10.2307/3586994
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Bilingualism: Language and Cognition
  • ISSN: 1366-7289
  • EISSN: 1469-1841
  • URL: /core/journals/bilingualism-language-and-cognition
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 16
Total number of PDF views: 300 *
Loading metrics...

Abstract views

Total abstract views: 947 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 18th August 2018. This data will be updated every 24 hours.