Skip to main content

Innovative assessment of aviation English in a virtual world: Windows into cognitive and metacognitive strategies

  • Moonyoung Park (a1)

Aviation English proficiency is a core competency in the global air traffic controller profession. There is, however, growing concern about the current ineffective paper-based assessment methods and the severe lack of interactive online testing for such a critical profession, one that should be ideally assessed in an authentic task and situation (Alderson, 2010; Douglas, 2013). The tests, which lack validity and authenticity, cannot capture how knowledge of aviation English communication and strategic competence are used through valid means and, therefore, inevitably fail to predict how test takers actually perform in the target language use situations (Douglas, 2000). In the present study, the researcher examines the potential use of verbal report data produced by test takers interacting in a virtual testing environment. The research seeks to answer two specific questions: (a) What types of strategies are used in the virtual aviation English task performance? and (b) How can the assessed strategies be interpreted in relation to test takers’ performance? The analysis of the test takers’ verbal reports from stimulated recalls indicates that various cognitive, metacognitive, and communication strategies were used while performing the Virtual Interactive Tasks for Aviation English Assessment, and that there is a positive relationship between the total number of cognitive and metacognitive strategies adopted and the test scores. The findings suggest that the use of an immersive interface and simulated tasks in a virtual world could provide language learners with more authentic opportunities to perform the target tasks and promote strategic, as well as linguistic, competence.

Hide All
Abdallah, M. M. S. and Mansour, M. M. (2015) Virtual task-based situated language-learning with Second Life: Developing EFL pragmatic writing and technological self-efficacy. Arab World English . Journal, 2: 150182.
Alderson, J. C. (2006) Final report on a survey of aviation English tests. Lancaster: Lancaster University and the European Organisation for the Safety of Air Navigation (Eurocontrol).
Alderson, J. C. (2009) Air safety, language assessment policy, and policy implementation: The case of aviation English. Annual Review of Applied Linguistics, 29: 168187.
Alderson, J. C. (2010) A survey of aviation English tests. Language Testing, 27(1): 5172.
Alderson, J. C., Clapham, C. and Wall, D. (1995) Language test construction and evaluation. Cambridge: Cambridge University Press.
Bachman, L. F. and Palmer, A. S. (1996) Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Bachman, L. F. and Palmer, A. S. (2010) Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford: Oxford University Press.
Berns, A., Gonzalez-Pardo, A. and Camacho, D. (2013) Game-like language learning in 3-D virtual environments. Computers & Education, 60(1): 210220.
Brusso, R. C., Wisher, R. A., Paddock, A. and Hatfield, J. (2014) Best practices and provisional guidelines for integrating mobile, virtual, and videogame-based training and assessments. Fairfax: ICF International.
Can, T. and Simsek, I. (2015) The use of 3D virtual learning environments in training foreign language pre-service teachers. Turkish Online Journal of Distance Education, 16(4): 114124.
Canto, S., Jauregi, K. and van den Bergh, H. (2013) Integrating cross-cultural interaction through video-communication and virtual worlds in foreign language teaching programs: Is there an added value? ReCALL, 25(1): 105121.
Chamot, A. U. and O’Malley, J. M. (1987) The cognitive academic language learning approach: A bridge to the mainstream. TESOL Quarterly, 21(2): 227249.
Chapelle, C. A. (1998) Construct definition and validity inquiry in SLA research. In Bachman, L. F. & Cohen, A. D. (eds.), Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press, 3270.
Chapelle, C. A. (2005) Interactionist SLA theory in CALL research. In Egbert, J. L. & Petrie, G. M. (eds.), CALL research perspectives. Mahwah, NJ: Lawrence Erlbaum, 5364.
Chuang, M.-T., Chang, B. and Chen, H.-C. (2014) Investigating reported social and affective strategy use by EFL learners in virtual and real learning environments. International Journal of Applied Linguistics & English Literature, 3(2): 2733.
Clarke-Midura, J. and Dede, C. (2010) Assessment, technology, and change. Journal of Research on Technology in Education, 42(3): 309328.
Corbin, J. and Strauss, A. (2008) Basics of qualitative research: Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks: Sage.
Creswell, J. W. and Plano Clark, V. L. (2007) Designing and conducting mixed methods research. Thousand Oaks: Sage.
Crystal, D. (2003) English as a global language (2nd ed.). New York: Cambridge University Press.
Cushing, S. (1994) Fatal words: Communication clashes and aircraft crashes. Chicago: University of Chicago Press.
da Silva, K. T. F. (2012) Preparing language teachers to teach in virtual worlds: Analyzing their content, technological, and pedagogical needs. Iowa State University, unpublished PhD.
Dörnyei, Z. and Scott, M. L. (1997) Communication strategies in a second language: Definitions and taxonomies. Language Learning, 47(1): 173210.
Douglas, D. (2000) Assessing languages for specific purposes. Cambridge: Cambridge University Press.
Douglas, D. (2001) Language for specific purposes assessment criteria: Where do they come from? Language Testing, 18(2): 171185.
Douglas, D. (2013) ESP and assessment. In Paltridge, B. & Starfield, S. (eds.), The handbook of English for specific purposes. Malden: John Wiley & Sons, 367383.
Hammick, J. K. and Lee, M. J. (2014) Do shy people feel less communication apprehension online? The effects of virtual reality on the relationship between personality characteristics and communication outcomes. Computers in Human Behavior, 33: 302310.
Holliday, L. (2007) Theory and research: Interaction and awareness. In Egbert, J. & Hanson-Smith, E. (eds.), CALL environments: Research, practice, and critical issues (2nd ed.). Alexandria, VA: Teachers of English to Speakers of Other Languages, 163170.
International Civil Aviation Organization (ICAO) (2004) Manual on the implementation of ICAO language proficiency requirements (ICAO Doc 9835). Chicago: Author.
Khalsa, D. K., Maloney-Krichmar, D. and Peyton, J. K. (2007) Theory and research: Interaction via computers. In Egbert, J. & Hanson-Smith, E. (eds.), CALL environments: Research, practice, and critical (2nd ed.). Alexandria, VA: Teachers of English to Speakers of Other Languages, 1931.
Lai, E. R. (2011) Metacognition: A literature review: Research report. Upper Saddle River: Pearson Education.
Lin, T.-J., Wang, S.-Y., Grant, S., Chien, C.-L. and Lan, Y.-J. (2014) Task-based teaching approaches of Chinese as a foreign language in Second Life through teachers’ perspectives. Procedia Technology, 13: 1622.
Long, M. H. (2005) Methodological issues in learner needs analysis. In Long, M. H. (ed.), Second language needs analysis. Cambridge: Cambridge University Press, 1976.
Melchor-Couto, S. (2017) Foreign language anxiety levels in Second Life oral interaction. ReCALL, 29(1): 99119.
Mell, J. (1992) Study of verbal communication between pilot and air traffic controller in standard and non-standard situations. Ecole Nationale de l’Aviation Civile, unpublished PhD.
Messick, S. (1989) Validity. In Linn, R. L. (ed.), Educational measurement (3rd ed.). New York: Macmillan, 13103.
Oliver, M. and Carr, D. (2009) Learning in virtual worlds: Using communities of practice to explain how people learn from play. British Journal of Educational Technology, 40(3): 444457.
Paltridge, B. and Phakiti, A. (2015) Research methods in applied linguistics: A practical resource. London: Bloomsbury.
Peterson, M. (2012) EFL learner collaborative interaction in Second Life. ReCALL, 24(1): 2039.
Purpura, J. E. (1999) Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge: Cambridge University Press.
Sadler, R. (2011) Virtual Worlds for language learning: From theory to practice. Bern: Peter Lang.
Sadler, R. (2012) Virtual worlds: An overview and pedagogical examination. Bellaterra Journal of Teaching & Learning Language and Literature, 5(1): 122.
Schwienhorst, K. (2002) Why virtual, why environments? Implementing virtual reality concepts in computer-assisted language learning. Simulation & Gaming, 33(2): 196209.
Stevenson, D. K. (1985) Authenticity, validity and a tea party. Language Testing, 2(1): 4147.
Swier, R. (2014) Tasks for easily modifiable virtual environments. The JALT CALL Journal, 10(3): 203219.
Tajima, A. (2004) Fatal miscommunication: English in aviation safety. World Englishes, 23(3): 451470.
Tseng, J.-J., Tsai, Y.-H. and Chao, R.-C. (2013) Enhancing L2 interaction in avatar-based virtual worlds: Student teachers’ perceptions. Australasian Journal of Educational Technology, 29(3): 357371.
Van Moere, A., Suzuki, M., Downey, R. and Cheng, J. (2009) Implementing ICAO language proficiency requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32(3): 27.127.17.
Verhagen, T., Feldberg, F., van den Hooff, B., Meents, S. and Merikivi, J. (2012) Understanding users’ motivations to engage in virtual worlds: A multipurpose model and empirical testing. Computers in Human Behavior, 28(2): 484495.
Vuopala, E., Hyvönen, P. and Järvelä, S. (2016) Interaction forms in successful collaborative learning in virtual learning environments. Active Learning in Higher Education, 17(1): 2538.
Wang, C. X., Calandra, B., Hibbard, S. T. and Lefaiver, M. L. M. (2012) Learning effects of an experimental EFL program in Second Life. Educational Technology Research and Development, 60(5): 943961.
Zokić, M., Boras, D. and Lazić, N. (2012) Say again. International Journal of Education and Information Technologies, 6(4): 363370.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0958-3440
  • EISSN: 1474-0109
  • URL: /core/journals/recall
Please enter your name
Please enter a valid email address
Who would you like to send this to? *