Hostname: page-component-76fb5796d-45l2p Total loading time: 0 Render date: 2024-04-29T05:01:30.272Z Has data issue: false hasContentIssue false
  • English
  • Français

How useful are native language tests for research with advanced second language users?

Published online by Cambridge University Press:  23 June 2023

Hanke Vermeiren Open the ORCID record for Hanke Vermeiren [Opens in a new window]  and
Marc Brysbaert Open the ORCID record for Marc Brysbaert [Opens in a new window]
Hanke Vermeiren*
Affiliation:
Department of Experimental Psychology, Ghent University, Ghent, Belgium
Marc Brysbaert
Affiliation:
Department of Experimental Psychology, Ghent University, Ghent, Belgium
*
Corresponding author: Hanke Vermeiren, KU Leuven campus Kulak Kortrijk, Etienne Sabbelaan 51, 8500 Kortrijk, Belgium E-mail: hanke.vermeiren@kuleuven.be
Get access Rights & Permissions [Opens in a new window]

Abstract

We investigated the extent to which language tests developed for native speakers (L1) can be used with advanced speakers of a second language (L2). We compared the performance of Dutch–English bilinguals with that of native English speakers on a series of English language tests, looking at vocabulary knowledge, crystallized intelligence, reading comprehension, and reading speed. It was found that advanced L2 speakers know fewer L2 words than native speakers and take longer to read texts but perform equally well on text comprehension. Tests optimized for native English-speakers predicted text comprehension less well than tests better adapted to the skill level of the bilinguals (which include the Lextale test). An exploratory graphical analysis suggested that L2 users’ performance on challenging vocabulary tests, along with performance on an English author recognition test, forms a distinct cluster – arguably also measuring interest in English language and culture besides knowledge in general (also called crystallized intelligence).

Keywords

vocabulary testsecond language researchreading comprehensiongeneral knowledge
Type
Research Article
Information
Bilingualism: Language and Cognition , Volume 27 , Issue 1 , January 2024 , pp. 204 - 213
Check for updates
Open Practices
Open data Open materials
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This article has earned badges for transparent research practices: Open Data and Open Materials. For details see the Data Availability Statement.

References

Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011). Multiple imputation by chained equations: what is it and how does it work?. International Journal of Methods in Psychiatric Research, 20(1), 1–62.CrossRefGoogle ScholarPubMed
Bishara, A. J., & Hittner, J. B. (2015). Reducing bias and error in the correlation coefficient due to nonnormality. Educational and Psychological Measurement, 75(5), 785–804.CrossRefGoogle ScholarPubMed
Boyd, J. K., & Goldberg, A. E. (2011). Learning what not to say: The role of statistical preemption and categorization in a-adjective production. Language, 87(1), 55–83. https://doi.org/10.1353/lan.2011.0012CrossRefGoogle Scholar
Brysbaert, M. (2019a). How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables. Journal of Cognition, 2(1), 16.CrossRefGoogle ScholarPubMed
Brysbaert, M. (2019b). How many words do we read per minute? A review and meta-analysis of reading rate. Journal of Memory and Language. 109.CrossRefGoogle Scholar
Brysbaert, M., & Ellis, A. W. (2016). Aphasia and age-of-acquisition: Are early-learned words more resilient? Aphasiology, 30, 1240–1263.CrossRefGoogle Scholar
Brysbaert, M., Keuleers, E., & Mandera, P. (2021). Which words do English non-native speakers know? New supernational levels based on yes/no decision. Second Language Research, 37(2), 207–231.CrossRefGoogle Scholar
Calloway, R., Helder, A., & Perfetti, C. (2022). A measure of individual differences in readers’ approaches to text and its relation to reading experience and reading comprehension. Behavior Research Methods, 55(2), 899–931.CrossRefGoogle ScholarPubMed
Cargnelutti, E., Tomasino, B., & Fabbro, F. (2019). Language brain representation in bilinguals with different age of appropriation and proficiency of the second language: A meta-analysis of functional imaging studies. Frontiers in Human Neuroscience, 13, 154.CrossRefGoogle Scholar
Christensen, A. P., & Golino, H. (2021). Estimating the Stability of Psychological Dimensions via Bootstrap Exploratory Graph Analysis: A Monte Carlo Simulation and Tutorial. Psych, 3, 479–500.CrossRefGoogle Scholar
Cobb, T. (2007). Computing the vocabulary demands of L2 reading. Language Learning & Technology 11(3), 38–63.Google Scholar
Costa, A., Vives, M. L., & Corey, J. D. (2017). On language processing shaping decision making. Current Directions in Psychological Science, 26(2), 146–151.CrossRefGoogle Scholar
de Vos, J. F., Schriefers, H., & Lemhöfer, K. (2020). Does study language (Dutch versus English) influence study success of Dutch and German students in the Netherlands?. Dutch Journal of Applied Linguistics, 9(1–2), 60–78.CrossRefGoogle Scholar
De Wilde, V., Brysbaert, M., & Eyckmans, J. (2020). Learning English through out-of-school exposure. Which levels of language proficiency are attained and which types of input are important? Bilingualism: Language & Cognition, 23, 171–185.Google Scholar
de Winter, J. C., Gosling, S. D., & Potter, J. (2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21(3), 273–290.CrossRefGoogle ScholarPubMed
Dhaene, S., & Woumans, E. (2023). Text recall and use of advance organisers in first and second language. Studies in Second Language Acquisition, 45(1), 264–275.CrossRefGoogle Scholar
Dirix, N., Vander Beken, H., De Bruyne, E., Brysbaert, M., & Duyck, W. (2020). Reading Text When Studying in a Second Language: An Eye-Tracking Study. Reading Research Quarterly, 55(3), 371–397.CrossRefGoogle Scholar
Ferré, P., & Brysbaert, M. (2017). Can Lextale-Esp discriminate between groups of highly proficient Catalan-Spanish bilinguals with different language dominances? Behavior Research Methods, 49, 717–723.CrossRefGoogle ScholarPubMed
Golino, H., & Epskamp, S. (2017). Exploratory Graph Analysis: A new approach for estimating the number of dimensions in psychological research. PLoS ONE, 12.CrossRefGoogle ScholarPubMed
Grant, A. M., Fang, S. Y., & Li, P. (2015). Second language lexical development and cognitive control: A longitudinal fMRI study. Brain and Language, 144, 35–47.CrossRefGoogle ScholarPubMed
Griffin, T. D., Wiley, J., & Thiede, K. W. (2008). Individual differences, rereading, and self- explanation: Concurrent processing and cue validity as constraints on metacomprehension accuracy. Memory & Cognition, 36, 93–103.CrossRefGoogle ScholarPubMed
Guasch, M., Sanchez-Casas, R., Ferre, P., & García-Albea, J. E. (2011). Effects of the degree of meaning similarity on cross-language semantic priming in highly proficient bilinguals. Journal of Cognitive Psychology, 23(8), 942–961.CrossRefGoogle Scholar
Hadjichristidis, C., Geipel, J., & Keysar, B. (2019). The influence of native language in shaping judgment and choice. Progress in Brain Research, 247, 253–272.CrossRefGoogle ScholarPubMed
Indefrey, P. (2006). A meta-analysis of hemodynamic studies on first and second language processing: Which suggested differences can we trust and what do they mean?. Language Learning, 56, 279–304.CrossRefGoogle Scholar
Isvoranu, A.-M., & Epskamp, S. (2021). Which estimation method to choose in network psychometrics? Deriving guidelines for applied researchers. Psychological Methods.Google ScholarPubMed
Izura, C., Cuetos, F., & Brysbaert, M. (2014). Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35, 49–66.Google Scholar
Kane, M. J., & Miyake, T. M. (2007). The validity of “conceptual span” as a measure of working memory capacity. Memory & Cognition 35, 1136–1150.CrossRefGoogle ScholarPubMed
Kim, H., & Krashen, S. (1998). The author recognition and magazine recognition tests, and free voluntary reading as predictors of vocabulary development in English as a foreign language for Korean high school students. System, 26(4), 515–523.CrossRefGoogle Scholar
Kuperman, V., Siegelman, N., Schroeder, S., Acartürk, C., Alexeeva, S., Amenta, S., Bertram, R., Bonandrini, R., Brysbaert, M., Chernova, D., Da Fonseca, S. M., Dirix, N., Duyck, W., Fella, A., Frost, R., Gattei, C. A., Kalaitzi, A., Lõo, K., Marelli, M… & Usal, K. A. (2022). Text reading in English as a second language: Evidence from the Multilingual Eye-Movements Corpus. Studies in Second Language Acquisition, 1–35.Google Scholar
Lemhöfer, K., & Broersma, M. (2012). Introducing Lextale: A quick and valid Lexical Test for Advanced Learners of English. Behavior Research Methods. 44, 325–343.CrossRefGoogle ScholarPubMed
McCarron, S. P., & Kuperman, V. (2022). Is the author recognition test a useful metric for native and non-native English speakers? An item response theory analysis. Behavior Research Methods, 53(5), 2226–2237.CrossRefGoogle Scholar
McLean, S., Stewart, J., & Batty, A. O. (2020). Predicting L2 reading proficiency with modalities of vocabulary knowledge: A bootstrapping approach. Language Testing.CrossRefGoogle Scholar
Moore, M., & Gordon, P. C. (2015). Reading ability and print exposure: Item response theory analysis of the author recognition test. Behavior Research Methods, 47(4), 1095–1109.CrossRefGoogle ScholarPubMed
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2022). Data quality of platforms and panels for online behavioral research. Behavior Research Methods, 54(4), 1643–1662.CrossRefGoogle ScholarPubMed
Revelle, W. (2021). psych: Procedures for Personality and Psychological Research, R package Version 2.1.9. Retrieved from https://CRAN.R-project.org/package=psychGoogle Scholar
Schafer, J. L., & Graham, J. W. (2002). Missing data: our view of the state of the art. Psychological Methods, 7, 147–177.CrossRefGoogle ScholarPubMed
Schipolowski, S., Wilhelm, O., & Schroeders, U. (2014). On the nature of crystallized intelligence: The relationship between verbal ability and factual knowledge. Intelligence, 46, 156–168.CrossRefGoogle Scholar
Stanovich, K. E., & West, R. F. (1989). Exposure to print and orthographic processing. Reading Research Quarterly, 24(4), 402–433.CrossRefGoogle Scholar
Steger, D., Schroeders, U., & Wilhelm, O. (2019). On the dimensionality of crystallized intelligence: A smartphone-based assessment. Intelligence, 72, 76–85.CrossRefGoogle Scholar
Sulpizio, S., Toti, M., Del Maschio, N., Costa, A., Fedeli, D., Job, R., & Abutalebi, J. (2019). Are you really cursing? Neural processing of taboo words in native and foreign language. Brain and Language, 194, 84–92.CrossRefGoogle ScholarPubMed
van Buuren, , S. (2021). Package ‘mice’ Version 3.14.0. Cran: https://cran.r-project.org/web/packages/mice/mice.pdfGoogle Scholar
Vander Beken, H., & Brysbaert, M. (2018). Studying texts in a second language: The importance of test type. Bilingualism: Language and Cognition, 21(5), 1062–1074.CrossRefGoogle Scholar
Vander Beken, H., Woumans, E., & Brysbaert, M. (2018). Studying texts in a second language: No disadvantage in long-term recognition memory. Bilingualism: Language and Cognition, 21(4), 826–838.CrossRefGoogle Scholar
Vander Beken, H., De Bruyne, E., & Brysbaert, M. (2020). Studying texts in a non-native language: A further investigation of factors involved in the L2 recall cost. Quarterly Journal of Experimental Psychology, 73(6), 891–907.CrossRefGoogle Scholar
van Hell, J. G., & Dijkstra, T. (2002). Foreign language knowledge can influence native language performance in exclusively native contexts. Psychonomic Bulletin & Review, 9(4), 780–789.CrossRefGoogle ScholarPubMed
Vermeiren, H., Vandendaele, A., & Brysbaert, M. (2022). Validated tests for language research with university students whose native language is English: Tests of vocabulary, general knowledge, author recognition, and reading comprehension. Behavior Research Methods, 55(3), 1036–1068. https://doi.org/10.3758/s13428-022-01856-x.CrossRefGoogle ScholarPubMed
Yeari, M., van den Broek, P., & Oudega, M. (2015). Processing and memory of central versus peripheral information as a function of reading goals: Evidence from eye-movements. Reading and Writing: An Interdisciplinary Journal, 28(8), 1071–1097.CrossRefGoogle Scholar
Zhang, S., & Zhang, X. (2020). The relationship between vocabulary knowledge and L2 reading/listening comprehension: A meta-analysis. Language Teaching Research, https://doi.org/10.1177/1362168820913998Google Scholar