The past decade has witnessed some dramatic methodological changes in the wider disciplines of psycholinguistics, psychology, and experimental linguistics. One such set of changes comprises the development of open and transparent research practices, which have increasingly been adopted in response to concerns that empirical results often fail to replicate and may not generalise across samples and experimental conditions (Gibson & Fedorenko, Reference Gibson and Fedorenko2013; Maxwell, Lau, & Howard, Reference Maxwell, Lau and Howard2015; McElreath & Smaldino, Reference McElreath and Smaldino2015; Yarkoni, Reference Yarkoni2020). Another important set of changes concerns the use of sophisticated statistical techniques, such as mixed-effects models (Baayen, Davidson, & Bates, Reference Baayen, Davidson and Bates2008) and Bayesian analyses (Vasishth, Nicenboim, Beckman, Li & Kong, Reference Vasishth, Nicenboim, Beckman, Li and Kong2018), which can provide much more information about magnitudes of effects and sources of variation than the more traditional statistical approaches.
Some of these changes have already trickled down to the field of bilingualism, while others are under way. The time is ripe to document these changes and take a broad view of the state-of-the-art, while at the same time examining the added value that such methodological advances can provide to bilingualism and second language research. These are the main goals of this mini-serie of seven peer-reviewed articles; altogether they review a wide range of novel and emerging statistical approaches and research practices in our field.
The first two contributions cover open science research practices. In the first of these, Bolibaugh, Vanek, and Marsden (Reference Bolibaugh, Vanek and Marsden2021) call for a “credibility revolution” in bilingualism research. They argue that complete disclosure of data, analysis code, and materials, is a minimum standard for assessing (and improving) the reproducibility and replicability of empirical results. The article also provides examples and recommendations for good practices in bilingualism research.
The second article by Mertzen, Lago, and Vasishth (Reference Mertzen, Lago and Vasishth2021), explains the benefits of preregistration, another research practice that can greatly improve the openness and transparency of bilingualism research. The authors provide practical examples of how preregistration can firmly distinguish hypothesis-testing from data exploration, and, more generally, how it can counter questionable research practices and unconscious biases on the part of researchers.
The two contributions that follow are at the border between statistical approaches and research practices. They present statistical techniques that are used in the service of study design and for drawing inferences from the published empirical literature. Brysbaert's (Reference Brysbaert2021) article discusses the potentially serious consequences of low statistical power in bilingualism research. The author presents a series of compelling simulations that demonstrate how small sample sizes can turn our statistical inferences into very blurry reflections of reality – especially when estimating between-group interactions, a common situation in bilingualism research.
In the next article Plonsky, Sudina, and Hu (Reference Plonsky, Sudina and Hu2021) review the application of meta-analysis (perhaps the oldest of the ‘new’ approaches presented here) to research on bilingualism. They discuss the strengths and advantages of meta-analysis over traditional literature reviews and provide a comprehensive overview of the research syntheses that have already been conducted in the field. Moreover, the authors provide a practical introduction to the major stages involved in conducting a meta-analysis, including data collection, analysis, and interpretation.
The last three contributions are short tutorials on specific statistical techniques and their application to various measures. Miwa and Baayen (Reference Miwa and Baayen2021) present an introduction to generalized additive mixed models a powerful and flexible regression-based approach that allows modelling nonlinear relationships between predictors and outcomes. The authors reanalyse two datasets of bilingual lexical decision and show that the technique is particularly useful for examining nonlinear interactions between continuous predictors, as well as for modelling the timecourse of lexical effects.
In the next contribution, Stone, Lago, and Schad (Reference Stone, Lago and Schad2021) present a novel method for estimating the onset of divergences in the visual world eye-tracking paradigm. The article discusses and compares various approaches to timecourse analysis and proposes a new bootstrapping procedure that overcomes the limitations of previous methods. The authors also exemplify how their method can be used to estimate timecourse differences between conditions or speaker groups, which may be crucial for distinguishing between theoretical accounts of L2 processing.
Finally, in my own contribution (Veríssimo, Reference Veríssimo2021), I apply Bayesian ordinal models to the analysis of rating scales (e.g., of grammatical acceptability or language proficiency). I first show how the statistical methods that are commonly used suffer from important flaws, and then illustrate how ordinal models can provide more valid, accurate, and informative inferences about graded constructs such as language proficiency.
The articles that feature in this mini-series all share three important attributes. First, they acknowledge the particular characteristics of bilingualism research: for example, a reliance on between-group comparisons, the measurement of latent constructs (e.g., language proficiency), the possibility of nonlinear effects, and the importance of timecourse analysis for testing theories of bilingual processing. Second, they collectively illustrate the large variety of data types, tasks, and measures that bilingualism researchers routinely deal with and care about (e.g., response times, target fixations, rating scales). Third the articles in this mini-series all speak from a practical standpoint. That is, they focus on the steps and challenges that are involved in the actual implementation of these technique.
I hope that this mini-series can serve not only as a systematic collection of resources, but also as a ‘call-to-action’: an invitation for students and researchers to integrate these techniques into their own methodological toolboxes. Taken together, the statistical approaches and research practices presented here hold the potential to improve our inferences, increase the openness and transparency of our field, and help establish bilingualism research as a cumulative scientific enterprise.
