What is replication research?

Replication is a research method used to confirm, consolidate, and extend knowledge and understanding in empirical fields of study (Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; Isager et al., Reference Isager, Van Aert, Bahník, Brandt, DeSoto, Giner-Sorolla and Lakens2023; Porte & McManus, Reference Porte and McManus2019). A replication study works toward this goal by repeating a study’s research design and methods with or without changes, collecting new data, and systematically comparing the previous study’s results with those from the new study (Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; McManus, Reference McManus, Gurzynski-Weiss and Kim2022b; Nosek & Errington, Reference Nosek and Errington2020). As a result, a replication study is a new empirical study with new data that purposefully addresses the same or similar research question and/or hypothesis as a previous study (McManus, Reference McManus, Mackey and Gass2023; National Academies of Sciences, Engineering, and Medicine, 2019; Schmidt, Reference Schmidt2009). Because one aim of carrying out a replication study is to determine the extent to which repeating a previous study’s design and methods leads to comparable patterns of results and conclusions, replication provides a systematic framework for reconsidering, refining, extending, and sometimes limiting prior research findings. This is why replication has long been considered an essential part of the research process (King & Mackey, Reference King and Mackey2016; Peterson & Panofsky, Reference Peterson and Panofsky2021; Porte, Reference Porte2012). Replication studies are key to scientific progress precisely because they systematically revisit prior claims to confirm their stability and detect potential anomalies. It is through this essential process of confirming results, detecting anomalies, and understanding their nature that new lines of research can emerge.

Given these features of the replication research process, it is troubling that replication studies in SLA are still claimed to be infrequent and poorly designed, reported, and disseminated (Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; Polio & Gass, Reference Polio and Gass1997; Porte, Reference Porte2012). One outcome of this situation is that calls for replication are now relatively common across the social sciences (Language Teaching Review Panel, 2008; Plucker & Makel, Reference Plucker and Makel2021; Zwaan, Etz, Lucas, & Donnellan, Reference Zwaan, Etz, Lucas and Donnellan2018), with journal special issues highlighting the specific ways that replication studies support disciplinary growth (Godfroid & Andringa, Reference Godfroid and Andringa2023; Tschichold, Reference Tschichold2023). In SLA, field-specific resources now exist to support the conduct of replication studies (Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; McManus, Reference McManus, Mackey and Gass2023, Reference McManus2024b; Porte & McManus, Reference Porte and McManus2019).

When conducting a replication study, the aim is to design and report a new empirical study that repeats a previous piece of research in some way, with systematic comparison throughout (Brown, Reference Brown and Porte2012; Porte & McManus, Reference Porte and McManus2019). By conducting a close replication study, in which almost all elements of the initial study’s design and methods are retained without modification (Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; LeBel, McCarthy, Earp, Elson, & Vanpaemel, Reference LeBel, McCarthy, Earp, Elson and Vanpaemel2018; McManus, Reference McManus, Gurzynski-Weiss and Kim2022b), the researcher’s aim is to understand how an intentional, carefully selected, well-motivated, and principled modification to the initial study’s method impacts the results, if at all. For example, a replication that modifies a prior study’s pedagogical intervention while retaining all other aspects of the research design and methods can be used to understand how that intentional modification influenced the study’s findings and conclusions (e.g., McManus & Marsden, Reference McManus and Marsden2018). In this way, replication research allows the stability of a study’s claims to be examined in a principled manner.

Even though determining the stability of previous results is recognized as an essential first step in developing any research program (Gass, Loewen, & Plonsky, Reference Gass, Loewen and Plonsky2021; Isager et al., Reference Isager, Van Aert, Bahník, Brandt, DeSoto, Giner-Sorolla and Lakens2023; Mackey, Reference Mackey and Porte2012; Schmidt, Reference Schmidt2009), this feature is frequently absent in how SLA research is conducted and reported. In other words, systematically revisiting a prior study to confirm its results before extending that line of research is rarely practiced (but see Ellis, Hafeez, Martin, Chen, Boland, & Sagarra, Reference Ellis, Hafeez, Martin, Chen, Boland and Sagarra2014; Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa Ii, Leow, Mikhaylova and Szudarski2018). Integrating replication into the research process (i.e., how we develop research questions, design studies, generate claims, and assess theories) is required to support the growth and longevity of the discipline in a principled and systematic way. For example, even though it is well understood that the amounts and types of prior knowledge and experiences that learners bring to learning a new language can substantially influence the trajectories of new language learning (Duff & Byrnes, Reference Duff and Byrnes2019; Ellis, Reference Ellis2019; Lantolf, Poehner, & Swain, Reference Lantolf, Poehner and Swain2018), no single study can examine all contextual influences on learning outcomes (e.g., L2 proficiency, L1 background, prior immersion and instructed experiences, social networks). This is because each study can only ever provide a snapshot of learners’ knowledge and use of language at that one point in time. Replication is needed to determine the stability of important and influential claims as well as to investigate variables unaccounted for in previous research. The more stable a claim turns out to be through replication, the more confident we can be that what has been observed and reported is a confirmed trend (rather than a chance finding, for instance). By not probing the stability of claims made in prior research through replication, we continue to build a discipline “on the basis of scanty evidence” (Polio & Gass, Reference Polio and Gass1997, p. 500).

Originality and innovation in replication research

Unfortunately, claims that replication studies lack originality and/or innovation are not only common but they also misguided. Furthermore, these criticisms serve only to harm the discipline. Does repeating aspects of a prior study’s research design and methods mean that replication studies lack originality and innovation (Martin & Clarke, Reference Martin and Clarke2017; Zwaan et al., Reference Zwaan, Etz, Lucas and Donnellan2018)? Regrettably, the field’s practices reinforce this view. For example, some journals (e.g., Applied Psycholinguistics) distinguish between “original research article” and “replication research article,” which, while likely unintentional, suggests that replication studies are not to be considered original. While concerns about originality are thought to have limited the conduct of replication studies in SLA research (Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; Porte, Reference Porte2012; but see McManus, Reference McManus2022a), it is misleading and incorrect to claim that replication studies cannot make original contributions.

Originality refers to studies designed to detect problems “in the existing archetype to the extent that it provokes other researchers and their understanding to further study and analyze so as to answer the questions raised” (Alajami, Reference Alajami2020, p. 4; see also Dean, Hender, Rodgers, & Santanen, Reference Dean, Hender, Rodgers and Santanen2006; Gordon, Reference Gordon2007). In other words, originality describes work designed to advance knowledge and understanding by investigating questions and/or problems and/or detecting anomalies in prior work. In addition, original research brings together new and existing information, including, but not limited to, new research questions/hypotheses, previously-reported methods, and new results (Dirk, Reference Dirk1999; Guetzkow, Lamont, & Mallard, Reference Guetzkow, Lamont and Mallard2004; Shibayama & Wang, Reference Shibayama and Wang2020). Based on this understanding, it is difficult to see how a replication study that meets these criteria could be claimed to lack originality. As a reminder, a replication study repeats a previous study’s research design and methods with or without modification, collects new data, and systematically compares the earlier study’s findings with those from the new research (Mackey, Reference Mackey and Porte2012; Nosek & Errington, Reference Nosek and Errington2020; Polio & Gass, Reference Polio and Gass1997; Porte & McManus, Reference Porte and McManus2019). In this way, a key aim of replication research is to confirm, consolidate, and extend knowledge and understanding within empirical fields of study by confronting existing claims and ideas using new evidence (Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; Isager et al., Reference Isager, Van Aert, Bahník, Brandt, DeSoto, Giner-Sorolla and Lakens2023; Long, Reference Long1993; Santos, Reference Santos1989). Thus, claims that replication research is unoriginal are misguided and not consistent with contemporary understandings of originality.

Innovation is also vital to the growth and strengthening of the discipline, but it, too, is misunderstood as describing something completely new, untested, and radical in nature with incremental innovation not counting (Kahn, Reference Kahn2018; Nind, Wiles, Bengry-Howell, & Crow, Reference Nind, Wiles, Bengry-Howell and Crow2013; Wiles, Crow, & Pain, Reference Wiles, Crow and Pain2011). Clearly, setting this as a target for a research study would be both impossible to achieve and undesirable, rendering most SLA research lacking innovation. Rather, innovation exists along a continuum, is not a binary phenomenon, and is defined by small, gradual changes to current approaches (Ledgerwood, Reference Ledgerwood2014; Long, Reference Long1993). This view of innovation is consistent with contemporary understandings of innovation as a cumulative process in which new lines of research emerge by building on established knowledge. In the next section, I review some of the ways that innovation and originality apply to the design and conduct of replication studies in SLA, highlighting how we can better support scientific progress by using replication to systematically revisit prior claims to confirm their stability and detect potential anomalies. I conclude this section by stressing that we must eliminate outdated notions and misconceptions of replication as lacking originality and innovation. These attitudes are unhelpful, misguided, and harmful to the discipline.

Approaches to replication

Because a replication study can come in different forms, several categorizations have been developed to convey the amount of intentional change implemented in the replication (Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; LeBel et al., Reference LeBel, McCarthy, Earp, Elson and Vanpaemel2018; Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; Porte & McManus, Reference Porte and McManus2019). In an exact replication, for example, the selected study’s entire design, methods, and procedure are followed without alteration. In a close replication, one major variable is intentionally modified, while an approximate replication intentionally modifies no more than two major variables (for approaches to variable selection and modification in replication studies, see Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; LeBel et al., Reference LeBel, McCarthy, Earp, Elson and Vanpaemel2018; Porte & McManus, Reference Porte and McManus2019). In a conceptual replication, however, almost all aspects of the previous study can be modified because the aim here involves “repetition of a test of a hypothesis or a result of earlier research with different methods” (Schmidt, Reference Schmidt2009, p. 91). It is important to note, however, that drawing meaningful comparisons between studies (i.e., initial and replication) is difficult when several major modifications are integrated into the replication. This makes conceptual replications an inappropriate choice for revisiting prior studies to confirm results. At the same time, exact replication in the social sciences is probably one of the most difficult, if not impossible, types of replication study to conduct (Stroebe & Strack, Reference Stroebe and Strack2014). Even though it might be possible to locate and use the previous study’s materials and analyses, keeping the data sample the same is very difficult when working with human subjects (Nosek & Errington, Reference Nosek and Errington2020). Of course, there could be similarities (e.g., L2 English learners in Austria and Germany), but several factors make the samples different (e.g., contexts, times, individual backgrounds).

In terms of using replication to revisit prior studies to confirm results and conclusions, close and approximate replication studies that include the least intentional modification are the most desirable because they allow for the greatest amount of systematic comparison (see Figure 1 for an overview of replication approaches).

Figure 1.

Overview of approaches to replication.

The amount of modification between the initial study and the replication should always be reported in the study’s title, abstract, and main text (Appelbaum et al., Reference Appelbaum, Cooper, Kline, Mayo-Wilson, Nezu and Rao2018; McManus, Reference McManus2022a, 2024), with labels such as close replication or approximate replication as well as summary tables to document between-study differences and similarities. This means that a replication should be clearly identified as a replication study from the outset. Furthermore, it is important for authors to identify, categorize, and report elements of the replication that are (a) the same as, (b) close to, or (c) different from the initial study (Brandt et al., Reference Brandt, Ijzerman, Dijksterhuis, Farach, Geller, Giner-Sorolla and van ’t Veer2014; LeBel et al., Reference LeBel, McCarthy, Earp, Elson and Vanpaemel2018; Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; McManus, Reference McManus, Mackey and Gass2023). In a close replication study, for instance, in which one variable is intentionally modified and all other differences are minimal, a minor modification refers to elements of the replication that are very close to those in the initial study, whereas a major modification refers to variables that are different between the studies. In addition, a major modification will represent the research focus of the replication, whereas a minor modification should not. In a close replication investigating whether web-based instruction influences the learning outcomes reported in a classroom-based study, for example, modifying the modality of instruction from in-person to web-based would represent an element of the initial study’s design that is different in the replication. In this example, the modality of the instruction (from in-person to web-based) can be categorized as a major variable modification. Communicating and justifying all between-study differences can also serve several benefits. For example, readers can judge if a modification appears justified or not and also where the replication is on the continuum from close to conceptual. Reporting all between study differences also allows future work to evaluate the boundary conditions on a particular effect and to develop guidance for evaluating magnitudes of change in replication research.

Supporting future replication efforts in the field

Supporting and improving the conduct and dissemination of replication studies is vital to the growth of the discipline. Doing so requires the field to develop and apply standards and expectations in design and reporting so that replication studies can meet their aims to confront and revisit existing understanding with new evidence. As a result, replication reporting must include the following components, as a minimum: a clear rationale for why the replication was carried out, full and transparent descriptions of differences and similarities in the design, methods, and results, and a framework for determining and evaluating the replicability of effect. Critically, however, these components are neither encouraged nor discussed in general reporting guidelines for empirical research in SLA, including research methodology textbooks (e.g., Mackey & Gass, Reference Mackey and Gass2022; Rose, McKinley, & Baffoe-Djan, Reference Rose, McKinley and Baffoe-Djan2020) and journal reporting guidelines (e.g., Language Learning, Studies in Second Language Acquisition; see McManus, Reference McManus2024a). This is because the aims of most empirical studies (or extension studies) are different from those of replication studies. For one, extension studies aim to extend a current line of research in new ways (e.g., contexts, methods, populations), whereas a replication study aims to critically revisit one study to better understand its findings and impact on the discipline. This difference in aims impacts how the study is designed and reported, which explains why the information reported in replication studies is often piecemeal, incomplete, and difficult to understand and evaluate. There is thus a compelling need for disciplines to develop resources to support the design, conduct, and reporting of replication studies (e.g., Language Teaching Review Panel, 2008; Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; McManus, Reference McManus2024a; Porte, Reference Porte2012; Porte & McManus, Reference Porte and McManus2019). In the next sections, I focus on three areas of current and future investment central to the growth of the discipline and the role of replication studies in that growth: reporting, design, and evaluation.

Replication reporting

In SLA research, improvements to the quality and transparency of reporting are needed (see Plonsky, Reference Plonsky2013, Reference Plonsky2024), and this situation has been noted for replication studies as well (Brown, Reference Brown and Porte2012; Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; Polio & Gass, Reference Polio and Gass1997; Porte & McManus, Reference Porte and McManus2019). The task is to develop and implement reporting strategies to maximize the transparency and openness of a replication project, including, but not limited to, transparent study labeling, definition, and description of the replication approach, description and justification of between-study modifications, comparative reporting in the replication study’s design and methods, and evaluation of the initial study’s results in light of those reported in the replication. One resource contributing to innovation in this area includes a new standard for replication studies in applied linguistics (McManus, Reference McManus2024a, see also Reference McManus, Mackey and Gass2023; Porte & McManus, Reference Porte and McManus2019). This standard outlines basic expectations in replication reporting so that outcomes can be better understood and evaluated. Table 1, adapted from McManus (Reference McManus2024a), outlines the minimum requirements in replication reporting for each section of a publication. The replication studies included in this special issue closely align with this standard for replication reporting.

Table 1.

Reporting expectations for replication studies in SLA research (adapted from McManus Reference McManus2024a)

The special issue

To date, replication studies in SLA research have been rare and of varying quality with a preference for conceptual over approximate and close replication designs. In addition, the transparency and completeness of replication reporting have—in many instances—been low. This special issue responds to these limitations in replication design and reporting with eleven replication studies. Thus, this special issue aims to showcase and highlight exemplary standards in replication study design and reporting to benefit the field. Indeed, each study can be regarded as a role model usable by researchers and students in conducting their own replication studies and teaching others about replication design and reporting.

In compiling the special issue, a call for proposals was launched in December 2022, in which prospective contributors were invited to submit short proposals no longer than four pages that stated (i) the study to be replicated with justifications, (ii) the type of replication study to be completed, (iii) modifications to be implemented in the replication with justifications, (iv) design of the replication study, (v) data analysis plan, and (vi) potential impact. Forty-seven proposals were received, and each was independently evaluated by two reviewers (Kevin McManus and an SSLA editorial board member) using the criteria provided in the call. Following this process, nineteen proposals were selected for potential inclusion, subject to favorable evaluation following external peer review. At that point, authors were provided with recommendations for reporting replication studies as presented in Appelbaum et al. (Reference Appelbaum, Cooper, Kline, Mayo-Wilson, Nezu and Rao2018) and Porte and McManus (Reference Porte and McManus2019), including:

• • Report the type of replication in the title and abstract, as follows: exact, close, approximate, or conceptual.

• • Report whether the replication has conditions, materials, and/or procedures that were not part of the initial study.

• • Present justifications and descriptions for all additions and variable modifications.

• • Summarize all changes between the initial and replication studies in a table.

• • Clearly label results from additional or different analyses.

• • State the criteria for deciding whether the initial study’s results were replicated. Examples of criteria include statistical significance testing, effect sizes, confidence intervals, and Bayes factors in Bayesian methods.

• • Include in the discussion a section titled “Future replication research” where suggestions for future replication studies building on the current replication study are made.

This process culminated in a special issue of eleven replication studies involving a variety of topics, populations, replication designs, motivations for replications, and initial studies that integrate open science practices (e.g., open data, open materials, preregistration). In the remainder of this introduction, I provide an overview of each study in this special issue, highlighting its main components and discussing how the replication study strengthens the field and advances knowledge and understanding about the topic.

Barlow et al. (Reference Barlow, Beardsley, Bsharah, Crofts, De La Rosa, Gutierrez and Hayes-Harb2024) examined the ways that “native” and “non-native” listeners judge “foreign-accented” speech, contributing to an established line of SLA research suggesting that statements produced by “foreign-accented” speakers are less likely to be judged as true than those produced by “native speakers” (e.g., Hanzlíková & Skarnitzl, Reference Hanzlíková and Skarnitzl2017; Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010). While some studies suggest that these results might be attributable to processing difficulty, accent-based prejudice, or both, findings are mixed, at best, resulting in very little clarity and consensus on this topic. To address this limitation in the field, Barlow et al. (Reference Barlow, Beardsley, Bsharah, Crofts, De La Rosa, Gutierrez and Hayes-Harb2024) carried out a close replication study of Boduch-Grabka and Lev-Ari (Reference Boduch-Grabka and Lev-Ari2021), a study that isolated the impact of processing difficulty on veracity judgments by exposing listeners to “native” or “non-native” accents prior to completing a veracity judgment task. The close replication retained all aspects of the initial study’s design, with the addition of an explicit bias task to independently examine the contributions of processing difficulty and accent-based prejudice to listeners’ judgments of veracity. The initial study’s audio materials were accessed from a public repository (Open Science Foundation, https://osf.io), with additional materials provided by the authors via an email request from Barlow et al. Overall, the initial study’s findings were not replicated: “we did not reproduce the effect of accent on veracity judgments that have been previously reported (Boduch-Grabka & Lev-Ari, Reference Boduch-Grabka and Lev-Ari2021; Lev-Ari & Keysar, Reference Lev-Ari and Keysar2010) or the Boduch-Grabka and Lev-Ari (Reference Boduch-Grabka and Lev-Ari2021) finding that prior exposure to Polish-accented English speech improves veracity judgments of Polish-accented statements.” Furthermore, results from the explicit bias task did not appear to shed further light on listeners’ judgments. In terms of why Barlow et al.’s findings did not replicate the initial study, the authors suggest that methodological differences, including “insufficient methodological detail provided by the original researchers,” between the studies may explain the different patterning of results (e.g., time frame differences, social, cultural, and political effects, participant recruitment, potential inclusion criteria differences). Going forward, Barlow et al. (Reference Barlow, Beardsley, Bsharah, Crofts, De La Rosa, Gutierrez and Hayes-Harb2024) recommend that previous studies reporting an accent-based reduction in veracity judgments “may not be robust to replication” and that future replication studies are needed to determine the factors that influence the presence/absence of such effects in listening.

Gudmestad, Edmonds, Henderson, and Lindqvist (Reference Gudmestad, Edmonds, Henderson and Lindqvist2024) examined the extent to which L2 learners’ interpretation of verbal moods in L2 Spanish varied as a function of L2 proficiency and L1 background through a close replication of Kanwit and Geeslin (Reference Kanwit and Geeslin2014), an important study that “initiated L2 research on the interpretation of variable structures and because it demonstrated that the traditional interpretations of verbal moods in adverbial clauses in Spanish are variable for learners and NSs.” To examine the role of L1 background in the replication, Gudmestad et al. (Reference Gudmestad, Edmonds, Henderson and Lindqvist2024) recruited Swedish-speaking and French-speaking learners of L2 Spanish who completed the same verbal mood interpretation task and grammar test (to assess L2 proficiency) as the initial study, whereas Kanwit and Geeslin (Reference Kanwit and Geeslin2014) recruited English-speaking learners only. To assess proficiency effects, participants were grouped by (i) institutional course level and (ii) performance on the grammar test, leading to two groups Level 1 and Level 2. In terms of evaluating the replicability of the findings, Gudemestad et al. used three criteria: crosstabulations of responses, statistical significance in the replication, and the direction of effect estimates for statistically significant results in the replication. Overall, the findings broadly confirmed those reported in the initial study. However, some differences emerged from the variable modification (L1 background). For example, while French-speaking learners’ patterns of interpretation of verbal mood patterns did not shift substantially between proficiency levels for interpretation, Swedish-speaking learners showed patterns of interpretation more consistent with the initial study. As discussed in the replication, this pattern of results (among others) suggests caution in the initial study’s proposed developmental stages for mood because L1 background was found to be an important variable not examined in the initial study.

Hamada, Shimizu, Hoshino, Takaki, and Ushiro (Reference Hamada, Shimizu, Hoshino, Takaki and Ushiro2024) conducted a secondary meta-analysis of Jeon and Yamashita (Reference Jeon, Yamashita, Jeon and In’nami2022), a frequently cited and pedagogically important study in the field, to (a) ascertain whether the initial study’s metanalytic findings can be reproduced and (b) explore the extension of their findings to the L2 SVR model using a meta-analytic structural equation modeling (MASEM) approach. Furthermore, reproducing meta-analysis is widely agreed to be important (but seldom undertaken) to account for and explore unresolved questions from prior meta-analytic works (Plonsky, Reference Plonsky and Porte2012; see also Boers, Bryfonski, Faez, & McKay, Reference Boers, Bryfonski, Faez and McKay2021). Hamada et al. (Reference Hamada, Shimizu, Hoshino, Takaki and Ushiro2024) worked toward this goal by using the same data set and meta-analytic calculations and procedures from the initial study. By adding MASEM (as an extension of the initial study), this secondary meta-analysis has the potential to provide a more nuanced, comprehensive, and robust account of effects on L2 reading outcomes than explored in the initial study (e.g., metalinguistic skills, language comprehension skills, word decoding skills). This is because the initial study used a correlation-based analysis that limits conclusions to a single effect, whereas MASEM allows for more than just one effect to be investigated simultaneously while also managing latent variables. In terms of evaluating the comparability of the results, Hamada et al. compared the average correlation coefficients and their statistical significance as well as standardized mean differences between the studies. Overall, some previously reported effects on reading comprehension were confirmed (e.g., decoding, orthographic knowledge), while others were not (e.g., L2 phonological awareness, L1 reading comprehension). The authors discuss the interpretation of the different sets of results as a potential consequence of data recoding.

Liu and Lu’s (Reference Liu and Lu2024) close replication study of Römer and Berger (Reference Römer and Berger2019) addressed long-standing concerns with data sparsity in learner corpora and the resulting claims about group-level L2 developmental trajectories. Data sparsity, Liu and Lu discuss, refers to the small-scale nature of most existing learner corpora, which can compromise statistical analyses. The issue of data sparsity can also be found in Römer and Berger (Reference Römer and Berger2019), a theoretically and empirically important study in the field showing that the associations between verbs and verb argument constructions (VACs) used by German and Spanish learners of English move closer to L1 usage norms as the learners’ proficiency increases. In that study, however, which involved correlation analyses of the first version of the EF-Cambridge Open Language Database (Alexopoulou, Geertzen, Korhonen, & Meurers, Reference Alexopoulou, Geertzen, Korhonen and Meurers2015) and the British National Corpus (BNC Consortium, 2007), a large number of combinations between verbs and VACs in the BNC were absent in the learner corpus. This issue was addressed with log transformation, an approach with several known limitations, especially for counting data with many zeros. Liu and Lu’s replication study addressed this concern by retaining the initial study’s overall design and methods, making one variable modification: using the EFCAMDAT2 corpus, an updated and larger version of the corpus used in the initial study. Also, to address analytical concerns, Liu and Lu (Reference Liu and Lu2024) carried out both correlational analyses (as in the initial study) as well as zero-inflated negative binomial regression analyses, a statistical approach that accounts for count data with overdispersion and excess zeros (Winter, Perlman, & Majid, Reference Winter, Perlman and Majid2018). Liu and Lu’s findings replicated those reported in the initial study. For example, both studies reported that learners produced a wider variety of verbs in specific construction types (e.g., V about N) as their L2 proficiency increased. While not discussed in the initial study, Liu and Lu’s (Reference Liu and Lu2024) analysis also indicated an influence of writing topics on the learners’ use of verbs. In addition, Liu and Lu (Reference Liu and Lu2024) found that the use of zero-inflated negative binomial (ZINB) models to account for count data with overdispersion and excess zeros corroborated the correlation analyses, showing that (i) L1 verb–VAC frequency was significantly positively associated with L2 verb–VAC frequency and (ii) that increased overlap between the verb–VAC combinations used by L1 users and higher proficiency L2 learners. Taken together, this replication corroborated the initial study’s findings while showcasing the advantages of ZINB models in analyzing corpus frequency data with overdispersion and excess zeros.

Huensch’s (Reference Huensch2024) preregistered close replication study of Darcy, Mora, and Daidone (Reference Darcy, Mora and Daidone2016), a pioneering study in the field of SLA that explored the relationships among general cognition and language abilities, examined relationships between L2 speech perception and production and inhibitory control. In that study, Darcy et al. (Reference Darcy, Mora and Daidone2016) investigated to what extent variability in L2 pronunciation outcomes could be explained by individual differences in inhibitory control. Using a retrieval-induced inhibition task and measures of perception and production, Darcy et al. found important relationships between inhibition and perception, with little impact of inhibition on production, a finding that might be explained by the type of inhibition task used. Huensch thus addressed this limitation by adding a different measure of inhibition (Stroop task, Simon task) to the study design and keeping all other aspects of the study unmodified. To support replication, the initial study’s authors provided their materials and tasks on the Instrument Repository for Research in Second Language Studies (IRIS; see Marsden, Mackey, & Plonsky, Reference Marsden, Mackey, Plonsky, Mackey and Marsden2016). To determine the replicability of an effect, means, and standard deviations from the descriptive statistics of the two studies were used to compute Hedge’s g effect sizes and corresponding CIs. Hedge’s g <. 40 and corresponding CIs passing through zero were taken to indicate negligible differences. Statistical tests were interpreted using the directionality (positive vs. negative) and magnitude of the effect size. Overall, the initial study’s findings were not replicated: “no strong, clear, or consistent relationship emerges between inhibitory control and L2 perception/production skills.” Thus, Huensch explores several factors that may explain these findings, including a lack of statistical power in the initial study.

Given substantial evidence to date about the benefits of providing aural support during reading (or reading-while-listening) for vocabulary learning (e.g., R. Brown, Waring, & Donkaewbua, Reference Brown, Waring and Donkaewbua2008; Vu & Peters, Reference Vu and Peters2022), very little is known about the ways in which L1 background may interact with the effectiveness of this instructional method, especially when the transparency of grapheme–phoneme correspondences varies between L1 and L2. To this end, the Twilex Group (2024) conducted a preregistered close replication study of Malone (Reference Malone2018), a methodologically rigorous, transparent, and important study in research examining the effectiveness of reading-while-listening, which systematically integrated L1 differences into the study design. In the initial study, the main research goal was to compare the effectiveness of reading-while-listening (RWL) and reading-only (RO) on incidental vocabulary learning, including to what extent does learning depend on the amount of exposure (two vs. four exposures) and working memory (WM). The replication’s major variable modification was the L1 background of the learners, achieved by recruiting L1 speakers of Chinese (Mandarin, Cantonese) and Germanic languages (German, Dutch), who were L2 learners of English. The initial study’s participants spoke a range of L1s. Any other modifications to the initial study’s design and methods were considered minor in scope (e.g., online instead of in-person data collection). The close replication study obtained all materials either through personal communication with the author of the initial study or from the IRIS database. Claims about the replicability of an effect were determined if the replication obtained the same direction and size of the correlations as those reported in the initial study. Overall, this close replication study obtained mixed results in comparison to the initial study. For example, the Twilex Group found few meaningful differences between the RWL and RO reading conditions, regardless of the amount of exposure, whereas the initial study found RWL to be more effective than RO after two exposures. Between-study similarities were found for working memory, though, showing that higher working memory capacities resulted in greater learning gains. In terms of the main objective of the replication, to assess what role L1 background played, the results indicated important interactions among L1 background, WM, and reading condition (RWL, RO). For example, the demand of WM was greater for L1 Chinese learners in RWL conditions. Taken together, this replication study sheds important light on the effectiveness of bimodal input on L2 learning outcomes and the potentially mediating role that L1 background can play.

Revisiting the role of transfer in third language (L3) acquisition, Parrish (Reference Parrish2024) reports an approximate replication of Rothman (Reference Rothman2011), a theoretically and empirically important study in the field examining how the perceived typological similarity of previously learned languages shaped L3 learning. Indeed, the Typological Primacy Model, an important theory in L3 research, is based on Rothman (Reference Rothman2011). In addition, Parrish (Reference Parrish2024) notes limitations with the initial study’s small sample size and the statistical tests used. In the approximate replication study, two major changes were implemented. First, new materials were created because the ones used in the initial study were lost and thus not available. These materials were recreated based on the descriptions and examples provided in the initial study. Second, new speaker groups were created. In the initial study, speakers were (a) L1 English–L2–Spanish–L3 Brazilian Portuguese and (b) L1 Italian–L2 English–L3 Spanish. In the replication study, mirror image groups of Spanish–English bilinguals were recruited: (a) L1 English–L2 Spanish–L3 Brazilian Portuguese and (b) L1 Spanish–L2 English–L3 Brazilian Portuguese. The sample size was also increased. Overall, Parrish’s (Reference Parrish2024) analysis showed that some aspects of the initial study’s findings replicated while others did not. For example, whereas the initial study showed that both learner groups showed evidence of transfer by Spanish in their L3, these results were only partially replicated by Parrish (Reference Parrish2024). The importance of adequate sample sizes is noted as a possible explanation for these findings, along with the use of different data collection materials. All in all, this replication calls for future research to consider the role of sample size in decisions about what to replicate going forward.

Revisiting a central question in bilingualism research about the ways in which L2 speakers access and activate words in their known languages, Pelzl and Van Hell (Reference Pelzl, Jonczyk and van Hell2024) conducted an approximate replication of Van Hell and Dijkstra (Reference Van Hell and Dijkstra2002), an influential and frequently cited study in the bilingual word processing literature. In addition, Van Hell and Dijkstra (Reference Van Hell and Dijkstra2002) focused on effects from L2 to L1, providing a particularly strong test of language nonselectivity, which serves as an empirical foundation for many theoretical models of bilingual processing in the field (e.g., Dijkstra, Wahl, Buytenhuijs, Van Halem, Al-Jibouri, De Korte, & Rekké, Reference Dijkstra, Wahl, Buytenhuijs, Van Halem, Al-Jibouri, De Korte and Rekké2019; Shook & Marian, Reference Shook and Marian2013). For these reasons, determining the replicability of Van Hell and Dijkstra’s (Reference Van Hell and Dijkstra2002) findings is important for continued theorization and growth in the field. In this approximate replication study, Pelzl and Van Hell (Reference Pelzl, Jonczyk and van Hell2024) made two major modifications to the initial study. First, the target population was modified from Dutch–English–French trilinguals to Dutch–English bilinguals. This change was implemented to test whether the initial study’s findings held when there was no L3 present, thus strengthening and refining the claims made in the initial study. Second, the study was conducted online rather than in-person in a lab context to recruit a more diverse and larger group of participants, thus addressing small sample size concerns in the initial study. All other aspects of the initial study’s design and methods were not intentionally modified, with any other changes considered to be minor in scope. Also, since Van Hell was involved in both the initial study and this replication, additional information about the study and access to the materials was available (e.g., providing stimuli not provided in the initial study), thus providing a critical resource for the replication study. In terms of determining the replicability of the findings, Pelzl and Van Hell’s (Reference Pelzl, Jonczyk and van Hell2024) replication used two indices: the direction and statistical significance of effects. Overall, Pelzl and Van Hell (Reference Pelzl, Jonczyk and van Hell2024) found the same patterns of results as those reported in the initial study, confirming that L2 knowledge can influence L1 performance in important and detectable ways. Furthermore, by recruiting a large sample of participants, the replication study showed stronger effects than the initial study, some of which did not research statistical significance. For example, the initial study observed a 10-ms facilitation effect, a result that was not statistically significant, but a larger sample of participants resulted in a facilitation effect that was 19 ms, which did reach statistical significance. Taken together, this replication study not only confirmed the foundational claims made in the initial study but also strengthened them in many cases, thus providing a strong validation of the initial study claims.

In line with Pelzl and Van Hell (Reference Pelzl, Jonczyk and van Hell2024), the close replication study conducted by Ryan et al. (Reference Ryan, Foster, Wang, Fester and Yap2024) also included a researcher from the initial study. The study selected for replication was Tavakoli and Foster (Reference Tavakoli and Foster2008), a theoretically and empirically important study in the field of SLA that is also frequently cited in the literature. That study investigated the ways in which L2 oral performance was influenced by narrative task design, showing that narratives with both foreground and background information elicited significantly greater syntactic complexity than those with only foreground information. In this close replication study, Ryan et al. (Reference Ryan, Foster, Wang, Fester and Yap2024) included one intentional modification only by adding L1 literacy. This was achieved by recruiting adult refugees to New Zealand with low L1 literacy. This variable modification is important for several reasons, not only because most SLA research is based on highly literate populations (see Andringa & Godfroid, Reference Andringa and Godfroid2020), but also because L1 literacy can strongly influence language processing and metalinguistic knowledge. Indeed, advanced levels of L1 literacy have been associated with more accurate L2 performance (Havron & Arnon, Reference Havron and Arnon2017). Thus, this close replication study set out to examine to what extent L2 oral performance can also be influenced by low levels of L1 literacy, thereby impacting many theories of L2 learning concerning explicit teaching, metalinguistics knowledge, the provision of corrective feedback, and key aspects of language instruction that depend on metalinguistic knowledge and its processing. All other aspects of the initial study’s design and methods were not intentionally modified, with all other differences considered minor in scope. Ryan et al. (Reference Ryan, Foster, Wang, Fester and Yap2024) used indices of direction and statistical significance to determine the replicability of the initial study’s findings. Overall, the replication study did not confirm the initial study’s findings about interactions between narrative task design and L2 oral performance because background information in the narrative tasks had no impact on the syntactic complexity, lexical diversity, or fluency of learners’ performance. For example, participants in the replication recounted narrative tasks with and without background events in comparable ways, whereas the initial study found that background events led to performance with longer and more complex syntactic patterns. Thus, Ryan et al.’s (Reference Ryan, Foster, Wang, Fester and Yap2024) findings indicated that learners let background events in the narrative pass unremarked, instead focusing on foreground events. Informed by these findings, Ryan et al. (Reference Ryan, Foster, Wang, Fester and Yap2024) suggest that schooling experience and levels of visual literacy may explain these findings better than levels of low literacy. This is because, the authors argue, the replication did not require literacy-supported cognitive tools (e.g., awareness of word boundaries) to follow sequences of events as presented in picture-based narratives. Thus, as an active area for future SLA research, Ryan et al. (Reference Ryan, Foster, Wang, Fester and Yap2024) encourage replication of key studies in task-based learning and teaching, which tend to presume experience and familiarity with visual prompts. Such study designs would appear not to work as well with all learner populations.

The last study in this special issue that included a researcher from the initial study is Wong and Prange (Reference Wong and Prange2024), a close replication of Wong, Zhao, and MacWhinney (Reference Wong, Zhao and MacWhinney2018), an important study in instructed SLA about the specific contribution of instruction couched in cognitive linguistics when compared with “traditional” instruction. In the initial study, Wong et al. (Reference Wong, Zhao and MacWhinney2018) developed an experimentalized computer-assisted language learning (eCALL) system called the English Preposition Tutor, to investigate the effectiveness of a cognitive linguistics approach to learning English prepositions, which indicated promising findings for this instructional approach. However, as noted by Wong and Prange (Reference Wong and Prange2024), the initial study included no delayed posttest, and the statistical analyses (analysis of variance, ANOVA) focused on analyzing separate effects only, leading to a limited understanding of the initial study’s effectiveness in supporting L2 development. Thus, in this close replication study, Wong and Prange (Reference Wong and Prange2024) integrated two intentional modifications to the initial design. First, rather than using ANOVAs to analyze separate effects, Bayesian mixed-effects models were used. Importantly, the initial study’s findings were used to inform the study’s priors (see Garcia, Reference Garcia2021; Norouzian, de Miranda, & Plonsky, Reference Norouzian, de Miranda and Plonsky2018). Second, a three-week delayed posttest was added to examine the potential for extended effects of learning in the replication. In line with recommended practice, the sample sizes in the instructional groups were also increased. All other aspects were not intentionally modified. Overall, Wong and Prange’s results are broadly consistent with those reported in the initial study. For example, all groups improved significantly following the instruction, as also reported in the initial study. The incorporation of the delayed posttest was able to show that learning gains documented immediately after the instruction were broadly maintained after the instruction. Thus, in sum, Wong and Prange’s replication study was able to broadly confirm the findings reported in the initial study.

In an approximate replication study, Wang and Sun (Reference Wang and Sun2024) revisited the Second Language Motivational Self System (Dörnyei, Reference Dörnyei2005, Reference Dörnyei2009), an influential theoretical framework in programs of empirical research and theory-building about language learner’s motivation and its role in second language learning. However, recent commentary has expressed concerns with this framework, highlighting issues such as “the fantasy problem,” the “ought-to L2 self problem,” and the “context problem” (Al-Hoorie, Hiver, & In’nami, Reference Al-Hoorie, Hiver and In’nami2024). Papi et al. (Reference Papi, Bondarenko, Mansouri, Feng and Jiang2019) responded to these concerns about the Second Language Motivational Self System with the 2 × 2 model, designed to offer a more nuanced account of motivational dynamics. Indeed, subsequent research across educational and cultural contexts has indicated that the 2 × 2 model appears to more effectively predict language learning behaviors and outcomes (e.g., Zhou & Papi, Reference Zhou and Papi2023). Indeed, Papi and Khajavy (Reference Papi and Khajavy2021) showed how this model can account for language learning outcomes in an Iranian context, thus providing useful information about the applicability of the 2 × 2 model in a novel context within this line of research. In the current approximate replication, Wang and Sun (Reference Wang and Sun2024) sought to revisit and extend this line of research in a Chinese English as a foreign language (EFL) context given the relatively different educational and cultural contexts in Iran and China. For example, students receive English language instruction at a younger age in China, and China’s promotion of English aligns with its broader strategy for global competitiveness, factors that could impact motivational dynamics toward language learning outcomes. In their approximate replication study, Wang and Sun (Reference Wang and Sun2024) intentionally modified three variables: learner background, the language used in the questionnaire, and the method of measuring L2 achievement. The replication team used the same questionnaire as used in the initial study (obtained from IRIS) and increased the sample size from 324 in the initial study to 855. Using this design, Wang and Sun (Reference Wang and Sun2024) examined to what extent Papi and Khajavy’s (Reference Papi and Khajavy2021) findings hold in a Chinese EFL context. Overall, Wang and Sun (Reference Wang and Sun2024) findings broadly replicated those reported in the initial study, especially in terms of how regulatory focus plays a critical role in language learning. One notable difference, however, was that the replication reported stronger impacts of ought-to-selves, anxiety, and enjoyment on language learning outcomes. The authors discuss these different patterns of results in light of the educational and contextual differences between Iran and China.