Internal and external validity are foundational constructs that serve as critical criteria for evaluating research quality (Shadish, Cook, & Campbell, Reference Shadish, Cook and Campbell2002). Internal validity refers to the extent to which the design, measurement, and analytic procedures employed within a study support a causal interpretation of findings by ruling out alternative explanations. Readers evaluate the internal validity of studies by assessing the strength of research designs; studies with strong internal validity allow readers to be confident that the findings are credible. In other words, internal validity concerns whether findings accurately reflect the variables and relationships of interest. External validity, by contrast, refers to the degree to which research findings can be generalized beyond the study at hand. Because there is a great deal of precision involved in SLA research designs, it is important to question the degree to which results may be valid beyond the boundaries of the methodology. Therefore, when readers evaluate the external validity of a study, they are assessing the degree to which the research findings are transferable across different learner populations, target items, materials, learning conditions, tests, and procedures. This editorial argues that insufficient attention to external validity in SLA research limits the interpretability and responsible use of research findings and that more systematic reporting and design practices are needed to address this issue.

One way to assess external validity is to consider whether and to what extent changes in key design variables are likely to influence findings. These variables include learner, material, learning condition, target item, measurement, and procedural factors.

1. 1. Learner variables

   To what extent can the findings be generalized to learners who differ in age, proficiency, L1 background, amount of formal instruction, level of exposure to the L2, metalinguistic awareness, learning strategies, motivation, and so forth?

2. 2. Material variables

   To what extent can the findings be generalized to learning with materials that differ in length, intended audience, difficulty, lexical complexity, lexical sophistication, topic familiarity, and mode of input (e.g., spoken, written, multimodal)?

3. 3. Learning-condition variables

   To what extent can the findings be generalized to spoken and written conditions that differ in length, frequency, learning context, dyadic or group configuration, mode (speaking, listening, reading, writing), degree of meaningful engagement, and feedback?

4. 4. Target item variables

   To what extent can the findings be generalized to target items that differ in form, meaning, frequency of exposure, difficulty, salience, and degree of L1–L2 formal and semantic congruency?

5. 5. Measurement variables

   To what extent can the findings be generalized across assessment formats that differ in length, task type, sensitivity to knowledge, mode of input and output, scoring criteria, and time allocated per item?

6. 6. Contextual and procedural variables

   To what extent can the findings be generalized across instructional contexts (e.g., classroom vs. out-of-class learning; EFL, ESL, ESP, EAP) and procedures with variations in instructional time, task instructions, level of instructional support, and number of exemplars?

The peer review process is well-designed to evaluate the internal validity of studies submitted for publication. Reviews must place emphasis on whether research designs meet the requirements necessary to effectively answer the research questions. Most studies are rejected for publication due to shortcomings in their research design. Although reviewers seldom label these issues explicitly as threats to internal validity, doing so would be useful because it would make their feedback clearer and easier for authors to apply. The external validity of studies of SLA is likely to receive less attention in the review process. This pattern is to some extent understandable. If research does not have strong internal validity, the results have little value and so there is little need to assess external validity. Moreover, any single SLA study will necessarily have limited external validity, because its findings cannot encompass the wide range of learner, target-item, material, learning-condition, test, and procedural variables that may influence outcomes. There is an expectation that authors will address limitations in external validity within discussion, limitation, and conclusion sections of a manuscript. However, because authors must often prioritize addressing multiple reviewer concerns within strict word limits, discussion of external validity may be neglected. Moreover, there is good reason why authors may include relatively little detail about the external validity of their research. The review process can be very negative and carefully describing limits on a study’s external validity, while useful, may reduce the potential for reviewers to recommend the manuscript be accepted for publication. Together, these factors may discourage authors from engaging in detailed discussion of external validity.

A lack of attention to external validity in published SLA research can have significant consequences, particularly through the misinterpretation or overgeneralization of findings. Such limitations are often compounded when claims are restated in subsequent publications. Research on lexical coverage and reading comprehension provides a clear example. Lexical coverage—the proportion of known words in a text—is among the most frequently discussed constructs in SLA; the five most cited studies of the topic (Hu & Nation, Reference Hu and Nation2000; Laufer, Reference Laufer, Lauren and Nordman1989, Reference Laufer, Arnaud and Béjoint1992; Laufer & Ravenhorst-Kalovski, Reference Laufer and Ravenhorst-Kalovski2010; Schmitt et al., Reference Schmitt, Jiang and Grabe2011) have together accrued over 7,000 citations (Google Scholar, March 22, 2026). It is therefore unsurprising that the claim that readers can understand a text when they know approximately 98% of its words has become widely accepted in the literature. However, empirical support for this claim is considerably weaker than is often assumed, particularly when evaluated across diverse contexts.

Early studies suggested that readers might achieve comprehension at coverage levels of 95% (Laufer, Reference Laufer, Lauren and Nordman1989) or 98% (Hu & Nation, Reference Hu and Nation2000), but subsequent research has shown that lexical coverage alone is a weak predictor of reading comprehension. Schmitt et al. (Reference Schmitt, Jiang and Grabe2011) concluded that while high coverage is necessary, it is insufficient, and that individual comprehension remains difficult to predict. More recent partial replications have found little evidence for a clear lexical coverage threshold and indicate that results are sensitive to learner characteristics, text features, and the measures used to assess comprehension (Kremmel et al., Reference Kremmel, Indrarathne, Kormos and Suzuki2023; Webb et al., Reference Webb, Pellicer-Sánchez and Wang2025). More importantly, the external validity of this body of research is highly constrained. Confidence in the generalizability of these findings would require evidence that the texts, learners, lexical items, assessments, and procedures employed were representative of the wide range of conditions under which L2 reading occurs. This includes variation in text length, genre, difficulty, and topic; learner proficiency, background knowledge, and reading experience; lexical characteristics and densities of unknown words; the quality and sensitivity of comprehension measures; and the ecological validity of research procedures. Even if consistent results were observed across these studies, generalizability would remain limited. Collectively, this body of research has examined comprehension using only five short texts, despite the many text-level variables known to influence reading comprehension. Although these studies clearly demonstrate the importance of vocabulary knowledge, they also underscore the need for substantially more research before lexical coverage can be considered a reliable predictor of reading comprehension across contexts. Moreover, these limitations in external validity raise wider concerns about how findings are interpreted and reported beyond the original studies.

More broadly, limited attention to external validity is evident in how research findings are reported in the SLA literature. Reviewers frequently suggest expanding the limitations sections of submitted manuscripts. However, very few published studies comprehensively address limitations in external validity. Limitations that are often provided tend to note that the findings should not be generalized to a wider and more diverse learner population. This is less likely to negatively influence the potential for publication because individual studies are unlikely to extend to diverse learner populations and address the large number of participant variables (e.g., age, proficiency, background knowledge, L1 background, L2 instruction). In contrast, if authors acknowledge limited generalizability across target items, materials, tests, learning conditions, and procedures, reviewers may recommend improving the research design, additional data collection, or rejecting the study for publication.

Researchers who have been trained in programs that place strong emphasis on research design and methodological rigor may be less influenced by the limited attention given to external validity in published studies. However, access to such training is unevenly distributed, with most leading programs concentrated in a relatively small number of WEIRD (Western, Educated, Industrialized, Rich, and Democratic) contexts. As a result, research communities that may already have fewer institutional resources or mentoring opportunities may be disproportionately affected. Novice researchers may have difficulty identifying limitations in external validity and consequently misinterpret research findings. They may also be less aware of opportunities to design well-justified follow-up studies that systematically test the generalizability of earlier results. Increased attention to external validity in research reporting would support more accurate interpretation of findings and help identify clear and methodologically motivated pathways for further investigation. Moreover, explicitly framing follow-up studies in terms of extending external validity, whether through formal replication studies or otherwise, may reduce barriers to conducting new research by providing a transparent and compelling rationale grounded in both the original findings and their limitations.

Established researchers often use limitations in the external validity of their findings as the motivation for subsequent studies. A clear example of this approach can be seen in a series of studies on the comprehensibility of L2 speech conducted by Kazuya Saito, Pavel Trofimovich, Talia Isaacs, and colleagues. In an early study, Saito, Trofimovich, and Isaacs (Reference Saito, Trofimovich and Isaacs2016) examined how native speakers evaluated the comprehensibility and accentedness of L2 learner speech. Recognizing both the substantial societal and pedagogical relevance of L2 speech comprehensibility and the possibility that findings based on native speaker ratings might not generalize to other populations, the researchers conducted a set of theoretically motivated follow-up studies. These studies systematically examined whether the original findings extended across different rater groups, including L2 learners (Saito et al., Reference Saito, Tran, Suzukida, Sun, Magne and Ilkan2019), expert and non-expert raters (Saito, Trofimovich, Isaacs, & Webb, Reference Saito, Trofimovich, Isaacs and Webb2016), and native speakers of North American English and Singapore English (Saito & Shintani, Reference Saito and Shintani2016). This line of research illustrates the importance of not assuming that findings related to L2 speech comprehensibility generalize across populations. Similar research programs aimed at extending external validity can be seen in studies of interactional feedback (e.g., Nassaji, Reference Nassaji2007, Reference Nassaji2009, Reference Nassaji2017; Sarandi & Nassaji, Reference Sarandi and Nassaji2024), errorless learning (e.g., Boers, Yu, & Wang, Reference Boers, Yu and Wang2025; Elgort, Baliaeva, & Boers, Reference Elgort, Beliaeva and Boers2020; Stengers & Boers, Reference Stengers and Boers2015), and lexical profiling (e.g., Dang & Webb, Reference Dang and Webb2014; Paribakht & Webb, Reference Paribakht and Webb2016; Webb, Reference Webb2011). Thus, recognizing limits in the external validity of prior research can help novice researchers formulate clearly motivated replication or follow-up studies that extend findings beyond the constraints of the original designs.

In recent years, three initiatives have been promoted as ways of increasing the external validity of studies in SLA: replication studies, meta-analytic studies, and multisite studies. Calls for replication emphasize the need to examine the extent to which research findings are consistent across repeated investigations (e.g., Marsden et al., Reference Marsden, Morgan-Short, Thompson and Abugaber2018; McManus, Reference McManus2024; Porte, Reference Porte2012; Porte & McManus, Reference Porte and McManus2019). Meta-analyses reveal the extent to which external validity holds across learner, material, learning condition, measurement, context, and procedure variables. Notably, perhaps the most common conclusion across meta-analyses is the need for further research to increase external validity across diverse variables. Multisite studies, in contrast, focus primarily on determining whether findings generalize across different learner samples (e.g., Morgan-Short et al., Reference Morgan-Short, Marsden, Heil, Issa Ii, Leow, Mikhaylova, Mikołajczak, Moreno, Slabakova and Szudarski2018; Vitta et al., Reference Vitta, Nicklin and McLean2022). Increasing generalizability across participant groups is important because it raises the likelihood that findings with one group of learners also occur in other populations and reduces the extent to which findings are driven by characteristics of a single participant sample. While this represents an important advance, multisite studies typically address only learner variables and do not, on their own, rule out the influence of other design features such as target item variables, material variables, dependent measures, or procedural variables. Consequently, although replication, meta-analyses, and multisite studies make valuable contributions to understanding generalizability, they do not on their own resolve the broader issue of external validity.

Moving towards increased external validity in SLA research

There are several ways in which the external validity of studies of SLA could be made more transparent. First and most importantly, there is a need for authors to include comprehensive discussion of the external validity of their findings. This is easier said than done because authors are understandably wary of describing how their findings may not extend beyond the constraints of their research design; those who do this could be penalized by reviewers while those who do not might find an easier pathway to publication. In order to achieve a fair and transparent process that benefits authors and the research community, it may be necessary to have required limitations sections that include focus on learner, material, target item, learning condition, test, and procedural variables. If this were a requirement in submission guidelines, then both authors and reviewers could follow a process benefitting the wider research community that was fair and transparent. Table 1 provides an initial framework that could be used, with modifications as needed, to guide evaluations of external validity.

Table 1.

Framework for Evaluating the External Validity of SLA Research Designs

Table 1. Long description

From the top row, the left column lists six design variables: learner variables, material variables, interactional and learning-condition variables, target item variables, measurement variables, and contextual and procedural variables. The right column details key considerations for external validity for each variable. Learner variables address transferability across learners differing in age, proficiency, L1 background, formal instruction, L2 exposure, metalinguistic awareness, learning strategies, and motivation. Material variables focus on transferability across instructional materials differing in length, audience, difficulty, topic familiarity, and input mode such as spoken, written, or multimodal. Interactional and learning-condition variables consider transferability across conditions differing in length, frequency, learning context, group configuration, mode (speaking, listening, reading, writing), and meaningful engagement. Target item variables examine transferability across items differing in form, meaning, exposure frequency, difficulty, salience, and L1–L2 congruency. Measurement variables address transferability across assessment formats differing in length, task type, sensitivity, input and output modes, scoring criteria, and time per item. Contextual and procedural variables cover transferability across instructional contexts (classroom, out-of-class, E F L, E S L, E S P, E A P) and procedures including instructional time, task instructions, instructional support, and number of exemplars.

A second way to improve the external validity of studies of SLA would be to continue to advocate for more replication studies. The journal Language Teaching has long included replication as a category for submission and Language Learning and Studies in Second Language Acquisition also publish replication studies. Studies in Second Language Acquisition recently put out a call for registered report replications of five studies with an aim to publish multiple replications of the same study. This useful initiative may reveal both the need to further examine the robustness and credibility of earlier findings, as well as the many ways in which one or two variables from one study can be modified to test the external validity of earlier findings.

A third way in which external validity can be increased is to continue to advocate for multisite studies. It is important to note, however, that while multisite studies do help to extend the generalizability of findings to different populations, generalizability across learner populations is still likely to be limited. This is because it would be difficult to account for variation across all learner variables (e.g., age, proficiency, L1 background, years of formal language instruction, previous exposure to the L2, vocabulary knowledge, working memory, metalinguistic awareness, learning strategies, motivation) in a single study. In addition, multisite studies do not account for variation across target items, materials, learning conditions, tests, and procedures. For this reason, we should also advocate for research that increases external validity through research designs that include more than one material (multimaterial studies), more than one type of target item (multitarget item type studies), and more than one learning procedure (variation could occur across procedural variables such as amount of L2 exposure/use, number of sessions, time on task, learning support provided: multiprocedure studies). Although studies often include multiple tests and learning conditions, greater emphasis on a wider range of learning conditions and measurement across different test formats would be beneficial. For example, Morgan and Rinvolucri (Reference Morgan and Rinvolucri2004) described 118 activities for vocabulary learning, and Webb and Nation (Reference Webb and Nation2017) outlined 23 activities that they reported were effective for learning. Nevertheless, a meta-analysis on word-focused vocabulary found that there were only four activities that met inclusion requirements (Webb et al., Reference Webb, Yanagisawa and Uchihara2020). One of the key values of meta-analyses is their ability to highlight such gaps. Taken together, this underscores the need for cautious interpretation of the external validity of research findings; while there are typically many ways to learn an L2, researchers have investigated relatively few learning conditions with most studies including the optimal conditions for learning with an aim towards finding significant results.