Literature search

A thorough and systematic literature search, following the guidelines of Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) 2020 checklist (Page et al., Reference Page, Moher, Bossuyt, Boutron, Hoffmann, Mulrow and McKenzie2021), was conducted to identify studies published up to September 2024, aimed at providing a comprehensive understanding of the strand. The initial search was focused on leading academic databases in applied linguistics and educational psychology, expanding later to include supplementary databases, online repositories, publisher platforms, and relevant social media outlets. To ensure exhaustive coverage, the review also incorporated an analysis of the Tables of Contents of the top 50 applied linguistics journals, as ranked by Scimago Journal & Country Rank [accessible at https://www.scimagojr.com/journalrank.php?category=3310&area=3300&type=j] (see Appendix 1 for the full list of databases, online resources, publisher platforms, and journals). This phase involved conducting keyword searches on each journal’s website and reviewing the Tables of Contents for relevant studies. In addition to database searches, the first author conducted targeted keyword searches within the WorldCat database, utilizing the specific search terms outlined in Appendix 2. To further enhance the search, references from all identified studies were reviewed. No meta-analyses on DoP in ISLA were found in the existing literature.

Forward searches were conducted by exploring Listserv archives, ResearchGate, and Google Scholar. Recognizing the importance of dissertations and theses due to their rigorous research methodologies, these were also included in the review (Vuogan & Li, Reference Vuogan and Li2023). Despite these efforts, no additional studies were identified through forward searches. Additionally, the first and third authors employed social media platforms to access unpublished or restricted-access theses, dissertations, and articles, ensuring a broad selection of studies for the research synthesis and meta-analysis (Abdi Tabari et al., Reference Abdi Tabari, Johnson and Farahanynia2024, Reference Abdi Tabari, Zhuang and Farahanynia2025). As part of the search strategy, the initial WorldCat search highlighted 14 prominent scholars in ISLA. The first author then conducted a detailed examination of each scholar’s faculty web page, while the third author investigated their personal and professional web pages. Despite these efforts, no further studies were identified.

Inclusion criteria

Our initial database searches, using key search terms, identified a total of 4,798 potential studies (3,131 from Scopus, 544 from Linguistics and Language Behavior Abstracts [LLBA], 383 from Web of Science, 335 from ProQuest Linguistics, 247 from ProQuest Dissertations and Theses Global, and 158 from the Educational Resources Information Center [ERIC]). To ensure a rigorous and methodologically sound synthesis of the literature, we employed a systematic screening process, reviewing the titles and abstracts of these studies based on a predefined set of eligibility criteria. Studies were included in the present review if they met all of the following criteria:

1. 1. The study must examine the impact of or the relationship between DoP and L2 performance or development.

2. 2. The study must operationalize DoP with clear, measurable indicators used within ISLA settings.

3. 3. The study must be conducted in a formal educational context. Specifically, the report must clearly describe the research setting (e.g., university, college, school, or language institute) or participant population (e.g., students enrolled in a course or program), allowing the study to be unambiguously identified as taking place within an instructional environment. Studies conducted in non-instructional contexts (e.g., informal learning, self-study, or immersion without pedagogical intervention) were excluded.

4. 4. The study must be a peer-reviewed journal article, book chapter, or dissertation.

5. 5. The study must provide sufficient methodological detail to support replication and analysis.

6. 6. The study must be written in English.

After the initial screening based on titles and abstracts, 688 studies (37 book chapters, 443 journal articles, and 208 theses/dissertations) remained for full-text review. Subsequently, a rigorous process further refined the selection to 30 studies eligible for full analysis. The first and third authors independently coded all studies. Interrater reliability was high (α = 0.95), and any inconsistencies were resolved through discussion. The full list of included studies is provided in Appendix 3. Figure 1 illustrates the PRISMA procedure adopted in this study.

Figure 1.

The PRISMA flow chart for the search and screening process.

Figure 1. Long description

The flowchart details the process of identifying, screening, and including studies for final analysis in instructed second language acquisition research. The process begins with a literature search involving various databases, journals, online resources, and publisher platforms, resulting in 4,798 records after duplicate removal. The next step involves applying inclusion criteria to exclude 4,110 studies based on title and abstract screening. The inclusion criteria specify that the study must examine the impact of or the relationship between depth of processing and L2 performance or development, operationalize depth of processing with clear indicators, be conducted in educational settings, be peer-reviewed, provide sufficient methodological detail, and be written in English. After title and abstract screening, 688 studies undergo full text downloading. Following full text screening, 658 studies are excluded for not meeting the inclusion criteria, leaving 30 studies for final analysis.

Dataset building and qualitative coding

Dataset Building. A thorough literature review led to the creation of a structured dataset to organize the selected studies based on key aspects such as publication type, study design, and findings. The first step involved systematically extracting detailed information from each study. For every study, the following elements were recorded: title, authors, publication year, source (names of journals, books, and institutions for articles, chapters, and dissertations, respectively), study design (e.g., experimental, quasi-experimental, observational, pretests, posttests, and treatment intervals), sample characteristics (e.g., size, age, gender, proficiency, linguistic setting, and country), proficiency assessments, treatment details (e.g., language and duration), statistical methods (e.g., regression models and t-tests), and statistical results (e.g., means, standard deviations, and effect sizes), and outcome measures.

Qualitative Coding. Following the final selection of studies (N = 30), a systematic qualitative coding procedure was conducted to categorize studies according to emergent themes and key variables related to DoP in ISLA. The coding framework was developed inductively and refined iteratively through multiple rounds of analysis to ensure conceptual clarity, internal consistency, and analytical robustness. A detailed overview of this coding scheme, outlining the specific definitions, constructs, and the breadth of operationalization used to examine DoP, is provided in Table 1.

Table 1.

Coding scheme and categories for depth of processing in ISLA

To establish the reliability of the coding scheme, the first and third authors independently coded a subset of the studies (n = 10; 33% of the dataset) during an initial calibration phase. Inter-rater reliability (IRR) for this double-coded subset was high (an overall agreement of 95% / TEMP) Cohen’s κ = .95), indicating strong consistency across the coding scheme. Aligning with established perspectives on meta-analytic research (Norouzian, Reference Norouzian2021), this initial IRR assessment served not only to quantify agreement but also fulfilled a diagnostic function. Specifically, initial coding discrepancies were examined in detail and completely resolved through systematic discussions to reach full consensus. This consensus-building process led to the refinement and consolidation of coding categories and operational definitions.

Once the coding framework was finalized through consensus, it was systematically applied to the remaining dataset. Each study was coded for (a) definitions and conceptualizations of DoP, (b) operationalization and measurement of DoP, and (c) key methodological characteristics. In addition, methodological limitations, such as small sample sizes, short treatment durations, and reliance on self-reported measures, were explicitly documented to inform the interpretive synthesis and to contextualize the strength of the evidence base. The subsequent thematic analysis enabled the identification of recurring patterns and cross-study trends, which formed the basis for the qualitative synthesis and directly informed the structure and interpretation of the meta-analytic component of the study.

Inclusion and exclusion criteria

From the 30 identified studies, an additional screening process was conducted to determine eligibility for the meta-analysis. Beyond the inclusion criteria used in the research synthesis, studies were required to provide descriptive statistics necessary for effect size (ES) calculation (Hedges’ g), such as means, standard deviations, and sample sizes. These statistics were needed for either within-group ES calculations (e.g., pre-test and post-test for the treatment group) or between-group ESs (e.g., post-test comparisons between treatment and control groups), or both. As a result of this additional screening, 18 studies were excluded due to insufficient quantitative data, leaving 12 studies eligible for the meta-analysis.

Dataset building, effect size calculation, and quantitative coding

Dataset Building. During the review of the 12 selected studies for meta-analysis dataset construction, we examined whether they included more than one independent sample or comparison. For instance, Peng (Reference Peng2024) included two experimental conditions for DoP, resulting in two independent within-group samples. Similarly, Shao and Liu (Reference Shao and Liu2022) featured three experimental groups and a control group, creating three independent comparisons for between-group analysis. Through this process, we identified a total of 47 independent samples (including independent comparisons) across the 12 studies. An overview of the dataset along with the title and details of the included 12 studies is provided in Appendix 4.

Effect Size Calculation. Once the dataset structure was finalized, we computed effect sizes as standardized mean differences (Hedges’ g, small-sample corrected) from the outcome measures reported in each study. This allowed us to address RQ2—What impact does DoP have on L2 learning outcomes?—by quantifying (a) within-group pre–post change under a given DoP condition and (b) between-group differences between DoP and comparison conditions (at posttest or in gains). Because the included studies used different dependent variables, effect-size estimation placed outcomes on a common metric; Appendix 4 provides study-level details of the learning outcomes, including the target domain and outcome construct. Given that 12 samples had fewer than 50 participants, and the average sample size across the 47 samples was approximately 67, we used Hedges’ g as the ES metric due to its reduced bias for small sample sizes. Hedges’ g, an adjusted version of Cohen’s d, applies a correction for small sample size bias $\left(J = 1 - \left( {\frac{3}{{4\left( {{n_1} + {n_2} - 2} \right) - 1}}} \right)\right)$ , and is interpreted using the same standard benchmarks as Cohen’s d: 0.2 represents a small ES, 0.5 indicates a medium ES, and 0.8 signifies a large ES.

While computing effect sizes, we noted three key characteristics of the dataset: (a) two study designs—within-group (pre–post) and between-group (treatment–comparison)—for which we estimated standardized mean differences (Hedges’ g; within-group ES indices pre–post change, between-group ES indices post-test or gain differences); (b) two timings—short-term (immediate post-test) and long-term (delayed/retention); and (c) multiple outcome measures in many samples. For studies with multiple measures, we first computed an ES for each eligible measure and then averaged the effect sizes within each sample to obtain a single ES per sample for the main analyses. Accordingly, effect sizes were organized in a 2 × 2 structure by design and timing, with cell counts reported below. Consequently, a total of 228 ESs were computed across the dataset:

1. (1) 133 short-term ESs for 47 within-group samples.

2. (2) 31 short-term ESs for 27 between-group samples.

3. (3) 33 long-term ESs for 17 within-group samples.

4. (4) 17 long-term ESs for 11 between-group samples.

Quantitative Coding. After calculating ESs, studies were coded for moderator variables, including variations in the definition and operationalization of DoP. It is important to note that most features of variation across studies described in the research synthesis could not be included in the meta-analysis due to missing data. Because meta-analytic moderator analyses require complete data for the variables being examined, only the following four variables were selected as moderators:

1. (1) Publication year: 2008∼2024

2. (2) Publication type: journal article, book chapter, or dissertation

3. (3) Definition of DoP: explicitly defined or not clearly defined

4. (4) Operationalization of DoP: single-type, dual-type, or multi-type operationalization

Publication information, such as publication year and type, was treated as control variables to improve the accuracy of data analysis, as these were not central to the research focus.

Regarding the definition and operationalization of DoP, we emphasize that these variables function as study-level descriptors of conceptual clarity and implementation fidelity rather than experimental manipulations intended to affect learner performance. Accordingly, we treated them as moderators. It is also imperative to clarify that L2 learning gains in this synthesis were operationalized based on the outcome measures reported in the primary studies, as presented in Appendix 4. These outcomes span four target domains (i.e., grammar, writing, reading, and vocabulary) and encompass constructs such as grammatical knowledge, grammatical accuracy, syntactic knowledge, complexity/accuracy/fluency (CAF), reading comprehension, and vocabulary knowledge. Each study’s ES reflected gains in one of these specific constructs. Our moderator analyses, therefore, examined whether the explicitness with which studies defined or operationalized DoP was associated with differences in the magnitude of these distinct types of learning gains and whether DoP showed differential short-term or long-term effects depending on the domain or construct assessed.

For the definition moderator, some studies explicitly defined DoP with reference to prior literature—most commonly Leow’s (Reference Leow2015) formulation of DoP as “the relative amount of cognitive effort, level of analysis, and elaboration of intake, together with the usage of prior knowledge, hypothesis testing, and rule formation employed in decoding and encoding some grammatical or lexical item in the input” (p. 204)—whereas others used the construct without providing a formal definition.

For the operationalization of DoP, we initially prepared four distinct values for coding based on qualitative findings: (1) cognitive effort and elaboration, (2) hypothesis testing and rule formation, (3) error detection and correction, and (4) attention and time spent on task. However, we found that studies did not restrict themselves to a single operationalization category; instead, they often combined multiple categories, reflecting the complexity of their approaches. To account for this, we coded studies based on the number of operationalization types present in each sample (e.g., single-type, dual-type, or multi-type). For instance, Cerezo et al. (Reference Cerezo, Caras and Leow2016), Leow et al. (Reference Leow, Hsieh and Moreno2008), and Peng (Reference Peng2024) employed “(4) Attention and time spent on task,” while Bergsleithner (Reference Bergsleithner and Leow2019) used “(3) Error detection and correction.” As each study utilized only one category, we coded them as ‘single-type.’ In contrast, Morgan-Short et al. (Reference Morgan-Short, Heil, Botero-Moriarty and Ebert2012) incorporated both “(1) Cognitive effort and elaboration” and “(4) Attention and time spent on task,” leading us to classify it as ‘dual-type.’ Furthermore, Shao and Liu (Reference Shao and Liu2022) employed three distinct operationalization types—“(1) Cognitive effort and elaboration,” “(3) Error detection and correction,” and “(4) Attention and time spent on task”—resulting in a ‘multi-type’ classification. This coding system maintains the conceptual integrity of the original categories while enabling a more structured and quantifiable analysis of whether the breadth of operationalization types influences study outcomes. Details on the number of samples corresponding to each moderator variable are provided in Table 2, and the complete dataset for the quantitative analysis is available at https://osf.io/atgdv/.

Table 2.

Number of categorical values for the three moderator variables

Note: “Publication Year” was coded as a continuous variable (Mean = 2016.04, SD = 4.88).

When available, we coded treatment duration (total minutes/hours or sessions) and the retention interval (days between treatment and assessment) and retained these variables in the dataset. Coverage was limited and uneven (duration: 9/47 short-term within-group, 3/17 short-term between-group; interval: 8/17 within-group, 8/11 between-group, with 7 of the 8 delayed tests at 14 days and 1 at 7 days), so these variables were not entered as moderators.

Statistical analyses

We estimated overall and subgroup average effect sizes using a random-effects meta-analytic model with inverse-variance weighting (see Lee & Lee, Reference Lee and Lee2024a; Lee et al., Reference Lee, Warschauer and Lee2019). Observed effect sizes ${g_i}$ (Hedges’ g, small-sample corrected) served as the dependent variable, and their sampling variances ( ${v_i} = SE_i^2$ ) provided the inverse-variance weights $\left( {{w_i} = 1/\left( {{v_i} + {\tau ^2}} \right)} \right)$ , where ${\tau ^2}$ denotes between-study heterogeneity. The model intercept yields the pooled mean effect; we report estimates overall and for the four Design × Timing cells (within- vs. between-group; short- vs. long-term). To examine moderators, we fit weighted meta-regressions using the same inverse-variance weights to test study-level coding of DoP definition (explicit vs. not explicit) and DoP operationalization (single-, dual-, and multi-type). Given the modest and uneven evidence base (i.e., 12 studies), we did not add further controls to avoid overfitting. All analyses were conducted in Stata 16 with random-effects estimation (pooled effects) and weighted least squares (meta-regression).