Elaborating on the discussion for applied linguistics – Quality of reporting

In our 2025 article, we were unable to fully unpack all five sources of research waste due to space constraints. Therefore, this article focuses specifically on the fifth source: reporting quality, which received scant attention in the original piece. In this paper, we conceptualize reporting quality as a key dimension of the broader construct of research transparency. In the context of research dissemination, transparency denotes clear (unambiguous) reporting of how a study was designed, conducted, and interpreted, with sufficient information provision to enable synthesis, replication, and reproducibility (see Marsden, Reference Marsden, McKinley and Rose2020). Foundational to this is openly and clearly communicating research aims and questions together with comprehensive documentation of instruments, participant characteristics, contextual variables, data, code, and analytic procedures in line with open science norms in applied linguistics (Plonsky, Reference Plonsky and Plonsky2024). Notably, we define reporting quality not in terms of rhetorical polish or stylistic elegance, but rather in reference to comprehensiveness and clarity – that is, full and accurate reporting of key methodological information, potentially in reference to research reporting guidelines (e.g., American Psychological Association, 2025). This is a prerequisite for broader transparency practices, including data sharing, replication, and cumulative knowledge building (Plonsky, Reference Plonsky and Plonsky2024).

Reporting quality is arguably the most straightforward of the five sources of waste for the applied linguistics research community to address, partly because successful models already exist in other fields. Improving research reporting quality in applied linguistics should not be difficult, costly, or time-consuming. However, heeding Čulina et al.’s (Reference Čulina, Purgar, Tarandek, Vidić and Žanko2026) warning of systemic challenges ahead, it will require commitment, compliance, and follow-through on the part of authors and publishers of research. Assuming this kind of momentum can be generated, we believe that the use of design-specific, applied linguistics-focused reporting guidelines stands to improve the completeness and transparency of research reporting, ameliorate the associated waste, and improve the informative value of research in the field.

Why reporting quality matters

With interdisciplinarity in mind, we look to an observation by education researcher Stephen Gorard:

Given the substantial resources required to conduct research – financial investment, researcher time, and the goodwill of participants – it is troubling that final reports sometimes lack the information necessary for their findings to be understood or used. Even well-designed studies can become wasteful if they are not reported clearly.

High-quality reporting matters for several groups. For systematic reviewers, inadequate reporting impedes decisions about eligibility, interpretation, and synthesis. For practitioners and policymakers, poor reporting limits their ability to interpret or apply findings, meaning that publicly funded research may fail to support public benefit. For replication researchers, insufficient reporting hampers attempts to reproduce or scrutinize prior findings. Across the social sciences, lack of methodological transparency has been highlighted as a contributor to the replication crisis (e.g., Forbes et al., Reference Forbes, Travers, Johnson, Cox, Syed, Brodhead and Quigley2023; Wiggins & Christopherson, Reference Wiggins and Christopherson2019), and similar concerns have been raised in applied linguistics (McManus, Reference McManus2024). When methods or analyses are insufficiently described, other researchers cannot understand or replicate a study and its contribution to cumulative knowledge is weakened.

Examples and consequences of poor reporting in applied linguistics

Incomplete or opaque reporting is most clearly revealed in the process of research synthesis. One purpose of systematic reviews is not just to conclude what a body of evidence says about a phenomenon, but to assess the level of trust one can have in that conclusion. This is typically achieved by assessing the assembled literature against a quality appraisal rubric such as the Cochrane Risk of Bias tool (Cochrane, n.d.) or the Mixed Methods Appraisal Tool (Hong et al., Reference Hong, Fàbregues, Bartlett, Boardman, Cargo, Dagenais, Gagnon, Griffiths, Nicolau, O’Cathain, Rousseau, Vedel and Pluye2018). These rubrics ask the reviewer to identify key information from eligible reports to help judge the internal validity of each study. These items differ based on the type of research being appraised and the specific tool being used. By way of example, when appraising experiments and quasi-experiments, reviewers are often directed to identify how many people took part in the study, whether any attrition occurred, what the prespecified outcome measures were, how comparison groups were generated, and so on. Having clarity on these points is crucial to understanding what was done in the study, critically evaluating it, and interpreting its implications for practice and theory. Simply put, a study that cannot be judged because of unclear reporting contributes little to our understanding of a phenomenon.

Examples exist in much social science research, and applied linguistics is no different. In their review of research on the effects of singing songs on linguistic development, Hamilton et al. (Reference Hamilton, Schulz, Chalmers and Murphy2024) note several omissions in the primary reports: failure to report the research question being addressed, failure to name or describe outcome measures, incomplete data reporting, and failure to describe the intervention or the comparator. Huang and Chalmers (Reference Huang and Chalmers2023) found similar shortcomings in their systematic review of pedagogical translanguaging in EFL settings. They noted missing outcome data, inadequately described interventions and comparators, failure to describe how groups were allocated to conditions, and lack of clarity over the duration of interventions. In both cases, in part because of incomplete reporting, these reviewers were unable to confidently articulate the pedagogical or theoretical implications of the research they had assembled.

In a different context, de Bruin et al. (Reference de Bruin, Treccani and Della Sala2014) identified tendencies to incomplete reporting of outcomes in research investigating relationships between bilingualism and executive functioning. Their comparison of conference papers on this topic with their subsequent published article versions suggested that, while planned outcomes may have been many, reported outcomes tended to be few. This suggests selective outcome reporting – that is, performing a number of tests but reporting only those that reinforce the orthodoxy (in this case that there is a positive association between bilingualism and executive functioning). While “questionable research practices” (John et al., Reference John, Loewenstein and Prelec2012) cannot be ruled out as an explanation for this artifact in their data, it is possible that some authors were not aware of the importance of reporting information for all outcomes sought. de Bruin et al. (Reference de Bruin, Treccani and Della Sala2014) and de Bruin & Della Sala (Reference de Bruin, Della Sala, Schwieter and Paradis2019) also observed that published papers in which multiple outcomes were reported (including those with null or negative findings) were more likely to challenge the orthodoxy than those with only one (confirmatory) outcome reported.

So, at a minimum, complete and transparent reporting helps consumers of research draw informed conclusions about a piece of research. In some cases, it can turn our understanding of a field on its head.

A proposed solution

Although not a panacea, adopting standard design-specific reporting guidelines for applied linguistics research would help address much of the problem. Reporting guidelines are structured checklists specifying essential information that authors should include to ensure that research reporting is comprehensive, replicable, and usable. Their value has been demonstrated most clearly in healthcare, where reporting guidelines have transformed the clarity and completeness of published research.

Lessons from healthcare: CONSORT and EQUATOR

In 1996, the first reporting guideline for use in healthcare research, the Consolidated Standards of Reporting Trials (CONSORT; Begg et al., Reference Begg, Cho, Eastwood, Horton, Moher, Olkin, Pitkin, Rennie, Schulz, Simel and Stroup1996), was promulgated by healthcare researchers frustrated by what they described as a “wide chasm between what a trial should report and what is actually published in the literature” (p. 637). Begg and colleagues established the SORT group (Standards of Reporting Trials), consisting of trials methodologists, journal editors, clinical epidemiologists, and statisticians, who met for a Delphi-style discussion over two days to agree on a set of items considered essential for the transparent reporting of clinical trials. At the time, they became aware of another group doing similar work, so they brought the two groups together to consolidate their efforts. Begg and colleagues note that this was not just a pragmatic decision. It increased the potential for consensus about the content of the guideline, which they hoped would maximize its uptake in the wider research community. The result was the CONSORT Statement: a checklist of 21 essential items for trial reporting focused on enabling readers to assess a study’s internal and external validity.

They acknowledged at the time that CONSORT would need updating as new sources of potential bias in the conduct of trials were identified and as methodological approaches to experimental research evolved. The CONSORT group was thereby established and has periodically updated CONSORT to account for this evolution (Hopewell et al., Reference Hopewell, Chan, Collins, Hróbjartsson, Moher, Schulz, Tunn, Aggarwal, Berkwits, Berlin, Bhandari, Butcher, Campbell, Chidebe, Elbourne, Farmer, Fergusson, Golub, Goodman, Hoffmann and Boutron2025; Moher et al., Reference Moher, Schulz and Altman2001; Schulz et al., Reference Schulz, Altman and Moher2010). In addition, the group has produced a number of extensions to the guideline for specialized trial designs (see consort-sprit.org).

CONSORT has been widely influential: it is endorsed by major medical journals, used internationally, and has demonstrably improved reporting completeness of trials in healthcare (Shamseer et al., Reference Shamseer, Hopewell, Altman, Moher and Schulz2016), especially when endorsed by journals. For example, Turner et al. (Reference Turner, Shamseer, Altman, Weeks, Peters, Kober, Dias, Schulz, Plint and Moher2012) synthesized evidence from studies that compared the completeness of reporting in journals that did or did not endorse CONSORT. They operationalized “endorsement” as “(a) requirement or recommendation in journal’s ‘Instructions to Authors’ to follow CONSORT guidelines; (b) journal editorial statement endorsing the CONSORT Statement; or (c) editorial requirement for authors to submit a CONSORT checklist and/or flow diagram with their manuscript” (n.p.). They found that, while endorsement did not guarantee completeness of reporting across the board, papers in CONSORT-endorsing journals were more likely to be more completely and transparently reported.

Building on its success, the CONSORT group founded the Enhancing the Quality and Transparency Of health Research (EQUATOR) Network in 2006 (see equator-network.org). The network began by bringing together reporting guideline development groups, journal editors, peer reviewers, medical writers, and funders to understand the processes and challenges associated with guideline development. This collaborative approach informed its ongoing mission to support and promote “accurate, complete, and transparent reporting of all health research studies to support research reproducibility and usefulness” (EQUATOR Network, n.d.). EQUATOR now hosts hundreds of reporting guidelines covering diverse research designs, from clinical trials and systematic reviews to observational studies and qualitative research, and provides support for guideline development and dissemination.

CONSORT has not been without criticism. The checklist has been described as overly complicated and lacking the nuance needed for specialist trial designs (although extensions have addressed this to some extent). Even in CONSORT-endorsing journals there are gaps in the completeness of reporting. Kwong et al. (Reference Kwong, Nguyen, Khondker, Li and Kulkarni2025, p. 1) suggest that this “reflects a systemic failure: researchers are either unaware of these guidelines or are choosing not to implement them, and journals appear unwilling or unable to enforce them.” Nonetheless, together, CONSORT and EQUATOR illustrate the potential impact of coordinated, consensus-based reporting standards developed across institutions, disciplines, and national boundaries. They also demonstrate that reporting guidelines can gain traction when journal editors endorse them and when checklists are explicitly required at submission.

Reporting guidelines in applied linguistics

Our field does not currently have widely adopted, design-specific, applied linguistics-focused reporting guidelines comparable to those provided by the EQUATOR network. Manuscript preparation guidelines for applied linguistics journals often refer to American Psychological Association (APA) style guidelines (2020) or to an in-house style. However, the emphasis is primarily on ensuring presentational consistency through following formatting and citation conventions. To our knowledge, no journal in our field links explicitly to APA’s Journal Article Reporting Standards (JARS), although some do provide their own reporting guidance. Consequently, the adoption of APA conventions is driven mainly by the desire to standardize surface presentation rather than to enhance the transparency, completeness, and rigor of research reporting.

As a temperature check, we searched the instructions to authors of a random selection of 10 Q1 applied linguistics journalsFootnote ¹ for reference to any reporting guidelines, applied linguistics-specific or otherwise. The picture is mixed. Some journals in our sample provide specific guidance on the reporting of some research designs in their instructions to authors. Others do not.

Guidance on word length, referencing style, use of pronouns, definitions of sex and gender, funding sources, use of inclusive language, and data sharing was common. However, more specific advice on the structure and content of reports was harder to find. We found no such guidance in the instructions to authors’ pages of Applied Linguistics, Studies in Second Language Acquisition, Annual Review of Applied Linguistics, IRAL, and System. Journal of Second Language Writing states that authors should “Divide your manuscript into clearly defined sections covering all essential elements using headings” (n.d.) but does not say what those essential elements are. Interestingly, given our argument that applied linguistics can learn from healthcare research, Language Teaching Research “recommends that authors follow the Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals formulated by the International Committee of Medical Journal Editors” and that manuscripts “must follow the relevant EQUATOR Network reporting guidelines” (Language Teaching Research, n.d., emphasis original). Language Testing has detailed guidance for the reporting of systematic reviews, recommending that authors include a copy of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist (Page et al., Reference Page, McKenzie, Bossuyt, Boutron, Hoffmann, Mulrow, Shamseer, Tetzlaff, Akl, Brennan, Chou, Glanville, Grimshaw, Hróbjartsson, Lalu, Li, Loder, Mayo-Wilson, McDonald, McGuinness, Stewart and Moher2021) with submissions, but does not have similar guidance for primary research. The first reference to an applied linguistics-specific guideline that we found was in Language Learning: “When reporting research using quantitative and statistical methods, authors submitting to Language Learning should follow the Guidelines for Reporting Quantitative Methods and Results in Primary Research by Norris, Plonsky, Ross, and Schoonen” (Godfroid & Trofimovich, Reference Godfroid and Trofimovich2022, p. 4). This document was commissioned by the board of directors of Language Learning and includes detailed guidance on what authors should include in the methods and results sections of their manuscripts. Similarly, TESOL Quarterly directs prospective authors to an article entitled TESOL Quarterly Research Guidelines (Mahboob et al., Reference Mahboob, Paltridge, Phakiti, Wagner, Starfield, Burns, Jones and De Costa2016), which provides guidance on what to include in the different sections in reports of experimental research, survey research, ethnographic research, discourse analysis, and practitioner research. Though not in our sample of Q1 journals, we are also aware that a new journal, Research Synthesis in Applied Linguistics, has produced the Synthesis Methods and Reporting Tool (SMART; Chong, Reference Chong2025), designed specifically for syntheses in applied linguistics, which it encourages authors to adhere to when submitting syntheses for publication.

While far from exhaustive, our survey suggests that reporting guidance across the field remains uneven and journal-specific. Existing guidance typically applies to a small selection of design types and there is limited evidence of widespread or consistent uptake. As the editor of Language Teaching notes, such guidance that does exist is “more honoured in the breach than the observance” (G. Porte, personal communication, November 14, 2025). This patchiness contrasts sharply with the coordinated, field-wide approach seen in healthcare and contributes to variability in the clarity and completeness of published research. That said, the green shoots of change are evident. Some editorial boards recognize the need for clearer guidance and have begun to act.

Toward design-specific reporting guidelines in applied linguistics

Given this burgeoning recognition of the value of clear and complete reporting, we argue that applied linguistics is well positioned to develop its own standard reporting guidelines, modeled on the successful consensus-building approach of CONSORT and EQUATOR. Their success can be attributed to their international, collegiate, and collaborative approaches, which benefit from representation of a variety of relevant domains of expertise and stakeholders. This provides not only gravitas but also an assurance that the resulting guidelines are consensus-driven, not externally imposed. For CONSORT, the consensus is further underscored by the fact that every iteration of the guideline since 2001 has been simultaneously published in five high-profile medical journals (The BMJ, JAMA, The Lancet, Nature Medicine, and PLOS Medicine) as well as a number of specialist journals. It is no small victory for champions of clear and complete reporting that the CONSORT group was able to break through usual proprietary publishing practices to ensure that this important contribution was disseminated widely. The contributions of journal editors here will have been key.

We have noted that, where they exist (TESOL Quarterly, Language Learning), guidelines for applied linguistics research have not been created through consensus beyond the original commissioning editorial boards and the authors they commissioned. An exception is the approach taken by Chong (Reference Chong2025), who submitted drafts of his SMART guideline for reporting research syntheses in RSAL to 24 applied linguists/synthesis specialists for scrutiny and comment before finalizing it. We can envisage a project, modeled on the CONSORT/EQUATOR approach, that builds on the consensus-oriented strategy adopted by Chong (Reference Chong2025), to bring together individuals with the expertise, willingness, and clout to contribute to the creation of standard reporting guidelines for applied linguistics research.

Academic journals, potentially supported by learned societies, have a key role to play in driving positive change in research reporting practices. For example, in an effort to improve research reporting quality and the transparency of evidence syntheses, Language Testing, where the second author serves as co-editor, solicited methodological feedback from expert reviewers on submitted manuscripts. The resulting insights formed the basis for a top-down requirement that authors include a PRISMA flow diagram, along with several additional recommendations (e.g., submitting a PRISMA checklist, prospectively registering the synthesis, conducting and reporting a methodological quality appraisal), with the intention of moving from these initial recommendations to more standardized reporting requirements after a trial period (Isaacs & Yan, Reference Isaacs and Yan2025). The approaches taken by Language Testing and RSAL offer examples of how journals can contribute to advancing reporting standards, with implementation potentially ranging from light-touch suggestions to mandatory requirements. Training workshops for researchers should complement such initiatives by helping embed these practices into the research culture, as could the earlier integration of stakeholder perspectives (e.g., through consensus-building methods) to identify effective ways forward and secure broader buy-in.

Such a project would begin with a systematic review of existing reporting guidance in the field, mapping what already exists (such as the guidelines we located in TESOL Quarterly and Language Learning), and scrutinizing their content. This would help identify common areas of agreement as well as gaps. This review would also help identify key individuals and stakeholder groups with relevant methodological expertise, editorial influence, and disciplinary knowledge.

A working group should then be convened, including methodologists, research synthesists, practitioners, and editors of major applied linguistics journals straddling different content areas and traditions. Through structured consultation and consensus-building processes, the group would determine whether design-specific, applied linguistics-focused reporting guidelines are feasible and warranted and, if so, what they should include. To maximize adoption, journal editors should be involved throughout, from shaping the guidelines to supporting their dissemination and endorsement. As in healthcare, simultaneous publication across several leading journals could help signal consensus and encourage consistent uptake. Buy-in from learned societies such as the British Association of Applied Linguistics and the American Association of Applied Linguistics will also help to establish authority and consensus around their use.

Such an initiative would strengthen the field by addressing a key source of avoidable research waste. Research that is incompletely reported cannot be synthesized, replicated, or used in evidence-informed policy or practice. By creating and adopting design-specific, applied linguistics-focused reporting guidelines, applied linguistics can ensure that research is communicated more clearly, is usable by practitioners, researchers, and policymakers, and is able to contribute fully to cumulative knowledge.