Identifying studies for inclusion in realist syntheses using title and abstract screening is challenging given the need to unearth explanatory insights to build context–mechanism–outcome configurations. Such insights may only be uncovered through full-text paper reading. To address this issue, a novel approach for identifying studies has been developed called Reverse Chronology Quota Record Screening (RCQRS). Results of database searches are screened in reverse chronology, and in stages, to fill quotas matching the requirements of the review. RCQRS may be beneficial in any of the following circumstances: (a) the timeframe to complete the synthesis is short; (b) the scope of inquiry is not fully defined, (c) the availability of suitable literature is beyond the screening capacity of the reviewers; or (d) the availability of literature is sparse and reviewers seek to extrapolate insights from other areas. In contrast to RCQRS, exhaustive title and abstract screening (i.e., screening entire database results) may hamper study selection due to being overwhelming in volume and time-intensive, resulting in a causally thin cohort of papers for analysis. RCQRS used in stages, and in conjunction with other search strategies (e.g., hand searching, backward citation tracking, and expert solicitation) can support creative, robust analytical insights with causally rich extrapolations. Using the Horizon-EU funded SERENITY study on deprescribing in palliative care as a case example, the benefits and limitations of RCQRS are explored. Finally, a checklist template is offered for teams who wish to reflect on, and transparently report, the use of RCQRS in their realist synthesis.

Automating data extraction from full-text randomized controlled trials for meta-analysis remains a significant challenge. This study evaluates the practical performance of three large language models (LLMs) (Gemini-2.0-flash, Grok-3, and GPT-4o-mini) across tasks involving statistical results, risk-of-bias assessments, and study-level characteristics in three medical domains: hypertension, diabetes, and orthopaedics. We tested four distinct prompting strategies (basic prompting, self-reflective prompting, model ensemble, and customized prompts) to determine how to improve extraction quality. All models demonstrate high precision but consistently suffer from poor recall by omitting key information. We found that customized prompts were the most effective, boosting recall by up to 15%. Based on this analysis, we propose a three-tiered set of guidelines for using LLMs in data extraction, matching data types to appropriate levels of automation based on task complexity and risk. Our study offers practical advice for automating data extraction in real-world meta-analyses, balancing LLM efficiency with expert oversight through targeted, task-specific automation.

A meta-analysis of diagnostic test accuracy (DTA) studies typically synthesizes study-specific test sensitivity ($Se$) and specificity ($Sp$) to quantify the accuracy of an index test of interest. The bivariate linear mixed effects model with logit transformation of $Se$ and $Sp$ (BLMM-Logit) is commonly used to make statistical inferences, but may lead to misleading results due to the need for Haldane–Anscombe correction and an approximate estimation of variance within the study. Alternative models based on the arcsine square root and Freeman–Tukey double arcsine transformation have been proposed to address these issues; however, they still rely on approximate variance estimation, which is suitable only for large sample sizes. The bivariate generalized linear mixed effects model (BGLMM) is another option, but it faces convergence issues with small meta-analyses or sparse primary studies. To address these limitations, we proposed an exact within-study variance calculation method that does not require Haldane–Anscombe correction and is applicable regardless of the transformation used or the number of studies and participants. We evaluated this method against existing approaches using real-life and simulated DTA meta-analyses. The methods were comparable for large meta-analyses. However, BLMM-Logit demonstrated substantial negative bias in estimating variances between studies and consistently underestimated summary $Se$ and $Sp$ in all simulation scenarios. In contrast, the proposed exact methods (Exact-Logit, Exact-ASR, and Exact-FTDA) and BGLMM had minimal bias and better performance metrics, particularly for meta-analyses with sparse primary studies. Thus, the proposed exact methods should be preferred for DTA meta-analyses with small or sparse studies.

The evaluation of the role of face masks in preventing respiratory infections is a paradigm case in synthesising complex evidence (i.e. extensive, diverse, technically specialised, and with multilevel chains of causality). Primary studies have assessed different mask types, diseases, populations, and settings using different research designs. Numerous review teams have attempted to synthesise this literature, in which observational (case–control, cohort, cross-sectional) and ecological studies predominate. Their findings and conclusions vary widely.

This article critically examines how 66 systematic reviews dealt with mask efficacy studies. Risk-of-bias tools produced unreliable assessments when—as was often the case—review teams lacked methodological expertise or topic-specific understanding. This was especially true when datasets were large and heterogeneous, with multiple biases playing out in different ways and requiring nuanced adjustments. In such circumstances, tools were sometimes used crudely and reductively rather than to support close reading of primary studies and guide expert judgments. Various moves by reviewers—excluding observational evidence altogether, assessing risk but not direction of biases, omitting distinguishing details of primary studies, and producing meta-analyses that combined studies of different designs or included studies at critical risk of bias—served to obscure important aspects of heterogeneity, resulting in bland and unhelpful summary statements.

We draw on philosophy to question the formulaic use of generic risk-of-bias tools, especially when the primary evidence demands expert understanding and tailoring of study quality questions to the topic. We call for more rigorous training and oversight of reviewers of complex evidence and for new review methods designed specifically for such evidence.

1. Meta-analyses are a reliable method for a quantitative research synthesis. They are, however, prone to specific biases that can be introduced in the process. Such a bias could exist if primary literature produces similar results if coming from the same authors. Authorship network bias is the non-independence of effect sizes introduced by the overlap of authors of primary studies. If not accounted for, it can severely impact the quality of meta-analysis and the conclusions drawn from it.

2. To account for such non-independence, multilevel models with author clusters as an additional hierarchy level were recently suggested. We propose a new method for the detection of non-independent effect sizes based on authorship networks and for their correction.

3. An analysis of simulated data demonstrates the effectiveness of the here-suggested new method. We further applied our new method to nine exemplary meta-analyses.

4. Our new method for detection and effective correction can be easily integrated in existing meta-analysis workflows, using the functionality already offered by R’s metafor package.

5. Our goal is to enhance the reliability of meta-analyses by highlighting potential authorship network bias and offering a method to address this often-overlooked bias.

A key output of network meta-analysis (NMA) is the relative ranking of treatments; nevertheless, it has attracted substantial criticism. Existing ranking methods often lack clear interpretability and fail to adequately account for uncertainty, overemphasizing small differences in treatment effects. We propose a novel framework to estimate treatment hierarchies in NMA using a probabilistic model, focusing on a clinically relevant treatment-choice criterion (TCC). Initially, we define a TCC based on smallest worthwhile differences (SWD), converting NMA relative treatment effects into treatment preference format. These data are then synthesized using a probabilistic ranking model, assigning each treatment a latent “ability” parameter, representing its propensity to yield clinically important and beneficial true treatment effects relative to the rest of the treatments in the network. Parameter estimation relies on the maximum likelihood theory, with standard errors derived asymptotically from the Hessian matrix. To facilitate the use of our methods, we launched the R package mtrank. We applied our method to two clinical datasets: one comparing 18 antidepressants for major depression and another comparing 6 antihypertensives for the incidence of diabetes. Our approach provided robust, interpretable treatment hierarchies that account for a concrete TCC. We further examined the agreement between the proposed method and existing ranking metrics in 153 published networks, concluding that the degree of agreement depends on the precision of the NMA estimates. Our framework offers a valuable alternative for NMA treatment ranking, mitigating overinterpretation of minor differences. This enables more reliable and clinically meaningful treatment hierarchies.

Meta-analysis synthesizes evidence from multiple randomized clinical trials and informs evidence-based practices across various medical domains. Recently, causally interpretable meta-analysis has been proposed and applied to treatment evaluations for target populations, requiring individual participant data (IPD). Standard meta-analysis assumes transportability or exchangeability of a (conditional) relative effect (such as relative risk or odds ratio), which may be violated when the relative effects are correlated with the baseline risks across clinical trials. In addition, the weighted average of some study-specific effect measures such as the (log) odds ratios or the (log) hazard ratios is non-collapsible and does not correspond to any target population. Furthermore, when the randomization ratios between treated versus untreated arms vary across trials, confounding bias may occur. To address these challenges, we propose a causal meta-analysis (CMA) framework using only aggregated data, enabling causally interpretable and accurate estimation for different target populations. The CMA adjusts its weights for treatment effect across various target populations, including the average treatment effect (ATE), the ATE on the treated (ATT) population, the ATE on the control (ATC) population, and the ATE in the overlap (ATO) population. Mathematically, we discover the connection between traditional meta-analysis estimators and CMAs. For example, the Mantel–Haenszel weighted meta-analysis is equivalent to the CMA with ATO.

The 2019 ICH E9(R1) addendum highlights the importance of estimands, including the specification of post-randomization events that may affect the interpretation of clinical trial outcomes (i.e., intercurrent events; ICEs) and strategies to handle these events. Compared to trial protocols, there is limited discussion of estimands in the context of evidence synthesis. We conducted a comprehensive review of the Cochrane Library for pairwise meta-analyses of immuno-, targeted, hormone, and other novel oncology therapies. Dates were restricted to 2021 and onwards to allow time for addendum adoption. Outcomes of interest were progression-free survival (PFS) and overall survival (OS). Information on treatment switching and analytic strategies to address treatment switching were extracted from each meta-analysis and the RCTs they included. Out of 162 oncology reviews published in the Cochrane Library since 2021, eight pairwise meta-analyses and 68 RCTs met selection criteria. Most RCTs were Phase 3 (68%; n = 46) and/or open-label (76%; n = 52). More than half of RCTs explicitly allowed switching (59%; n = 40), while more than one third (38%; n = 26) did not report on treatment switching. Among trials that allowed treatment switching, censoring mechanisms for treatment switching varied in analyses of PFS. No included RCTs censored OS at the time of treatment switching. Despite the high prevalence of treatment switching in included trials, none of the identified meta-analyses addressed treatment switching analytically. Poor reporting regarding treatment switching in the RCTs themselves hinders the utility of aggregate-level meta-analyses. To ensure accurate interpretation of meta-analytic results, improved reporting of ICEs and ICE handling strategies is needed.

Meta-analytic-predictive (MAP) priors have been proposed as a generic approach to deriving informative prior distributions, where external empirical data are processed to learn about certain parameter distributions. The use of MAP priors is also closely related to shrinkage estimation (also sometimes referred to as dynamic borrowing). A potentially odd situation arises when the external data consist only of a single study. Conceptually, this is not a problem, it only implies that certain prior assumptions gain in importance and need to be specified with particular care. We outline this important, not uncommon special case and demonstrate its implementation and interpretation based on the normal–normal hierarchical model. The approach is illustrated using example applications in clinical medicine.

The exponential growth of scientific literature poses increasing challenges for evidence synthesis. Systematic reviews (SRs) usually rely on keyword-based database searches, which are limited by inconsistent terminology and indexing delays. Citation searching—identifying studies that cite or are cited by known relevant articles—offers a complementary route to uncover additional evidence but remains poorly automated and integrated into screening workflows. We developed BibliZap, an open-source, fully automated citation-searching tool built on Lens.org data, performing multi-level forward and backward citation searches with relevance-based ranking. Its performance was evaluated across 66 published SRs, comparing five approaches: (1) PubMed-only searches; (2) PubMed followed by BibliZap restricted to the top 500 ranked results; (3) PubMed followed by full BibliZap screening; and (4–5) two exploratory early-stop strategies where BibliZap was initiated after identifying the first or the first three PubMed relevant records. The primary outcome was sensitivity, with secondary assessments of screening workload and precision. When used after PubMed screening, BibliZap increased mean sensitivity from 75% to 97%, achieving complete recall in over half of the reviews. Screening only the top 500 outputs still allowed over 90% of reviews to reach or exceed 80% recall. BibliZap recovered a median of three additional included articles per review, not retrieved by PubMed, while adding a median of 6,450 additional records. Citation searching via BibliZap enhances the completeness of evidence retrieval in SRs based on restricted database searches and supports transparent, scalable workflows adaptable to rapid and exploratory review contexts.

In random-effects meta-analysis, the between-study heterogeneity variance, $\tau ^2$, is often reported but is not easy to interpret. For meta-analyses of differences (such as mean differences, standardized mean differences, or risk differences), the standard deviation (SD), $\tau $, indicates the extent to which studies’ true effects vary about their average. For meta-analyses of (natural) log-transformed measures of effect (such as log risk ratios [RRs]), we explain how the geometric SD, $\exp (\tau )$, is helpful to understand how untransformed measures (such as RRs) vary multiplicatively about their average. We recommend that authors and software developers report $\tau $ for differences and $\exp (\tau )$ for ratios, rather than $\tau ^2$. This will facilitate the interpretation of the magnitude of heterogeneity values, for example, the interpretation of heterogeneity estimates and confidence intervals beyond simple binary statements about the presence or absence of heterogeneity.

Network meta-analysis (NMA) is a vital methodology for synthesizing evidence across multiple treatments and informing medical decision-making. However, effective visualization and interpretation of results from large networks of interventions remain challenging, particularly for non-specialists. NMAstudio 2.0 is an innovative, interactive web application designed to address these difficulties by streamlining NMA workflows and enhancing result visualization. Developed using Python and R, NMAstudio 2.0 seamlessly integrates with established NMA frameworks. Our exemplar application of NMAstudio 2.0 using a Cochrane Review comparing several treatments for chronic plaque psoriasis demonstrates its capacity to facilitate all crucial steps of an NMA. The application features an intuitive interface for uploading data, automating analyses, generating interactive visualizations such as network diagrams, forest plots, ranking plots, and producing unique outputs like boxplots for transitivity checks and bidimensional forest plots. Most outputs are dynamically linked with the network diagram, enabling users to interactively explore evidence networks, apply advanced filtering, and highlight specific features by selecting nodes or edges within the diagram. While NMAstudio 2.0 aims to simplify NMAs, it also incorporates steps during the data upload process to mitigate the risk of producing poorly reported NMAs. NMAstudio 2.0 represents a significant step forward in improving the usability and accessibility of NMA, offering researchers a robust, versatile platform for evidence synthesis. Its integration of advanced features with an emphasis on user experience positions it as a valuable resource for enhancing decision-making and promoting evidence-based practice across diverse contexts.