To estimate accurately the parameters of a regression model, the sample size must be large enough relative to the number of possible predictors for the model. In practice, sufficient data is often lacking, which can lead to overfitting of the model and, as a consequence, unreliable predictions of the outcome of new patients. Pooling data from different data sets collected in different (medical) centers would alleviate this problem, but is often not feasible due to privacy regulation or logistic problems. An alternative route would be to analyze the local data in the centers separately and combine the statistical inference results with the Bayesian Federated Inference (BFI) methodology. The aim of this approach is to compute from the inference results in separate centers what would have been found if the statistical analysis was performed on the combined data. We explain the methodology under homogeneity and heterogeneity across the populations in the separate centers, and give real life examples for better understanding. Excellent performance of the proposed methodology is shown. An R-package to do all the calculations has been developed and is illustrated in this article. The mathematical details are given in the Appendix.

Mapping reviews (MRs) are crucial for identifying research gaps and enhancing evidence utilization. Despite their increasing use in health and social sciences, inconsistencies persist in both their conceptualization and reporting. This study aims to clarify the conceptual framework and gather reporting items from existing guidance and methodological studies. A comprehensive search was conducted across nine databases and 11 institutional websites, including documents up to January 2024. A total of 68 documents were included, addressing 24 MR terms and 55 definitions, with 39 documents discussing distinctions and overlaps among these terms. From the documents included, 28 reporting items were identified, covering all the steps of the process. Seven documents mentioned reporting on the title, four on the abstract, and 14 on the background. Ten methods-related items appeared in 56 documents, with the median number of documents supporting each item being 34 (interquartile range [IQR]: 27, 39). Four results-related items were mentioned in 18 documents (median: 14.5, IQR: 11.5, 16), and four discussion-related items appeared in 25 documents (median: 5.5, IQR: 3, 13). There was very little guidance about reporting conclusions, acknowledgments, author contributions, declarations of interest, and funding sources. This study proposes a draft 28-item reporting checklist for MRs and has identified terminologies and concepts used to describe MRs. These findings will first be used to inform a Delphi consensus process to develop reporting guidelines for MRs. Additionally, the checklist and definitions could be used to guide researchers in reporting high-quality MRs.

A fundamental pillar of science is the estimation of the effect size of associations. However, this task is sometimes difficult and error-prone. To facilitate this process, the R package metaConvert automatically calculates and flexibly converts multiple effect size measures. It applies more than 120 formulas to convert any relevant input data into Cohen’s d, Hedges’ g, mean difference, odds ratio, risk ratio, incidence rate ratio, correlation coefficient, Fisher’s r-to-z transformed correlation coefficient, variability ratio, coefficient of variation ratio, or number needed to treat. Researchers unfamiliar with R can use this software through a browser-based graphical interface (https://metaconvert.org/). We hope this suite will help researchers in the life sciences and other disciplines estimate and convert effect sizes more easily and accurately.

The number of published systematic reviews has increased over the last years, with a non-negligible proportion displaying methodological concerns. We aimed to develop and evaluate a tool to assess the reported methodological quality of medical systematic reviews. The developed tool (ReMarQ) consists of 26 dichotomous items. We applied an item response theory model to assess the difficulty and discrimination of the items and decision tree models to identify those items more capable of identifying systematic reviews with higher reported methodological quality. ReMarQ was applied to a representative sample of medical systematic reviews (excluding those published in the Cochrane Database of Systematic Reviews) to describe their methodological quality and identify associated factors. We assessed 400 systematic reviews published between 2010 and 2020, of which 196 (49.0%) included meta-analysis. The most discriminative items were (i) conducting a risk of bias assessment, (ii) having a published protocol and (iii) reporting methods for solving disagreements. More recent systematic reviews (adjusted yearly RR=1.03; 95%CI=1.02 −1.04, p<0.001) and those with meta-analysis (adjusted RR=1.34; 95%CI=1.25 −1.43, p<0.001) were associated with higher reported methodological quality. Such an association was not observed with the journal impact factor. The items most frequently fulfilled were (i) reporting search dates, (ii) reporting bibliographic sources and (iii) searching multiple electronic bibliographic databases. ReMarQ, consisting of dichotomous items and whose application does not require subject content expertise, may be important (i) in supporting an efficient quality assessment of systematic reviews and (ii) as the basis of automated processes to support that assessment.

Systematic reviews play important roles but manual efforts can be time-consuming given a growing literature. There is a need to use and evaluate automated strategies to accelerate systematic reviews. Here, we comprehensively tested machine learning (ML) models from classical and deep learning model families. We also assessed the performance of prompt engineering via few-shot learning of GPT-3.5 and GPT-4 large language models (LLMs). We further attempted to understand when ML models can help automate screening. These ML models were applied to actual datasets of systematic reviews in education. Results showed that the performance of classical and deep ML models varied widely across datasets, ranging from 1.2 to 75.6% of work saved at 95% recall. LLM prompt engineering produced similarly wide performance variation. We searched for various indicators of whether and how ML screening can help. We discovered that the separability of clusters of relevant versus irrelevant articles in high-dimensional embedding space can strongly predict whether ML screening can help (overall R = 0.81). This simple and generalizable heuristic applied well across datasets and different ML model families. In conclusion, ML screening performance varies tremendously, but researchers and software developers can consider using our cluster separability heuristic in various ways in an ML-assisted screening pipeline.

Researchers would often like to leverage data from a collection of sources (e.g., meta-analyses of randomized trials, multi-center trials, pooled analyses of observational cohorts) to estimate causal effects in a target population of interest. However, because different data sources typically represent different underlying populations, traditional meta-analytic methods may not produce causally interpretable estimates that apply to any reasonable target population. In this article, we present the CausalMetaR R package, which implements robust and efficient methods to estimate causal effects in a given internal or external target population using multi-source data. The package includes estimators of average and subgroup treatment effects for the entire target population. To produce efficient and robust estimates of causal effects, the package implements doubly robust and non-parametric efficient estimators and supports using flexible data-adaptive (e.g., machine learning techniques) methods and cross-fitting techniques to estimate the nuisance models (e.g., the treatment model, the outcome model). We briefly review the methods, describe the key features of the package, and demonstrate how to use the package through an example. The package aims to facilitate causal analyses in the context of meta-analysis.

Bibliographic aggregators like OpenAlex and Semantic Scholar offer scope for automated citation searching within systematic review production, promising increased efficiency. This study aimed to evaluate the performance of automated citation searching compared to standard search strategies and examine factors that influence performance. Automated citation searching was simulated on 27 systematic reviews across the OpenAlex and Semantic Scholar databases, across three study areas (health, environmental management and social policy). Performance, measured by recall (proportion of relevant articles identified), precision (proportion of relevant articles identified from all articles identified), and F1–F3 scores (weighted average of recall and precision), was compared to the performance of search strategies originally employed by each systematic review. The associations between systematic review study area, number of included articles, number of seed articles, seed article type, study type inclusion criteria, API choice, and performance was analyzed. Automated citation searching outperformed the reference standard in terms of precision (p < 0.05) and F1 score (p < 0.05) but failed to outperform in terms of recall (p < 0.05) and F3 score (p < 0.05). Study area influenced the performance of automated citation searching, with performance being higher within the field of environmental management compared to social policy. Automated citation searching is best used as a supplementary search strategy in systematic review production where recall is more important that precision, due to inferior recall and F3 score. However, observed outperformance in terms of F1 score and precision suggests that automated citation searching could be helpful in contexts where precision is as important as recall.