Highlights

• • What is already known?

  1. – Meta-analysis is a critical tool to synthesize clinical safety data and evaluate the harms of new interventions.

  2. – Adverse event (AE) data in clinical trials are often incompletely reported, particularly for rare events.

  3. – Conventional approaches, such as complete-case analysis, can lead to biased estimates of AE incidence by ignoring or mishandling missing data.

• • What is new?

  1. – We present Shiny-MAGEC, an R Shiny application that implements a Bayesian meta-analysis model designed for censored AE data.

  2. – This interactive, user-friendly app enables automated input of study-level data with reporting cutoffs, facilitates bias-corrected safety meta-analyses, and compares results with conventional methods without bias correction.

• • Potential impact for RSM readers

  1. – To our knowledge, Shiny-MAGEC is the first software designed for the meta-analysis of drug safety AE data.

  2. – By addressing the limitations of conventional AE reporting, it improves transparency, accuracy, and methodological rigor in safety evidence synthesis, supporting better-informed clinical and regulatory decisions.

2 A brief review of the Bayesian MAGEC model

To address the challenge of left-censored AE data in meta-analysis, a Bayesian MAGEC random-effects meta-analytic model was developed to account for censored AE outcomes.Reference Xinyue, Shouhao and Christine ⁶ It explicitly accounts for censoring by incorporating study-specific reporting thresholds into the likelihood function, enabling more accurate estimation of AE incidence rates while appropriately reflecting the uncertainty introduced by missing data.

Let $Y_i$ denote the count of AEs under a target severity interval (e.g., all-grade or grade 3 and above) in study i, and $ N_i $ represent the number of patients treated in that study. $Y_i$ is assumed to follow a binomial distribution:

$$\begin{align*}Y_i \sim \text{Binomial}(N_i, \theta_i), \end{align*}$$

where $ \theta _i $ represents the study-specific incidence probability. To account for left-censored AE data, the full likelihood function integrates both completely observed and censored data:

$$\begin{align*}\mathcal{L} = \prod_{o=1}^{O} f_Y(y_o) \prod_{l=1}^{L} F_Y(c_l) = \prod_{o=1}^{O} f_Y(y_o) \prod_{l=1}^{L} \sum_{k_l=0}^{c_l} f_Y(k_l), \end{align*}$$

where O represents the set of studies with fully-observed AE counts, and L represents the set of studies where the AE count is left-censored below a study-specific cutoff $c_l$ . The censored probability component, $F_Y(c_l)$ , ensures that the analysis incorporates information about studies where AEs were unreported due to cutoff-based omission rather than true absence.

The incidence probability parameter $\theta _i$ can be modeled based on the link function $g(\cdot )$ that can transform values in the probability space into the real-value space. In the Shiny application, a logit link was used, such that

$$ \begin{align*} g(\theta_i) = \text{logit}(\theta_i) = \mu + \alpha_i + \mathbf{X}_i \boldsymbol{\beta}, \end{align*} $$

where $\mu $ is the overall log-odds of AE incidence, $\alpha _i$ are the study-specific random effects, $\mathbf {X}_i$ is the design matrix of study-level covariates in the ith study, and $\boldsymbol {\beta }$ are the effect parameters corresponding to the study-level factors. In the current version of the Shiny application, specifications of the study-level covariates are not available. The $\alpha _i$ parameters follow conditionally independent normal distributions with mean 0 and variance $\tau ^2$ , where $\tau $ quantifies the between-study heterogeneity of the log odds. A noninformative normal prior distribution $\mathcal {N}(0, v_0^2)$ is placed on the overall mean parameter $\mu $ ( $v_0^2=10^4$ by default). A half-Cauchy prior distribution $C^+(0,A)$ is placed for the between-study standard deviation (SD) parameter $\tau $ . Such weakly-informative prior with its scale parameter A equal to 2.5, 10, or 25 was generally recommended for the SD parameters in hierarchical models.Reference Gelman ¹⁸ The estimation for posterior inference is implemented using Just Another Gibbs Sampling (JAGS),Reference Plummer ¹⁹ and carried out using a data-augmentation strategy for censored data.Reference Qi, Zhou and Plummer ¹¹

This MAGEC model mitigates bias in incidence estimation while incorporating between-study variability. The hierarchical Bayesian framework avoids relying on asymptotic normality for inference, improves estimation stability through MCMC sampling,Reference Hamza, van Houwelingen and Stijnen ²⁰ and enhances the reliability of drug safety assessments. Simulation studies have demonstrated its robustness in small-sample scenarios and for rare events.

3 The R shiny application: Shiny-MAGEC

We have developed an R Shiny application named Shiny-MAGEC that implements the Bayesian meta-analysis model reviewed in Section 2. Users can upload their raw AE data collected from multiple clinical studies to obtain meta-analytic estimates of the overall AE incidence probability and the between-study heterogeneity.

In this section, we provide a walk-through of the application, outlining its main features and functions. Users may refer to Section 4 for an illustrative example using a sample data excerpted from the real data applicationReference Wang, Zhou and Yang ² and subsequently reanalyzed in a methodological study.Reference Xinyue, Shouhao and Christine ⁶ The application consists of an operation panel on the left and a result panel on the right. The right panel is organized into two primary tabs: (1) “User Guide,” providing basic information about the software and showing an overview of the uploaded dataset and (2) “Results,” displaying the analysis outputs. The accepted data format, advanced settings tunable by users, and the result presentations supported by the application are introduced below. The general layout and features are summarized in Figure 1.

Figure 1 An overview of the operation and result panels in Shiny-MAGEC.

3.1 Data preparation

To familiarize themselves with the accepted data format, users can download the sample dataset at the operation panel for reference. The sample data, a subset of an AE meta-analysis dataset,Reference Wang, Zhou and Yang ² is displayed in Table 1. The input data should be provided as a CSV file with mandatory column names: “study,” “N,” “Y,” and “cutoff.” The “study” column is a unique identifier for each study. We recommend using the abbreviated names of different studies (i.e., character strings) to label the studies since these labels will be used in presenting and plotting the study-specific results as shown in Section 3.2. The “N” column is the sample size assigned to the treatment in each study, and the “Y” column (when not missing) represents the observed AE count given the prespecified severity grade and category of interest. “Y” should be coded as NA or left blank (preferred) when an AE count is unreported. The “cutoff” column gives the study-specific left-censoring threshold, which can be extracted from a footnote or methods description in the original publication. Each cutoff is the largest integer not reported; for example, in a study with $N=459$ and a footnote stating that “AE counts $\ge 2\%$ of the treated size are disclosed,” the study-specific cutoff would be $9$ , since any count larger than $459\times 2\%=9.18$ would be reported. Of note, some studies’ reporting threshold might differ by AE categories (e.g., “2017-Peters-J Clin Oncol” in Table 1 Reference Peters, Gettinger and Johnson ²¹ ). Users should carefully collect the “cutoff” information in order to match their AE category of synthesis interest.

Table 1 An example AE data subsetReference Wang, Zhou and Yang ² including the grade 3–5 pneumonitis counts for patients treated with Atezolizumab. “-” indicates unreported (left-censored) in the original publication. The left-censored cutoffs are calculated specific to different studies. Given a cutoff of 0 (e.g., in 2018-Colevas-Ann Oncol), the actual pneumonitis count, though unreported, was exactly 0

After uploading the meta-analysis data following the required format, the application will automatically display the data content in the “Data Overview” section in the “User Guide” tab for a routine check. In addition, the app will automatically check the uploaded data and provides error messages to the user in the following situations: (1) if any data entry in the “N,” “cutoff,” or “Y” columns contains non-numeric symbols instead of numeric values, NA, or a blank; (2) if the column names do not match the required format; and (3) if “cutoff” $>$ “N” or “Y” $>$ “N.” This immediate feedback allows users to verify that their data has been read correctly (e.g., all studies are listed and the columns are interpreted as intended) before proceeding.

4 An illustrative example

We present an illustrative example to demonstrate the use of Shiny-MAGEC based on the built-in sample dataset. A meta-analysis of 125 clinical studies was conducted to evaluate the incidence probabilities of pneumonitis (a type of AE involving inflammation of lung tissue) for several types of PD-1 and PD-L1 inhibitors.Reference Wang, Zhou and Yang ² As shown in Table 1 (Section 3.1), our sample data is a small subset of the full data focused on the incidence of Grade 3–5 pneumonitis for patients who received the PD-L1 drug Atezolizumab. Note that in the practical use of this current version of the Shiny app without accounting for study-level factors, such division of meta-analysis data by specific grade interval and specific AE category, as illustrated in the example, is highly recommended for delivering evidence synthesis within meaningful subgroups.

Once the data are uploaded, a data overview can be found in the “User Guide” tab. By clicking the “Advanced Settings” checkbox on the left, a panel allowing customization of prior hyperparameters and MCMC options will show up. In this illustrative example, we use the default settings. After clicking the “Run Model” button, a progress bar will approximately indicate the models’ execution statuses. In this illustrative example, the running time on an AMD R9-6900HS CPU is approximately 10 seconds. Once the posterior sampling is completed, we can navigate to the “Results” page to review the model outputs.

The main results generated based on the MAGEC methodology are displayed in the “Results” tab, while the parallel results derived from the complete-case analysis are summarized briefly in a separate paragraph only for supplementary comparison purposes. Figure 2 illustrates the summary statistics of the MAGEC model estimates (results associated with the default and alternative prior choices would be paralleled). Furthermore, it will automatically generate paragraphs summarizing the key meta-analytic results as exemplified in Figure 3 for users’ references.

Figure 2 Shiny app outputs of the summary statistics table.

Figure 3 Result descriptions based on the illustrative example.

Additionally, a forest plot will be generated to illustrate both the study-specific and overall AE incidence probability estimates (in percentages), as presented in Figure 4. This visualization will follow established scientific reporting structures exemplified in prior literature.Reference Wang, Zhou and Yang ^{2 ,} Reference Xinyue, Shouhao and Christine ⁶ As demonstrated in prior research, this presentation approach provides a clear and structured summary of results, enhancing interpretability and comparability of the results.

Figure 4 Comprehensive forest plot of the incidence probabilities (in percentage) of Grade 3–5 pneumonitis related to the PD-L1 drug Atezolizumab based on the MAGEC meta-analysis model.

It is straightforward that by incorporating richer information provided by the censored studies, the MAGEC model provides different point and interval estimates compared to the complete-case analysis that solely uses the non-censored studies. By applying the MAGEC modeling approach, the AE incidence is estimated at 0.38% (95% Crl [0.05%, 0.87%]) in this example, whereas the complete-case analysis yields a higher estimate of 0.51% (95% Crl [0.10%, 1.13%]). The discrepancy reflects a 34% over-estimation bias inherent in the complete case approach, reinforcing findings from previous studies that have shown complete case methods tend to inflate incidence estimates due to missing data.Reference Xinyue, Shouhao and Christine ⁶