Hostname: page-component-89b8bd64d-z2ts4 Total loading time: 0 Render date: 2026-05-09T14:25:51.255Z Has data issue: false hasContentIssue false
  • English
  • Français

NMAstudio 2.0: An interactive tool for network meta-analysis to enhance understanding, interpretation, and communication of the findings

Published online by Cambridge University Press:  06 March 2026

Tianqi Yu ,
Silvia Metelli ,
Theodoros Papakonstantinou  and
Anna Chaimani Open the ORCID record for Anna Chaimani [Opens in a new window]
Tianqi Yu
Affiliation:
Center of Research in Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, Inserm, France
Silvia Metelli
Affiliation:
Center of Research in Epidemiology and Statistics (CRESS-U1153), Université Paris Cité, Inserm, France
Theodoros Papakonstantinou
Affiliation:
Laboratory of Hygiene, Social & Preventive Medicine and Medical Statistics, Aristotle University of Thessaloniki School of Medicine, Greece
Anna Chaimani*
Affiliation:
Oslo Center for Biostatistics and Epidemiology, Department of Biostatistics, University of Oslo, Norway
*
Corresponding author: Anna Chaimani; Email: anna.chaimani@medisin.uio.no
Rights & Permissions [Opens in a new window]

Abstract

Network meta-analysis (NMA) is a vital methodology for synthesizing evidence across multiple treatments and informing medical decision-making. However, effective visualization and interpretation of results from large networks of interventions remain challenging, particularly for non-specialists. NMAstudio 2.0 is an innovative, interactive web application designed to address these difficulties by streamlining NMA workflows and enhancing result visualization. Developed using Python and R, NMAstudio 2.0 seamlessly integrates with established NMA frameworks. Our exemplar application of NMAstudio 2.0 using a Cochrane Review comparing several treatments for chronic plaque psoriasis demonstrates its capacity to facilitate all crucial steps of an NMA. The application features an intuitive interface for uploading data, automating analyses, generating interactive visualizations such as network diagrams, forest plots, ranking plots, and producing unique outputs like boxplots for transitivity checks and bidimensional forest plots. Most outputs are dynamically linked with the network diagram, enabling users to interactively explore evidence networks, apply advanced filtering, and highlight specific features by selecting nodes or edges within the diagram. While NMAstudio 2.0 aims to simplify NMAs, it also incorporates steps during the data upload process to mitigate the risk of producing poorly reported NMAs. NMAstudio 2.0 represents a significant step forward in improving the usability and accessibility of NMA, offering researchers a robust, versatile platform for evidence synthesis. Its integration of advanced features with an emphasis on user experience positions it as a valuable resource for enhancing decision-making and promoting evidence-based practice across diverse contexts.

Keywords

comparative effectiveness researchdata synthesisinteractive visualizationsmultiple treatmentsopen-access software

Information

Type
Software Focus
Information
Research Synthesis Methods , First View , pp. 1 - 14
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Open Practices
Open data Open materials
Copyright
© The Author(s), 2026. Published by Cambridge University Press on behalf of The Society for Research Synthesis Methodology

Highlights

What is already known?

  • • Network meta-analysis (NMA) is a crucial method for synthesizing evidence across multiple treatments and guiding medical decision-making.

  • • Visualizing and interpreting results from large networks of interventions remain challenging, especially for non-specialists.

What is new?

  • • NMAstudio 2.0 is an interactive web application designed to simplify NMA workflows and improve result visualization.

  • • It features a user-friendly interface for uploading data, automating analyses, and creating interactive visualizations.

  • • Outputs are dynamically linked to the network diagram, enabling users to explore evidence networks, apply filters, and highlight key features interactively.

  • • To support open research, NMAstudio 2.0 includes functions for saving and sharing uploaded data, making it easier for users to manage and share research projects.

Potential impact for research synthesis methods readers

  • • NMAstudio 2.0 enhances understanding and interpretation of results, accounting for risks of bias and data limitations, while fostering data sharing and open science.

  • • Its advanced features and focus on usability make it a valuable tool for evidence-based decision-making across various research contexts.

1 Background

Network meta-analysis (NMA) is a powerful tool for synthesizing evidence on multiple healthcare interventions and is widely used to support clinical recommendations and guidelines. By combining for every comparison evidence that comes directly and indirectly from the available studies within a connected network of interventions, it usually offers more comprehensive and precise findings than individual studies or pairwise meta-analysis do.Reference Nikol akopoulou, Mavridis and Salanti1 Beyond estimating relative effects, NMA offers the ability to rank the competing interventions based on specific outcomes and to facilitate decision-making, particularly in the presence of many alternative interventions for the condition of interest.

Despite the growing popularity of NMAs and the availability of extensive guidance, criticisms persist mainly due to their complex methodology and assumptions, which are not always examined carefully.Reference Cipriani, Higgins and Geddes2– Reference Salanti5 Several empirical studies have identified deficiencies in published NMAs, although an improvement in the overall quality has been observed over time.Reference Petropoulou, Nikolakopoulou and Veroniki6, Reference Veroniki, Florez and Hutton7 The significant level of statistical expertise required and the long outputs that NMA typically produces pose a major barrier in understanding and properly considering the strengths and limitations of the findings.Reference Cipriani, Higgins and Geddes2 Additionally, even though several software tools are available for conducting NMAs,Reference Neupane, Richer and Bonner8, Reference Shim, Yoon and Shin9 they do not offer visualizations that allow to easily make the connection between data and results.

Complete reporting is a crucial aspect of every NMA to avoid misinterpretations or over-confidence in uncertain findings. The PRISMA-NMA checklistReference Hutton, Catala-Lopez and Moher10 assists researchers toward this direction, but often it is only partially followed by published NMAs. On the other hand, comprehensive NMAs usually put a lot of the required information into Supplementary Materials, which can extend to hundreds of pages.Reference Hutton, Salanti and Chaimani11, Reference Bafeta, Trinquart and Seror12 Navigating through these extensive and complex outputs can be both time-consuming and confusing, thereby impeding effective communication and decision-making. Typically, the volume of information in an NMA is significantly greater than that of a pairwise meta-analysis. This presents significant challenges to the principles of open science, which emphasize transparency, accessibility, and reproducibility.Reference Foster and Deardorff13, Reference Allen and Mehler14 since it makes it difficult to share data and findings in a user-friendly manner.

In recent years, several new graphical and tabular visualizations for NMA have been introduced.Reference Chaimani, Higgins and Mavridis15, Reference Salanti, Ades and Ioannidis16 For example, contribution plots highlight the balance between direct and indirect information in the results;Reference Krahn, Binder and Konig17 radial ranking plots integrate network graphs with the relative ranking of the interventions;Reference Nevill, Cooper and Sutton18 and the Kilim plot depicts absolute risk for each outcome and treatment alongside the strength of statistical evidence.Reference Seo, Furukawa and Veroniki19 However, all these visualizations typically need to be run separately in a scripting environment such as R or STATA and thus lack interactivity and interoperability.

In this paper, we introduce NMAstudio 2.0, an online interactive user interface for conducting, reviewing, and managing network meta-analyses. Compared to existing applications, NMAstudio 2.0 integrates several innovative features such as transitivity assessment via box plots and scatter plots, analysis and visualizations for multiple outcomes, and interactive graphics that make the complexity of NMAs more accessible and transparent. Through novel visualizations, our tool enhances understanding and interpretation of findings in light of risk of bias and other potential limitations of the data, while promoting data sharing and open science. At the same time, it supports transparency by guiding researchers to go through all the required steps for a complete report. NMAstudio 2.0 also simplifies the full NMA process by offering a “point-and-click” environment that completely removes the need for scripting. In the rest of the article, we first describe the main characteristics of the tool, and we introduce our illustrative example dataset. Then, we go through the different steps of a typical NMA and show how NMAstudio 2.0 can address existing challenges in the entire process. A brief description of the key terms used throughout the paper is available in Box 1 .

Box 1.

Explanation of key NMA terms.

PRISMA-NMA

PRISMA-NMA is an extension of the PRISMA statement, created specifically to improve the reporting of systematic reviews that incorporate network meta-analyses (NMA). This extension includes a 32-item checklist, developed to address key elements considered essential for the accurate and thorough reporting of NMAs.

Transitivity

Transitivity is a key assumption in network meta-analysis that ensures the validity of indirect comparisons. It requires that studies comparing different interventions are similar, on average, with respect to all potential effect modifiers. It also implies that intervention A, as assessed in A versus B studies, is equivalent to intervention A in A versus C studies.

Prediction interval

A prediction interval is the interval within which the underlying effect of a new study similar to those included in a meta-analysis is expected to lie. Prediction intervals provide a way to assess the impact of the estimated heterogeneity and its uncertainty on the findings of the meta-analysis.

CINeMA

CINeMA (confidence in network meta-analysis) is a methodological framework for evaluating the confidence (or certainty) of the results from network meta-analyses. A key element of CINeMA is the percentage contribution matrix, which indicates the proportion of information contributed by each study or direct comparison to each network estimate.

P-score

The P-score is a statistical measure used to rank treatments based on a specific outcome and shows the mean extent of certainty that a treatment is better than its competitors. It is calculated using the probabilities that one treatment is better than another, considering all direct and indirect comparisons within the network. These are determined based on the estimated relative effects and their variance.

Consistency

The consistency assumption refers to the agreement between different sources of evidence—direct and indirect—in a network meta-analysis. It reflects the assumption of transitivity, which considers potential clinical and methodological variations across comparisons. Consistency is assessed through statistical tests, such as the design-by-treatment interaction model and the side-splitting approach. The former assesses consistency in the entire network, whereas the latter at the comparison level.

Comparison-adjusted funnel plot

A comparison-adjusted funnel plot is a variation of the standard funnel plot used in network meta-analysis to assess small-study effects while accounting for multiple comparisons. The horizontal axis shows the difference between study-specific effects and the corresponding comparison-specific summary effect, while the vertical axis represents the precision of the study, often measured by the inverse of its standard error. For proper interpretation, comparisons should be defined consistently (e.g., placebo versus active treatment). A symmetric comparison-adjusted funnel plot around the zero line indicates no indication of small-study effects.

2 Implementation

NMAstudio 2.0 is a Python web application integrated into the Python “Dash”Reference Dabbas20 environment and further connected to the R packages “netmeta”Reference Balduzzi, Rücker and Nikolakopoulou21 for NMA and “meta”Reference Schwarzer22 for pairwise meta-analysis. NMA results are imported from R into Python, where interactive and downloadable visualizations are created using “Plotly” modules. Users of NMAstudio 2.0 do not need any prior familiarity with these software packages. The application is open-source and accessible via a web browser at https://nmastudio.uiocloud.no/. The source code and data are available in the following GitHub repository (https://github.com/CER-METHODS/NMAstudio-2.0) under the MIT license. We also include instructions on how to self-host the application for users with data privacy concerns, although the process is not trivial.

Currently, NMAstudio 2.0 consists of three pages (see screenshots in the Supplementary file): the homepage, the analysis setup page, and the results page. In the analysis setup page, users upload their data and specify the variables that represent outcomes, risk of bias, and potential effect modifiers. Once these steps are completed, the analysis is performed using the R netmeta package,Reference Balduzzi, Rücker and Nikolakopoulou21 and additional customizable visualizations are generated automatically. The results page then displays all produced plots and tables in an interactive format. Users can explore data and results by interacting with all the outputs, or can export them and prepare their NMA report.

3 Results

3.1 Illustrative example

To illustrate the use and the benefits of our tool, we use throughout this manuscript an exemplar NMA comparing systemic pharmacological treatments for chronic plaque psoriasis.Reference Sbidian, Chaimani and Guelimi23 The dataset contains 158 randomized controlled trials (RCTs) involving 20 treatments, including placebo. The full names and abbreviations of the treatments are provided in the Supplementary file. This NMA focuses on eight outcomes (two primary and six secondary). For our illustration, we will use four outcomes:

Primary outcomes:

  • • Efficacy: Proportion of participants achieving psoriasis area severity index ≥90 (PASI 90)

  • • Safety: The proportions of participants with serious adverse events (SAE)

Secondary outcomes:

  • • Quality of life measured by the dermatology life quality index (DLQI).

  • • The proportions of participants with any kind of adverse events (AE).

PASI90, SAE, and AE are binary outcomes, and DLQI is a continuous outcome. Example data for these four outcomes are provided in the Supplementary file.

For all binary outcomes, we use the risk ratio (RR) as the effect measure; for the continuous outcome, the standardized mean difference (SMD) is used due to the presence of different scales. The main interest here lies in the comparisons between the different active drugs rather than comparisons against placebo. However, placebo-controlled trials provide a large amount of data (58% of the trials) and thus have been included in the network to supplement the analysis.

3.2 Uploading the data

NMAstudio 2.0 streamlines the NMA process with an interactive interface that lets users upload datasets and run analyses through simple point-and-click actions, guided by embedded instructions. It supports three data formats—long, wide, or contrast data (see the Supplementary file for an explanation of the different data formats)—and allows users to upload data for several outcomes. Users need to follow the instructions on the data upload page and select a few options. Then, NMAstudio 2.0 automatically runs the analysis, generates results, and displays key graphs and tables. Step-by-step guidance for the general data upload process is available on the Setup Analysis page by clicking the “See here explanations for each step” button, and instructions for the illustrative data are provided in the Supplementary file (Illustrative_data_upload_instruction.pdf).

While NMAstudio 2.0 provides flexibility in data uploading, it also involves steps that reassure a comprehensive and transparent reporting. Specifically, users are recommended to include study risk of bias information and provide variables representing potential effect modifiers in their data, which are essential for evaluating the credibility of NMA results. A further crucial aspect of high-quality systematic reviews is the development of a protocol that outlines the main objectives, study design, and planned analyses of the review.Reference Moher, Shamseer and Clarke24 Registering the protocol is important for ensuring transparency, reducing reporting bias, avoiding data-driven decisions, and preventing the duplication of reviews.Reference Stewart, Moher and Shekelle25 Therefore, NMAstudio 2.0 asks users to provide the link to their protocol, which will then appear on the results page. This allows all end-users as well as peer-reviewers to easily access and compare the final review with the original protocol.Reference Silagy, Middleton and Hopewell26 If information on study risk of bias, potential effect modifiers, or the protocol link is not provided, warnings appear on the results page (Supplementary Figure S1). All settings for the statistical analysis can be found on the results page by clicking the “Download settings of statistical analysis” button. Additional warnings related to the dataset may appear during the data checks, but these do not appear on the results page (see Supplementary Figure S2 and the associated explanations in the Supplementary File).

3.3 Exploring the structure of the data

A powerful feature of the application is the interactive network diagram. The function of the graph is to give an outline of the structure of the data and the included studies. It also acts as a global filter and selector for the rest of the application. The graph is linked with an adjacent table of the selected studies. In the graph, each node represents an intervention, while each edge represents a direct comparison. Users can select interventions or comparisons by clicking on the corresponding nodes or edges. When a node is selected, the table highlights only the treatments that are directly compared with the selected intervention, helping users quickly identify comparisons supported by direct evidence from the available studies. Once nodes or edges are selected, the rest of the visualizations will display information specific to the corresponding interventions and comparisons.

Users can customize the network diagram to display distinct types of information using the provided options. Figure 1 presents network graphs for the four outcomes of our example, each illustrating specific features. In Figure 1A, the network diagram for PASI90 uses edge size to represent the number of trials for each comparison. In Figure 1B, the network diagram for SAE demonstrates additional customization options. Users can also adjust the node size to reflect the number of trials for both interventions and comparisons and include labels on the edges to indicate the number of trials. Node colors can also be customized by specifying a color name or code (e.g., “pink” or “#FFC0CB”). Further, node colors can be modified to represent the risk of bias. For example, in the graph of DLQI (Figure 1C), each node is displayed as a pie chart, showing the proportion of trials with low (green), moderate (yellow), and high (red) risk of bias for the corresponding intervention. Additionally, if treatment class information is included in the data, the diagram can group interventions by color to indicate that they belong to the same treatment class, as shown in Figure 1D for AE.

Figure 1

Network graphs for different outcomes. (A) Outcome PASI90: The edge width is proportional to the number of studies for the corresponding comparison. (B) Outcome SAE: Numbers on the edges indicate the number of studies for the corresponding comparisons; node size is proportional to the number of studies that include the corresponding interventions. (C) Outcome AE: Pie charts within the nodes represent the percentage of studies with different risks of bias for the corresponding interventions (red = high, yellow = moderate, green = low). (D) Outcome DLQI: Node colors represent different treatment classes.

At the top of the results page, users can choose from a dropdown list to display different outputs. The first option, “Data & Transitivity,” presents data tables, box plots, and scatter plots that are typically used in NMA to evaluate the transitivity assumption, which is explained in detail in the Assessing transitivity section. Both raw data and data converted to the contrast-based format are shown prior to the analysis. Additionally, on the setup analysis page, if users upload time-related information such as publication year or study start and end dates, a timeline bar will be displayed on the results page, allowing users to visualize how the evidence has evolved over time.

3.4 Assessing transitivity

Before conducting the analysis, it is crucial to assess the validity of the transitivity assumption. For an indirect comparison (e.g., A versus B) to be considered valid, it is necessary for the direct comparisons (e.g., A versus C and B versus C) to exhibit similar distributions of key clinical and methodological characteristics (i.e., effect modifiers).Reference Song, Altman, Glenny and Deeks27 However, substantial empirical evidence suggests that the evaluation of this assumption is often inadequately reported, or not performed, in published NMAs.Reference Hutton, Salanti and Chaimani11, Reference Bafeta, Trinquart and Seror12, Reference Bafeta, Trinquart and Seror28

While the transitivity assumption cannot be statistically tested, it can be assessed clinically and epidemiologically.Reference Cipriani, Higgins and Geddes2, Reference Salanti5, Reference Jansen and Naci29, Reference Donegan, Williamson, D'Alessandro and Tudur Smith30 During the data upload process, users are prompted to select potential effect modifiers from their dataset. Currently, effect modifiers must be in continuous format. For dichotomous variables, users should convert them into proportions (e.g., the percentage of participants with a given characteristic). For categorical variables, users can assign numerical codes to represent categories (e.g., 1, 2, and 3). At the results page, box plots and scatter plots are generated for each selected effect modifier in the dropdown list, allowing users to compare their distribution across the available direct comparisons. In scatter plots, point sizes are proportional to the sample size when that information is provided for each. For example, Figure 2 shows for the psoriasis network box plots for age and percentage of male participants, and scatter plots for BMI and mean weight. Users can highlight specific box plots or scatter plots by selecting the corresponding comparisons in the network diagram. Overall, the distributions of these characteristics appear relatively similar across the comparisons, raising no concerns with respect to transitivity for this dataset. However, for some comparisons, the number of available studies is very small (one or two studies only) and no distributions can be obtained.

Figure 2

Boxplots of age and male percentage, and scatter plots for BMI and weight for the PASI90 outcome. Seven comparisons (highlighted in blue) are selected by choosing the corresponding edges (shown in red) in the network diagram.

4 Visualizing and exploring results

4.1 Forest plots for direct and network summary effects

The psoriasis example includes 20 distinct treatments, which can provide 20 different NMA forest plots, each with a different reference treatment. In the “Forest plots” tab of NMAstudio 2.0, users can click on any node in the diagram, such as “placebo”, to set it as the reference treatment. This generates the corresponding “standard” NMA forest plot (Figure 3) and a bidimensional forest plot showing the relative effects for two outcomes with the same reference treatment (Supplementary Figure S3) simultaneously. The standard forest plot shows all effect sizes along with confidence intervals, prediction intervals (optional), heterogeneity estimates for each comparison (optional), and the common heterogeneity estimate of the NMA, preventing the overemphasis of highly uncertain findings. The bidimensional forest plot enables users to have a more complete view of the results and identify treatments that perform well on two outcomes simultaneously (e.g., efficacy and safety). Given that this graph can become busy and less visible for large networks, users may remove treatments by clicking the respective point in the legend of the graph. For example, in the psoriasis example, we kept the ten most effective treatments using placebo as the reference; those that appear in the upper-right corner are those that perform best both for efficacy and safety (Supplementary Figure S4).

Figure 3

Forest plots for different outcomes with “PBO” as the reference treatment. (A) Outcome PASI90. (B) Outcome SAE. (C) Outcome AE. (D) Outcome DLQI. The black line represents the confidence interval, and the red line represents the prediction interval. CI: Confidence Interval; PI: Prediction Interval; tau2: heterogeneity for each comparison in pairwise meta-analysis.

Although summary effects involving both direct and indirect evidence are the main output from NMA, direct evidence and study-level effects also provide important information and need to be easily accessible. For example, high heterogeneity within a certain comparison or the presence of outlying studies may be the cause of increased heterogeneity or inconsistency in the entire network.Reference Donegan, Williamson, D'Alessandro and Tudur Smith30, Reference Imrey31 The psoriasis example includes 58 comparisons with direct evidence, which is challenging to present clearly using traditional formats. NMAstudio 2.0 addresses this problem by allowing users to click on any edge in the diagram, such as the one connecting “PBO” and “ETA” to generate the corresponding pairwise forest plot (Supplementary Figure S5). This feature makes it easy and fast to navigate and review the pairwise forest plots for all or selected direct comparisons.

4.2 League tables presenting all NMA relative effects

League tables typically display the summary relative effects along with the respective confidence intervals for every treatment contrast in the network. They often present the results of two outcomes, one in the upper triangle and the second in the lower triangle. In NMAstudio 2.0, two types of league tables are offered. The first is for a single outcome, with the upper triangle showing direct estimates and the lower triangle showing mixed (NMA) estimates. The second is for two outcomes, where both the upper and lower triangles display the mixed effects for each outcome. Users can enhance league tables by coloring the cells based on either the overall risk of bias of the respective contrast or the overall confidence rating. The latter is obtained using the CINeMA framework;Reference Nikolakopoulou, Higgins and Papakonstantinou32 users need first to upload the final report as obtained from the CINeMA software (cinema.med.auth.gr). Users can also hover over the colored cells of the table to see reasons for downgrading the evidence as these appear in the CINeMA report (Figure 4). This makes it straightforward to identify interventions and comparisons with higher and lower confidence in the evidence. The league table is also connected to the network diagram, allowing users to select specific interventions in the diagram and display the relevant comparisons in the table in the desired order (Supplementary Figure S6).

Figure 4

League table with color-coded CINeMA ratings. The upper triangle represents PASI90, and the lower triangle represents SAE. Hovering over the cell comparing MTX and BIME displays the reasons for downgrading.

4.3 Ranking of the interventions

NMAstudio 2.0 generates two ranking visualizations: a P-score heatmap for all the outcomes analyzed and a two-dimensional scatter plot displaying the P-scores for two selected outcomes (Supplementary Figures S7 and S8). P-scores rank treatments based on all relative effects in the league table while also considering their precision, reflecting the average certainty that a treatment is better than its competitors.Reference Rücker and Schwarzer33 Consideration of ranking results for different outcomes jointly is crucial for proper decision-making. In the two-dimensional scatter plot, the size of the points is proportional to the number of studies that include the corresponding interventions, and the different colors represent different clusters of interventions with similar performance based on the p-score values. In our example, IFX demonstrates the highest score for the efficacy outcome (PASI90) but has a notably low P-score for the safety outcome (SAE). Also, ranking should always be considered along with the NMA relative effects and other characteristics, such as risk of bias or amount of evidence.Reference Chaimani, Porcher and Sbidian34 Examining the ranking alongside the network diagram, with risk of bias indicated by color and node size reflecting the number of trials for each intervention (Supplementary Figure S9), reveals that IFX has substantial information coming from low risk of bias studies, while it is mainly compared with placebo, and only one study connects it directly with ETA. The two-dimensional scatter plot suggests that BIME might be the best choice, as it is positioned in the top-right corner of the graph (Supplementary Figure S8).

5 Consistency and reporting bias

Consistency refers to the statistical agreement between direct and indirect evidence for a given comparison.Reference Cipriani, Higgins and Geddes2, Reference Salanti5 There are two types of tests for inconsistency: local and global tests. A local test can only be performed when a certain intervention comparison is included in one or more closed loops in the network.Reference Dias, Welton and Caldwell35 If direct and indirect evidence are inconsistent, there are concerns with respect to the validity of the NMA results.Reference Cipriani, Higgins and Geddes2, Reference Donegan, Williamson, D'Alessandro and Tudur Smith30 NMAstudio 2.0 offers two tests for inconsistency: the side-splitting approach that evaluates inconsistency for every comparison informed both directly and indirectlyReference van Valkenhoef, Dias and Ades36 and the design-by-treatment interaction model that provides a global test for the entire network.Reference Jackson, Barrett and Rice37 For the outcome PASI90 of the psoriasis example, the global test indicates no significant inconsistency. However, the local test highlights potential issues for the IXE versus USK comparison (p-value = 0.04), suggesting that combining direct and indirect effects for this comparison might be problematic and requires further investigation.Reference Cipriani, Higgins and Geddes2, Reference Salanti5 The local consistency results table is also connected to the network diagram, allowing users to select specific comparisons displayed in the table (Supplementary Figure S10). Here, the suspicious comparison is only informed by one study, which might explain the difference between direct and indirect effects.

Reporting bias poses a significant threat to the validity of results in systematic reviews. Small-study effects are a common issue and refer to the systematic differences in the results between smaller and larger studies.Reference Rücker, Carpenter and Schwarzer38 Small-study effects can be visually assessed using funnel plots, where asymmetry may indicate the presence of small-study effects and requires further investigation.Reference Copas and Shi39 NMAstudio 2.0 provides comparison-adjusted funnel plots to examine small-study effects across the entire networkReference Chaimani and Salanti40 and standard funnel plots for specific comparisons. For comparison-adjusted funnel plots, users should select one treatment in the network diagram to be used as a reference for all comparisons. This prevents misleading interpretations that may arise when there is not a consistent direction for all comparisons in the graph. In the psoriasis example, we chose PBO as the reference treatment; in this way, the produced funnel plot assesses whether smaller studies tend to favor active interventions against PBO more than larger studies do. For the four outcomes of the psoriasis network, the funnel plots appear symmetrical, raising no concerns with respect to the presence of small-study effects (Supplementary Figures S11–S14). For standard funnel plots, users can select a comparison that includes at least two studies in the network diagram to generate the plot. However, to avoid misleading interpretations, asymmetry should be judged only for comparisons with at least ten studies. For example, Supplementary Figure S15 presents the standard funnel plot for the comparison “PBO versus ETA” for the PASI90 outcome, showing no visible asymmetry, which suggests no evidence of small-study effects for this comparison.

6 Data sharing and open science

Transparency, openness, and reproducibility are essential features of science that should be standard practice.Reference Foster and Deardorff13 In a typical NMA, authors present key findings in the main text while additional results, raw data, code, and other materials are placed in the supplementary material. However, the supplementary material often receives less attention due to its large volume, requiring substantial time for thorough exploration. The fixed format of supplementary material also limits readers’ flexibility in exploring results. These challenges hinder the efficient sharing of data and results and prevent readers from deeply delving into the findings.

To promote an open research culture, NMAstudio 2.0 provides easy functions for saving and loading uploaded data, allowing users to store and share their projects. On the results page, users can click the Save/Load Project button to generate a unique token for data sharing (Supplementary Figure S16). This token can be shared by authors alongside their published article. Readers, peer-reviewers, or other stakeholders can then use the “Load Project” function in NMAstudio 2.0 to enter the token, reload all results of the previously saved project, and explore the results interactively (Supplementary Figure S17). With its interactive and user-friendly features, NMAstudio 2.0 gives access to original data and results, supporting independent verification and offering reviewers flexibility to explore findings and draw evidence-based conclusions. Of course, this feature is not intended to replace traditional supplementary materials entirely, but rather to complement them by offering an interactive and user-friendly way to explore data, results, and assumptions.

7 Discussion

In this paper, we introduce NMAstudio 2.0, a web application that aims to facilitate the production, visualization, and communication of NMAs. Its underlying motivation is to continuously address existing and new challenges that most investigators encounter when conducting or using NMAs. We illustrate how NMAstudio 2.0 can simplify the entire analytical process and enhance the interpretation and understanding of NMA findings using a Cochrane Review that involves data on multiple treatments and multiple outcomes.Reference Sbidian, Chaimani and Guelimi23 NMAstudio’s 2.0 unique design combines a user-friendly “point-and-click” environment with extensive interactivity and a direct connection between the network diagram and all outputs. Other web-based applications for NMA are also available.Reference Owen, Bradbury and Xin41– Reference Chiocchia, Nikolakopoulou and Higgins44 However, to our knowledge, NMAstudio 2.0 is the only tool, to date, that allows the analysis of multiple outcomes simultaneously, visualizing the evolution of evidence over time, evaluating transitivity, facilitating data sharing, and offering such a high level of interaction with data and results. Moreover, NMAstudio 2.0 allows flexibility in the preparation of the data file with no restrictions on variable names or the number of study arms. Users benefit from a wide range of interactive, high-quality plots that are unique to NMAstudio 2.0 and not derived from other packages. These plots are generated in a single analysis run, making the process efficient, fast, and user-friendly.

A usual concern for tools that aim to simplify NMAs and statistical analyses, in general, is the risk of an increased number of poorly conducted studies and invalid results. NMAstudio 2.0 involves steps during the data upload process that aim to mitigate, to some degree, this risk. First, users are asked to provide the link to their protocol to prevent data-driven decisions. Second, they need to upload risk of bias information for the included studies, which then appears automatically in the league table. Third, users are asked to upload data on the potential effect modifiers, which are used for the evaluation of transitivity. If any of these steps are skipped, warnings appear on the results page to inform end-users about the omission of these steps and the associated risks. Despite these steps, it is essential to emphasize, though, that involvement of statisticians and methodologists in NMA projects remains indispensable. One limitation of the current version of the tool is that it does not allow viewing the actual R functions executed during analysis. Additionally, software sustainability is an important consideration. It refers to the capacity of a software tool to remain reliable, maintainable, and accessible over time, independent of changes in technology or the duration of specific funding sources. NMAstudio 2.0 will address this by offering self-hosting options, which allow users to run the software independently of external servers, and by ensuring that users retain ownership of their data.

Further updates are under development. NMAstudio 2.0 currently offers only the conduct of NMA in the frequentist framework using the netmeta package in RReference Balduzzi, Rücker and Nikolakopoulou21 and the inverse-variance model. Additional models available in netmeta, such as the Mantel–Haenszel or the penalized likelihood models,Reference Efthimiou, Rücker and Schwarzer45, Reference Evrenoglou, White and Afach46 as well as Bayesian NMA, will be implemented in upcoming versions of the application. We are also working on allowing the conduct of sensitivity and subgroup analyses based on pre-defined (during data uploading) study and intervention characteristics. Comparison of results between primary and exploratory analyses will be enhanced through the interactive features that NMAstudio 2.0 offers. Additionally, NMAstudio 2.0 will allow for a full order of the treatments in the comparison-adjusted funnel plots. Finally, we plan to allow users to upload NMA results obtained from external analyses and use NMAstudio 2.0 for visualization purposes. To further enhance the comprehensiveness and interpretation of NMA findings for end-users, a knowledge translation tool is being developed for integration into NMAstudio 2.0. This tool is designed to provide an interactive, flexible, and user-friendly approach to summarizing and presenting NMA results within a single page. By featuring two versions—a standard version and an advanced version—it aims to improve the accessibility and usability of NMA findings for a broad range of stakeholders.

8 Conclusions

We believe NMAstudio 2.0 serves as a valuable resource for researchers and clinical practitioners embarking on an NMA. Its unique interactive features facilitate the communication of NMA outputs and enhance the clarity of results for diverse audiences. At the same time, it offers increased flexibility and interactivity, while prompting users to adhere to high standards for conducting and reporting NMAs. Future versions of NMAstudio 2.0, together with the upcoming knowledge translation tool, have the potential to further promote transparency, reproducibility, data sharing, open science, and broad research collaboration.

Acknowledgments

We sincerely thank Isabelle Boutron, Toshi Furukawa, Emily Karahalios, Tianjing Li, Michael McCaul, Adriani Nikolakopoulou, Haliton Oliveira, Georgia Salanti, Guido Schwarzer, Ian Saldanha, Nicky Welton, and Sally Yaacoub for their valuable suggestions and feedback on this project.

Author contributions

T.Y., S.M., and A.C. conceptualized and designed the project. T.Y., S.M., and T.P. wrote the code and developed the software. T.Y. performed all analyses. T.Y. and A.C. drafted the manuscript. All authors revised the manuscript and accepted the final version.

Competing interest statement

The authors declare that no competing interests exist.

Data availability statement

All data and code can be obtained via https://doi.org/10.5281/zenodo.18368497 or https://github.com/CER-METHODS/NMAstudio-2.0

Funding statement

T.Y. received funding from the French National Research Agency under the project ANR-22-CE36–0013-01.

Supplementary material

To view supplementary material for this article, please visit http://doi.org/10.1017/rsm.2026.10074.

Footnotes

This article was awarded Open Data and Open Materials badges for transparent practices. See the Data availability statement for details.

References

Nikol akopoulou, A, Mavridis, D, Salanti, G, et al. Living network meta-analysis compared with pairwise meta-analysis in comparative effectiveness research: empirical study. BMJ. 2018;360:k585.10.1136/bmj.k585CrossRefGoogle Scholar
Cipriani, A, Higgins, JP, Geddes, JR, et al. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159(2):130–137.10.7326/0003-4819-159-2-201307160-00008CrossRefGoogle ScholarPubMed
Dias, S, Ades, AE, Welton, NJ, et al. Network Meta-Analysis for Decision-Making. Hoboken, NJ: John Wiley & Sons; 2018.10.1002/9781118951651CrossRefGoogle Scholar
Dias, S, Sutton, AJ, Ades, AE, et al. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Mak. 2013;33(5):607–617.10.1177/0272989X12458724CrossRefGoogle ScholarPubMed
Salanti, G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3(2):80–97.10.1002/jrsm.1037CrossRefGoogle ScholarPubMed
Petropoulou, M, Nikolakopoulou, A, Veroniki, A, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20–28.10.1016/j.jclinepi.2016.11.002CrossRefGoogle ScholarPubMed
Veroniki, AA, Florez, I, Hutton, B, et al. Two decades of network meta-analysis: roadmap to their applications and challenges. Res Synth Methods. 2024;15(5):741–746.10.1002/jrsm.1744CrossRefGoogle ScholarPubMed
Neupane, B, Richer, D, Bonner, AJ, et al. Network meta-analysis using R: a review of currently available automated packages. PLoS One. 2014;9(12):e115065.10.1371/journal.pone.0115065CrossRefGoogle Scholar
Shim, S, Yoon, BH, Shin, IS, et al. Network meta-analysis: application and practice using Stata. Epidemiol Health. 2017;39:e2017047.10.4178/epih.e2017047CrossRefGoogle ScholarPubMed
Hutton, B, Catala-Lopez, F, Moher, D. The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA. Med Clin (Barc). 2016;147(6):262–266.10.1016/j.medcli.2016.02.025CrossRefGoogle ScholarPubMed
Hutton, B, Salanti, G, Chaimani, A, et al. The quality of reporting methods and results in network meta-analyses: an overview of reviews and suggestions for improvement. PLoS One. 2014;9(3):e92508.10.1371/journal.pone.0092508CrossRefGoogle ScholarPubMed
Bafeta, A, Trinquart, L, Seror, R, et al. Reporting of results from network meta-analyses: methodological systematic review. BMJ. 2014;348:g1741.10.1136/bmj.g1741CrossRefGoogle ScholarPubMed
Foster, ED, Deardorff, A. Open science framework (OSF). J Med Libr Assoc. 2017;105(2):203.10.5195/jmla.2017.88CrossRefGoogle Scholar
Allen, C, Mehler, DM. Open science challenges, benefits and tips in early career and beyond. PLoS Biol. 2019;17(5):e3000246.10.1371/journal.pbio.3000246CrossRefGoogle ScholarPubMed
Chaimani, A, Higgins, JP, Mavridis, D, et al. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8(10):e76654.10.1371/journal.pone.0076654CrossRefGoogle Scholar
Salanti, G, Ades, AE, Ioannidis, JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64(2):163–171.10.1016/j.jclinepi.2010.03.016CrossRefGoogle ScholarPubMed
Krahn, U, Binder, H, Konig, J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013;13:35 10.1186/1471-2288-13-35CrossRefGoogle ScholarPubMed
Nevill, CR, Cooper, NJ, Sutton, AJ. A multifaceted graphical display, including treatment ranking, was developed to aid interpretation of network meta-analysis. J Clin Epidemiol. 2023;157:83–91.10.1016/j.jclinepi.2023.02.016CrossRefGoogle ScholarPubMed
Seo, M, Furukawa, TA, Veroniki, AA, et al. The Kilim plot: a tool for visualizing network meta-analysis results for multiple outcomes. Res Synth Methods. 2021;12(1):86–95.10.1002/jrsm.1428CrossRefGoogle Scholar
Dabbas, E. Interactive Dashboards and Data Apps with Plotly and Dash: Harness the Power of a Fully Fledged Frontend Web Framework in Python—No JavaScript Required. Birmingham: Packt Publishing Ltd; 2021.Google Scholar
Balduzzi, S, Rücker, G, Nikolakopoulou, A, et al. Netmeta: an R package for network meta-analysis using frequentist methods. J Stat Softw. 2023;106:1–40.10.18637/jss.v106.i02CrossRefGoogle Scholar
Schwarzer, G. Meta: an R package for meta-analysis. R News. 2007;7(3):40–45.Google Scholar
Sbidian, E, Chaimani, A, Guelimi, R, et al. Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis. Cochrane Database Syst Rev. 2023;7(7):CD011535.Google ScholarPubMed
Moher, D, Shamseer, L, Clarke, M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1–9.10.1186/2046-4053-4-1CrossRefGoogle ScholarPubMed
Stewart, L, Moher, D, Shekelle, P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:1–4.10.1186/2046-4053-1-7CrossRefGoogle ScholarPubMed
Silagy, CA, Middleton, P, Hopewell, S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287(21):2831–2834.10.1001/jama.287.21.2831CrossRefGoogle ScholarPubMed
Song, F, Altman, DG, Glenny, AM, Deeks, JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003;326(7387):472.10.1136/bmj.326.7387.472CrossRefGoogle ScholarPubMed
Bafeta, A, Trinquart, L, Seror, R, et al. Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review. BMJ. 2013;347:f3675.10.1136/bmj.f3675CrossRefGoogle ScholarPubMed
Jansen, JP, Naci, H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 2013;11:159.10.1186/1741-7015-11-159CrossRefGoogle ScholarPubMed
Donegan, S, Williamson, P, D'Alessandro, U, Tudur Smith, C. Assessing key assumptions of network meta-analysis: a review of methods. Res Synth Methods. 2013;4(4):291–323.10.1002/jrsm.1085CrossRefGoogle ScholarPubMed
Imrey, PB. Limitations of meta-analyses of studies with high heterogeneity. JAMA Netw Open. 2020;3(1):e1919325.10.1001/jamanetworkopen.2019.19325CrossRefGoogle ScholarPubMed
Nikolakopoulou, A, Higgins, JP, Papakonstantinou, T, et al. CINeMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med. 2020;17(4):e1003082.10.1371/journal.pmed.1003082CrossRefGoogle ScholarPubMed
Rücker, G, Schwarzer, G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:58.10.1186/s12874-015-0060-8CrossRefGoogle ScholarPubMed
Chaimani, A, Porcher, R, Sbidian, É, et al. A Markov chain approach for ranking treatments in network meta-analysis. Stat Med. 2021;40(2):451–464.10.1002/sim.8784CrossRefGoogle ScholarPubMed
Dias, S, Welton, NJ, Caldwell, DM, et al. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29(7–8):932–944.10.1002/sim.3767CrossRefGoogle ScholarPubMed
van Valkenhoef, G, Dias, S, Ades, AE, et al. Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis. Res Synth Methods. 2016;7(1):80–93.10.1002/jrsm.1167CrossRefGoogle ScholarPubMed
Jackson, D, Barrett, JK, Rice, A, et al. Design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Stat Med. 2014;33(21):3639–3645.10.1002/sim.6188CrossRefGoogle Scholar
Rücker, G, Carpenter, JR, Schwarzer, G. Detecting and adjusting for small-study effects in meta-analysis. Biom J. 2011;53(2):351–368.10.1002/bimj.201000151CrossRefGoogle ScholarPubMed
Copas, J, Shi, JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1(3):247–262.10.1093/biostatistics/1.3.247CrossRefGoogle ScholarPubMed
Chaimani, A, Salanti, G. Using network meta-analysis to evaluate the existence of small-study effects in a network of interventions. Res Synth Methods. 2012;3(2):161–176.10.1002/jrsm.57CrossRefGoogle Scholar
Owen, RK, Bradbury, N, Xin, Y, et al. MetaInsight: an interactive web-based tool for analyzing, interrogating, and visualizing network meta-analyses using R-shiny and netmeta. Res Synth Methods. 2019;10(4):569–581.10.1002/jrsm.1373CrossRefGoogle ScholarPubMed
Brown, S, Hutton, B, Clifford, T, et al. A Microsoft-excel-based tool for running and critically appraising network meta-analyses—an overview and application of NetMetaXL. Syst Rev. 2014;3:110.10.1186/2046-4053-3-110CrossRefGoogle ScholarPubMed
Papakonstantinou, T, Nikolakopoulou, A, Higgins, JP, et al. CINeMA: software for semiautomated assessment of the confidence in the results of network meta-analysis. Campbell Syst Rev. 2020;16(1):e1080.10.1002/cl2.1080CrossRefGoogle ScholarPubMed
Chiocchia, V, Nikolakopoulou, A, Higgins, JP, et al. ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis. BMC Med. 2021;19:1–13.10.1186/s12916-021-02166-3CrossRefGoogle ScholarPubMed
Efthimiou, O, Rücker, G, Schwarzer, G, et al. Network meta-analysis of rare events using the mantel-Haenszel method. Stat Med. 2019;38(16):2992–3012.10.1002/sim.8158CrossRefGoogle ScholarPubMed
Evrenoglou, T, White, IR, Afach, S, et al. Network meta-analysis of rare events using penalized likelihood regression. Stat Med. 2022;41(26):5203–5219.10.1002/sim.9562CrossRefGoogle ScholarPubMed
Figure 0

Figure 1 Network graphs for different outcomes. (A) Outcome PASI90: The edge width is proportional to the number of studies for the corresponding comparison. (B) Outcome SAE: Numbers on the edges indicate the number of studies for the corresponding comparisons; node size is proportional to the number of studies that include the corresponding interventions. (C) Outcome AE: Pie charts within the nodes represent the percentage of studies with different risks of bias for the corresponding interventions (red = high, yellow = moderate, green = low). (D) Outcome DLQI: Node colors represent different treatment classes.

Figure 1

Figure 2 Boxplots of age and male percentage, and scatter plots for BMI and weight for the PASI90 outcome. Seven comparisons (highlighted in blue) are selected by choosing the corresponding edges (shown in red) in the network diagram.

Figure 2

Figure 3 Forest plots for different outcomes with “PBO” as the reference treatment. (A) Outcome PASI90. (B) Outcome SAE. (C) Outcome AE. (D) Outcome DLQI. The black line represents the confidence interval, and the red line represents the prediction interval. CI: Confidence Interval; PI: Prediction Interval; tau2: heterogeneity for each comparison in pairwise meta-analysis.

Figure 3

Figure 4 League table with color-coded CINeMA ratings. The upper triangle represents PASI90, and the lower triangle represents SAE. Hovering over the cell comparing MTX and BIME displays the reasons for downgrading.

Supplementary material: File

Yu et al. supplementary material 1

Yu et al. supplementary material
Download Yu et al. supplementary material 1(File)
File 1.7 MB
Supplementary material: File

Yu et al. supplementary material 2

Yu et al. supplementary material
Download Yu et al. supplementary material 2(File)
File 18.4 KB
Supplementary material: File

Yu et al. supplementary material 3

Yu et al. supplementary material
Download Yu et al. supplementary material 3(File)
File 24.2 MB