Background
The use of evidence from economic evaluations (EEs) is likely to play an increasingly important role in decisions related to the introduction and implementation of healthcare interventions, supporting their sustained and optimal use (Reference Roberts, Healey and Sevdalis1). However, many countries, especially low and middle-income countries, often lack the data, resources, or expertise needed to conduct EEs using complex modeling approaches. Until such capacity is developed, decision makers in these settings may need to rely on existing EEs conducted in other countries. This raises concerns about the transferability of evidence across different policy contexts and healthcare systems (Reference Goeree, Burke and O’Reilly2). In addition, decision makers frequently face challenges in navigating large volumes of information and conflicting findings from EEs. Systematic reviews of EEs offer an efficient way to synthesize existing economic evidence to address specific value questions. However, systematic reviews typically provide only descriptive or qualitative summaries. Meta-analysis, by contrast, can pool results from multiple studies to produce more reliable quantitative estimates (Reference Veettil, Syeed, Noviyan, Thakkinstian and Chaiyakunapruk3), a method that is now being explored for application to EEs (Reference Saint, Veenstra and Sullivan4).
One example of such an effort to pool results from EEs is the incremental net benefit (INB) approach, which builds on the statistical methods proposed by Crespo et al. (Reference Crespo, Monleon, Díaz and Ríos5). Bagepally et al. later refined this approach into a structured, step-by-step data harmonization method designed to integrate inconsistently reported data (such as monetary units) and address missing data (e.g., information needed to estimate the variance of INB) from different EEs, thereby facilitating the conduct of meta-analysis of economic evaluations (MAEEs) (Reference Bagepally, Chaikledkaew, Chaiyakunapruk, Attia and Thakkinstian6). Since then, the INB-based approach has been utilized in MAEEs covering a broad spectrum of healthcare interventions and disease areas, such as vaccinations, metabolic and cardiovascular conditions, asthma, biologic treatments, and various surgical procedures (Reference Veettil, Harris and Syeed7–Reference Lambach, Silal and Sbarra18). Across these applications, MAEEs have demonstrated several potential benefits, such as enabling quantitative synthesis of economic evidence, improving comparability across studies conducted in different settings, and reducing reliance on single study estimates (Reference Veettil, Syeed, Noviyan, Thakkinstian and Chaiyakunapruk3). In addition, the quantitative evidence generated from MAEEs may support clearer policy recommendations and may facilitate decision-making in resource-constrained settings where context-specific EEs are unavailable (Reference Lambach, Silal and Sbarra18).
At the same time, important methodological challenges remain. In March 2024, the World Health Organization’s Immunization and Vaccine-related Implementation Research Advisory Committee (IVIR-AC) reviewed several examples of MAEEs and highlighted key limitations of the INB-based approach (Reference Lambach, Silal and Sbarra18). These included the need to better document and explain heterogeneity across included studies and to develop standardized approaches for addressing such heterogeneity. The committee also noted uncertainty regarding how studies should be weighted within MAEEs, as uncertainty intervals in individual EEs often reflect the extent of uncertainty explored by modelers rather than inherent variability in cost-effectiveness estimates. Furthermore, there is currently no consensus on which outcome indicators from EEs are most appropriate for meta-analysis. The committee acknowledged that MAEEs may support decision-making in countries lacking context-specific EEs; however, it emphasized the need to first better understand the value and demand for such evidence among end users and subsequently to address the methodological challenges involved.
Although methodological developments continue to evolve, limited evidence exists to understand end users’ perceptions of, demand for, and the applicability of MAEE findings in real-world decision-making (Reference Dhippayom, Meraz and Lee17). Understanding these perspectives is essential to guide methodological refinement, capacity building, and effective communication of MAEE results. We conducted a cross-sectional online survey among participants of the ISPOR Real-World Evidence Summit (Japan, October 2025), following a workshop that introduced MAEE concepts, presented case studies, and facilitated discussions on the benefits, limitations, and practical applicability. Insights from this survey aim to inform future methodological improvements of the INB-based approach, enhance its utility for decision support, and guide the design of broader surveys involving diverse stakeholder groups across different contexts and healthcare settings. Accordingly, the objective of this study was to examine end users’ perceptions, demand, and the practical applicability of MAEE findings in real-world decision-making.
Methods
Study design and participants
We conducted a cross-sectional online survey among attendees of an ISPOR Real-World Evidence Summit breakout session titled “Informing Health Policy Decisions Through Meta-Analysis of Economic Evaluations: Why and How to Do It? How to Connect with Users?” held on September 29, 2025. This session was part of the ISPOR Real-World Evidence Summit 2025: Through the Lens of Asia Pacific.
Participation was voluntary, and responses were collected anonymously using the Poll Everywhere® platform (https://www.polleverywhere.com/). Participants entered self-generated screen names, and no identifying information was collected. The target population included all individuals attending the breakout session; therefore, no formal sample size calculation was performed. Ethical approval for this study was obtained from the University of Utah Institutional Review Board (IRB_00179956). Participants were informed at the start of the survey that participation was voluntary, that responses would be collected anonymously, and that submission of the completed survey implied their informed consent to participate. No incentives were provided for participation.
Survey instrument development
The survey instrument was developed collaboratively by the study team through an iterative, structured process. Initial survey items were informed by key MAEE concepts introduced in the workshop, including the rationale for the INB-based approach, applied case studies, and facilitated discussions on the benefits, limitations, and practical applications of MAEEs. Together, these materials helped identify the user perceptions and information needs the questionnaire sought to assess. Insights from prior discussions of the WHO Immunization and IVIR-AC on MAEEs (March 2024) were incorporated to ensure that the questionnaire reflected relevant end-user perspectives, particularly around awareness, expected value, and use cases of MAEEs (Reference Lambach, Silal and Sbarra18).
Draft questions were refined through multiple rounds of team discussion to enhance clarity, content validity, and appropriateness of response formats. The final questionnaire consisted of 10 items spanning four domains: (1) respondent characteristics (role, region, and experience with EEs); (2) awareness and prior exposure to MAEEs; (3) perceived usefulness and potential applications of MAEEs; and (4) perceived end users and likelihood of future use (Supplementary Table 1). Items included single-response, multiple-response, and one open-ended question. The instrument was refined through iterative team discussions to ensure clarity and content validity and was internally tested by study team members on the Poll Everywhere® platform to confirm usability and comprehension prior to administration.
Data collection and analysis
The survey link was displayed live during the 60-minute breakout session, with the link shown on the presentation slides. Participants were invited to respond using their personal devices while the session was ongoing, completing each survey section as it was presented. Poll Everywhere® automatically captured responses in real time. No duplicate-response restrictions were applied; however, each device was limited to one response per question. All survey data were subsequently exported into Microsoft Excel for Microsoft 365 (Version 2408) for descriptive analysis.
Counts and percentages were calculated for each question, using the number of respondents to that specific question as the denominator. For multiple-selection items, frequencies and proportions were reported for each selected option. To explore whether participants’ outlook toward MAEEs differed by respondent characteristics (e.g., familiarity with EEs, involvement in decision-making), responses were compared across groups. Chi-square tests and odds ratios (OR) with 95 percent confidence intervals (CIs) were calculated using Stata version 17 to examine associations between categorical variables, with statistical significance defined as p < 0.05.
One open-ended question invited participants to describe any prior use of MAEEs in decision-making or research contexts. Contingent upon a sufficient number of responses, open-ended survey questions were qualitatively reviewed and summarized.
Results
A total of 76 participants completed the survey, of whom 29 answered all questions (Supplementary Table 2). Most respondents were based in the WHO South-East Asia Region (72.9 percent), followed by the Western Pacific Region (20.0 percent), and Europe (7.1 percent). Participants were primarily from Industry/Pharma/MedTech (36.8 percent) and Academic/Research (34.0 percent), with smaller proportions from Students/Early Career (13.6 percent), Consultants (11.8 percent), Policymakers/Government Agencies (2.6 percent), and Other (1.2 percent). The Industry/Pharma/MedTech category includes respondents engaged in research, development, manufacturing, and other related activities within these sectors. Regarding experience with EEs, 46 percent had less than 5 years, 24 percent had 5–1 years, 16 percent had more than 10 years, and 13 percent reported no experience.
Awareness and prior use of MAEEs in decision-making or research
Among the 75 participants who answered the question on whether they had heard of this MAEE approach before, 35 (46.7 percent) reported encountering it for the first time during the workshop, while 40 (53.3 percent) had prior awareness of the approach. Despite this awareness, practical experience remained very limited, as none of the respondents (0/75) had conducted an MAEE using the methodology presented. Among the 40 participants who answered the question on prior use of MAEE in decision-making, research support, or citing a paper, 36 (90.0 percent) had never utilized MAEE evidence, whereas four (10.0 percent) reported having used it. These findings indicate that while awareness of MAEEs exists among participants, practical experience in conducting MAEE remains limited. The open-ended question, which asked participants to describe any direct use of MAEEs in decision-making or research contexts, received no responses; therefore, no qualitative analysis was conducted. Detailed results for these items are provided in Table 1.
Awareness and prior use of MAEEs
INB, incremental net benefit; MAEE, meta-analysis of economic evaluation; N, number of participants who selected each response.
Perceived benefits of MAEE
Perceptions regarding the value of MAEEs were generally positive. Among the 68 respondents who addressed this item, eight (11.8 percent) considered MAEEs “definitely” beneficial, while the majority (n = 45, 66.2 percent) perceived them as “possibly” beneficial. Twelve participants (17.6 percent) were unsure about the benefits, and only three (4.4 percent) did not consider MAEEs beneficial. Sixty-three participants answered the question on the potential advantages of conducting MAEEs in situations where multiple cost-effectiveness analyses from the same country exist on the same topic. Among them, 46 (73.0 percent) indicated that MAEEs could help identify variation across studies, 42 (66.7 percent) reported that MAEEs could reduce single-study bias, and 36 (57.1 percent) felt that MAEEs could improve precision by pooling results (Figure 1). A small minority (n = 4, 6.4 percent) reported being unsure or perceiving little benefit.
Perceived potential benefits of MAEE.
Potential users and likelihood of use
Participants identified a broad range of potential users for MAEEs, with multiple responses permitted. Sixty participants answered the question on the potential users of MAEEs. National health authorities and policy makers were the most frequently cited (n = 42; 70.0 percent), followed by countries with limited EE capacity (n = 37; 61.7 percent), healthcare organizations such as hospitals and insurers (n = 33; 55.0 percent), and the pharmaceutical or MedTech industry (n = 24; 40.0 percent). Regarding the likelihood of using MAEEs in policymaking, 56 participants responded: six (10.7 percent) indicated they would be very likely to use MAEEs, 25 (44.6 percent) likely, 18 (32.1 percent) were unsure, and smaller proportions reported being unlikely (n = 4, 7.1 percent) or very unlikely (n = 3, 5.4 percent) to apply MAEEs in their decision-making. Detailed responses for the likelihood of use are provided in Table 2, and the distribution of potential users is illustrated in Figure 2.
Likelihood of use of MAEEs in policymaking (n = 56)
MAEE, meta-analysis of economic evaluation; N, number of participants who selected each response.
Potential users of MAEE.
Association of respondent characteristics and perceptions toward MAEEs
We explored whether respondent characteristics, specifically experience with EEs and prior awareness of MAEEs, were associated with perceptions of MAEE benefits or likelihood of future use (Table 3).
Associations between participant characteristics and MAEE perceptions
EE, economic evaluation; MAEE, meta-analysis of economic evaluation.
Participants with ≤5 years of experience in EE were slightly more likely to view MAEEs favorably compared with those with more than 5 years of experience (84.2 percent vs. 71.4 percent). However, this difference was not statistically significant (OR, 2.13, 95 percent CI: 0.65–7.06; p = 0.209). Similarly, experience level did not show a significant relationship with likelihood of future MAEE use. Although participants with less experience had a somewhat higher proportion of favorable responses than those with greater experience (59.4 percent vs. 50.0 percent), the association was nonsignificant (OR, 1.46, 95 percent CI: 0.50–4.25; p = 0.485).
Prior awareness of MAEEs did not meaningfully influence respondents’ perceptions. Participants who were previously aware of MAEEs reported favorable views at nearly the same rate as those who were not (77.1 percent vs. 78.1 percent; OR = 0.95, 95 percent CI: 0.30–2.99; p = 0.923). Similarly, prior awareness was not associated with a greater likelihood of using MAEE findings for decision-making, with favorable responses remaining comparable across groups (51.6 percent among those aware vs. 62.5 percent among those not aware; OR = 0.64, 95 percent CI: 0.22–1.90; p = 0.419).
Discussion
The volume of EEs has expanded substantially across diverse health conditions and interventions since the 1990s (Reference Kim and Basu19). While systematic reviews collate and summarize this evidence, they typically provide only descriptive insights (Reference Gomersall, Jadotte and Xue20). The INB-based MAEE method enables quantitative pooling of results, offering evidence that may be more directly useful for policy decisions (Reference Bagepally, Chaikledkaew, Chaiyakunapruk, Attia and Thakkinstian6). Despite recognition of the importance of stakeholder perceptions (Reference Lambach, Silal and Sbarra18), no studies have systematically examined end users’ perceptions of the value and applicability of MAEEs. This preliminary study addresses that gap by collecting real-time feedback from participants after a workshop on MAEE methods, providing a basis for a larger survey across diverse contexts.
Our survey findings indicate that, despite limited prior awareness or experience with MAEEs, nearly 90 percent of participants had never conducted one using this approach, yet stakeholders generally viewed the method positively once it was introduced. When asked about the likelihood of using MAEEs in policymaking, around 55 percent of participants indicated they would be likely or very likely to apply the method, while a smaller proportion were unsure or unlikely to do so. These findings can be understood through the lens of innovation adoption, consistent with diffusion of innovations theory (Reference Dearing and Cox21). Even though participants lacked prior experience with MAEEs, the workshop demonstrated the method’s usefulness, clarity, and relevance, which likely shaped positive perceptions. For many participants, the intention to use MAEEs was positive, but some, while recognizing the method’s value, were uncertain or hesitant, likely due to the complexity of handling heterogeneity when pooling EEs, the data requirements for pooling, or because the method’s application may be limited to specific contexts rather than broadly generalizable. Future efforts should focus on providing clearer guidance and capacity building to address methodological challenges and promote the effective use of MAEEs across diverse contexts.
Participants identified national health authorities and policy makers as the most frequent potential users of MAEEs, followed by countries with limited EE capacity, healthcare organizations such as hospitals and insurers, and the pharmaceutical or MedTech industry. This is expected, as policy makers rely on economic evidence for resource allocation, while countries with limited EE capacity, healthcare organizations, and the pharmaceutical/MedTech industry may use MAEEs to inform decisions and supplement local context evidence. We did not find significant associations between respondents’ experience with EEs or their prior awareness of MAEEs and their perceptions of the benefits or likelihood of using MAEE findings. This may imply that interest in MAEE is broad-based, encompassing users of all experience levels, and that barriers may be structural rather than user specific. However, participants with ≤5 years of experience were slightly more likely to view MAEEs favorably, though not statistically significant, indicating a possible trend of greater enthusiasm among less experienced users. Future studies with larger respondent pools are needed to further explore potential associations between experience level and perceptions of MAEEs.
Stakeholders often report challenges in interpreting EEs due to complex models, variable assumptions, and conflicting findings (Reference Garrison, Neumann and Willke22–Reference Biskupiak, Oderda and Brixner25). MAEEs offer a structured way to address these issues by systematically identifying relevant studies, assessing study quality and potential bias, and combining results to explore variation across assumptions, perspectives, or population subgroups (Reference Veettil, Syeed, Noviyan, Thakkinstian and Chaiyakunapruk3). This quantitative synthesis helps clarify inconsistent findings and reduces reliance on single studies. Respondents in this survey similarly highlighted the value of MAEE, particularly when multiple evaluations from the same country analyze the same topic, citing their ability to reveal heterogeneity, minimize bias, and improve precision. These insights highlight MAEE’s potential to inform priority setting and strengthen the use of evidence in real-world policy decisions, particularly in regions with increasing but heterogeneous EE outputs. However, when EEs on the same topic are drawn from different countries, MAEEs often exhibit substantial heterogeneity, even when analyses are stratified by income level, perspective, or other relevant assumptions (Reference Veettil, Syeed, Noviyan, Thakkinstian and Chaiyakunapruk3). This variability highlights a major methodological gap, particularly regarding the transferability of economic evidence, and underscores the need for improved methods to manage heterogeneity to enhance MAEEs’ relevance for policy decisions.
This study has several limitations. First, most respondents were from the WHO South-East Asia and Western Pacific regions, which may limit the generalizability of the findings to other geographic contexts. This group, particularly those from the South-East Asia region, likely included a mix of low- and middle-income countries that may lack the data, resources, or expertise to conduct EEs. As a result, respondents may have viewed the MAEE methodology favorably because it provides an accessible framework for synthesizing quantitative evidence from countries with varying income levels, supporting policy decisions without relying exclusively on locally generated EEs or evidence from contexts that differ substantially from their own. MAEEs may therefore be perceived as a practical tool for informing decisions in settings where context-specific evidence is limited. Second, the voluntary nature of the survey and low response rates for some items highlight variability in familiarity with MAEE methods and limited confidence in answering technical questions, which may influence interpretation. There is a potential for response bias because respondents had attended the workshop, which may have influenced their perceptions of MAEE. Exposure to the workshop content and engagement with facilitators could have led to more favorable views than those of individuals without such prior exposure, limiting generalizability. This emphasizes a key limitation of online opinion polls conducted during a conference, as the results reflect subjective perceptions rather than objective measures of MAEE applicability, and caution is needed when interpreting the findings. These observations underscore the need for clearer practical guidance and targeted training for users of EEs. In future work, it is important for participants to understand both the methodological principles of MAEE and its practical applications in policy decision-making in different contexts to ensure meaningful engagement. Third, the relatively small sample size may have limited the ability to detect associations between respondent characteristics and perceptions of MAEEs. Future surveys could explore strategies to enhance participation, such as offering incentives, providing preconference materials, and more effectively disseminating MAEE studies to increase awareness and engagement among potential respondents. Fourth, the study included relatively few policy makers, which may limit insights into decision-making perspectives. Additionally, because the survey reflects an opinion poll among a specific workshop cohort, findings may overestimate positive perceptions of MAEEs and cannot be generalized to the broader population of potential users. Future research should aim to conduct a larger, more representative survey across multiple regions to enhance generalizability and capture a wider range of perspectives. Additionally, incorporating qualitative approaches, such as focus group discussions or in-depth interviews with key stakeholders from diverse contexts could provide richer insights into perceptions, challenges, and potential strategies. Such mixed-method approaches would improve the development and refinement of the methodology, supporting its utility for policy decisions and practice.
Overall, our results suggest that while MAEEs remain relatively unfamiliar to many stakeholders, they are generally well-received, highlighting the opportunity to strengthen targeted capacity building and provide clearer methodological guidance to facilitate wider adoption.
Conclusion
This study provides preliminary insights into stakeholders’ perceptions of MAEEs. Respondents generally expressed positive views of the methodology and noted that it could be valuable for informing policy decisions. Although the findings should be interpreted cautiously due to the modest sample size and voluntary participation, the results indicate some level of interest in MAEEs across various professional backgrounds. Respondents indicated that national health authorities, policy makers, and organizations in countries with limited EE capacity are the most likely users. The survey also suggests that recognition of potential benefits in MAEEs is widespread across different experience levels, and adoption may be constrained more by system-level barriers, such as lack of guidance, than by individual users’ knowledge or skills. These findings highlight the opportunity to strengthen capacity building and provide clearer methodological guidance to support wider adoption. Larger surveys and qualitative studies with diverse stakeholders are needed to explore context-specific applicability, refine methodology, and strengthen their utility for informed policy decisions across settings.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0266462326103742.
Funding statement
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interests
The authors declare none.
Open access
