Eligibility criteria

We were primarily interested in three types of studies, namely studies that transferred an HTA from one setting to another, along with a description of experiences arising; studies that described a tool, approach, or method for transferring an existing HTA, including an explanation of how the tool could be used in practice; and systematic reviews of various tools for HTA transfers. There were no restrictions applied to the type of HTA report, and the HTAs could include multiple domains or only a specific domain such as clinical effectiveness, safety, or economic evaluation. We excluded studies providing methods that mainly reused HTA recommendations and not the underlying assessments. For instance, Multiple-Criteria Decision Analysis (MCDA) or deliberative processes for priority setting are tools primarily developed to be used by decision-makers (HTA users) and were considered outside of the “HTA-analysis” step, as well as mini-HTAs, hospital HTAs, and HTA fast-tracking. Lastly, studies needed to be accessible in full text, related to public health, and published in English after 2005 to be eligible for inclusion.

Search strategy

To identify relevant literature, an information specialist helped to develop the search strategy and performed literature searches in November 2020 in six electronic databases: Ovid Medline, Embase, Cochrane Database of Systematic Reviews, Scopus, Epistemonikos, and the Cochrane Methods Methodology Register (Supplementary Material S2). We did not search in reference lists nor for grey literature but included conference abstracts from the past three years. Additionally, we sent a short questionnaire to members of the International Network of Agencies for Health Technology Assessment (INAHTA) about whether they knew or used any transfer tools or had any transfer experiences, in order to identify tools in development or not publicly available (Supplementary Material S3).

Study selection, data extraction, and charting

Articles fulfilling the inclusion criteria were uploaded to Covidence for title and abstract screening performed by four reviewers (LF, EP, IS, and LC), where at least two reviewers independently screened references. EP and LF read and screened the full texts, while IS and LC verified their decisions. Disagreements were resolved through discussion or by consulting a third independent reviewer. EP and LF used Microsoft Excel (2016) to document, extract, and chart data. Items on the standardized data extraction sheet were guided by the research questions (Supplementary Material S1). We extracted or summarized data on all relevant items from each article. Afterwards, articles were grouped per HTA domain(s) and the data were narratively summarized. The strengths and weaknesses were identified, thematically grouped, and described, with specific attention to LMICs.

The “common” structure of HTA transfers

There is no consensus on how to structure HTA transfers; however, our findings indicated overlap and common features between HTA transfers and processes for de novo HTA production. Figure 2 illustrates the overall structure of an HTA transfer. To summarize, after having decided to opt for an HTA transfer (step 1), a relevant HTA or research underlying a specific HTA domain needs to be identified. This process commonly uses pre-defined selection criteria and follows procedural recommendations as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement (Reference Page, McKenzie and Bossuyt39). The checklist by Grutters et al. (Reference Grutters, Seferina and Tjan-Heijnen40) can help to determine and frame the relevance, or applicability, of an HTA to the identified research question (step 2). The identified HTA(s) is then assessed for methodological quality, risk of bias, and transferability (step 3). Step 3 is crucial in the transfer process, as transferring a low-quality product might lead to ambiguous, misleading, or even wrong conclusions. At this step, approaches start to diverge, for instance in the order of the quality, risk of bias, or transferability assessments. Some approaches even fuse steps 2 and 3, combining applicability and partial quality assessments into one. What most tools have in common, however, is that they prepare for transfers like a catalyst in a chemical reaction. An additional step might be added to identify transferability factors, which refers to factors that need to be adapted or replaced with local data. After identifying transferability factors, and when necessary, adapting or replacing them (sub-step 3), relevant data can be extracted (step 4). Then, the results are reported (step 5), discussed, and interpreted (step 6). Finally, the appraisal (step 7) and implementation in the local setting (step 8) conclude the HTA transfer process.

EUnetHTA adaptation toolkit

The EUnetHTA Adaptation Toolkit consists of two sections (Reference Turner, Chase and Milne4;Reference Guegan, Milne and Pordage14;Reference Chase, Rosten, Turner, Hicks and Milne31–Reference Kristensen, Lampe and Chase33). First, eight speedy sifting questions (on topic, language, description of the technology, methods, scope, peer review, conflict of interest, and timeliness) appraise the applicability of the identified HTA. After deciding to continue the adaption process, the second section includes questions on relevance, reliability, and transferability for five “transferable” HTA domains (use of technology, safety, effectiveness, economic evaluation, and organization) (Reference Guegan, Milne and Pordage14). The MAST Application is derived from the Adaptation Toolkit, but with guidance questions specific to HTAs on telemedicine applications (Reference Kidholm, Ekeland and Jensen34).

Macpherson and Thompson (Reference Macpherson and Thompson24) describe their experiences using the toolkit to adapt two relative effectiveness assessments produced by EUnetHTA to inform decision-making in the National Health Service in Scotland. The modular structure of the toolkit makes it possible to use the whole or only parts thereof (Reference Guegan, Milne and Pordage14); and Gorry, McCullagh, and Barry (Reference Gorry, McCullagh and Barry23) used questions from the economic evaluation domain to assess the generic and specific transferability of published economic evaluations on treatments for advanced melanoma to the Irish setting.

TRANSFER approach guides interpretation of transferability in systematic reviews

The TRANSFER approach mirrors the standard systematic review process in seven stages and provides guidance to assess and integrate the transferability of review findings. For instance, templates for collaborating with decision-makers and identifying context-specific “transfer” factors are included (Reference Munthe-Kaas, Nøkleby, Lewin and Glenton21).

Systematic reviews of economic evaluations

A systematic review of economic evaluations collects economic evidence about specific health interventions to inform evidence-based decisions. The article series by van Mastrigt et al. (Reference Thielen, Van Mastrigt and Burgers18–Reference Wijnen, Van Mastrigt and Redekop20) describes the process, with expert best-practice recommendations for each step, as follows: (i) initiate the review; (ii) identify and select economic evaluations; (iii) data extraction and assessment of the risk of bias and transferability; (iv) reporting results; (v) discussion and interpretation of results. Economic evaluations must “pass” the quality and risk of bias assessment before an assessment of transferability is relevant. Wijnen et al. (Reference Wijnen, Van Mastrigt and Redekop20) provide a comprehensive overview of checklists that can facilitate these quality assessments. Notably, they recommend the Philips Checklist (Reference Philips, Bojke, Sculpher, Claxton and Golder41) for assessing the risk of bias in model-based economic evaluation. However, the Philips Checklist is rather long, so, when over ten economic evaluations are under review, the checklist by Caro et al. (Reference Caro, Eddy and Kan42) was recommended. For trial-based economic evaluations, Wijnen et al. (Reference Wijnen, Van Mastrigt and Redekop20) recommended the BMJ (Reference Drummond and Jefferson43) or CHEC-extended (Reference Evers, Goossens, De Vet, Van Tulder and Ament44) checklist. Further, identified economic evaluations may be compared to local guidelines to fit methodological standards and ensure relevancy. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has collected thirty-three country-specific guidelines that may be used for this purpose (Reference Eldessouki and Smith45) [access at: https://tools.ispor.org/peguidelines/]. Lastly, Wijnen et al. (Reference Wijnen, Van Mastrigt and Redekop20) highlighted the Welte checklist (Reference Welte, Feenstra, Jager and Leidl46) as a convenient transferability checklist.

The search strategy identified three papers conducting systematic searches to find and transfer economic evaluations. Checklists were primarily used to assess if results could be adopted (referring to a direct transfer without any changes) in the local setting. Only the article by Nystrand et al. (Reference Nystrand, Gebreslassie, Ssegonja, Feldman and Sampaio25) used the “Van Mastrigt” guidance (Reference Thielen, Van Mastrigt and Burgers18–Reference Wijnen, Van Mastrigt and Redekop20). They used multiple tools for the transferability assessments, including a country-specific checklist from the Swedish HTA Agency (SBU) that covered transferability items other checklists did not consider (e.g., the feasibility of implementation, standard practice in the comparator, and health financing mechanism) (47). Van Haalen, Severens, Tran-Duy, and Boonen used a combination of checklists and approaches like the disease-specific reference case, the Welte checklist (Reference Welte, Feenstra, Jager and Leidl46), the Drummond-ISPOR approach (Reference Drummond, Barbieri and Cook48), and the Phillips checklist (Reference Philips, Bojke, Sculpher, Claxton and Golder41). Ruggeri et al. (Reference Ruggeri, Drago, Rosiello, Orlando and Santori27) used pre-defined criteria and the Augustovski checklist (Reference Augustovski, Iglesias and Manca5) to assess generalizability. All studies found that economic evaluations could be transferred although adaptations were required, for instance replacing unit costs.

Catalyst transfer tools for economic evaluations

Another way to transfer economic evaluations is exemplified by the more ad-hoc approaches that identified or had access to existing economic evaluations and adapted underlying models to fit the new setting (Reference Ademi, Tomonaga and van Stiphout28–Reference Alshreef, MacQuilkan and Dawkins30). These approaches followed the steps from Figure 2, using a variety of tools to assess applicability, quality, bias, or transferability. Ademi et al. (Reference Ademi, Tomonaga and van Stiphout28) used multiple tools and adapted the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist (Reference Husereau, Drummond and Augustovski49) to predefine exclusion criteria for the selection process (step 2), while Essers et al. (Reference Essers, Seferina and Tjan-Heijnen29) used the knock-out criteria in the Welte checklist (Reference Welte, Feenstra, Jager and Leidl46) as a quick-sifting tool. If economic evaluations failed to report items from the CHEERS (Reference Husereau, Drummond and Augustovski49) or Welte checklist (Reference Welte, Feenstra, Jager and Leidl46), they were deemed non-transferable and excluded. After relevant economic evaluations were identified and selected, Ademi et al. (Reference Ademi, Tomonaga and van Stiphout28) used quality assessments to judge the methods’ characteristics and other aspects, while neither Alshreef et al. (Reference Alshreef, MacQuilkan and Dawkins30) nor Essers et al. (Reference Essers, Seferina and Tjan-Heijnen29) included quality assessments. Both used models previously applied in another context, and this validation was considered appropriate. However, Essers et al. (Reference Essers, Seferina and Tjan-Heijnen29) did use the Urdahl (Reference Urdahl, Manca and Sculpher50) and EURONHEED Checklist (Reference Boulenger, Nixon and Drummond35), and Alshreef et al. (Reference Alshreef, MacQuilkan and Dawkins30) used the Mullins checklist (Reference Mullins, Onwudiwe and Branco de Araújo51) to identify factors that could limit the transfer (step 3). Identified factors included costing parameters, resource use, discount rates, and outcome measures (i.e., mortality, utility, and health-related quality of life). All papers adjusted factors that could limit transferability by replacing them with local data if available. Otherwise, expert assumptions or international studies were used. Table 2 provides an overview the identified checklist that can aid the transfers of economic evaluations.

HTA transferability issues to consider

The articles included in this mapping all reported on factors and issues that may affect the transferability of HTAs. For instance, it is essential to identify and access relevant HTAs, research papers, or models to avoid duplication (Reference Wijnen, Van Mastrigt and Redekop20;Reference Gorry, McCullagh and Barry23;Reference Nystrand, Gebreslassie, Ssegonja, Feldman and Sampaio25;Reference Ademi, Tomonaga and van Stiphout28;Reference Essers, Seferina and Tjan-Heijnen29;Reference Goeree, He and O’Reilly38). Differences in the context of the technology under review (i.e., resources available, the standard of care, type of health system, and population) further hinder the transferability of HTAs (Reference Alshreef, MacQuilkan and Dawkins30;Reference Kidholm, Ekeland and Jensen34;Reference Brozek, Canelo-Aybar and Akl37). Replacing key parameters with local data can help to ensure relevant estimates, but the country-specific data are often lacking in resource-limited settings. Sensitivity analyses and other estimation techniques could help fill any data gaps, but will require knowledge and skills (Reference Kidholm, Ekeland and Jensen34;Reference Boulenger, Nixon and Drummond35;Reference Goeree, He and O’Reilly38). Additionally, knowledge of HTA, and specifically appraisal skills, is essential. HTAs that use poor-quality data will only have limited use for a transfer, as this might perpetuate poor HTA recommendations (Reference Turner, Chase and Milne4;Reference Chase, Rosten, Turner, Hicks and Milne31;Reference Turner, Chase and Milne32). Lastly, differences in HTA practices also affect the ease of transferability. Hence, the challenges of HTA transfers should not be underestimated as they still require time, scientific expertise, and other resources.

Table 3 presents an overview of all the identified transferability issues, which we summarized in five overarching themes: (i) lack of access to original studies or models, (ii); poor quality of HTAs, underlying studies, and models; (iii) lack of local data limits possibility to adapt existing HTAs; (iv) local HTA requirements ensure relevancy, but diversify end-products; and (v) significant resources are required to conduct HTA transfers.

Table 3.

Factors that may influence the transferability of HTAs

HTA, health technology assessment; PRISMA, preferred reporting items for systematic reviews and meta-analyses.

Implications for practice

Our findings indicated that the EUnetHTA Adaptation Toolkit was the most comprehensive and versatile, including separate checklists for five domains (use of technology, safety, effectiveness, economic evaluation, and organization). However, other tools might fit better depending on the type of evidence to be transferred. For instance, Alshreef et al. (Reference Alshreef, MacQuilkan and Dawkins30) omitted the EUnetHTA Adaptation Toolkit (Reference Guegan, Milne and Pordage14) because it was not specific to model adaptation. The Mullins checklist (Reference Mullins, Onwudiwe and Branco de Araújo51) was selected given it fitted best with their criteria, which included relevance to model adaptation, endorsement from a respected organization, compatibility with the International Decision Support Initiative’s reference case for economic evaluations, transparency, inclusiveness, length, and external validity. Hence, which tool to select to transfer HTAs in any setting is not straightforward and careful consideration is required.

All HTA transfers will face trade-offs between minimizing bias and maximizing precision when selecting evidence. These issues are often magnified in LMICs, which have limited access to appropriate local data, and where research from HICs is usually not as easily applicable or generalizable. Therefore, low resource settings must be pragmatic, as HTA transfers will never be perfect. Local or generalizable studies from other settings might not exist, whereas transferring and adapting applicable studies that answer the right question, even if the study population is small or quality is low, can still provide valuable insights (Reference Hawkins, Heggie, Wu, Glassman, Giedion and Smith56). There is a need to strengthen capacity and provide pragmatic guidance which discusses the trade-offs, benefits, and disadvantages of HTA transfers. Additionally, involving stakeholders is essential as they can provide expert information, ensure relevant research questions are asked, inform about transferability, and comment on the uncertainty of transferred HTAs (Reference Pinho-Gomes, Stone and Shaw7;Reference Trowman, Powers and Ollendorf8;Reference Munthe-Kaas, Nøkleby, Lewin and Glenton21).

HTA transfers are only one of the approaches which could benefit LMICs. There are various other pragmatic and rapid HTA approaches, also referred to as Adaptive HTA (aHTA), that reuse existing evidence to inform decisions, such as expedited processes, adaptation of global datasets, reuse of reviews, and price-benchmarking (Reference Nemzoff, Ruiz and Chalkidou2). These aHTA approaches require a diverse range of knowledge and skills to interpret the findings and understand uncertainty potentially added by the adaptation, or transfer, processes. There is a need for more research to understand how and when HTA transfers, as well as other aHTA approaches, are appropriate and helpful in LMICs, and what type of guidance is relevant for settings with fewer capacities, resources, and defined processes for HTA.

Strengths and limitations

The main strengths of this scoping review are the comprehensive search and up-to-date overview of tools, checklist, and approaches that can be used when considering an HTA transfer. We also addressed key issues related to definitions and missing guidance which may interfere with HTA transfers, specifically in LMIC settings.

This scoping review also has some limitations. Firstly, we focused on tools relevant to HTA analysis rather than the whole HTA process. Consequently, possible transfer tools related to “transferability” such as MCDA, deliberative processes for priority setting, appraisal, and frameworks for topic selection or stakeholder engagement that could be part of the transferability toolbox were outside of the scope of this review and excluded. Secondly, to reduce the volume of literature, the search strategy was focused on the clinical effectiveness and economic domains, and less explicitly on other HTA domains. Transferring existing evidence is complex, as it addresses many components and disciplines. The heterogeneity of the included studies reflects this complexity and made it difficult to generalize findings. We are aware that other relevant tools (e.g., country guidelines for rapid HTA methods, appraisal checklist, etc.) exist. Nevertheless, by including conference abstracts and the questionnaire to INAHTA members, we are confident that we have captured the most used transferability tools for HTA analysis currently available.