Background

Health technology assessment (HTA) is a crucial tool for informing evidence-based policy decisions regarding the adoption and reimbursement of health technologies (HTs) (Reference O’Rourke, Oortwijn and Schuller1) and is widely recognized as an essential component of modern healthcare systems.

In Ukraine, HTA has become increasingly integrated into the national health policy framework, specifically to substantiate key policy decisions. This integration was accelerated by the adoption of the reimbursement framework under the Ministry of Health (MOH) (2). However, these efforts have primarily focused on pharmaceuticals, creating a significant gap in the evaluation of MD, especially given the limited resources for conducting HTA (Reference Daubner-Bendes, Kovács and Niewada3).

The MD market in Ukraine is experiencing rapid growth, driven by the introduction of numerous new and innovative HT across various healthcare areas and the acceleration of domestic MD development (4). Considering that the MD market is comparable in scale to the pharmaceutical sector in Ukraine, and that a substantial portion of the state budget is allocated to the procurement of MD, amendments were introduced in December 2023 to the Resolution of the Cabinet of Ministers of Ukraine dated December 23, 2020, No. 1300 “On Approval of the Procedure for Conducting State Health Technology Assessment.” These amendments stipulate that, by January 1, 2024, methodological guidelines for conducting state HTA of MD must be developed and approved. In line with this directive, Ukraine has been actively working on the development of specific methodological guidance for HTA for MD. A draft guideline has already been prepared, representing a significant step toward institutionalizing HTA for MD, and is currently under review by relevant national interest holders.

At this stage, the application of HTA for MD in Ukraine remains voluntary and is primarily intended to support informed decision-making in public procurement and healthcare planning.

Conducting HTA for MD often differs significantly from pharmaceuticals in terms of evaluation criteria, evidence requirements, and life cycle management. Assessing MD presents unique challenges due to its iterative nature, with frequent post-market design modifications. Clinical evidence at launch is often limited, lacking randomized controlled trials (Reference D’Angela, Migliore, Gutiérrez-Ibarluzea, Polistena and Spandonaro5). Therefore, real-world data plays a crucial role. Unlike pharmaceuticals, MDs follow different regulatory pathways, relying more on post-market surveillance. Their cost-effectiveness assessment requires tailored approaches, as pricing and adoption patterns differ from those of pharmaceuticals (Reference Chalkidou, Marten and Cutler6). Additionally, factors like learning curves, incremental innovation, dynamic pricing and organizational impact vary by device type (Reference Basu, Eggington, Hallas and Strachan7).

Recognizing these challenges, the Ukrainian government has sought to expand HTA expertise beyond pharmaceuticals, ensuring that decision-makers have the tools to evaluate MD rigorously. Findings indicate that structured capacity-building programs (Reference Pichler, Oortwijn, Ruether and Trowman8) can significantly enhance the ability of local experts to evaluate MD, leading to improved decision-making and alignment with international best practices (2). To address this need, the MOH requested the Management Science of Health (MSH), through the Safe, Affordable, and Effective Medicines for Ukrainians (SAFEMed) project, to organize an online training for Ukrainian HTA experts focused on building capacity for assessing MD. SAFEMed has been working in Ukraine since 2017 in close partnership with the Government of Ukraine to modernize and strengthen the country’s pharmaceutical governance and financing systems. MSH contracted HTA International (HTAi) to undertake this work. This initiative aims to equip participants with the practical tools, methodological knowledge, and international best practices necessary to support evidence-based evaluation and decision-making related to MD in Ukraine.

This article reports on a contract between MSH and HTAi, which involved designing and delivering an online training course for Ukrainian HTA professionals. These experts had experience in pharmaceutical evaluation but lacked formal training in assessing MD. The training initiative was launched against the backdrop of Ukraine’s healthcare reforms and its efforts to integrate HTA more comprehensively into decision-making.

Approach to the program development

HTAi developed the training program through a structured, multiphase approach that included needs assessment, curriculum design, delivery and evaluation. At the onset, a detailed training needs assessment identified competency gaps and resulted in defined learning objectives. This foundational process involved extensive consultations with the SAFEMed team and international and local experts, including medical doctors and pharmacists with HTA expertise, to ensure the curriculum was contextually relevant to the Ukrainian healthcare environment. Findings indicated that while most participants were familiar with pharmaceutical HTA, few had experience or formal education in assessing MD. Based on these findings, the curriculum was designed to equip participants with the skills to understand how HTA supports decision-making for MD, critically assess its current status and conduct evaluations with a medium level of autonomy (Reference Schnell-Inderst, Mayer and Lauterberg9). It also emphasized adapting assessments to national contexts and keeping up with evolving HTA methodologies. Special attention was given to Ukraine’s local context, where HTA for MD is still developing, by incorporating sessions on adapting international assessments and emergency preparedness (Reference Miglietta, de Waure and Chronaki10).

The course was delivered entirely online via Zoom and included simultaneous interpretation, a key feature that ensured language was not a barrier to participation. This allowed for broad geographic participation and flexibility. Each session included live lectures, segments for questions and answers, case studies and home tasks (Reference Merriam and Bierema11). Case studies and home tasks gave participants the opportunity to independently navigate each phase of an assessment. Solutions to the home tasks were provided and discussed collectively. To support learning, participants received supplementary materials, including peer-reviewed articles, HTA reports and analytical tools. The course material was managed through a shared folder to which all participants had access. To ensure that the program was contemporary and had practical applications, the curriculum included modules on digital health, disinvestment and emergency preparedness.

Faculty and participants

The program brought together a diverse faculty team (11 lecturers from 8 countries), including experts in various HTA domains, all having at least 15 years of individual experience in HTA, previous publications on the lecture’s topic and current or previous leadership roles in HTAi (e.g., Interest Group chair and advisory committee member). Their collective expertise ensured comprehensive coverage of technical content and enriched participant learning through real-world insights and diverse perspectives. The multidisciplinary nature of the faculty was especially valuable for topics like engagement of interest holders and hospital-based HTA.

The training participants were selected based on their engagement in HTA and MD-related activities. This included professionals from the HTA Department of the State Expert Center of the MOH, nongovernmental HTA agencies, representatives of the MD industry and business associations, as well as experts from academic and research institutions. This inclusive approach ensured broad outreach and captured the diversity of interest holders within the HT ecosystem.

Participants’ assessment and feedback

Assessment was a core component of the program. All participants completed a baseline knowledge assessment (a 21-item questionnaire designed by the lecturers) at the start of the course. At its conclusion, they took the same test to measure learning gains. Participants were evaluated according to their performance in the post-training survey. A threshold of 75 percent was established before the training, in accordance with SAFEMed standards for demonstrating sufficient competency in the training objectives. Participants scoring at or above this threshold were considered to have successfully completed the training program and received a certificate.

The post-training survey also gathered qualitative feedback on content, lecturers, logistics and overall experience to evaluate participant satisfaction and identify areas for improvement in future training programs.

Program delivery and results

The training program was delivered over 4 months, from September to December 2024, enrolling 71 participants and 41 hours of virtual face-to-face lectures. The modular-based curriculum covered diverse topics such as economic evaluation, life cycle approaches, adaptation of HTA reports, engagement of interest holders and the use of real-world data. Innovative topics like HTA for digital HTs and emergency preparedness were also integrated to reflect global priorities (Table 1).

Table 1.

Sessions of the online training program on health technology assessment of medical devices

Results of the post-training test indicated a significant increase in knowledge. The average correct response rate improved from 75 percent to 91 percent. The number of questions answered correctly by over 80 percent of participants rose from 7 to 20. For questions with over 90 percent accuracy, the number increased from 1 to 14. Of the 71 participants, 62 achieved a passing score of the established 75 percent threshold or higher and received a certificate of completion. Participants significantly improved their understanding of key HTA domains, particularly organizational and economic aspects, where methodologies differ from those used for pharmaceuticals. They also gained specialized competencies in evaluating digital health solutions. Feedback highlighted the program’s relevance, particularly in areas such as regulatory frameworks, economic evaluations and quality of evidence. Participant feedback was overwhelmingly positive, with high ratings for content relevance, instructional quality and practical applicability. Suggestions for improvement included more interactive workshops and follow-up sessions.