Defining the value and impact of HTA

The value and impact of HTA can be defined in a multitude of ways and quantification is challenging. Determining a useful counterfactual (i.e., what would happen in the same health system without HTA) is problematic due to the many contemporaneous changes that may have occurred in the health system after HTA’s inception and the fact that technologies may be implemented irrespective of the work of a HTA body (Reference Bouttell, Craig, Lewsey, Robinson and Popham14). It was noted, however, that retrospective reviews of the anticipated health outcomes (as estimated through the HTA process) compared with the actual health outcomes could be considered as a possible proxy for a counterfactual, particularly in areas where longer-term data may now be available.

Furthermore, value and impact assessment can have many uses, from informing internal process and methods improvement to reporting toward continued funding or submitting business cases for additional funding (Reference Greenhalgh, Raftery, Hanney and Glover15). Therefore, what is measured, when, with what methods, and by whom will all be influenced by the purpose and audience for such assessments. The perspective that the value is assessed from is also an important a priori consideration. For example, a patient may place greater value on rapid access to innovative treatments without fully demonstrated clinical benefit, whereas a government, payer or even society as a whole, may place greater value on access to technologies that will maximize population health more broadly, with some patients potentially disadvantaged as a result (although it is acknowledged that this is a nuanced concept and there may be government and societal preferences toward technologies that may benefit disadvantaged groups). There are also examples of industry members who have developed and published statements supporting the value of HTA, an example of which was presented in the panel session (16).

The value and impact of HTA is also closely linked to the remit of the HTA body within its healthcare system. Clearly defining and understanding the remit that a HTA body has must occur before the value of the HTA activities can be determined (e.g., considering whether a HTA body must provide reimbursement advice on all new technologies within a specific timeframe from regulatory approval). The remit will also likely have a direct influence on how recommendations and reports from a HTA body are used and implemented (e.g., in systems where HTA recommendations are mandated) and this will have a substantial effect on the downstream impact and subsequent value of the HTA. It is possible that the greatest impact of HTA bodies comes when the HTA body is well integrated into the health system, so that the HTA recommendations are more likely to be effectively disseminated and implemented. Where this integration is less effective, stakeholder management with relevant stakeholders such as the public, providers, payers, and others, is often a cornerstone of the HTA process to facilitate adoption and implementation of final HTA recommendations (Reference Werkö, Merlin and Lambert17).

Measuring the value and impact of HTA

As noted, the value that an individual places on a particular outcome or impact may differ according to personal beliefs, cultures, and a range of possible other factors such as the overall health and wealth of a country. Therefore, value and impact assessment need to be relevant to the local context, disease or condition, and the perspective of the stakeholder(s) should be considered within the assessment.

Despite this, there are some general considerations pertaining to value and impact assessment. Firstly, the reason for conducting the assessment should be clear; for example, determining accountability of public funds and/or undertaking learning and process improvements. Secondly, tools such as the INAHTA impact framework (Reference Schuller and Söderholm Werkö18;Reference O’Rourke, Werkö, Merlin, Huang and Schuller19) were noted as particularly useful. This framework has been adapted and utilized by a number of HTA bodies already. Such a framework could therefore serve as a starting point and could be expanded to include consideration of certain interdependencies such as interaction with regulatory authorities and other health sector stakeholders and decision makers (where relevant). Additionally, system level metrics and a checklist could be added to help apply the framework across multiple jurisdictions. Such developments could enable the INAHTA framework to be applied at a local (HTA body) context (as is the case now) and then used more widely to share learnings across jurisdictions.

GPF members further suggested that value and impact assessments should be done regularly to track progress, as part of a continuous learning cycle. However, it was noted that there should be flexibility in the timing of the assessments and that conducting assessments in a short-time frame would not allow for alterations and subsequent changes to be made and impacts felt. GPF members felt that 5-yearly increments would allow sufficient time to implement recommendations and process changes.

Finally, if HTA recommendations are not implemented, then it stands to reason that the full potential impact and subsequent value of the HTA recommendations on the health system will not be realized. This is not a new challenge as noted by Henshall et al. in 2002 (Reference Henshall, Koch and von Below20); it has been observed that this aspect is one of the weakest elements of the HTA process. There needs to be greater effort by both the HTA bodies but also and other health system stakeholders such as policy and decision makers to carefully consider the feasibility of implementing the recommendations at a system level (with notable examples of how this is done by ZIN and Health Technology Wales discussed at the GPF and outlined on their respective web sites). These considerations should explore whether there are any supportive recommendations or actions that could facilitate implementation of HTA recommendations through closer consultation with key stakeholders (such as patients, clinicians – including medical societies who often develop clinical guidelines, payers, and health system managers) to increase the likelihood of successful implementation. Potential barriers to implementing HTA recommendations should be highlighted and explored. An example of such a challenge includes enforcing eligibility criteria for certain technologies that are not realistic for patients to achieve (e.g., patients who manage a condition primarily at home, but need a certain number of emergency department consultations to be considered eligible for a drug). Where technologies require system reorganization or behavioral changes that have not been captured within a clinical trial setting, then implementation may also need to be considered within the evaluation itself (Reference Heggie, Boyd and Wu21). Careful consideration of the HTA recommendations and whether they could unintentionally lead to health inequities (e.g., a technology being recommended after testing which may incur out of pocket costs for patients, or treatments being made available at centers not easily accessible for patients based remotely) is critical. Steps to ensure equitable access to all technologies (e.g., including patient accessibility as a domain of HTA particularly when considering technology implementation) are important for all HTA bodies and must be closely monitored and reviewed. Closer attention to the uptake of HTA recommendations with a qualitative review of any barriers or challenges to their uptake (as routinely conducted by Health Technology Wales) could also facilitate this process.

The role of HTA in the wider health system

One of the key recurring points of discussion throughout the 2023 GPF concerned the current role of HTA in many countries, particularly in higher-income countries where HTA is increasingly a mechanism to aid reimbursement decisions for (typically new) drugs and devices. However, drugs and devices only account for 15–20 percent of a typical health budget. While some HTA bodies have a broader remit and look at procedures, vaccines, and other public health interventions, HTA concepts are not consistently applied at the broader health system level. GPF members argued that the value and impact of HTA could be greater and more widely recognized if the remit of HTA bodies, or at least the application of HTA principles, was extended to include broader organizational and budget allocation decisions. Examples were provided of countries with waitlists to see primary care physicians and other specialists that are considered unacceptable. Shortages of staff and beds, as well as limited resources for palliative care were also provided as examples that have not been subject to evidence-based decision making or principled allocation of resources. Without such an expansion of the HTA lens, there are likely to be broader funding issues that may prove catastrophic; for example, an increasing number of people dying on waiting lists each year. Expanding the remit of HTA to the broader heath system to include all aspects of healthcare provision and decision making may be warranted. HTA bodies may be able to see where more resources are needed to allow for the adoption of new technologies, however, this would require additional resources for HTA bodies to undertake and could be subject to factors outside of the scope of the health system.

The role of societal values

One element that is critical in determining the broader value of HTA is considering the values and preferences of the society or the taxpayer – meaning patients, caregivers, citizens, and companies – it represents. Where HTA recommendations align with the values held by a society, then the ultimate value is likely to be higher with a probable greater uptake and implementation in the recommendations as a result. However, establishing societal preferences is not a widespread or straightforward activity (Reference Breslau, Cohen, Diaz, Malcolm and Neumann22), with many methodological challenges. Further exploration into democratic, deliberative decision making and methods to elicit societal values is needed to ensure that the values of the people that HTA is being conducted on behalf of are represented. Aligning assessment and appraisal of technologies to societal values, for example, prioritizing severity or reducing health inequities will require trade-offs and preferences to be elicited and how these factors are considered within a HTA should be reflective of these values and objectives and conducted transparently. Without this, HTA recommendations may be ignored, and people may ultimately lose trust in HTA decisions and bodies.

As previously mentioned, an additional element of elevating the value and impact of HTA may be to minimize health inequalities or, at the very least, not unintentionally exacerbated through HTA recommendations. More research is needed to determine the societal preferences around health inequities; that is, do different societies have a preference for minimizing gaps in health inequalities, or a preference to improve health outcomes universally, even if this extends gaps in health outcomes and how this trade off can be addressed (Reference Cookson, Mirelman and Griffin23). A white paper from the Institute for Clinical and Economic Review (ICER) on HTA methods and processes that may advance health equity was referenced during the GPF discussions (Reference Agboola and Pearson24). There was broad agreement that this is an area where further empirical research is needed, and this is one aspect where the impact of HTA could be increased.

Engaging stakeholders

As with the HTA process itself, participation of stakeholders from the beginning of the assessment of the value and impact of HTA is important; however, identification of who should be involved is needed, and this will be context and assessment specific (Reference Concannon, Grant and Welch25). Considering the perspectives of the “7Ps” (patients, providers, payers, purchasers, product makers, policy makers, and principal investigators) (Reference Boaz, Hanney, Borst, O’Shea and Kok26), plus the public and caregivers, is considered an appropriate base from which to begin, with associations representing these stakeholder groups as relevant. Additional “nontraditional” stakeholder groups such as judicial/legal systems and the media could also help increase the reach and therefore potential impact of HTA. Each stakeholder group will need different levels and methods of engagement, and those that are typically most engaged are the patients, providers (clinicians), and product makers (industry). Given the variety of stakeholder groups, a range of formats and messages should be developed and tailored to the interests, needs, and understanding of each audience. More proactive, rather than reactive, messaging about the value of HTA is also needed, with one key (sometimes overlooked) stakeholder being the media. Greater use of novel digital and online tools for communicating and engaging stakeholder groups (such as podcasts and social media) (Reference Walsh, Hyett and Juniper27;Reference Pluye, El Sherif and Granikov28),may facilitate stakeholder engagement with the assessment of the impact of HTA.

Resourcing value and impact assessment

It was acknowledged that value and impact assessment can be resource intensive and that resource constraints are only increasing with the number of emerging technologies and new requirements on HTA bodies (such as the EU HTA Regulation). Ensuring that HTA bodies have sufficient time, resources and expertise for conducting internal impact assessment activities is important as most HTA bodies are conducting their own assessment activities. This may, however, bias the results of the assessment (particularly if the results are tied to re-financing and future budgets) and may divert resources from core business. External assessments of value and impact assessment; undertaken by independent, neutral, and unbiased parties (e.g., national audit offices) could be an alternative approach, funded outside of the HTA body. The concept of sharing learnings in a resource-constrained environment could be expanded to include HTA methods and process training courses such as undertaken by organizations like the European Patients’ Academy on Therapeutic Innovation (EUPATI) (Reference Warner, See and Haerry29). However, this would require the evaluation of such materials first to ensure that they also add value and have a positive impact on the HTA community.

Communicating the value and impact of HTA

One area that was reported as challenging by nearly all GPF members was the communication of the value and impact of HTA. The purpose, methods, and processes of HTA can be perceived as overly complex, with often technical and opaque language used to describe it (Reference Trowman, Powers and Ollendorf30). HTA has essentially been created as a social construct to handle a basic ethical dilemma of making difficult decisions regarding the allocation of limited resources (Reference Daniels and van der Wilt31). This means that “negative” recommendations are inevitable, and this needs to be better explained to patients, the public, and others. Better communication of the work and outputs of HTA bodies is needed and the value proposition of HTA should be clearly articulated, including the notion that HTA is more than cost-effectiveness analysis and that different HTA bodies may look at the same evidence base and come to different conclusions. Describing the role of HTA in a post-pandemic setting is especially important as HTA was circumvented in some countries during the pandemic and may now be seen by some as an unnecessary hurdle. The role that HTA can and should play where regulatory decisions are made on scant evidence should be better explained.

Resourcing better communication is a challenge for many HTA bodies who are currently experiencing acute staff and skills shortages. Despite this, clearer communication is something that the HTA community can strive for. Use of unhelpful language, such as “rationing” and “cost-containment,” should be avoided and replaced with sensitive and respectful communication strategies. Further, there may be a role for organizations such as HTAi to assist in developing some core messages (e.g., around the value proposition of HTA). The risks of not adequately communicating the value and impact of a HTA body to all stakeholders may include legal challenges; problems in obtaining and/or sustaining funding; and loss of trust and reduced implementation of HTA recommendations. However, if the value and impact of HTA is communicated effectively, this could build trust, establish legitimacy, and ultimately increase the uptake and potential impact of HTA recommendations.