Introduction
Health technology assessment (HTA) is a multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its lifecycle (Reference O’Rourke, Oortwijn and Schuller1). The purpose is to inform decision making in order to promote an equitable, efficient, and high-quality health system. This information is used by health system decision makers for resource allocation, such as decisions to reimburse a particular health technology or whether to include it in the benefits package.
HTA has increasing influence in health systems, which carries with it the need to evolve and improve these processes to support better decision making. In Latin America, HTA continues to be in a state of heterogeneous development, with some countries in the initial stages of adoption and others with decision-making processes strongly linked to assessment. Across the region, the drive to advance toward strengthening good practices in HTA and the creation of informed, deliberative decision-making processes is both a reality and a challenge (Reference Lessa, Caccavo, Curtis, Ouimet-Rathé and Lemgruber2;Reference Pichon-Riviere, Soto, Augustovski and Sampietro-Colom3).
Over the last few years, the participants of the Health Technology Assessment International (HTAi) Policy Forum have identified and discussed principles and processes that are closely associated with the implementation of deliberative processes. These principles and processes are considered essential to improve HTA in the region: stakeholder involvement, clear prioritization mechanisms, value frameworks, and clear links between assessment and decision making (Reference Pichon-Riviere, Soto, Augustovski and Sampietro-Colom3–Reference Pichon-Riviere, Garcia-Marti, Oortwijn, Augustovski and Sampietro-Colom7).
Stakeholder involvement is one of the good practice principles by which it is possible to improve the quality of recommendations and decision making (Reference Bond, Stiffell and Ollendorf8). However, there are gaps and differences in how HTA processes involve various stakeholders and how they are linked to resource allocation decision making. In many agencies, particularly in Latin America, stakeholder involvement remains limited to specific requests for opinion or mechanisms for experts or interested parties to submit information (Reference Oortwijn, Jansen and Baltussen9–Reference Wale, Thomas, Hamerlijnck and Hollander11). Deliberative processes are increasingly recognized as a way to improve the quality of recommendations and decision making in HTA (Reference Goetghebeur and Cellier12). They are characterized by their participative nature that focuses on open dialogue among parties in order to reveal and understand the assumptions, values, and underlying arguments that define the perspective of each of the participants (Reference Oortwijn, Husereau and Abelson13). This implies active participation on behalf of all those involved in the deliberative process, creating an environment that aims to enrich the understanding of the different stakeholders about a particular topic, with arguments and evidence brought to light that may not be recognized outside the deliberative context (Reference Bond, Stiffell and Ollendorf8).
Deliberative processes can have a role in all stages of HTA, from the design of the HTA structure and governance to enriching the conduct of the HTA; from topic identification to the preparation of recommendations and decision making. However, to deliver the participation that is required to improve the legitimacy of decision making these processes must be designed and carried out appropriately (Reference Oortwijn, Husereau and Abelson13;Reference Oortwijn, Jansen and Baltussen14). This is particularly relevant in Latin America, as the countries in the region are very heterogeneous and in many cases, there are no formal or explicit mechanisms defined regarding stakeholder involvement in HTA (Reference Pichon-Riviere, Soto, Augustovski and Sampietro-Colom3).
This paper presents the main results of the 6th Latin America Policy Forum (2021), which explored the implementation of deliberative processes in HTA frameworks and how stakeholder involvement could be improved in the processes used by HTA agencies in Latin America. This paper is not a formal consensus document by the participants and it does not necessarily represent the views of participants nor the organizations where they work.
Methods
The scientific secretariat (Institute for Clinical Effectiveness and Health Policy, IECS, Argentina) created a background document describing deliberative processes for decision making informed by HTA (Reference Pichon-Riviere, Augustovski, Alfie, Martí and Alcaraz15). This document was based on a systematic literature search that was conducted along with a review of documents and websites of HTA agencies and governments to identify basic concepts about deliberative processes and to compile a selection of international examples. The background document was to “level set” the knowledge of participants, harmonize definitions of key terminology, and support the discussions held during the Forum.
To supplement the background document, a survey was conducted of the representatives of participating countries to gather information about the characteristics of stakeholder involvement and deliberative processes in HTA agencies in the region.
The 6th Forum was held in virtual format on the 25–27 October 2021 and included a total of ninety-four participants in the first open session, with fifty-two participants involved in the closed sessions. There were ten representatives from HTA agencies and eleven representatives from payers of public, social security, and private sectors; seventeen representatives from industry (pharmaceuticals, medical devices, and diagnostics); one representative of the Pan American Health Organization; two representatives from patient groups; three representatives from HTAi; along with eight academics, plus organizers, and staff from the scientific secretariat for the event. In total, there were twelve countries in the region represented (Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, El Salvador, Mexico, Panama, Paraguay, Peru, and Uruguay). The “Acknowledgments” section is presented the detailed list of participants, their affiliations, and country.
The format included plenary presentations, breakout group sessions, and plenary discussions.
Ahead of the synchronous component of the Forum, three oral presentations were made by prominent speakers (Wija Oortwijn, HTAi; Pilar Pinilla, NICE International; and Mireille Goetghebeur, University of Montreal) to introduce the topic from an international perspective. These presentations were available in advance to all Forum participants (https://htai.org/policy-forum/latin-america-policy-forum/presentations/). The first day of the event, the speakers provided a summary of these presentations and participated in a question-and-answer session. Next, presentations were made about the current state and future plans regarding stakeholder involvement in Brazil, Colombia, and Chile. To close, perspectives from patients and users, as well as industry, were heard.
During the two following days, a series of breakout group activities were conducted. These were adapted to the virtual format and based in design thinking methodology in which discussion and debate follow a set of cognitive, strategic, and practical processes that are used to redefine problems and to find innovative solutions (Reference Melles16). Participants were divided into groups with an equal mix of countries and stakeholders represented and after each breakout group the findings were presented and debated in plenary sessions.
The activities of the second day concentrated on the discussion about the current situation in the region. What are the reasons? Why most of the countries in the region have not implemented formal deliberative processes in HTA and decision making? What consequences/problems currently exist in this situation? What are the main weaknesses/barriers and main strengths/facilitators to establish deliberative processes in the region? This discussion followed a participatory method with voting supported by the virtual platform on which the Forum was held.
The third day followed a similar methodology and considered the debates and results of the previous day to consider the following questions: Which stages of HTA are of higher priority to advance deliberative processes in the region? Which stakeholders should participate in deliberative processes and what would be their role? What recommendations could be made to health systems in Latin America and what would be key actions to take?
The Forum was held according to Chatham House Rules (17). All materials were produced in Spanish and English. The Spanish version of this paper is available at (Reference Pichon-Riviere, Augustovski, García Martí, Alfie and Sampietro-Colom18).
Results
Situation in the region
There were responses received from eleven countries to the pre-Forum survey on the status of stakeholder involvement, with the agencies represented providing coverage to the majority of their populations. Table 1 presents the main results of the survey with additional results in the Supplementary Material.
Stakeholder involvement in different stages of health technology assessment in Latin America
Note: +++, have voice and vote in deliberative process; ++, active participants in deliberative process, but without vote (only voice); +, provide opinion/other type of involvement, but no active participation in the deliberative process; −, typically do not participate in this stage of the HTA process.
Abbreviation: HTA, health technology assessment.
While ten of eleven agencies indicate that they involve stakeholders in at least one stage of HTA (identification and selection of technologies, assessment, recommendation, and decision making), the degree of involvement varied.
In five of eleven countries, a formal and explicit process for stakeholder involvement in the selection technology selection stage was reported. The Ministry of Health and other public agencies are strongly represented across the board. In eight of eleven countries’ and nine of eleven countries’ patients and industry, respectively, do not participate in the technology identification and selection stage, or their participation is limited to information submissions or comments with no involvement in deliberative processes.
In the assessment stage, formal and explicit stakeholder involvement is present in eight of eleven countries, with scientific societies, payers, and providers taking more active roles. The role of payers is more preponderant in eight of eleven countries, while industry has a role in four. Patients and citizen participation, albeit at a low level, occurs in seven of eleven countries.
Finally, in the recommendation and decision-making stage, nine of eleven countries report having a formal process for stakeholder involvement. There are countries with a significant level of participation, such as Chile, for example, while in other countries, such as Costa Rica, involvement is minimal. The Ministry of Health and other public agencies have a strong role in this stage, followed by payers, scientific societies and experts, while the role of patients is very low and citizens were engaged only in Chile.
Seven of eleven countries reported the implementation of some type of deliberative process to define the governance, structure, and characteristics of the HTA process and decision making in the health system, see Table 1.
Regarding the selection of stakeholders to participate in deliberative processes, only in one country is this done through a public call; in another by an application process; in three through nomination by relevant stakeholders, and in three as a combination of approaches. In ten of eleven countries, the deliberations are open to the public, but in six of these, it is restricted to selected individuals or groups. The information used to support the deliberative process nearly always consists of a summary of all information available to HTA agency, and in six of ten countries, there are explicit methods to summarize this information. In five of eleven countries, the deliberative process produces a non-binding recommendation; in three countries, the agency recommendations are binding; and in two countries, only an opinion is issued. In six of ten countries, the results are disseminated generally, while in four, the dissemination is narrowly targeted to specific stakeholder groups. Once the deliberative process is completed, five countries have an appeal process on the results. Detailed information by country is provided in the Supplementary Material.
The early-stage development status of deliberative processes is observed also in the survey undertaken during the Forum to gather participant perceptions of the appropriateness of the current level of development and application of deliberative processes in the region. Of thirty-three votes, 79 percent of participants consider it of limited appropriateness, 9 percent not at all appropriate, 9 percent consider it appropriate, and 3 percent very appropriate.
Barriers to improved stakeholder involvement and deliberative processes development in Latin America
The breakout group discussions identified some of the main barriers to stakeholder involvement and deliberative processes such as the low level of HTA institutionalization in many of the countries in the region; a lack of education and knowledge about HTA; lack of a clear mandate from authorities so that HTA agencies can develop deliberative processes; and, concern about excessive influence of interest groups (see Table 2). Additional barriers that were identified include the lack of continuity or force in policies; changes in authorities; difficulties to find appropriate interlocutors; lack of clear mechanisms to manage conflicts of interest; lack of laws, norms, and platforms to enable deliberative processes; and the lack of communication in an appropriate language for the processes, particularly for patients.
Barriers to the implementation of deliberative processes in health technology assessment in Latin America
Abbreviation: HTA, health technology assessment.
Consequences due to lack of or deficiency in deliberative processes in the region
The breakout group participants on day 2 of the Forum discussed the consequences of decision-making processes and health systems in the region due to the lack of, or deficiency in, deliberative processes in HTA frameworks. The consequences included the lack of legitimacy in health resource allocation decision making; the lack of legitimacy in the process and structure of HTA; the inappropriate implementation of coverage decisions; the inequity in decisions; stakeholders who feel excluded from the assessment and decision-making process; and the lack of transparency. The lack of formal mechanisms for stakeholder involvement in many countries means that interest groups with the greatest power and influence will use non-formal channels (informal lobby power), whereas the other stakeholders may be completely excluded.
Facilitators in Latin America to advance toward better stakeholder engagement and deliberative processes
In nearly all countries, there are mechanisms to consider stakeholder involvement in at least some stages of HTA (in the survey, ten of eleven countries reported having formal mechanisms for stakeholder involvement in at least one stage, see Table 1). In light of this, participants agreed that – despite the barriers – it is a good time to deepen stakeholder involvement in HTA and to improve the quality and quantity of deliberative processes in the region. There are successful experiences in the region that, along with international experiences, can be taken as examples to guide the process of improvement. Initiatives such as the HTAi Global Policy Forum and the HTAi-ISPOR Joint Task Force on deliberative processes were recognized as helpful and valuable tools for the region.
Contextual factors were also identified that could support more active involvement of relevant stakeholders in HTA. For instance, in the region, there is a significant demand from the general public to participate in decision making about the allocation of health resources and heightened awareness about the importance of doing so. Other contextual factors identified include: many of the interlocutors and representatives of different sectors have training and experience in HTA; there is promotion and regulation in countries to facilitate greater public participation in decisions in other social spheres; and the experience gained during the COVID-19 pandemic has enabled public participation and discussions in virtual platforms that are more accessible to the population. Furthermore, some countries in the region have not yet established the structure, governance, and processes for HTA, which offers the invaluable opportunity to incorporate deliberative processes.
In the final plenary, one of the questions from the synchronous survey asked to what degree the advancement toward improved deliberative dialogues might affect the judicialization of coverage decisions – an important problem in the region. Of thirty-three responses, the majority of participants considered that deliberative processes do not carry a risk regarding judicialization, and they might even reduce it: 61 percent thought it would reduce judicialization; 3 percent thought it would increase; 15 percent said it would not be affected; and 21 percent thought the effect could not be known.
How to improve stakeholder involvement in HTA in Latin America? How to advance toward more and better deliberative processes?
During the third day of the Forum, different health systems in the region were discussed regarding how they might advance toward greater stakeholder involvement in HTA and, more specifically, how to arrive at more and better use of deliberative processes.
While there was no clear consensus, there was some agreement that agencies should make efforts to incorporate more deliberative processes, especially in the technology identification and prioritization stage (45.5 percent currently involve stakeholders in this stage, see Table 1). A vote was conducted to identify which stakeholders should be prioritized to be involved in HTA processes. Of the thirty-four respondents, there was agreement that all stakeholders could bring value to the decision-making process, with patients identified as the group whose participation should be the most expanded (Figure 1). They also examined the type of participation considered the most appropriate for patients, clinical experts, and industry (Figure 2). Of the thirty-three respondents who indicated the necessity to involve clinical experts, nineteen (57.6 percent) believed they should participate with voice and vote within the deliberative process, twelve (36.7 percent) that they should participate with voice only, and two (6 percent) that they should only provide an opinion without participating in the deliberative process. Of the twenty-seven respondents who expressed an opinion about patient involvement, twelve (44.4 percent) felt that they should participate with voice, while the same number felt they should participate with voice and vote, three (11.1 percent) felt that they should be consulted. Regarding industry, twenty-four respondents indicated that they should have some level of participation; they should be consulted according to seven responses (29.2 percent); be part of the deliberative process with voice only in fourteen responses (58.3 percent), and with voice and vote in three responses (12.5 percent).
Prioritization of stakeholders to include in deliberative processes in Latin America. Note: higher points = higher priority; 330 maximum – 0 minimum.
Necessity of participation and role suggested for stakeholders.
Table 3 provides a summary of the main comments and recommendations that emerged from the discussions about stakeholder involvement in the different stages. Overall, the role and the necessity to incorporate each type of stakeholder is very dependent on the stage of HTA that is being considered. For example, in the identification and prioritization of technologies for assessment, citizens and those who represent health system users should have a relevant role, and patient representatives, caregivers, and industry bring invaluable perspectives to inform decisions in the stage of scoping, assessment, and recommendation.
Considerations for stakeholder involvement in each stage of health technology assessment
Abbreviation: HTA, health technology assessment.
Recommendations for countries and health systems that decide to advance toward greater stakeholder involvement and improved deliberative processes
Forum participants discussed on the last day a series of recommendations that focused on general questions to advance toward improved stakeholder involvement, as well as specific aspects of the processes for implementation.
The importance of benefitting from regional and international experiences was noted, with the need to identify regional collaboration initiatives (e.g., via the World Health Organization) or international initiatives (e.g., HTAi). There is a need to sensitize different decision makers and social stakeholders about the importance of stakeholder involvement and the need to improve the legitimacy of HTA processes. This is a critical step that needs to be accompanied by permanent pathways for education, training, and communication. These efforts to promote increased empowerment and more mature and informed stakeholder representation will ensure that stakeholder involvement continues regardless of changes in governments or authorities.
Specific recommendations regarding which aspects of the processes to implement to arrive at improved stakeholder involvement emphasized the need for involvement to be planned and brought forward in all stages of HTA, particularly in governance and institutionalization. It was suggested to prioritize the stages of HTA where deliberative processes could begin to be implemented in a pilot project and from here, it could be expanded into the other stages. It was recommended to consider the appropriate mix of deliberative and consultative processes and carefully select stakeholders for specific stages so as to support the optimization of processes. It is important that processes are transparent, with clear incentives and objectives for the participation of each stakeholder; clearly defining the difference between patients and caregivers on the one hand, who bring perspectives of the lived experience of a condition, and citizens or health system users on the other, who bring a more general perspective on needs and priorities.
Policy implications
Countries in Latin America have made significant progress in recent years. Health systems in the region today have mechanisms to consider some degree of stakeholder involvement; however, this involvement remains limited. Many stakeholders are in fact not involved, with some involved only in certain stages of HTA and not others, or participation is limited to a consultative role. Stakeholder involvement using deliberative processes with authentic participatory characteristics and open dialogue among parties remains uncommon in the health systems in Latin America.
There are important barriers and limitations, both structural and contextual, in the region that have prevented or slowed the advance toward improved deliberative processes. Among these, the low levels of institutionalization and knowledge about HTA, and the lack of trust among different stakeholders can be emphasized. This low level of stakeholder involvement carries consequences for health systems, and it diminishes the legitimacy of the very structures and processes of HTA.
Nevertheless, there was agreement that today the conditions in Latin America are right to move forward to improve deliberative dialogues in HTA frameworks. Deliberative processes offer a valuable opportunity to understand the assumptions, values, and underlying arguments that determine the perspectives of the different parties in the context of decision making. This can enrich the understanding of all stakeholders both in regard to specific technologies, and also in other HTA topics, by bringing to light arguments and evidence that may not have been recognized outside of the deliberative space.
There remains a need to deepen discussions and analysis to determine in each country and context which stages of HTA should be prioritized to advance toward improved deliberative processes, and the role that each stakeholder should have in this.
There are important regional experiences and international initiatives that can serve as points of reference in establishing transparent and explicit mechanisms to provide an appropriate framework for greater stakeholder involvement in HTA.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0266462322003294.
Acknowledgments
We thank the participants of the Forum for their valuable contributions: Rob Abbott, HTAi, Canada; Johanna Acevedo, Ministry of Health – Health Planning Division (DIPLAS), Chile; Lizbeth Acuña, High Cost Account, Colombia; Débora Aligieri, Blog Diabetes and Democracy [Blog Diabetes e Democracia], Brazil; Larissa Andrade, Janssen Pharmaceuticals, Brazil; Fernando Argento, IECS-Institute for Clinical Effectiveness and Health Policy, Argentina; Elizabeth Barona, Ministry of Public Health, Ecuador; Virginia Becerra, F. Hoffmann-La Roche, Brazil; Jeremías Bernal, Social Security Fund, Panamá; Vania Cristina Canuto Santos, CONITEC, Brazil; William Dorling, Pfizer, USA; Graciela Fernandez, National Resource Fund, Uruguay; Verónica Gallegos, CENETEC, México; Pedro Galvan, Health Sciences Research Institute / Ministry of Public Health, Paraguay; Mireille Goetghebeur, School of Public Health, University of Montreal, Canada; Carlos Gonzalez Malla, Ministerio de Salud, Argentina; Alicia Granados, Sanofi Genzyme, Spain; Elizabeth Guambo, Ministry of Public Health, Ecuador; Aquiles Henriquez, Ministry of Public Health, Ecuador; Gerick Jiménez Pastor, Costa Rica Social Security Fund (CCSS), Costa Rica; Kyla Jones, Medtronic; Silvana Kelles, Unimed Bh, Brazil; Fernanda Laranjeira, Janssen Pharmaceuticals, Brazil; Alexandre Lemgruber, Pan American Health Organization (PAHO), USA; Homero Monsanto, Merck & Co., USA; Wija Oortwijn, Radboud University Medical Centre, The Netherlands; Manny Papadimitropoulos, Eli Lilly & Company, USA; Lucas Perelli, IECS-Institute for Clinical Effectiveness and Health Policy, Argentina; Pilar Pinilla, NICE, UK; Alicia Powers, HTAi, Canada; Andrea Rappagliosi, Edwards LifeSciences, Switzerland; Nora Reyes Puma, National Health Institute (INS), Peru; Adriana Robayo, Institute for Health Technology Assessment (IETS), Colombia; Gabriela R. Rodriguez Grimalt, IECS-Institute for Clinical Effectiveness and Health Policy, Uruguay; Cecilia Rodríguez, I Move Foundation (Fundación Me Muevo), Chile; Nicola Romanello, Pfizer, USA; Ricardo Ruano Arévalo, Ministry of Health, El Salvador; Ana Eduviges Sancho Jiménez, Ministry of Health, Costa Rica; Nicolás Schlumberger, Sanofi (Brazil Medtronic, United States); Dino Sepulveda, Ministry of Health, Chile; Silva, Biomarin, Brazil; Katerine Simbaña, Ministry of Public Health, Ecuador; Jose Thomaz, Biomarin, Brazil; Juan Valencia, Medtronic; Jice Valentim, F. Hoffmann-La Roche, Brazil; Victoria Wurcel, MSD, Argentina. We also thank Tara Schuller for preparing the English translation of this paper; to Mireille Goetghebeur, Pilar Pinilla, and Wija Oortwijn for their presentations and participation at the Forum, and a special thank you to Gabriela Rodriguez (IECS-Argentina) for all her collaboration in organizing the Policy Forum.
Author Contributions
AA y APR were involved in the activities design, implementation, and analysis. They led the writing of this article. SGM, FA, VA, and HC were involved in the activities’ design, implementation, and analysis. They reviewed, improved, and approved this manuscript.
Funding statement
The organization and attendance at the Forum were financed by HTAi. The preparation of the manuscript was the responsibility of the Institute for Clinical Effectiveness and Health Policy (IECS) with funding from HTAi.
Conflict of interest
H.C. received personal funding and support for attending meetings as Chair of the Latin-American Policy Forum at HTAi. The other authors declared no conflicts of interest.
Open access
