Background
Health Technology Assessment (HTA) is a multidisciplinary process using explicit methods to determine the value of a health technology. The purpose is to inform decision-making and promote equitable, efficient, and high-quality health systems (Reference O’Rourke, Oortwijn and Schuller1).
While processes vary, HTA bodies and policymakers increasingly recognize the importance of patient involvement to inform decision-making (Reference Weeks, Polisena and Scott2;Reference Facey, Single and Hansen3). Patient involvement in HTA enhances transparency, fairness, and legitimacy (Reference Scott and Wale4). Patient experience can provide unique insights into the effects, risks, and benefits of treatment (Reference Hunter, Facey and Thomas5). Patient organizations are being relied upon to effectively represent patients and caregivers in the HTA decision-making process (Reference Single, Cabrera and Fifer6).
Despite growing opportunities, barriers to effective patient involvement remain. Patient organizations often face challenges due to limited knowledge of HTA methods and difficulties understanding complex submission documents written using technical terms (Reference Dimitrova, Jakab and Mitkova7). They frequently receive little or unclear information about the medicines being assessed (Reference Jakab, Dimitrova and Houÿez8). In many countries, regulations restrict companies from sharing information with patient organizations, especially before regulatory approval, limiting access to information about treatments, their intended use, and supporting research. These challenges hinder meaningful participation and highlight the need for clearer, more accessible information to support patient input in HTA processes.
To meaningfully contribute to the HTA process, patient organizations need clear and accessible information about the medicine being assessed, leading to better quality and relevance of their input. To address this, a template to compile plain language information that can be shared with patient organizations was first introduced by the Scottish Medicines Consortium (SMC) in 2017 and has become a core part of the submission dossier (9). This is known as the Summary Information for Patient Groups or “Summary” for short.
HTA International (HTAi) is a professional society promoting the development, communication, understanding, and use of HTA around the world. The HTAi Patient and Citizen Involvement in HTA Interest Group (PCIG) established a project subcommittee to create an adaptable international Summary template (see Supplementary Material S1). This template, with guidance for companies, HTA bodies, and patient organizations, aims to support effective patient involvement across HTA processes (Reference Cook, Livingstone and Dickson10). Figure 1 illustrates an overview of the template’s development.
Overview infographic of the HTAi Patient and Citizen Involvement in HTA Interest Group (PCIG) development of an international Summary Information for Patient Groups template.
Methodology
The National Institute for Health and Care Excellence (NICE) in England and in Australia, the Consumer Evidence and Engagement Unit (CEEU) in the Office of HTA, along with the Pharmaceutical Benefits Advisory Committee (PBAC), piloted the International Summary in their HTA processes. The HTAi PCIG Project Subcommittee evaluated feedback from various stakeholders involved in the pilots, and this article examines the lessons learned and recommendations for future adaptation.
Three companies completed Summaries with their submissions, allowing both HTA bodies to evaluate and address challenges arising with a limited number of cases. In the pilots, NICE received nine submissions, and PBAC initially focused on two (Table 1).
List of submissions with completed Summaries as part of the pilots
Both NICE and CEEU reviewed the Summaries to verify that the information was accurate, provided a fair reflection compared to the full technical submission, and was not promotional. They then shared the Summaries with relevant patient organizations to inform their input and assist them in seeking patient perspectives on the medicine. The HTA bodies used a variety of methods to gather feedback during the pilots, as described below.
NICE pilot
To initiate the pilot process in England, a statement about the Summary was included in the public consultation on the NICE methods and process guide, allowing NICE to assess support for adopting the approach (11). A baseline survey was also conducted with NICE and industry representatives to obtain their opinions and perspectives on patient involvement (see Supplementary Materials S2 and S3).
To support the pilot, a multi-stakeholder Short-Life Working Group (SLWG) was established following a call for expressions of interest from patient organizations from different disease areas. NICE worked in partnership with the SLWG to coproduce their involvement processes (Reference Rasburn, Livingstone and Scott12). Involving four patient organizations, a lay member, and led by NICE staff, the role of the SLWG was to provide feedback and advise on the Summary implementation. This included evaluating and adapting the international Summary template (see Supplementary Material S4).
Additionally, an advisory board meeting was held with representatives from four patient organizations, led by the Patients Association, to gather feedback on the Summary, including the level of plain language used (13).
PBAC pilot
There was a recognized need by patient organizations to improve the information provided to support their participation in HTA (14). The CEEU at the Office of HTA within the Australian Department of Health and Aged Care collaborated with the PBAC to conduct an evaluation using an adapted HTAi template, focusing on an initial pilot in collaboration with Bristol Myers Squibb, with Summaries for two submissions. These were provided to relevant patient organizations, along with written and verbal briefings, as well as a guide on how to use the Summary.
In addition to gathering feedback during the pilots, 1-hour interviews were conducted by a third-party consultant with representatives of the submitting company, five patient organizations, and CEEU, gathering perspectives from different stakeholders on aspects such as ease of completion, whether they helped save time, and whether the process improved patient organizations’ input (Reference Single15).
Project subcommittee
Following the pilots, three reports were completed: one for the pilot in Australia (see Supplementary Material S5), one for the SLWG with NICE (see Supplementary Material S6), and one by the Patients Association. The HTAi PCIG International Summary of Information for Patients Project Subcommittee, composed of participants from patient organizations, HTA bodies, and industry, analyzed the reports and held regular meetings over 2 years to gather feedback on the pilots (see Supplementary Material S7). This included meetings with NICE and CEEU to discuss the analysis and elucidate a broad range of questions, such as how the Summary can help support patient organization input to HTA, how easy the template is for companies to complete at the appropriate level of plain language, and considerations for HTA bodies implementing the pilots and incorporating the Summary into existing processes. The learnings from the pilots have now been collated by the Project Subcommittee to form the recommendations and guidance presented in this article.
International conference panel
At the HTAi Annual Conference in Adelaide (June 2023), the panel “Outcomes and Lessons from Implementing the International Summary of Information for Patient Groups in Different HTA Systems” presented initial findings from the NICE and PBAC pilots. Feedback from the session was collated by the HTAi PCIG Project Subcommittee.
Results and evaluation
NICE evaluation
The public consultation on the NICE methods and process guide asked stakeholders to provide feedback on their support for the following statement using a Likert scale response option: “Companies will provide a ‘Summary of Information for Patients’ with their evidence submission.” More than 80 percent of the 103 respondents agreed or strongly agreed with this statement (11) indicating strong support from key stakeholders for implementing the approach (Figure 2).
Responses to the consultation statement: “Companies will provide a ‘Summary of Information for Patients’ with their evidence submission.”
The SLWG noted positive feedback from patient organizations regarding the Summary (11). The SLWG agreed that there were many benefits to implementing the approach at NICE. Benefits included helping patient organizations
-
a. as a source of background reading to help with participation.
-
b. understand the issues (such as evidence uncertainty).
-
c. understand the company submission and, thus, help with the committee papers and discussions.
Patient organizations involved in the pilot also provided positive feedback as part of an advisory board. There were several recommendations made to improve the use of the Summary. The SLWG identified some key issues (Table 2), covered in further detail below.
Key issues identified by the NICE Short-Life Working Group
PBAC evaluation
The five patient organizations involved in the PBAC pilot were largely positive about their involvement and found the Summary useful, with one commenting that “it’s gold!,” as it substantially reduced the time spent looking for information. In some cases, it helped patient organizations to seek particular insights from patients and carers. For example, one was able to design targeted surveys asking patients about quality of life and perspectives on the medicine being assessed. Patient organizations also reported improved levels of confidence in contributing to the HTA process. Information on the treatment, comparator(s), place in the management pathway, and setting was considered helpful, particularly when given in a clear, concise format. Feedback indicated there should be better use of diagrams, tables, and formatting options to improve clarity and reduce the length of the Summary. The collaboration between the company, HTA body, and patient involvement experts was found to be useful to ensure the Summary contained not just a simple version of the application but had meaningful information for patient organizations. It also helped inform how best to complete and use the Summary.
Industry representatives completing the Summaries were broadly supportive. A survey by Medicines Australia, the pharmaceutical industry association (see Supplementary Material S8), showed strong feasibility, with thirteen of fourteen companies confirming their capacity to complete the template (16).
Reflections by the CEEU emphasized that by providing patient organizations with clear and concise information, they were better able to contribute their perspectives relating to the impact of a new medicine and identify what additional information should be considered by PBAC. The initial pilot led to the completion of seven additional Summaries involving various products and patient organizations.
Discussion
Patient and industry perspectives on the Summary in England and Australia align with those sought by SMC during its development. The Summary improves submissions by providing context to patient organizations but requires further accessibility enhancements. Industry and HTA bodies emphasized resource challenges in implementing and reviewing Summaries, despite widespread support for the approach.
Feasibility
In the initial development of the international Summary, HTA bodies were interviewed about the feasibility of including Summaries in existing processes. Industry input and resources required were identified as key considerations. The pilots revealed significant resource needs for HTA bodies, including staff time for implementation, stakeholder involvement, and reviewing and evaluating Summaries. Future pilots should address these requirements. Securing buy-in from industry, senior leadership, and committee members is critical to ensuring sufficient resource allocation for successful implementation.
Governance and mitigating bias
Good governance and mitigating bias are two important factors when discussing the Summaries, particularly when they involve the provision of information prepared by submitting companies. Both pilots had clear guidance for companies when completing the template, and HTA bodies checked that the Summaries were accurate, provided a fair reflection compared to the full technical submission, and were not promotional before sending them to patient organizations. Guidance emphasized that the document should be no longer than twelve pages and in plain language. If not appropriately completed, the HTA body returns the documentation to the company for revision. To ensure equity of access, both SMC and NICE have made completion of the Summary mandatory for company submissions.
Adapting the summary template and guidance
HTAi PCIG developed the International Summary alongside a series of guidance documents providing instructions for companies completing the form and how patient organizations should use it (17). These guidance documents were adapted by both NICE and the CEEU to support the implementation of their pilots (Reference Single15;18).
A key consideration is the variation in expertise of patient organizations and how to meet the different types and levels of need in one plain language summary. Some will be regular contributors to HTA and understand the process well. Others will have less experience and may need support to understand not only the treatment being assessed but also their role in the process. Some patient organization representatives might have other accessibility challenges, such as loss of vision, dyslexia, or speaking a non-English language.
NICE used the SLWG to review and update the International Summary and guidance to support the pilot. The outputs of the SLWG were a Summary template for companies, a guide for companies, and a guide for patient organizations (18–20).
PBAC also updated the guidance for patient organizations to ensure they were clear on how they should be used. Further revisions were undertaken to reflect feedback and to refine content (Reference Single15).
Timing of the summary
In order to best support patient organizations preparing HTA input, they need to receive information as early as possible but there can be barriers to this including regulations preventing information from being shared with patient organizations ahead of a company receiving regulatory approval for the treatment.
NICE and PBAC pilot discussions included timings for the availability of the Summary and how it aligns with existing methods and timelines, as described below.
NICE summary timing
Issues were raised about when the Summaries are made available within the timelines of the NICE appraisal. This is principally because industry provides the Summary with their submission to NICE at the same time as patient organizations are asked to submit their written evidence. The SLWG outlined recommendations, including “that (NICE) timelines are amended so that patient organizations could receive the Summary before completing their submissions” (11). To address this, NICE is exploring the possibility of delaying the time patient organizations are required to submit their written evidence.
Additional areas of discussion included completing the Summary earlier in HTA processes to potentially be used at the scoping stage and updating it as required during the process.
PBAC summary timing
The Summary was provided to patient organizations when the agenda was published for an upcoming PBAC meeting. From this time, for ~8 weeks, “consumer comments” could be submitted ahead of the PBAC meeting. After this period, the information submitted was collated and summarized by the CEEU and PBAC’s consumer nominee and presented to PBAC. Eight weeks may not be sufficient for patient experts to gather information or survey patient communities to support their input. However, this must be balanced with avoiding a delay in the decision-making process.
The timing of the availability of the Summary and other plain language information is becoming increasingly important, raising a relevant point for industry to consider having plain language summaries available at an early stage as they develop their HTA dossiers. In the context of the new European Union (EU) HTA Regulation, which includes the development of a Joint Clinical Assessment (JCA) before HTA evaluation at the EU country level, it would be important for stakeholders to explore how plain language summaries could support patient organizations to provide input to the JCA. Furthermore, the inclusion of plain language summaries to strengthen the patient voice at a regional EU HTA level could also enhance patient involvement in local HTA processes.
Limitations
This study has several limitations. The findings and recommendations reflect the authors’ interpretations based on feedback from a limited number of pilots conducted by HTA bodies in high-income countries, leaving challenges in other contexts unexplored. Feedback was collected retrospectively using diverse evaluation methods rather than systematic prospective planning. Additionally, the focus on patient organizations, rather than individual patients and carers, may have excluded perspectives from those less familiar with HTA processes. Despite these limitations, the international Summary template, along with locally adapted resources, offers opportunities for other HTA bodies and patient organizations to adopt and refine the approach, fostering broader experience and insight.
Conclusion and recommendations
Pilots by NICE and PBAC confirmed the positive impact of providing plain language summary information to support patient organization input to HTA, and the Summary template has been incorporated into the NICE HTA process. Patient organization and industry perspectives on the Summary in England and Australia are consistent with those sought by SMC from patient organizations and their industry users during the development of the international template (Reference Cook, Livingstone and Dickson10). Issues and challenges remain for different stakeholders, which can be addressed and improved. Based on the experience and lessons learned from these pilots, the authors offer the following recommendations for future implementations by HTA bodies and other stakeholders.
-
1. Managing and implementing a pilot: Establishing an SLWG helps gather diverse perspectives and secures buy-in from internal and external stakeholders. The involvement of committee members, patient organizations, and industry is key to the successful implementation of a pilot.
-
2. Evaluating a pilot: Use multiple methods, such as surveys, to collect data that assess the usability of the Summary, the guidance provided, and its impact on decision-making.
-
3. Patient organization expertise: Guidance should clarify that the Summary provides standardized, accessible plain language information to support HTA decision-making. While it may not address every information need, it serves as a foundation for additional research and insights.
-
4. Clarity and accessibility: Summaries must be clear, concise, and accessible, with infographics, images, and diagrams enhancing readability. Guidance provided by the HTA body to companies on completing the Summary template is critical. Examples can assist companies in preparing high-quality summaries, and plain language reviews can also help.
-
5. Mitigating bias: HTA bodies might consider preparing Summaries themselves, though this would require significant resources. The SMC, NICE, and PBAC approach, where HTA bodies review company-prepared Summaries for balance, accuracy, and neutrality, is an effective bias mitigation strategy. Regulations and industry codes also minimize the risk of promotion.
-
6. Resource impact: Mitigate resource demands with good planning by weighing resource requirements against the significant benefits summaries provide.
-
7. Summary timing: Summaries should be shared early enough to support meaningful patient input. Integrating the Summary into existing methods and timelines is a practical first step, with adjustments made as necessary to optimize timing.
-
8. Local regulations: National policies and legislation may pose barriers to adopting the Summary. Strict rules on patient–industry interaction should be explored during pilots to identify and address real versus perceived challenges, enabling implementation.
Evaluating patient and public involvement programs in HTA is crucial for understanding their impact, identifying training needs, and driving improvements (Reference Weeks, Polisena and Scott2). Information on the international Summary’s impact supports its enhancement and securing stakeholder buy-in for broader implementation. NICE and PBAC will continue evaluating their pilots and refining forms, guidance, and requirements (11;21). HTAi PCIG will update international templates and guidance based on global learnings. Sharing information and learning empowers HTA bodies to implement the approach effectively, ensuring informed patient involvement and strengthening the HTA process.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0266462325100408.
Acknowledgments
The authors would like to thank the Public Involvement Team at the National Institute of Health and Care Excellence in the United Kingdom and the team at the Consumer Evidence and Engagement Unit, Department of Health, Office of Health Technology Assessment in Australia, for providing feedback and information on the pilots.
Author contribution
All authors are members of the Health Technology Assessment International (HTAi) Interest Group for Patient and Citizen Involvement in Health Technology Assessment (PCIG). The views and interpretations included in the article are those of the authors. The HTAi Scientific Development and Capacity Building (SDCB) Committee provided a thorough review of the draft manuscript, offering valuable comments and suggestions. The authoring team is solely responsible for the final content.
Funding statement
No specific funding was provided for this project. Under the direction of the authors, writing support was provided by Opened Door Limited. Bristol Myers Squibb provided funding for this support.
Competing interests
Martin Coombes is Director of Opened Door Limited, a consultancy firm that has contracts with AstraZeneca UK Limited, Roche Products Limited, and Bristol Myers Squibb Pharmaceuticals Limited. Kate Morgan is an employee of Myeloma Patients Europe (MPE). MPE receives funding from a number of pharmaceutical companies but retains full independence and editorial control over publications. Jose Diaz is an employee and shareholder of Bristol Myers Squibb. Elisabeth Oehrlein is employed by Applied Patient Experience, LLC, which has contracts with a variety of nonprofit organizations, pharmaceutical companies, and academic institutions. Hayley Andersen is an employee and shareholder of Bristol Myers Squibb. Ramiro Gilardino is a Merck & Co., Inc., Rahway, NJ, USA, shareholder and was employed by MSD Innovation & Development, GmbH, Zurich, Switzerland, during the development of this manuscript. Kawitha Helme is an employee and shareholder of Novartis Pharmaceuticals. Fiona Pearce has no disclosures. Antonella Cardone works with Cancer Patients Europe (CPE). CPE receives funds from several pharmaceutical industry companies and always maintains full editorial control over anything published.
Open access
