Impact on basis of HTA result or recommendation

This domain covers the impact on the evidence base and deliberation process that forms the basis of the HTA result or recommendation; it highlights ways in which patient involvement can contribute to broadening the evidence base, support contextualization, and strengthen the quality of deliberations and the appraisal process alongside other stakeholders.

The most commonly reported example of impact identified under this domain was improved data interpretation. This type of impact, which featured across nineteen of the twenty-four stories, focuses on instances where the patient perspective helped to contextualize, reframe, or otherwise provide an interpretation of evidence reviewed as part of the assessment. For example, understanding the experience of living with the condition may help data interpretation, support or counter claims made by others, or identify outcomes of importance to patients. In the joint HTA body-industry submission respondents cited, “In the absence of this patient input, the clinical expert view would likely have prevailed during the discussion… the patient input supported the alternative view, consistent with findings from the qualitative research” (response H11/I1). Another HTA respondent shared that “Information provided by the patient submission supported the company’s statements regarding the value placed on the therapy by patients and the degree of improvement in quality of life after successful treatment. It would have been near enough impossible for the Committee to get a feel for this without such direct experience” (response H2). Many HTA respondents emphasized that the meaningfulness of small clinical differences can be accentuated through patient input, “this helped to put the clinical and economic evidence into context” (response H9); “[patient input] provided a reminder of how small benefits may be transformational” (response H5); “this contributed to alleviating some of the doubts members had surrounding gaps in the clinical evidence” (response H10).

Relatedly, other commonly reported examples of impact under this domain were contributions from patient representatives that increased the understanding of the patient experience (mentioned in seventeen out of twenty-four stories) and highlighted patient needs (twelve mentions). Specifically, patient participants provided an unfiltered perspective of living with the condition, how it was affecting the (quality of) life for patients and caregivers, and insights into how the current treatments were not meeting their needs. For example, a patient organization contributed “personal perspectives, the reality of living with [condition], the symptoms and challenges it poses on a day-to-day basis and the reasons patients want to proceed with [procedure] rather than medication” (response P9); “the patient group submission brought a new dynamic to the meeting and showed the reality of what happens for someone who has [condition] and how [device] can help” (response H1). Without patient involvement, “the committee would have been less aware of the merits of the drug and missed some of the success stories amongst patients” (response P6).

The presentation of new data for the committee’s consideration (eleven mentions) and the demonstration of data limitations (four mentions) were two additional ways patient involvement made a difference during deliberation. For example, patient organizations doing research to present data to HTA bodies, “the survey the patient organization conducted… was particularly helpful… [in] underpin[ning] the willingness to travel and the likely uptake of the treatment” (response H2), or highlighting gaps in existing data, “Patient testimony at the meeting indicated that there were other aspects of the disease which were not captured in the model; indicating that QALY measurement may not be capturing full benefit [and that] impact on carers wasn’t properly captured” (response H5).

Other types of contributions under this domain included presentation of cost data (five mentions), “relevant aspects related to the uptake of the technology would have been disregarded, such as the costs that are [borne] by the patients and their families” (response H3), stressing patient acceptability (three mentions), including patient perspectives on the risk/benefits tradeoffs of new therapies; and shedding light on inequalities or special needs of subpopulations (six mentions), “[patient involvement] allowed to include information about how African Americans are more impacted by the disease” (response H6).

Lastly, ten out of twenty-four stories mention some direct impact of patient involvement on the recommendation itself, including changes in the HTA recommendation or support for an appeal process. “The Appeal Panel… was convinced by the argument and the patient input confirming that under treatment they have an almost normal life and upheld the Appeal point that the benefit of the treatment is not ‘small’” (response P2); “Patient participation is directly related to the recommendation that changed from initial recommendation to disinvest… to not to disinvest” (response P10). In contrast, four stories (P7, P8, H3, I2) noted a lack of clear impact based on patient involvement. “Had we not been in the room, or had we not done the study, would it have made any difference to the appraisal outcome?” (response P7); “Generally, I don’t feel that [patient involvement] has a large impact on decision-making. This may be because of how it is collected” (response I2).

Impact on HTA body

This domain covers the impact of patient involvement on the HTA bodies that conduct these assessments, including changes in staff perceptions and adaptations to processes based on patient input.

The most common element under this domain was HTA staff awareness about the importance of involving patients in HTA processes (seven mentions) and reminding staff about HTA’s purpose (five mentions). “Without patient involvement, we would notably be missing the user’s vision, which is key to the development and implementation of any single treatment” (response H7); patient participation “serves to remind members of the purpose that committees exist to serve – that is, to improve the quality of care” (response H9).

This increased awareness led to a heightened HTA engagement culture at the organizational level in four stories, “clear guidance has now been put in place for patient organizations attending meetings” (response H1), and interestingly, to a shift in HTA perceptions of patients as equal partners (four mentions). Active patient participation builds trust in the patient community’s ability to contribute meaningfully. One HTA body Patient and Community Advisory Committee representative mentioned how patient participation has “encouraged [HTA body] to move from a focus on ‘extracting perspectives’ to ‘growing relationships’ with patients. Recent projects have involved ‘patient collaborators’ interacting directly with project teams” (response P11).

The direct contact through patient participation was also cited as a key element of impact in three stories, “the ability to ask [patient representatives] supplementary questions before the meeting (and after their submission) was really helpful” (response H2).

Lastly, two stories noted that patient representatives suggested HTA process improvements; “we fed back that perhaps a video presentation could be considered for future meetings to demonstrate how devices are used” (response H1).

Impact on patient participants

This domain covers the impact on the patient representatives participating in HTA.

The most common impact identified under this domain was the feeling of acknowledgment, being heard and being valued (three mentions). “We felt our experiences and what mattered had been made somewhat visible and considered as part of the decision-making process… it alleviated a little of the stress and anxiety for me and my community that our voices were not being heard” (response P1); “I had the impression that patient input was valued and that many of the inputs and aspects of the testimonies and arguments put forward were taken up for the final determination” (response P3).

There was also the perception of co-construction (three mentions) as patient stakeholders felt as partners in the deliberation process, “patient input is not always about providing information about burden of disease and patient experience. It can often be about being an honest broker and advocating for practical and pragmatic solutions.” (response P4), which led in one case to better patient acceptance of the HTA recommendation, “being able to attend a hearing made a difference to how I, and to some extent my community, felt about the decision-making process” (response P1).

Moreover, the ability of patient representatives to participate in HTA processes led to broader patient awareness about HTA (one mention) and increased culture of patient participation in HTA (two mentions) across the community: “As a patient, I gained a greater understanding of HTA processes, how to have effective input and the need to educate and engage more patients about this process” (response P1).

Transparency and expectation-setting

The most significant area for improvement that surfaced was the need to increase transparency about HTA processes (nine mentions), including who is invited and how testimonies will be considered: “There was little transparency about the process – which patient groups had been invited and how the information was ‘weighted’ or how information would be valued against the other information that was placed in front of decision makers” (response P1); “The way that patient feedback is reported makes it difficult to know how it impacts decision-making. [HTA bodies] usually provide the names of the organizations submitting feedback, but there is no detail on how it factored into decision-making” (response I2); “the process… needs more clarification on how patient inputs are being considered; it needs criteria” (response P12). The importance of this transparency and its reporting is highlighted by patient organization response P8, “the challenge has always been what is the impact of our activity (we have limited time/resources) and, when cynical, [we] question if the outcome would be any different if we were not involved.”

There was also a call to improve feedback as part of the submission process (five mentions), for example, instituting a feedback process or following up post-hearing, “we would appreciate regular follow-ups with updates on actions and progress since the decision; knowing what stage it is currently at or a timeframe for when [procedure] will be introduced to [health system] is important to us and we would appreciate an update” (response P9).

Lastly, patient participants request more guidance be shared about the process (two mentions) so they can effectively participate, including type and length of testimony, setting, and audience. “I had little understanding of the hearing itself – the format, length, etc.” (response P1).

Systems and processes

Systematic involvement of patients across different stages of HTA processes was called for in seven stories, “we operate behind a wall, send a document into a black box, and occasionally get asked to send in a couple of additional documents; never a phone call or meeting to resolve any uncertainties or allow us to address any questions they are wrestling with. The manufacturer has an opportunity to respond to questions during the process, however the patient organization is not afforded any opportunity to give perspective on these uncertainties” (response P5).

There was also a call for a more systematic review of patient involvement processes (four mentions), “The analysis of contributions can be more in-depth and consistent from a theoretical and methodological point of view” (response H4); “try and somehow quantify and/or weight the impact on decision-making our participation had beyond just saying that it was really good that you were in the room, and the committee thought you were great” (response P7).

Lastly, the responses surfaced additional opportunities to improve participation, such as publishing and promoting opportunities to get involved (three mentions), increasing the representativeness of patient participants (three mentions), and improving submission templates for patient testimonies (one mention); “questions get repetitive (given frequent submissions) and do not get to the core of our message regarding that submission. We have to work outside the box to comment on what they are NOT asking us… This has made the template more or less useless. We use it as a blank piece of paper” (response P5).

Training and data readiness

Seven stories called for HTA personnel training to increase the awareness of patient involvement as a core value of HTA, how to incorporate patient insights into their work, and what is appropriate and reasonable to ask patients. “The [HTA] team noted that the most important factors that inhibited the patient involvement process were their lack of experience in previous work with patients and their inexperience using lived knowledge from patients in their prior assessments” (response H7); “more can be done to improve panel member’s confidence in speaking to and asking questions of patients” (response H9). Sensitivity training is especially needed in fostering productive dialogues, “some of the questions asked of me at the hearing were not appropriate and should have been asked of the appropriate subject matter experts rather than myself” (response P1), “sometimes we have to “dumb down” the language [of our submission] because we get feedback that it appears to be professionally written. Some of us are professionals” (response P5).

Four stories suggested patient training on the HTA process (what it is, how to contribute) would be necessary to support effective participation, “Patients are likely to have a wealth of experience and feedback that could be valuable, but they just don’t know what information to provide” (response I2). There was also a mention of the usefulness of developing a buddy system to support participation, “I was fortunate that the consumer representative was willing to speak with me directly to provide some guidance in how to prepare” (response P1). HTA officials that submitted responses offered specific suggestions about how the patient community could present more robust data (four mentions): “more quantitative information on the impact on families outside of QALY gains” (response H2); “more detailed accounts of the effect of [medicine] on patients’ quality of life, more specific patient stories on the transformative effect of the drug, better clarification of wide clinical severity presentation and impact of interpretation of QALY” (response H5).

Lastly, two entries mentioned the importance of decreasing resistance and negative perceptions of patient involvement in HTA amongst other stakeholders, for example, the medical community: “We saw some initial resistance from some health care professionals about the value of patient involvement, but most were pleasantly surprised with the experience” (response H7).