Literature review

A comprehensive review of scientific and grey literature was conducted to examine patient involvement frameworks implemented by overseas HTA agencies, with the aim of identifying best practices, methodologies, and key activities. Relevant resources were also gathered from the websites of Canada’s Drug Agency (CDA-AMC), England’s National Institute for Health and Care Excellence (NICE), HTAi, and INAHTA. In particular, INAHTA’s position statement on patient involvement (18), and HTAi PCIG’s Values and Quality Standards for Patient Involvement in HTA (17) were referenced. To stay up to date with global trends, the CEE team also joined patient working groups in INAHTA and HTAi.

Stakeholder mapping exercise

In the absence of a centralized database or national umbrella patient organization, two CEE staff conducted a comprehensive stakeholder mapping exercise (May–July 2021) to identify all patient and volunteer organizations in Singapore whose members could contribute – or expressed interest in contributing – to ACE’s HTA process. They consulted the Singapore Charity Portal (23) to identify registered patient or voluntary organizations, and reviewed websites of public and private healthcare providers (public hospitals, community hospitals, and polyclinics) to identify patient groups initiated by care teams, which are typically unregistered. Online search engines (e.g., Google) and social media platforms (e.g., Instagram, LinkedIn, and Facebook) were used to locate smaller, informal patient communities.

Identified groups were screened using predefined inclusion and exclusion criteria (Figure 1). Organizations were included if they represented one or more health conditions and comprised Singaporean patients, caregivers, or family members. The intent was to identify groups that represent individuals with lived experience of health conditions who are engaged in issues directly related to healthcare access, treatment, and outcomes. The following types of organizations were excluded: activity centers for the elderly, schools for adults or children with disabilities, Traditional Chinese Medicine societies, research groups, referral networks that link patients to relevant charities, fundraising and distribution charities, humanitarian aid or disaster relief organizations, and organizations that solely provide legal assistance, transportation services, befriending, or hotline support to patients. These groups were excluded because their missions and activities do not directly relate to evidence-based healthcare evaluation or ACE’s remit.

Figure 1. Flow diagram of the stakeholder mapping exercise to identify patient and voluntary organizations in Singapore.

Following screening, organizations were contacted to gather further information about their remit, size, and interest in providing patient input. Dormant organizations and those unreachable after multiple attempts were excluded from further consideration.

The stakeholder mapping exercise identified 106 patient organizations and groups in Singapore, representing 20 different health conditions (Table 1). Forty-eight percent (n = 51) were registered organizations listed in the Singapore Charity Portal, 39 percent (n = 41) were affiliated with public hospitals, and the remainder were smaller, informal groups founded by local patient advocates, with an online presence through websites or social media platforms. The size of the organizations ranged from 8 to 3,500 members. All organizations were consolidated into a database to identify relevant groups whose members can provide condition-specific input for each HTA. Invitations to contribute to ACE’s HTAs are extended based on the relevance of their members’ conditions to the technologies under evaluation, irrespective of organization size, governance structure, or formal registration status.

Table 1. Conditions supported by patient organizations in Singapore

To ensure comprehensiveness, the database was shared with key local stakeholders – including SingHealth Patient Advocacy Network, Duke-NUS Medical School Centre of Regulatory Excellence (CoRE), and the Singapore Association of Pharmaceutical Industries (SAPI) – for cross-verification and to identify any potentially overlooked organizations. The database is continually updated as new patient groups are found through direct engagement activities and referrals.

Early dialogue with key stakeholders

Early dialogue sessions were convened with select local patient organizations, Duke-NUS CoRE, and SAPI to introduce ACE’s patient involvement workstream, discuss processes, and explore strategies to encourage patient participation. The CEE team also consulted patient engagement experts from Australia and the UK to discuss potential challenges associated with including patient input in the HTA process. ACE and Duke-NUS CoRE co-organized an information session for local patient organizations in November 2021 to raise awareness about HTA and clarify how patients could meaningfully contribute to ACE’s assessments.

In April 2022, ACE established a consumer panel of fifteen patient leaders to represent the collective voice of local patients and caregivers and provide strategic advice to ACE and its technical advisory committees, supporting process improvements and ensuring that patient perspectives inform ACE’s work (Reference Teo, Pearce and Tan24). All patient involvement processes were co-developed with the Consumer Panel to ensure they met local patients’ needs and were feasible to implement.

Developing survey templates and support tools

The literature review revealed that frameworks for patient involvement in HTA vary across countries in terms of how, and at which stage of the process patients are engaged (Reference Abelson, Li, Wilson, Shields, Schneider and Boesveld7). Importantly, there is no one-size-fits-all approach; strategies must be adapted to local contexts, needs, and available resources.

Drawing on patient input templates used by overseas HTA agencies, the CEE team developed a structured survey adapted to the Singaporean context with 20 core questions to support patients share insights relevant to ACE’s HTA (see Supplementary Material). The survey could also be modified to incorporate additional questions to address specific information needs of ACE’s technical evaluation team. A “patient journey form” was also created as an alternate option for patients to complete, which covered the same key areas as the survey – patient information, experiences with the condition and treatments, expectations of new treatments, and conflict of interest declaration – but with fewer questions to allow for more flexible responses. All templates were piloted in English, with Chinese versions subsequently developed to improve accessibility and response rates.

Once a technical evaluation begins, surveys are emailed to patient organizations to distribute to their members. Unlike approaches used by many overseas HTA agencies – which require consolidated submissions from patient groups on behalf of their members – CEE allows individual patients to submit their responses directly, to reduce the administrative burden on local patient groups who are unable to coordinate collective input. All responses are incorporated into ACE’s HTA reports, which are used by MOH advisory committees to inform their funding recommendations.

From initial outreach efforts and feedback gathered during the information session, it became clear that many patient organizations were unfamiliar with ACE, the concept of HTA, and how they could contribute to the process. To address this knowledge gap, the CEE team developed additional support tools including an infographic and factsheets (25) to provide simple information about the HTA process and tips on the types of information that are useful to include in survey responses. All materials are published on ACE’s website and are sent via email to patient organizations with the survey when each HTA begins.

A dedicated email address (ACE_CEE@moh.gov.sg) enables patient organizations to enquire about contributing to ACE’s work. Patient organizations can also use this contact point to engage with the CEE team for advice on useful information to include in the survey to inform treatment funding decisions.

Piloting the new patient involvement process

Between March and August 2022, members from local patient organizations were invited to contribute lived experiences about their health conditions and treatments to inform ACE’s HTAs of drugs under consideration for government funding. Twenty-three local patient groups were invited to provide input into this pilot initiative and were encouraged to share the invitation with their members and within their networks to maximize reach and inclusivity. In total, ACE received 82 responses from patients aged 28–78 years from 10 registered organizations and groups affiliated with public hospitals. These inputs collectively informed seven evaluations of treatments for cancer, respiratory and kidney diseases, diabetes, and HIV, which were deliberated at advisory committee meetings in August and December 2022 (Reference Tan, Pearce, Quek and Teo26).

Following the committee meetings, guidance documents which described the funding recommendations and key evidence that informed the HTA, including the patient input received, were published on ACE’s website. A plain English summary (PES) of the guidance was also produced to explain the funding recommendations in non-technical language. The guidance and PES were shared with patient organizations who contributed to the HTA, along with feedback on which patient input was most helpful to the advisory committee to inform their recommendations.

Co-developing ACE’s process and methods guide for patient involvement

Drawing on good practice examples from international HTA agencies, insights from early stakeholder dialogues, and findings from the 2022 patient involvement pilot, the CEE team co-developed ACE’s Process and Methods Guide for Patient Involvement (21) with local patient organizations and the ACE Consumer Panel. The guide explains how patients and caregivers can participate in ACE’s work, serving as a comprehensive reference, and enhancing the transparency of ACE’s processes.

The pilot informed several important process considerations for the guide. These included providing clear information and advice on available patient involvement opportunities, co-developing processes with patient groups to ensure relevance and practicality, and enabling patient participation throughout the entire HTA lifecycle: topic prioritization, clinical and economic evaluation, deliberation and funding recommendations, and implementation. Equally important was the emphasis on ensuring that engagement is meaningful rather than tokenistic, supported by transparent communication, mutual respect, and the acknowledgement of patient contributions. Collectively, these elements formed the foundation for a structured yet flexible framework to support consistent and authentic patient involvement in ACE’s HTAs.

To support accessibility, a patient glossary (25) was developed to explain technical terms in simple language, and factsheets were produced to summarize key aspects of the guide – particularly how patients can contribute to the HTA process and how funding decisions for treatments are made – serving as quick-reference tools for patients who are unable to read the entire guide.

Consultations on the guide, glossary, and supporting factsheets were conducted between November 2022 and January 2023 with the ACE Consumer Panel, SAPI, ACE’s technical teams, and international patient engagement experts from CDA-AMC (Canada), NICE (England), the Australian Government Department of Health, Disability and Ageing, and Patient Voice Initiative (Australia). Many consultees provided positive, constructive feedback to help improve the documents. Overseas experts also expressed strong support for the CEE team’s collaborative approach, noting that co-developing processes with local patient organizations ensures alignment with shared values, builds trust in the process, promotes transparency, and supports capacity-building among patient groups.

Following the publication of these resources in January 2023, ACE hosted an in-person briefing and networking session in February 2023 with representatives from local patient organizations, industry, and academia. The event aimed to promote greater understanding of ACE’s new patient involvement processes and encourage participation from all patient groups.

Building on the success of initial patient involvement opportunities

Leveraging the success of the initial patient involvement opportunities for drug HTAs, ACE has since expanded patient participation to non-drug topics including gene therapies and home-based or wearable medical devices, recognizing the importance of incorporating patient perspectives across a broader range of health interventions.

Similarly, in 2023, a new process was co-developed with patient organizations to support patient and caregiver contributions to HTA topic prioritization by enabling them to suggest health technologies for evaluation (Reference Koh, Pearce, Quek and Tan27). This process known as the annual call for topics, marks a significant shift toward valuing patient and caregiver perspectives. It helps identify areas of unmet need, and informs the selection of health technologies, ensuring that HTAs are aligned with the priorities and lived experiences of the population.

Initial challenges and future priorities

When the CEE team was first established, awareness of patient involvement in HTA was limited among local patient organizations. There was uncertainty about how to do it well, in a way that fostered genuine partnerships and reflected the diversity of patient experiences. Insights from the ACE Consumer Panel and overseas experts helped identify perceived barriers to implementation and inform practical solutions to address them, ensuring that any approaches adopted were patient-centered, non-tokenistic, and accessible. Early processes had flexible approaches, so they could be piloted for select HTAs, then fine-tuned based on experience, to better meet patients’ needs.

Using patient input to inform HTAs and support decision making was also initially unfamiliar to ACE’s technical teams and MOH advisory committee members. Early feedback indicated some skepticism regarding whether patient input could offer evidence that was sufficiently robust to be considered alongside clinical trial data. However, as the number of HTAs with patient input has grown, both ACE staff and advisory committee members increasingly recognize the distinct yet complementary insights it provides, enriching the evidence base beyond what traditional clinical studies alone can offer. This initial barrier is not unique to ACE and has been reported by other HTA agencies. Misconceptions often stem from the perception that patient perspectives – being inherently subjective – lack the robustness of scientific evidence derived from clinical trials. However, a supportive organizational culture, along with dedicated time to discuss patient input during committee meetings, has been identified as a key enabler of meaningful patient involvement in HTA (Reference Menon and Stafinski6;Reference Facey, Boivin and Gracia28).

Patient organizations have responded positively to ACE’s patient involvement process, with many expressing strong interest in learning more about HTA. This enthusiasm was evident not only in the 82 patient survey responses received during the pilot initiative but also in the high rate of participation seen afterwards – at least one patient testimonial was submitted for 85 percent of HTAs in the first year, increasing to 89 percent in the second year (Reference Tan and Pearce29). While patient involvement is still emerging in Singapore, over half of the patient organizations identified through the stakeholder mapping exercise have already provided input into ACE’s HTAs. This reflects growing trust in the process and highlights the proactive efforts of the CEE team to raise awareness of patient involvement opportunities and provide support for patients wishing to participate.

Another challenge encountered was the limited availability of patient input in certain contexts – particularly for rare diseases, conditions for which no relevant local patient organization exists, or when invited organizations did not respond. These gaps posed a risk of excluding patient perspectives from HTAs where insights could be especially valuable due to limited clinical trial data or unique lived experiences. To address this, the CEE team proactively engages clinicians to identify and refer patients who can provide testimonials. So far, this strategy not only helped fill information gaps but also uncovered informal patient networks that had been established by clinical care teams. These groups have since been incorporated into ACE’s stakeholder database, helping to extend the reach of future efforts to ensure that voices from less-visible communities are also represented in decision making.

Throughout this journey, the CEE team has actively modified processes in response to early challenges, with the aim of strengthening and sustaining patient involvement. For example, the patient survey was initially disseminated as a Word document via email to patient organizations. In response to feedback that WhatsApp is the primary communication channel for many organizations and that patients prefer completing surveys on their mobile devices, the team developed an online survey form optimized for mobile phone use. Additionally, hard copies of the survey are available upon request to accommodate individuals who are less familiar with digital platforms. Beyond adapting the mode of survey distribution, the CEE team also revised their approach to sourcing input. To minimize burden on patient organizations, early efforts focused on collecting responses directly from individual patients. However, in response to requests from several groups, a new template was introduced to enable well-resourced organizations to interview their members and provide a consolidated submission. This approach aligns with established practices in agencies such as CDA-AMC (10), NICE (9), and the Australian Government Department of Health, Disability and Ageing (11). ACE remains committed to offering multiple formats for submitting testimonials, allowing patients to choose the method that best suits them.

ACE’s approaches to addressing barriers and identifying enablers of patient participation – including raising awareness of involvement opportunities, promoting participation from diverse yet representative patient groups, and adapting processes as needed – align with international frameworks that define meaningful patient engagement as participation supported by appropriate institutional structures, capacity-building and resources (18;Reference Facey, Boivin and Gracia28;Reference Pickaert30). While HTA agencies often use process indicators to measure patient contributions, such as the number, quality, and timeliness of submissions (Reference Pickaert30), patient organizations may define meaningful participation differently – emphasizing non-tokenistic and genuine engagement from HTA agencies, as well as timely and transparent feedback on how their input influenced HTA outcomes (Reference Mercer, Chambers, Mai, McDonald, McMahon and Chan31). These priorities have also been consistently highlighted by the ACE Consumer Panel and local patient groups (Reference Teo, Pearce and Tan24).

In response, the CEE team places a strong emphasis on openness and two-way communication with local patient organizations. To ensure engagement goes beyond tokenism, the team informs patient organizations of the outcomes of the HTAs they contributed to and shares which aspects of their survey responses were most helpful to the evaluation and the advisory committee’s deliberations. To support continuous improvement, a structured online feedback form is used to collect suggestions from patient organizations, helping ACE to improve its processes and resources to better meet patients’ needs.

Feedback to date has been both positive and insightful. While some suggestions cannot be implemented immediately, ACE is committed to addressing them overtime where feasible. One important gap identified by local patients is the lack of resources in languages commonly spoken by Singaporeans other than English – namely Chinese, Malay, and Tamil – limiting participation among non-English literate patients. The CEE team is currently exploring the translation of key materials to promote inclusivity, while also prioritizing the development of supportive English-language tools to address frequently asked questions. Additionally, patients have proposed conducting one-to-one interviews in spoken dialects such as Hokkien, Cantonese, or Teochew to engage elderly patients more effectively.

Evaluating the impact of patient involvement in HTA is an emerging focus, though current efforts are at an early stage of development. Initial findings indicate that patient input serves multiple purposes. When patient insights align with existing scientific evidence, they provide due diligence and assurance to the advisory committees that the evidence accurately reflects the views and experiences of local patients. Conversely, where evidence gaps exist, patient input helps explain unmet clinical needs and assess the extent to which new treatments address patients’ priorities and preferences.

ACE’s formative evaluation framework includes both process and impact measures. These cover the resources required, the number of patient inputs collected and the methods used, the therapeutic areas represented, the number of evaluations informed by patient input, and the ways in which input is used at different stages of the HTA process by technical evaluators and decision-making committees. Evaluation findings are helping to identify which communication and engagement strategies are most effective, highlighting approaches that effectively influence HTA, and signaling where mid-course adjustments may be necessary to strengthen and diversify patient involvement methods. For instance, the potential for patient organization disengagement due to survey fatigue is being monitored, along with the time required for patient organizations to provide submissions, and strategies are being adapted as needed to address these challenges. Additionally, a curated collection of case studies demonstrating the impact of patient involvement was recently published (32) with more to be added over time and shared with patient organizations. This will not only promote broader awareness of the value of patient input but may also encourage sustained participation. ACE Consumer Panel members and leaders from local patient groups have affirmed the importance of documenting impact stories, noting that “most patients would be willing to be involved if their input can bring value to other patients.”

An illustrative example of impact was ACE’s evaluation of continuous glucose monitoring systems for type 1 diabetes mellitus (T1DM). Input from the patient community provided important insights into real-world usage patterns, which directly informed assumptions underpinning the budget impact modeling. Furthermore, although the clinical evidence base for pediatric use was limited at the time of assessment, patient testimonials highlighted improved clinical outcomes for children and adolescents which supported the advisory committee’s recommendation to extend subsidy eligibility to both pediatric and adult populations. By ensuring the recommendation reflected unmet needs and patient expectations across age groups, the involvement process strengthened the relevance, credibility, and acceptability of the funding decision (Reference Tan, Pearce and Quek33).