Findings from the Online Survey

1. (i) Use of RWD/RWE in HTA

From the online survey, all respondents reported that their HTA agencies accept RWE either as standalone or supplementary evidence to estimate clinical effectiveness in local HTAs to inform reimbursement (Table 2). Seven health systems require explicit justification for the use of RWE to be included in the HTA dossier, while four health systems waive this requirement. Respondents confirmed that RWE from a variety of patient populations has already been submitted to inform reimbursement decisions for specific technologies in areas such as rare diseases, cancers, and immunology, among others. Several of the respondents indicated that RWE from other health systems is acceptable although local data are preferred. Nonetheless, any RWD should be collected systematically and transparently so that the RWE can be validated and undergo quality assessment. Seven of the health systems currently accept data from pragmatic clinical trials (PrCT) (Reference Chalkidou, Tunis, Whicher, Fowler and Zwarenstein15); the remaining four health systems indicated that they plan to accept such data over time, but not within the next six months. Some health systems only consider RWE or data from PrCT as supplementary evidence to clinical trial data, or on a case-by-case basis when clinical trial data are lacking (e.g. for rare diseases).

Table 2. Acceptance of RWD/RWE in HTA to inform reimbursement decision making* (N = 14)

RWD, real-world data; RWE, real-world evidence; PrCT, pragmatic clinical trial; HTA, health technology assessment.

^a In response to: “Does your HTA agency currently accept clinical effectiveness data (e.g. relative risk, odds ratio, sensitivity and specificity, etc.) from real-world data sources (e.g. registries, claim databases, observational studies) for HTA in informing or making reimbursement decisions?”

^b In response to: “Do you require people who submit HTA dossiers to provide justification(s) for the use of clinical effectiveness data from real-world data sources?”

^c In response to: “Does your HTA agency currently accept clinical effectiveness data from pragmatic clinical trials? If not, are you planning to?”

^d In response to: “Does your HTA agency think that pragmatic clinical trial should be conducted for every novel therapy to be considered for reimbursement?”

* As supplementary evidence to randomized controlled trials (RCTs) or in specific instances where RCTs are lacking (e.g. for rare diseases).

One-third of the survey respondents have encountered situations where clinical trial data are not available and they have made reimbursement decisions solely on clinical effectiveness data from other sources such as registries and observational studies. This predominantly occurred when it was unethical to conduct a clinical trial for the technology under assessment, or when the condition was very rare and there were insufficient patients to conduct a trial. For example, in one health system, the decision to reimburse cholic acid for the treatment of primary bile acid synthesis disorder (a very rare disorder) was informed by a case series.

• (ii) Challenges in using RWD/RWE

Table 3 summarizes the key challenges encountered by HTA agencies when using RWD/RWE for HTA. It was most commonly reported that there was insufficient evidence in the HTA dossier that the patients selected to generate the RWE truly reflect the patients in local routine clinical care, hence giving rise to potential selection bias, and reducing the external validity of the results. A large number of respondents also indicated that many HTA dossiers did not clearly report if the patients in the RWD studies were receiving other treatments or had other comorbidities. In addition, some respondents felt that they may not have the expertise to evaluate whether confounding had been properly accounted for when assessing RWD/RWE.

• (iii) Availability of RWD/RWE guidance documents in Asia

Table 3. Challenges encountered regarding use of RWD/RWE for HTA to inform reimbursement decision making (N = 13)

RWD, real-world data; RWE, real-world evidence; HTA, health technology assessment.

^a In response to: “What are the challenges that your HTA agency encounters with regards clinical effectiveness data from real-world data sources? Select all that apply.”

We asked respondents if they had guidance documents available in their health systems for the following areas: (i) circumstances under which clinical effectiveness data from RWD sources can be included in an HTA dossier; (ii) the minimum standards for collecting and submitting clinical effectiveness data from RWD sources for HTA; (iii) how to account for confounding factors when analyzing RWD; and (iv) how to reduce selection bias when designing a non-randomized study (Table 4). Five of eleven health systems have available guidance. South Korea has documents available in Korean for all four topics. In China, the China Real World Data and Studies Alliance (ChinaREAL) has issued five guidance documents on how to design observational studies (Reference Peng, Shu, Tan, Wang, Nie and Wang16), how to develop research databases using existing health and medical data (Reference Wang, Gao, Wu, Xuan, He and Hu17), how to develop patient registries (Reference Tan, Peng, Shu, Wang, Li and Zhang18), how to conduct PrCT (Reference Wen, Li, Liu, Guo, Li and Guo19), and how to appropriately analyze RWD (Reference Gao, Wang, Luo, Ren, Hu and Tang20). In January 2020, the National Medical Products Administration in China issued guidance on the use of RWE for drug development and assessment for pilot testing (21). In Singapore, while the existing HTA guidance documents describe the circumstances under which clinical effectiveness data from sources other than clinical trials may be used as supplementary evidence to inform decision making, most reimbursement decisions are predominantly informed by RCTs. In Malaysia, although RWD has been identified as a pharmacy research priority area for monitoring therapeutic outcomes (22), there is currently no specific plan to develop guidelines about how to use this data for HTA. In the Philippines, the Guidelines of the PHL National Formulary System (AO 2016-0034) does not explicitly state if RWD/RWE should be collected. However, in the Exemption section of the policy, there is nothing to preclude the Formulary Executive Council from requesting any type of data to inform their recommendations.

Table 4. Availability of guidance documents on the use of RWD/RWE

RWD, real-world data; RWE, real-world evidence; HTA, health technology assessment.

^a A document is considered as “Shareable in English” if it has an English version which is publicly available or available upon request.

Findings from Face-to-Face Meeting and/or Teleconference

The face-to-face meeting allowed for more in-depth and varied discussions about RWD/RWE beyond the scope of the online survey. From the examples of drug reimbursement decisions that involved RWD/RWE presented by each health system, it was clear that there was significant variation in: (1) sources and types of RWD/RWE being used in HTA; and (2) how RWD/RWE is being used for policy-making (e.g. initial reimbursement, re-assessment for price adjustments, or investment/disinvestment purposes and expanding coverage).

In general, respondents reported that RWD collected from a variety of sources, such as medical records, patient registries, and claims databases has been used for HTA. For example, in China, RWD is collected from regional EMR, EMR from single care institutions, disease registries, and claims databases. However, to ensure the quality of RWD, the HTA agency in China only accepts clinical effectiveness data collected by teaching hospitals, or used in published articles in peer-reviewed journals for quality assurance. In Taiwan, a cost-effectiveness analysis of a treatment for Alzheimer's disease was conducted using transition probabilities derived from observational data from a clinical sample, cost data from the National Health Insurance database, and a caregiver survey (Reference Fuh and Wang23). In Bhutan, the combination of data from the Health Management Information System (HMIS) and the Royal Centre for Disease Control (RCDC) surveillance was used to estimate the healthcare utilization (outpatient visit and hospitalization) related to a particular viral infection for the evaluation of a vaccine.

How RWD/RWE is used in HTA and decision making also varies across Asian health systems, with some using it primarily to inform pharmacoeconomics, pharmacovigilance, or pharmacoepidemiology, while others also use it for reassessments. In reassessments, RWE informs price adjustments and/or disinvestment decisions of the technologies that have gained an initial positive reimbursement decision. This is relevant, if new evidence regarding the technologies becomes available and highlights unforeseen benefits or harms that require reassessment, or if newer technologies with greater benefits enter the market, triggering decision makers to consider whether they should disinvest the existing technology (24). RWE also informs effectiveness of off-label use of drugs, which may result in the coverage expanded to more indications and is relevant in health systems that accept coverage with evidence development as a tool for facilitating potential expanded indications or use of therapies (Reference Kumluang, Ingsrisawang, Sangroongruangsri, Ratanapakorn, Ruamviboonsuk and Taweebanjongsin25).

There are some challenges that were not mentioned or not emphasized in the survey but were discussed in the meetings by attendees who were predominantly HTA agency representatives, academics, and clinicians, for, for example, (i) lack of infrastructure and human capacity to support data collection; (ii) lack of clinician, institutional, or legislative support for data collection; (iii) lack of experience in analyzing and interpreting RWD/RWE; and (iv) collecting RWD/RWE for drugs with multiple indications.

1. (i) Lack of infrastructure and human capacity to support data collection

Participants cited the lack of EMR in several health systems in Asia due to limited resources as a major barrier to generating RWD/RWE. In such cases, it is common to establish patient registries with manual data curation to make up for the lack of EMR. However, due to lack of funding and manpower, the completeness and quality of data in patient registries in these health systems also tend to be suboptimal (e.g. there are substantial inconsistencies found in registry data sets when checked against actual patient medical records), which can make them an unreliable source for RWD.

• (ii) Lack of clinician, institutional, or legislative support for data collection/sharing

Many healthcare providers do not see the collection of RWD within their scope of work. Administrative data are not typically organized or collected for the purpose of measuring patients' outcomes and experiences, thus posing IT challenges and potential data entry errors, when existing electronic record systems are modified to enable providers to capture relevant outcomes. In some health systems, there is no existing legislation to support the setting up of disease registries. Hence, these health systems lack registries that are commonly found in the US and Europe, for example, cancer registries. In one health system, a registry was set up to facilitate payment to clinicians. However, as the sponsor for the registry was not interested in effectiveness data, clinicians were not diligent in entering clinical data, resulting in captured information that was not useful to inform HTA decision making.

The conservative attitude of the Asian governments, who are often the custodians of RWD, was also mentioned as one of the challenges encountered in the use of RWD/RWE. It was agreed by most participants that Asian governments are more reluctant to release health data for quality assessment or research than in Europe and North America due to concern over data privacy and data ownership for publication purposes. This impedes data access and data linkage. In a few health systems, participants explained that their government was reluctant to link public claims data to demographics, socioeconomic factors, or health outcomes data out of concern over data privacy. In one health system, the government held comprehensive data but was unwilling to share the data with anybody including the HTA agency, which was independent from the Ministry of Health, to maintain control over how the data were analyzed and reported.

• (iii) Lack of experience in analyzing and interpreting RWE

The challenges of conducting analyses to mitigate confounding and selection bias on RWD/RWE were mentioned in the online survey and reiterated during the face-to-face and teleconference discussions. Also, participants highlighted that it is easy to be misled by the HTA dossier if the review committee does not have sufficient experience in assessing the quality of and interpreting RWD/RWE. For example, it is important for the review committee to recognize that anecdotal evidence, such as patient testimonials (a form of RWD/RWE), may be subject to bias, especially survivor bias as only those who have had the treatment and survived will live to tell the story. It is thus imperative that the review committee members are sufficiently trained and experienced to review RWD/RWE and guidance is in place to ensure that RWD/RWE is assessed and applied consistently in decision making.

• (iv) Lack of data quality assurance

In many institutions, there is no provision to ensure accurate data capture for drugs with multiple indications. For example, hospitals may code for the indication that will provide the larger reimbursement (if applicable) even though the drug was used for a different indication, and there are limited audit mechanisms in place to assess actual indication-specific use. An example was shared in one health system, where the use of angiotensin-receptor blockers (ARB) is covered by the national reimbursement scheme for hypertension but not for chronic kidney disease. Local hospitals apply the International Classification of Diseases codes for hypertension in all reimbursement claims to ensure that they get paid for all patients who are prescribed ARB irrespective of actual use. While decision makers are aware of this practice, effective audit practices are currently not in place to accurately capture prescribing practice.