Introduction
Currently, more than 200 million types of medical devices categorized into over 7000 generic groups are on the world market (1). The World Health Organization defined medical device as “An article, instrument, apparatus or machine that is used in the prevention, diagnosis or treatment of illness or disease, or for detecting, measuring, restoring, correcting or modifying the structure or function of the body for some health purpose” (2). Similarly, in China, medical devices refer to devices, machines, apparatus, in vitro diagnostic device including the relevant calibrator, material, and other items, and computer software directly or indirectly interacted with human body (3). In modern medicine, as an indispensable part of the new technologies, medical devices are inseparable from new medical technologies, complementing with each other. In China, the basic medical insurance program reimburses medical devices accordingly: for those nondurable medical devices, the reimbursement happens as common practice, reducing the price from payment and settlement, but for the fixed and permanent devices, reimbursement is converted as part of the procedural fee and medical service expense. In this study, medical devices are defined as implantable medical devices that are used for medical purposes and reimbursed by the basic medical insurance program with high value. Examples of high-value implantable medical devices include pacemakers, knee implants, stents, and so on.
With the rapid development of health services and the progress of medical science and technology, the consumption of medical devices, especially the high-value implantable medical devices, is increasing and new medical materials are constantly emerging. According to a latest unrevealed study, the market size of medical devices in 2020 were 772.1 billion yuan in China, among which high-value medical devices took up 59 percent of the total market share. From the perspective of medical insurance payment, the National Healthcare Security Administration (NHSA) has launched the DRG/DIP payment reform across the country in recent years (4). To hospitals, medical devices are paid by health insurance according to the price of each DRG/DIP group which is determined in advance and irrelevant to the number medical devices used. The rapid adoption and diffusion of new medical devices gradually become a great expenditure of the social basic health insurance, an important medical expense of hospitals, and a huge economic burden to patients (Reference Yingpeng, Xiang and Yue5).
Medical devices represent a very heterogeneous family of technologies. However, the innate characteristics of medical devices make it hard to perform value evaluation, as explained below. First, unlike pharmaceuticals which directly interact with patients by generating biochemical reactions, medical devices rely on their physicality for treatment (Reference Neugebauer, Rath and Antoine6). The different mechanism of action, thus, leads to reliance on different clinical evidence to prove its clinical effectiveness: high-level clinical evidence (randomized controlled trials) required by pharmaceuticals and relatively low-level clinical evidence on medical devices. Second, a new device would undergo continuous alteration throughout its lifecycle, which considerably reduces its average product lifetime. Third, the interaction between the device and users, commonly the medical doctors, known as the learning curve, also matters. The performance of medical devices changes as medical doctors become familiar and proficient with the operation. Thus, evaluating the performance of the devices and users’ interaction in practice is challenging (Reference Bitkina, Kim and Park7). Fourth, the introduction of medical devices can have a substantial organizational impact (Reference Tarricone, Callea and Ogorevc8), for example, new medical devices may require specific professional training or investments in infrastructure.
These special characteristics impede medical devices from unfolding the widely accepted cost-effectiveness evaluation. A few studies in China have explored the attributes that comprise the value of medical devices (Reference Hui, Chunlin and Xin9–Reference Huangying, Qiao and Wan11); however, it’s still too hard to fully understand what factors are considered from stakeholders’ perspective, especially how decision-makers handle the elements of the value of medical devices.
A discrete choice experiment (DCE) allows people to state their preferences in different hypothetical options (Reference de Bekker-Grob, Ryan and Gerard12). Nowadays DCE has been increasingly employed in the healthcare to elicit preference (Reference Pfarr, Schmid and Schneider13); however, few studies applied DCEs to quantify multistakeholders’ preferences on medical devices. Given the increasing demands for conferring objectivity, transparency, and consistency on the decision-making process, this study aims to gain insight into the attributes as presented in the value of medical devices and quantify their strengths by conducting a DCE. The results of this study will hopefully provide decision-makers with references on the value evaluation of medical devices and promote more transparent decision-making procedure and enable sponsors or researchers to collect relevant decision-making data.
Methods
Discrete choice experiment
DCE was conducted to quantitively evaluate the preferences of multistakeholders on medical devices. The under-reviewed medical devices are described by several attributes, and each attribute attaches several levels. By choosing between pairs of hypothetical medical device profiles with various combination of levels under each attribute, respondents’ preference thus could be determined.
Attribute and level identification
After collecting the relevant and possible attributes of medical device evaluation published previously, we discussed them thoroughly with 5 health technology assessment (HTA) experts and finally identified the following 6 as clinical effectiveness, clinical safety, innovation, disease severity, implementation capacity, and cost as shown in the Table 1.
Attributes and levels of the DCE
Note: Clinical effectiveness was described by two levels: low level and high level. Low meant the ability to improve the short-term outcome of the treatment, such as reduction of surgery duration, the amount of surgical bleeding, and the length of hospitalization, while high level was defined as the improvement of both short-term and long-term treatment outcome, like decreasing the recurrence rate, prolonging lifetime and improving patients’ quality of life.
Two levels were used to measure the clinical safety, the incidence rate of having adverse event (1 percent/8 percent). Initially, this attribute was described by three levels: 1 percent, 8 percent, and 15 percent, selected form a network meta-analysis exploring the adverse event related to stent (Reference Madhavan, Kirtane and Redfors14), one of the most representative implantable medical devices. During the pilot survey, however, medical doctors suggested that 8 percent of adverse event rate was overwhelming in real-world therapy, and 15 percent was an unacceptable and unimaginable scenario. We adjusted our levels accordingly.
Disease severity and innovation had low and high to define themselves (Table 1).
The cost of the medical devices used in every single treatment was divided into three levels (2000 yuan/20000 yuan/50000 yuan) according to the price distribution of high-value medical devices in Nanjing medical insurance list and experts consultation, comparable to the cost level defined in other studies (Reference Lee and Bae15).
Attributes like clinical effectiveness and clinical safety are repeatedly emphasized in several medical devices’ management rules issued by the authority and major considerations in real-world application. In this DCE survey, clinical effectiveness refers to the improvement of patients’ health outcome after treatment and short-term and long-term therapeutical effectiveness; clinical safety was defined as adverse event incidence rate of the medical devices and operational risk; innovation meant whether this device possessed new iterations or new indications to the existing devices or not; disease severity was defined as whether the targeted disease was life threatening or not; implementation capacity was assessed from three different sector, the health system, the medical institute, and medical doctors’ learning curve.
DCE design
Based on the settled 6 attributes and their levels used in this study, about 144 hypothetical profiles would be generated, an impractical number to be handled both by the investigators and the participants. Therefore, D-efficiency design was adopted to cultivate the most representative and effective combinations (Reference Reed Johnson, Lancsar and Marshall16); a total number of 27 sets with different combinations of attribute levels of the two alternatives were randomly divided into 3 blocks, each including 9 sets. Choice set no. 5 of each block was duplicated as a verification set, added into the questionnaire as set no. 10. Those 3 blocks were randomly assigned to the eligible participants with even distribution.
Considering the attention distraction and preference bias caused by the name of each alternative due to the participants’ well-grounded but different mindsets toward different medical devices or product names, we chose medical device A and medical device B in each set.
A pilot survey of 14 participants including HTA experts, medical doctors, and management personnel from hospital health insurance department and reimbursement department and local basic health insurance center was performed to ensure the feasibility and validity of the questionnaire. After receiving feedback, we adjusted the questionnaires’ language expression and formats accordingly. The survey questionnaire was then finalized and settled.
Target population and sample size
The rapid technology iteration, limited healthcare technology assessments, insufficient clinical evidence support and the individual differences of learning curve complicate the evaluation of high-value medical devices. To conduct comprehensive evaluation, the involvement of multiple stakeholders needs to be considered. The participants of this study, therefore, are chosen according to their roles in pricing and reimbursement decision-making, ranging from government department of basic health insurance management, medical institutes, HTA experts, and medical doctors to patients. Under the guidance of the thumb rule proposed by Orme, at least 75 participants were required in this study, based on the settled questionnaire of 2 alternatives, maximum 3 levels for attributes, and 10 choice sets (Reference Orme17).
To ensure the sufficient power and further subgroup analysis, a total of 134 participants completed the survey questionnaire but 16 participants were excluded for failing to satisfy the validation set (set no. 10 in very version of the survey). All respondents were given oral and written questionnaire instructions and finished individually via face-to-face interview conducted during the 21 May to 30 June 2023.
Analyses
In this study, stated preference was analyzed based on random utility theory (Reference Hensher, Rose and Greene18). There was
$ n $
assumed respondents and
$ i $
available combinations of the understudied medical devices’ levels, and n’s retrievable utility could be quantified by
$ {U}_{ni} $
, which contains two parts: the systematic relative utility
$ {V}_{ni} $
based on the attributes and an error term
$ {\varepsilon}_{ni} $
capturing individual-specific unexplained variation around the mean. We represent
$ {U}_{ni} $
as:
$$ {U}_{ni}={V}_{ni}+{\varepsilon}_{ni}={\beta}_0+{\beta}_{1i}{\chi}_{1i}+{\beta}_{2i}{\chi}_{2i}+\dots +{\beta}_{ni}{\chi}_{ni}+{\varepsilon}_{ni,} $$
where
$ {\beta}_0 $
is the constant telling the average preference for selecting participants’ favorable medical devices across the difference choice sets, and
$ {\beta}_1 to\;{\beta}_n $
are utility weight of every level in the population. Cost was estimated as random parameters, and dummy coding was used to describe other categorical attributes (Reference Hauber, González and Groothuis-Oudshoorn19). Initially, the DCE data were analyzed by mixed logit model and conditional logit model. Then the final model was settled based on several model selection criteria. Specifically, we chose Akaike information criterion and Bayesian information criterion to choose between mixed logit and conditional logit model (Reference Dziak, Coffman and Lanza20). Willingness to pay was calculated by taking the ratio of the mean coefficient for the attribute/level to the mean coefficient of the cost (Reference Hensher, Rose and Greene18). The relative importance of each attribute could be demonstrated by its highest coefficient minus its lowest coefficient. Subgroup analysis was performed in accordance with the participants’ career. All the statistical analysis were performed in the Stata 16.0.
Results
Respondent characteristics
The sample characteristics are reported in Table 2. Among the quantified respondents, approximately 75 percent (n = 88) were male, with a mean age of 45. Twelve percent were decision-makers from national, provincial, and city healthcare security administrations, responsible for managing medical devices Tendering and Bidding, National Drug Reimbursement List, medical insurance payment, and other relevant duties. Accordingly, hospital administrators were represented by staffs from medical insurance, medical affair, and medical devices price control departments, in charge of the whole process management of medical devices in the hospital. Well-experienced HTA experts were surveyed nationwide, taking up 16 percent. Approximately one-fourth of the respondents were experts titled with deputy directors or above in orthopedics, general surgery, thoracic surgery, neurosurgery, and cardiovascular medicine. The rest were patients suffering from specific medical conditions with implantable medical devices like heart disease with pacemakers.
Respondents’ characteristics (n = 118)
HTA, health technology assessment.
Preference
A total of 2360 observations from 118 respondents are analyzed in Table 2. All the six selected attributes had the expected statistical significance assessed by their p value, indicating that respondents were more likely to evaluate high medical devices from the perspective of clinical effectiveness, clinical safety, innovation, disease severity, implement capacity, and cost (Table 3). The positive sign of coefficients indicates that respondents prefer innovative medical devices with 1 percent adverse event incidence rate, long-term clinical improvements and outcomes after operation, and moderate implementation capacity that treat severe disease over the ones that without innovation, with 8 percent adverse event incidence rate, short-term clinical improvements, and low or high implementation capacity treating mild disease. The negative sign for then coefficient of cost (β = −0.0000203, p <. 001) indicates that respondents preferred a less costly device. Among the six attributes, clinical safety was the most important attribute (relative importance = 38 percent), followed by cost (relative importance = 16 percent), clinical effectiveness (relative importance = 16 percent), disease severity (relative importance = 10 percent), implement capacity (relative importance = 10 percent), and innovation (relative importance = 9 percent) (Figure 1).
Multistakeholders’ preference for high-value medical devices: main effects of mixed logit model results
Relative importance of each attribute.
The magnitude of the coefficient showed that the unit change in clinical safety (β = 2.193, p <. 001) would greatly impact the utility and probability of being chosen. Similar applies to the innovation of medical devices (β = 0.486, p <. 001), in which respondents show more preference on innovative technologies. Disease severity (β = 0.579, p <. 001) also had an expected effect on the choice of medical devices: a medical device that treats severe diseases is more preferrable than that treats mild diseases. The DCE result also revealed that among the levels of implement capacity (including application frequency and fields, impact on medical expenditure, and medical doctors’ learning curve), the middle level (β = 0.549, p =. 05) was most preferred than high level (p > .05) by the respondents.
Subgroup analysis
Subgroup analysis was conducted in terms of respondents’ professions, and the coefficients vary accordingly (Supplementary Table S1). All subgroups had the strongest preference on clinical safety, comparable to the full sample size. For hospital administrators, all the attributes and levels matter except disease severity (p =. 489). Besides, they laid more emphasis on the high level of implement capacity than other subgroups. As expected, decision-makers attached relatively more weight to cost (β = −0.0000279, p = 0) than others. Medical doctors also showed a substantial preference on cost (p =. 004). Innovation was considered important in most subgroups, but decision-makers (p =. 13), medical doctors (p =. 073), and patients (p =. 07) cared less about the high level of clinical effectiveness, but others cared more. HTA experts weighted cost less than the other respondents.
Willingness to pay
All respondents have the highest willingness to pay for the attribute of clinical safety (Supplementary Table S2). Taking low-level clinical safety as the reference, respondents would be willing to pay 107846.2 Yuan (95 percent CI, 66760.60–148931.80) for significant improvement in clinical safety. With respect to the clinical effectiveness, respondents were willing to pay 43512.44 Yuan (95 percent CI, 21454.62–65570.25) if they could choose a medical device with higher level of clinical effectiveness. In addition, they were willing to pay 28467.64 Yuan (95 percent CI, 12982.57–43952.72) for the one that treats severe diseases.
All subgroups showed the strongest willingness to pay for clinical safety, consistent with the results from the whole sample analysis. Among the subgroups, patients (180711.10 Yuan) and medical doctors (145344.50 Yuan) had the highest willingness to pay for clinical safety and HTA experts had the highest willingness to pay (CNY 96809.36 Yuan) for clinical effectiveness.
Discussion
This study investigated relevant stakeholders’ preferences regarding specific attributes constituting the value of medical devices and figured out which attributes significantly influenced the decisions-making of the pricing and reimbursement of medical devices. The analysis in this study also clearly revealed to what extent will the trade-off be made. As expected, the sign and comparative effect size of each attribute significantly impacted medical devices choice-making process.
The findings of this study advise that adjustments for stakeholders’ actions in medical devices pricing and reimbursement should be considered to promote the further development of medical devices in China. First, in terms of importance of the attributes, clinical safety, clinical effectiveness, and disease severity were the top three attributes, indicating that medical devices supported by evidence on clinical benefits and application scenarios are highly preferred. This is consistent with the results of the newly published study in which multistakeholders believed adverse events for the patient (defined as clinical safety in this study) and clinical effectiveness as critical for the evaluation of medical devices (Reference Lee and Bae15). Therefore, sponsors were expected to invest more in reducing the adverse event incidence rate and increasing long-term clinical benefits of medical devices rather than stressing micro innovation in procedural convenience. Nowadays, with the revolutionary change of concept brought by big data, real-world clinical data will be increasingly required by medical devices review and approval to promote the objective evaluation toward clinical effectiveness in China. In the year of 2019, National Medical Products Administration lunched a pilot project on the research and application of real-world data of medical devices in Hainan (21). The need for evidence-based evaluation is increasing. Second, respondents in this study also took implement capacity and innovation as important attributes into consideration, and the two attributes also could be observed in Queensland’s health technology assessment framework (Reference Howard, Scott and Ju22). This advises that sponsors should notice the impact of medical devices on the overall treatment and medical expenditure and invest more training resources to shorten the learning curve.
There are differences in preferences among subgroups. First, hospital administrators attached more weight to implement capacity than other groups. A reasonable explanation for this might be that the respondents from hospital administrators covered every sector of the medical devices management process, leading to more comprehensive consideration on medical devices. Second, the decision-makers gave higher preference to cost, which could be explained by the NHSA endless efforts on bulk-buying of drugs and high-value medical devices on a regular and institutionalized basis to improve operational efficacy and performance of the medical insurance funding (23). Third, HTA experts focused on the link between long-term clinical benefits and costs, which is in line with the cost-effectiveness concept of health technology assessment (Reference Mathes, Jacobs, Morfeld and Pieper24). Fourth, following the clinical safety, medical doctors cared more about cost for the DRG/DIP payment reform in China to improve their awareness on medical expenditure control. Fifth, medical doctors and patients were statistically less concerned about clinical effectiveness, which may be due to the setting of the clinical effectiveness levels. After consulting five HTA experts, we assumed the medical devices in the survey were clinically effective, so even the base level of clinical effectiveness showed short-term effectiveness. In other words, medical doctors and patients preferred the short-term clinical effectiveness and focused on the clinical outcome happened during hospitalization.
To our knowledge, this is the first DCE study of stakeholders’ preference for medical devices in China. The questionnaires were finished via face-to-face interview, so the attributes and their levels could be clearly understood by all the respondents and the quality of the survey also could be guaranteed. Meanwhile, the respondents involved in the survey comprehensively represent the multistakeholders related to medical devices decision-making. For example, decision-makers were chosen from national, provincial, and city healthcare security administrations in accordance with their different working concentration; nationwide HTA professionals who have worked or are working on medical devices were interviewed in this study; medical doctors with years’ experience of using medical devices were invited.
Currently, there are limitations in this study. First, the patient respondents in this study were required to meet specific criteria, and therefore, the conclusion could not apply to the whole patient population. Meanwhile, the questionnaire in this study was constituted by several professional terms unfamiliar to the public, so literate and highly educated patients who could fully understand the terms were elected to answer the questionnaire. Second, medical doctors in this study were from a tertiary medical institute of the regional medical center, without investigating the preference of medical doctors working at local hospitals. In China, high-value medical devices, especially the innovative ones, are chiefly used in tertiary medical institute; consequently, this deficiency will not greatly impact the result. Third, the decision-makers and HTA experts also made statements about the definition of disease severity. They proposed that disease severity should combine disease incidence rate into consideration, which would better reflect what impact medical devices could bring to a specific disease. As patients are included in this multistakeholder’ preference assessment study, it is important to ensure patients’ understanding of the attributes. Disease incidence rate in this study was believed to be an ambiguous concept and difficult for patients to handle, so we simplified the definition of disease severity accordingly.
Conclusion
We identified the relevant attributes of medical devices in pricing and reimbursement decisions and what trade-offs respondents were willing to make. We also found that clinical safety was the most important attribute, followed by cost and clinical effectiveness. The findings of DCE will help to make the existing evaluation of medical devices more transparent and consistent in China and provide decision-makers with the relative importance of medical devices attributes. Given the operating pressure of the medical insurance funding and the increasing need for evidence-based evaluation, we hope the findings of this study will support health budget allocation decisions for medical devices in China.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462323002799.
Funding statement
This research received no specific grant from any founding agency, commercial, or not-for-profit sectors.
Competing interest
The authors declare none.
Open access
