Health technology assessment (HTA) seeks to provide policy makers with information on the clinical and economic value of health technologies (including pharmaceuticals, medical devices (MDs), clinical procedures, and organizational systems used in health care) to inform their reimbursement or coverage decisions (Reference Banta1).
Several surveys have been undertaken to date that describe and compare the HTA activities across organizations and countries (Reference Perry, Gardner and Thamer2–6). For example, in 2008, the European Network for Health Technology Assessment (EUnetHTA) published an international survey of HTA organizations examining the barriers and solutions for establishment of the HTA units, their characteristics and processes (6). These previous surveys have tended to focus on drugs rather than non-drug technologies (Reference Franken, le Polain and Cleemput7), often because “the use of HTA information in the reimbursement of pharmaceuticals is currently the most advanced area of research in the discipline” (Reference Oortwijn, Broos and Vondeling3). However, the report by Wilsdon and Serota showed that several countries have HTA systems in place with a mandate to include non-drug technologies, including MDs (Reference Wilsdon and Serota8). This report also confirmed that the number of HTA reports devoted to non-drug technologies remain in the minority.
The evaluation of non-drug technologies, in particular MDs, may pose different challenges to those of pharmaceuticals. Compared with drugs, undertaking randomized controlled trials are often more difficult, product modification occurs more frequently, pricing is more dynamic, and clinical outcomes are more depending on training, competence, and experience (“learning curve”) of the operator (Reference Drummond, Griffin and Tarricone9;Reference Taylor and Iglesias10). It is often difficult to disentangle procurement costs (including associated infrastructure) and running costs (including maintenance and consumables) of MDs (Reference Kirisits and Redekop11).
The aim of this study was to undertake a survey of non-European Union (EU) HTA agencies to characterize and contrast their activities for MDs in terms of: (i) organizational structure; (ii) processes, that is, standard operating procedures in use; and (iii) methods, that is, scientific methodologies and assessment in place. This study was undertaken as part of the MedTecHTA project (www.medtechta.eu), funded under the EU's 7th Framework Program (FP7). Under the conditions of this EU-FP7 funding, the MedTecHTA project was directed to undertake a survey of MD activities in non-EU HTA agencies. The EU-FP7 funded ADVANCE-HTA project (www.advance-hta.eu) project is responsible for a parallel survey of MD activity in EU HTA agencies.
METHODS
Selection of HTA agencies
We considered institutionalized HTA activities (12) and selected HTA organizations based on their membership, as of February 2013, of European network for HTA (EUnetHTA), Health Technology Assessment International (HTAi), International Network of Agencies for Health Technology Assessment (INAHTA) or World Health Organization (WHO) Collaborating Centres for HTA. In addition, we performed a search to identify additional HTA agencies who were not potential members of these networks. To be included, organizations had to be an HTA agency based on the HTA definition proposed by Kristensen (Reference Kristensen13). Based on this definition, patient and industry organizations were excluded.
Development of an Evaluative Framework
Our assessment of HTA for MDs was based on the key principles for the improved conduct of HTA proposed by Drummond et al. (Reference Drummond, Schwartz and Jonsson14). HTA activities were evaluated from three perspectives: Organizational structure: what are the HTA roles of the organization and how is it organized and governed? (e.g., separate department or allocation of resources or deployment of people to the HTA of MDs versus drugs); Processes: how does the organization conduct HTA and involve stakeholders? (e.g., separate decision-making committee, degree of stakeholder interaction or amount of time it time of completion of HTA versus drugs); and Methods: what methodologies are used for an HTA? (i.e., specific scientific method guidelines for assessing evidence related to MDs versus drugs).
We undertook a two-stage data collection process. In stage 1, we performed: (i) a review of the literature to understand the HTA system in each country, (ii) a review of agency Web sites using a standardized questionnaire, and (iii) review and validation of the completed Web site questionnaires by a senior representative of the agency. In stage 2, we conducted semistructured telephone interviews with key informants of the agencies that were identified at stage 1 as having developed specific HTA activities for MDs (see Supplementary Figure 1).
Stage 1: Literature Review, Web Site Data Collection, and Verification of Results
First, we reviewed published literature describing the HTA system in each of the included countries to understand the interactions and players involved from a macro perspective. This involved a Medline (PubMed) search (“country” AND (“Health Technology Assessment” OR “HTA”)). From this information, we developed a “map” summarizing the links between HTA agencies and other key stakeholders (e.g., regulatory agencies) (Reference Rogowski, Hartz and John15;Reference Taylor, Drummond and Salkeld16).
A questionnaire based upon a literature review of HTA of MDs and previous HTA survey (Reference Wilsdon and Serota8;17;18) was developed to collate information available of the agencies' Web sites. The questionnaire consisted of fifty items (forty-four closed and six open questions) from four sections (i.e., Section A - agency information, Section B - structure of HTA agency, Section C - agency HTA processes, and Section D - agency HTA methods. A copy of this Web-site assessment questionnaire is available from authors). Data extraction was performed by one researcher and checked by a second.
For verification of the collected data, a key strategic individual within each agency (e.g., Chief Operating Executive, Head of HTA Agency) was e-mailed a copy of our completed Web site questionnaire assessment and the country level HTA map. After verification, if applicable, these individuals were asked if they would participate in a follow up semistructured interview.
Stage 2: Semistructured Interviews of Selected HTA Agencies
Interviews were conducted in the subgroup of HTA agencies identified from Stage 1 to have MD-specific processes in terms of their structure, processes, or methods. The objective of the interviews were: (i) to clarify items reported in the Web based data collection process and (ii) to determine how the organizational framework mitigates challenges associated with performing HTA on MDs. If individuals did not respond to the initial invitation e-mail letter and the reminders, other key informants of the agency were identified and asked if they would participate. Interviews were by telephone and semistructured in format, lasting approximately 1 hour, and conducted by two members of the research team. We sought permission to audio-record at the outset of each interview.
Data Analysis
Data collected from the stage 1 analysis of agency Web sites is presented descriptively. Categorical and continuous data were summarized as frequencies and percentages or as medians and ranges, respectively. Stage 1 results are aggregated across all agencies and presented in a detailed tabular form. Given the objective of this study, aggregate results are also presented separately for “MD specific” and “non–MD-specific” agencies.
Stage 2 audio-recorded interviews collected from MD-specific agencies were transcribed. Answers to open questions were reviewed to identify major topics that were recurrently reported by the interviewees. These topics were then mapped to the three HTA perspectives outlined above, that is, agency structure, processes, and methods. Individual interviews were then searched for certain expressions (e.g., “efficient,” “challenge,” or “suggest”) reflecting either positive or negative opinions/recommendations. Those expressions were then linked to the major themes and coded using NVivo 10 qualitative data analysis software. To summarize the frequency of item occurrence, the following categorization was assigned: “some” (10–30 percent of respondents), “many” (31–65 percent), and “most” (66–100 percent) (Reference Siegal and Schrimasha19).
RESULTS
Selection of Included HTA Agencies
A total of 135 HTA agencies were identified; after applying our exclusion criteria, 36 agencies across 20 non-EU countries were included for analysis (see Supplementary Figure 2).
Stage 1: Web Site Data Collection
Web site data collection was completed for all thirty-six HTA agencies between June and September 2013; thirty-one (86 percent) data collection sheets were validated by an agency representative. Twenty-seven of the thirty-six agencies were considered to be “MD-specific” because they reported MD-specific information about either their organizational structure (21; 58 percent), processes (18; 50 percent), or methods (1; 3 percent) in stage 1 (Figure 1). The most common model of MD-specific organizational structure was an agency that retained a broad HTA remit (i.e., assessed the full range of health technologies including drugs, diagnostic, public health interventions) but had also developed, within the organization, a specific staff grouping with collective expertise in medical device HTA (e.g., Canadian Agency for Drugs and Technologies in Health (CADTH)). In contrast, the Medicines Advisory Services Committee (MSAC) in Australia is an example of agency established with specific remit for the HTA of devices (along with medical services). Various MD-specific processes were seen across agencies but could be broadly characterized as development of specific HTA process pathways for assessment of devices processes, for example, specific committees for the appraisal of device technologies.
Figure 1. Classification of HTA agencies as non–MD-specific or MD-specific (in terms of process, structure, or methods).
Structure
The organizational structures of the MD- and non–MD-specific HTA agencies are compared in Table 1. The two groups of agencies appeared broadly similar in terms of their funding, staffing and other element of structure. There were two important differences related to handling of MDs. First, whereas the remit of all HTA agencies included the evaluation of MD technology, a higher proportion of MD reports were produced by MD-specific than non–MD-specific agencies (51 percent versus 10 percent). Second, MD-specific agencies were more likely to apply a classification system when assessing MDs (e.g., US Food and Drug Administration (FDA) class III - life-supporting and life-sustaining products and high-risk devices) than non–MD-specific agencies (56 percent versus 33 percent).
Table 1. Summary of Organizational Characteristics of Non-EU Agencies
a All proportions based on the N = 36 (all agencies), N = 27 (MD specific agencies), N = 9 (Non MD specific agencies) unless otherwise indicated.
b A positive response could be given to more than one item.
Process
A summary of agencies' processes for conducting HTA is presented in Table 2. As with structure, the majority of process characteristics appeared to be very similar across MD-specific and non–MD-specific agencies. However, there were some areas of difference. Compared with non–MD-specific agencies, MD-specific agencies were more likely to have a structured process (38 percent versus 80 percent), an internal unit (25 versus 67 percent) responsible for priority setting, to prioritize based on patient outcomes (33 percent versus 59 percent) and budget impact (33 percent versus 64 percent) and to externally commission reports (25 percent versus 44 percent).
Table 2. Summary of HTA Process Across Non-EU Agencies
a All proportions based on the N = 36 (all agencies), N = 27 (MD specific agencies), N = 9 (Non MD specific agencies) unless otherwise indicated.
b A positive response could be given to more than one item.
Methods
Table 3 summarizes the scientific methods adopted by HTA agencies. Sixty nine percent of all agencies had a methods guidance document or policy for evidence submission. However, only one agency, the Department of Science and Technology in Brazil, reported that they have developed scientific methods guidelines specifically for the HTA of MDs. No agencies stated that they were in process of developing MD-specific methods guidance. Areas of difference between MD-specific and non–MD-specific agencies were: consideration of organization aspects (63 percent versus 33 percent), observational studies (35 percent versus 14 percent), expert opinion (52 percent versus 14 percent), and MD-specific attributes (52 percent versus 33 percent), the use of cost consequence (48 percent versus 14 percent) and cost minimization methods of economic evaluation and specific processes to ensure transferability (47 percent versus 0 percent).
Table 3. Summary of HTA Methods of Non-EU Agencies
a All proportions based on the N = 36 (all agencies), N = 27 (MD-specific agencies), N = 9 (non–MD-specific agencies) unless otherwise indicated.
b A positive response could be given to more than one item.
Stage 2: Semi-structured Interviews
All twenty-seven HTA agencies that were judged in stage 1 to be MD-specific were invited to participate in a follow up interview process (stage 2). Eighteen (67 percent) agencies agreed to be interviewed. In most cases, the interviews were conducted by telephone with the same agency individual(s) who verified the Web assessment questionnaire. In one case, a telephone interview was not possible and instead responses were obtained by e-mail. After an iterative process of qualitative data analysis, the following seven broad themes were identified: (i) capacity; (ii) complex intervention and co-dependency; (iii) decision making; (iv) evidence; (v) coordination; (vi) decision problem; (vii) transferability. In a next step, these themes were subsumed under the three dimensions of HTA used earlier in this report (i.e., structure, process and methods). A tabulated summary of the results of the semi-structured interviews is available in Supplementary Table 1.
Structure
Decision Making
Some interviewees indicated that their organizational structure effectively evolves to meet the needs of decision makers as it relates to HTA of MDs. By adapting the organizational structure according to the objectives outlined in policy requirements rather than by an HTA “codebook,” agencies contend that they are better positioned to efficiently inform key funders about the medical, social, economic, and ethical issues associated with introducing a new technology into the health care system. One interviewee from an MD-specific agency purported:
“The structure of the HTA has evolved to meet the funding programs; it directly reflects that need and not the other way round (not build on an HTA-centric view). HTA serves the need of the policy requirement and its organizational structure fits with what is expected from it.”
They find that this approach reduces the time needed to translate MD-specific evidence into the health system.
Capacity
Many interviewees cited insufficient resources (e.g., budget, skilled economists) as a limitation, creating hurdles in the agency's ability to adapt to policy needs for HTA of MDs. For example, one key informant explained that the lack of resources has a trickle-down effect on the way in which their agency is able to examine MD evidence:
“Not enough [MD] experts – there is a very large gap there. So we only look at evidence in the sense of systematic reviews, meta-analysis.”
Process
Capacity
Some interviewees indicated that they integrate external academic partners into the HTA process for MDs when they do not have staff with the appropriate skills to perform evidence assessments. In fact, the use of external networks was even cited as an asset by some interviewees. For example:
“I think some of the real operational issues related to medical technologies are important to understand – How this piece of equipment works in the hospital, who's using it, what happens if it's in a different pair of hands? I think we don't necessarily have a lot of that expertise in house. I'd say one of our strengths is building those networks.”
Coordination
Some interviewees noted the value of integrating subject matter experts in the HTA process for MDs. The input of experts fosters stakeholder buy-in on policy decisions resulting from evidence appraisals. Furthermore, it supports the contextualization of MD evidence and the introduction of MDs into the health system.
Another issue identified by many interviewees is that the introduction of MDs into the healthcare system is diverse and not standardized:
“For devices, our health care system is extremely fractured. Devices can come into our system at a regional level, hospital level, within a particular doctors' office, at a provincial level, or even at a national level. So, for devices we have an extremely fractured system for entry points. Because of that, we have different kinds of evidence requirement at different kinds of levels. I would go so far to say at some levels there is no rigorous evidence assessment taking place.”
Although many interviewees acknowledge that the procedure used to review devices, including evidence requirements, is inextricably linked to the point of entry into the health care system, they expressed discontent with the poor levels of evidence required for pre-market access for MDs.
Many also found fault with the level of harmonization between the regulatory and reimbursement bodies' evidence requirements. In consequence, this imbalance leads to unnecessarily prolonged review processes.
“The fact of having an evaluation of sanitary registration for marketing that does not use the same principles as are used for incorporation in the Public Health System, are considerable obstacles in HTA for MD.”
Complex Interventions/co-dependency
Another topic was how agencies manage complex interventions, that by definition include MDs, and the evidence assessment process for co-dependent technologies, for example, delivery of drug using a device technology. Responses were suggestive in nature, including recommendations on how to improve the management of MDs in complex environments as well as harmonize the review process by having co-dependent technologies simultaneously reviewed by drug and device assessment committees.
Decision Making
Some agencies noted that the approval channels integrated into their standard operating procedures mitigate potential challenges associated with translating MD evidence into policy. For example, one key informant describes current review processes as sufficient for advising decision makers on whether to adopt a new technology:
“In health technology assessment term, I think there is not any raised problem of HTA of MDs. The applied HTA of MDs has been discussed and assessed by sub-committee which consist of qualified medical doctors in their specialty, and has been re-discussed internally whether to accept, modify or reject the assessment of sub-committee.”
Methods
Evidence
Most interviewees stated that there is a lack of methodological guidance for MDs. Furthermore, they identified several shortcomings with evidence requirements for MDs.
“The main problem of health technology assessment is systematic literature review that search, analyze and contemplate existing studies […] The main problems of literature review are that any newly emerging technologies which have potential benefit for patients but lack of literature evidence will not be approved as new technologies.”
The quality of evidence (e.g., lack of randomized trial evidence) was cited several times as the main challenge for performing HTA of MDs.
“The main challenge is that there is not enough good evidence […] I think that there is a broad societal question of how we will ever collect enough data to answer all the questions we need to answer. I think that that's really the biggest challenge.”
The quality of evidence was also described as being a limiting factor for the scope of device assessments.
“There are certainly again limitations on what we can do with the devices given the nature of the evidence base.”
The MD industry was identified by some interviewees as a “conspirator” to poor evidence generation. As a result, agencies experienced challenges with obtaining the appropriate evidence for HTA of MD.
Capacity
Some interviewees identified human resource constraints as a great challenge when attempting to apply scientific methods to critique MD evidence. The range of staff skills tend to be limited. An interviewee described how they are addressing skill deficiencies such as the ability to appropriately evaluate various types of evidence or data.
“We've also recognized the need to put a real focus in terms of getting our staff comfortable with qualitative research. I'm not sure we'll get it to where it needs to be in the next year, but at least getting people to understand what qualitative research can bring to an assessment on the non-drug side and especially with respect to the non-clinical, non-economic of HTA. Within the team that we have here, we're working to build awareness of what it is and why it's important and then to be able to take on some projects to get people actually using it and doing it and then get their comfort level up.”
Transferability
Many interviewees identified the inability to generalize the findings obtained from MD evidence reviews to their local context as highly problematic.
“The methodology of HTA of MDs can be really difficult. All we know is generally translated from drugs to devices but we know that differences exist, for instance the performance of devices in clinical practice can be very different from that assessed in controlled setting.”
This weakness was attributed to device-specific challenges, such as the learning curve. Interviewees believed that intervention evidence, typically generated in environments with highly skilled staff, may not be representative of the health system where the technology will be introduced:
“Such another issue, an issue that probably exists here and elsewhere is that the early use and early evidence on devices tend to be in individuals with “the best hands problem”- so specialized academic centers that either develop technologies themselves or have used it for a number of years. So it's very difficult to then generalize what the findings might be in the community.”
DISCUSSION
Main Findings of the Study
In addition to providing a detailed survey of the practices and methods of a comprehensive sample of non-EU HTA agencies, this study specifically sought to characterize and compare current HTA activities for MDs across these agencies. In particular, we assessed agencies in terms their organizational structure, processes, and scientific methods. All 36 HTA agencies identified within our sampling framework undertook the evaluation of MDs. A substantive proportion (27/36; 75 percent) of HTA agencies reported that they had developed specific approaches within their organizational structure or procedures for the assessment or appraisal of MD technologies (“MD-specific agencies”). For example, organizational developments included the allocation of specific staff to MDs assessment or the setup of a completely separate program or unit within the agency for device evaluation. Procedural developments could include convening a specific committee to appraise MDs evidence and provide policy advice. There appeared to be few differences in the organization, process, and methods of between MD-specific and non–MD-specific agencies. Although several agencies had a methods guidance document or policy for evidence submission, only one MD-specific agency (Department of Science and Technology in Brazil) had developed specific scientific methodological guidance for the HTA of MDs (20).
Our interviews confirmed several commonly cited challenges in the HTA of MDs (Reference Drummond, Griffin and Tarricone9–Reference Kirisits and Redekop11). These included: the relatively poor quality of evidence for MDs, problems in generalizing the MD evidence obtained in a specific setting to another, the “learning curve” effect and difficulties in defining the scope of the appropriate decision problem when conducting HTAs of MDs. This later challenge is particularly emphasized by the recent national review of HTA in Australia in 2011 (21). This review has led to reform the processes by which the MSAC undertake MDs evaluation, including the introduction of a formal scoping workshop that brings together key stakeholders (i.e., industry, clinicians, and patients) to have the opportunity to debate and finalize the scope of the decision problem (e.g., in terms of the population, comparator(s), and key outcomes) before the state of the actual assessment of evidence.
That the vast majority of agencies identified in this survey has not developed MD-specific methods guidelines indicates that these agencies applied the same methodological approach to the assessment of MDs evidence as non-device technologies, such as drugs. However, we did identify an innovative approach to evidence generation for MDs. The MaRS Excellence in Clinical Technology Evaluation (EXCITE) program recently established in Ontario (22), has been developed to integrate research and development directly into the health care system to collect the evidence necessary to support “parallel submission” with both regulators and HTA decision-making authorities. At the time we completed our survey, the EXCITE program was recruiting patients for its first randomized controlled trial of a MD (SymplicityTM catheter renal ablation therapy for treatment-resistant hypertension), specifically designed to facilitate parallel submission to Health Canada, for regulatory purposes, and to the Ontario Health Technology Advisory Committee (23). Other ongoing EXCITE projects include electrical stimulation for upper limb movements in stroke patients, home sleep apnea events detector, and RNA disruption assay for early prediction of complete response to chemotherapy in breast cancer patients (Reference Levin24).
Comparison to Previous Literature
Whereas there have been several previous surveys of international HTA agency practice, we believe this to be the first international survey of HTA practice to focus on MDs evaluation. Three previous studies have included aspects of their review of HTA practice to the consideration of MDs. In 2012, Stephens et al. published a survey of thirty HTA organizations representing sixteen countries including Europe, North and South America, and Australia (Reference Stephens, Handke and Doshi5). In accord with our findings, this report found that the majority of respondents assessed MDs: 94 percent in Europe and 76 percent in the United States. Similarly, thirty of the forty-one (75 percent) HTA organizations responding to a EUNetHTA survey reported to assess MDs (6). In contrast, the 2009 survey by Charles River Associates reported that almost half of the HTA systems (7/15; 47 percent) did not produce any HTA reports for MD technologies, unlike reports considering drugs that were published by all HTA systems (Reference Wilsdon and Serota8). The lower level of MD HTA activity identified in this later survey can be explained by the sample of HTA agencies which mainly consisted of drug specific HTA agencies (e.g., Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, Pharmaceutical Management Agency (PHARMAC) in New Zealand, The Dental and Pharmaceutical Benefits Agency (TLV) in Sweden). In addition, other components of HTA activity within those same countries were ignored (i.e., MSAC and Australian Safety and Efficacy Register of New Interventional Procedures-Surgical (ASERNIP) in Australia, New Zealand Health Technology Assessment (NZHTA) in New Zealand, Swedish Council on Technology Assessment in Health Care (SBU) in Sweden).
Strengths and Limitations
The main strengths of this study include the comprehensive and systematic approach to the selection of HTA agencies, high levels of response both to the request for verification of our agency Web-site surveys and interview invitations, and quality assurance measures for data collection (development of structured questionnaire to assess agency Web sites and a second researcher to verify the all Web-site based assessments and interviews).
However, we acknowledge several potential limitations. As the scope of this survey was limited by the funder of the project to consideration of non-EU HTA agencies, our findings may not be directly generalizable to the EU jurisdiction. A consequence of the comprehensiveness of our survey was the inclusion of a heterogeneous group of HTA agencies (e.g., size, age). Although we largely took a summative (“average”) approach to the presentation of results across agencies, where possible, we did report ranges. The assessment of agency Web sites for some countries (e.g., Norway) was limited to text and material available in English, excluding content in other languages. However, to minimize the risk of this retrieval bias, key informants reviewed the accuracy of the country-level analysis and Web-based assessment form. An alternative approach that may have led to more direct collection of data would have been to directly request agencies to self-complete a questionnaire. Although this later approach has been undertaken by several international surveys of HTA agencies, it has the major limitation of a low response rate. For example, the ISPOR and EUNetHTA surveys that both used questionnaires reported response rates of only 42 percent and 36 percent, respectively (Reference Stephens, Handke and Doshi5;6). Finally, to compare the approaches of HTA of MDs with drugs, a contrast between so-called “MD-specific” agencies versus “non–MD-specific” HTA agencies (i.e., between agency comparison) was conducted. It is possible that a within agency analysis (i.e., comparison of structure, processes, and methods of the MD and non-MD technology assessment within the same agency) may have been more sensitive to detecting differences. However, the data collection methods were not designed to undertake such a within agency comparison of HTA approaches as the available Web-site material does not often differentiate between the evaluation of differing health technologies.
Implications for Policy
Our findings raise some important implications. First, in contrast to the regulatory requirements that appear fairly consistent across international settings (Reference Tarricone, Torbica, Ferré and Drummond25), we found that HTA organizational structures, processes, and scientific methods for MDs can vary considerably across countries. Second, although it is well accepted that MDs differ from other health technologies, little evidence of differentiation in the HTA methods used by agencies for devices compared with drugs was found. This raises the question of whether these differences are such to require a fundamentally different methodological approach to the HTA of MDs compared with drugs or, instead, specific organizational or procedural adaption to current HTA activities, to consider MD-specific issues. Scientific methods to deal with usual challenges posed by MDs evaluation are being developed (Reference Craig, Dieppe and Macintyre26–Reference Turner, Spiegelhalter, Smith and Thompson28) and are part of the armamentarium of HTA analysts regardless of the type of technologies evaluated. Third, for other jurisdictions to develop innovative MD evaluation initiatives, such as Ontario's EXCITE programs, several factors need to be in place. For example, an infrastructure must exist to support trial development. The EXCITE program was introduced in a setting where academic centers with experience in field evaluation and coverage-with-evidence development activities already existed. Another important requirement is that a mutual understanding must exist between the various stakeholders, and in particular, the different incentives between device manufacturers, academics, and policy makers must be aligned. Finally, appropriate financing is necessary to support such activities. As many interviewees have noted, the issue is more about capacity building and training on advanced methodological tools for HTA MD assessors. The on-going EXCITE MD trial is funded by a device manufacturer although, importantly, the data collection, analysis and reporting of the studies is overseen by an independent Trial Steering Committee (23).
Conclusions
This survey provides a unique overview of organization and practices of non-EU HTA agencies. It is widely cited that MDs differ from other health technologies, in particular, they can change rapidly, clinical outcomes often depend on training and experience of operator, and pricing is more dynamic than drugs. The fact that all the non-EU HTA agencies included in this survey undertake the evaluation of MDs and do so alongside to other health technologies such as pharmaceuticals, supports the principles published by Drummond and colleagues for good HTA practice: “HTA should include all relevant technologies” (Reference Drummond, Schwartz and Jonsson14). However, given the lack of evidence for differentiation in scientific methods of HTA used by agencies for evaluation of devices compared with drugs, our study raises the question of whether the differences between medical technologies are such to require a fundamentally different methodological approach to their HTA, and whether organizational or procedural adaptation may suffice to accommodate specific challenges posed by devices' evaluation. In other words, existing primary and secondary scientific methods, including the analysis of observational studies and evaluation approaches for complex interventions (Reference Craig, Dieppe and Macintyre26–Reference Turner, Spiegelhalter, Smith and Thompson28), may provide a sufficient methodological basis for the HTA of MDs.
SUPPLEMENTARY MATERIAL
Supplementary material http://dx.doi.org/10.1017/S0266462315000185
CONFLICTS OF INTEREST
All authors declare no conflict of interest.
Open access
