This mini-theme contains six stories of health technology assessment (HTA) impact from member agencies of The International Network of Agencies for Health Technology Assessment (INAHTA), which were originally shared at the 2015 and 2016 INAHTA Congresses. The INAHTA impact story sharing is an innovative network activity where member agency representatives share experiences of HTA impact in a loosely structured story format. Through this process, members gain insights from other agencies on new ways of thinking about and approaching HTA impact assessment. A guide is provided to members to prepare their story, and the best story receives the David Hailey Award for Best Impact Story. This mini-theme contains stories of HTA impact from six member agencies in different parts of the world: the Health Assessment Division of the Ministry of Public Health (Uruguay), the Institute of Quality and Efficiency in Health Care (Germany), the Health Information and Quality Authority (Ireland), the Finnish Office for Health Technology Assessment (Finland), the Australian Safety and Efficacy Register of New Interventional Procedures-Surgical (Australia), and the Institut national d'excellence en santé et en services sociaux (Canada). Across the papers, common themes emerge about the importance of appropriate engagement of stakeholders and the broadening scope of HTA beyond reimbursement decision making.

Objectives: The impact of health technology assessment (HTA) in the judicialization of the right of health has not been deeply studied in Latin American countries. The purpose of this study is to review the process of judicialization of the access to high cost drugs in Uruguay and assess the impact HTAs have had on this process.

Methods: The methodology used for this study included a comprehensive literature search in electronic databases, local journals, internal documents developed in the Ministry of Health, as well as conducting interviews with key informants.

Results: Judicialization of the access of high cost drugs has been increasing since 2010. The strategy of the Ministry of Health of Uruguay to decrease this problem included the organization of roundtables with judges and other stakeholders on the basis of HTA, the training of defense lawyers in the use and interpretation of HTA, and the participation of a professional who develops HTA in the preparation of the defense arguments. A year after the implementation of this strategy, 25 percent of writs of protection were won by the Ministry of Health.

Conclusions: Even though the strategy implemented was effective in reducing the loss of litigations, it was not effective in reducing the growing number of writs of protection. It is essential to address this problem in a broad debate and to promote understanding between the parties.

Objectives: The Quebec Trauma Care Continuum (TCC) was initiated in 1991 with the objective of providing accessible, continuous, efficient, and high quality services for all injury cases in the province.

Methods: The TCC design relied on three key components: (i) the designation of a network of acute care and rehabilitation facilities with specific mandates and responsibilities; (ii) the elaboration of transfer protocols, standing agreements, and governing structures to ensure fluid and optimal patient flow; and (iii) the close monitoring of several indicators to facilitate the continuous evaluation and improvement of the network.

Results: Between 1992 and 2002, in-hospital mortality following major trauma decreased from 51.8 percent to 8.6 percent, followed by an additional 24 percent drop between 1999 and 2012. We also observed a 16 percent decrease in average LOS but no change in the incidence of complications or unplanned readmissions. These changes translate into 186 lives saved per year and cost savings, due to shorter LOS, of 6.3 million CD$ per year. The risk-adjusted incidence of in-hospital mortality following major injury between 2006 and 2012 (7 percent) was the lowest of all Canadian provinces.

Conclusions: Strategic transformation of a network's structure and processes, supported by continuous monitoring of validated quality indicators, can lead to significant and sustainable improvements in clinical outcomes. It is hoped that the Quebec trauma story will inspire other jurisdictions and other healthcare sectors.

Objectives: This study aims to describe how a negative reimbursement decision—based on the health technology assessment (HTA) report of a nondrug intervention—affects healthcare providers in Germany.

Methods: Knee arthroscopy was chosen as an example, because as of April 2016 this procedure is no longer reimbursed for osteoarthritis, but is still covered for other indications, including meniscal lesions. The exclusion followed an HTA report prepared by the Institute for Quality and Efficiency in Health Care (IQWiG). Here, we examine how the decision to revoke reimbursement for arthroscopy was perceived by the surgical community. Information was collected from official hospital statistics, the internet, and informal interviews with orthopedic surgeons.

Results: In 2015, a total of 37,920 arthroscopic procedures were performed for knee osteoarthritis in Germany. Several surgical societies were unhappy with the negative decision, which was issued as a directive in November 2015, and they challenged the decision-making process as well as the underlying scientific evidence. In March 2016, fifteen societies issued joint recommendations on how to differentiate osteoarthritis from other knee diseases and how to document other diseases in a way that inspections by representatives of health insurance funds would not detect any deficiencies. In informal interviews, orthopedic surgeons indicated that miscoding of the principal diagnosis (meniscal tear rather than knee osteoarthritis) is to be expected, especially in the hospital sector.

Conclusions: HTA can have a significant impact on the provision of health services, but various loopholes allow physicians to undermine policy decisions. Therefore, it is important to involve all stakeholders in HTA and to convince them of the benefits of evidence-based medicine.

Objectives: The aim of this study was to illustrate the contribution of stakeholder engagement to the impact of health technology assessment (HTA) using an Irish HTA of a national public access defibrillation (PAD) program.

Background: In response to draft legislation that proposed a PAD program, the Minister for Health requested that Health Information and Quality Authority undertake an HTA to inform the design and implementation of a national PAD program and the necessary underpinning legislation. The draft legislation outlined a program requiring widespread installation and maintenance of automatic external defibrillators in specified premises.

Methods: Stakeholder engagement to optimize the impact of the HTA included one-to-one interviews with politicians, engagement with an Expert Advisory Group, public and targeted consultation, and positive media management.

Results: The HTA quantified the clinical benefits of the proposed PAD program as modest, identified that substantial costs would fall on small/medium businesses at a time of economic recession, and that none of the programs modeled were cost-effective. The Senator who proposed the Bill actively publicized the HTA process and its findings and encouraged participation in the public consultation. Participation of key stakeholders was important for the quality and acceptability of the HTA findings and advice. Media management promoted public engagement and understanding. The Bill did not progress.

Conclusions: The HTA informed the decision not to progress with legislation for a national PAD program. Engagement was tailored to ensure that key stakeholders including politicians and the public were informed of the HTA process, the findings, and the advice, thereby maximizing acceptance. Appropriate stakeholder engagement optimizes the impact of HTA.

Objectives: The Managed Uptake of Medical Methods Program (MUMM) started 10 years ago as a joint venture of the Finnish Office for Health Technology Assessment (Finohta) and the twenty hospital districts in Finland. The aim is to offer information on the effectiveness, safety, organizational effects, and costs of new medical methods to support decisions concerning their uptake in secondary care. In this article, we discuss the successes and challenges of implementing the MUMM program.

Methods: A study of awareness and use of five MUMM recommendations for various medical specialties with short-form web-based surveys to hospitals.

Results: The recommendations were noticed and considered relevant. In overall assessment they received a mean rating of 8.4 (range: 4 to 10). Two thirds of the respondents thought MUMM recommendations were useful for practice, but only a third had actually used them in decision making.

Conclusions: HTA-based MUMM recommendations were well received by physicians but in practice they are less used than clinical practice guidelines. Short-form electronic surveys were a useful way of gathering information about awareness and implementation. The surveys also functioned as another method of informing key physicians about the recommendations.

Background: There is growing trend for some surgical procedures previously performed in hospitals to be done in alternative settings, including office-based facilities. There has been some safety concerns reported in the media, which document serious adverse events following procedures performed in an office-based setting. To understand the current regulatory oversight of surgery in this setting ASERNIP-S conducted a review of the legislative and accreditation process governing these facilities in Australia.

Methods: Using rapid review methodology, internet searches targeted government Web sites for relevant publicly-available documents. Use of consolidated versions of legislative instruments ensured currency of information. Standards were sourced directly from the issuing authorities or those that oversee the accreditation process.

Results: Within Australia, healthcare facilities for surgery and their licensing are defined by each state and territory, which results in significant jurisdictional variation. These variations relate to the need for anesthesia beyond conscious sedation and listing of procedures in legislative instruments. In 2013, Australia adopted National Safety and Quality Health Service standards (NSQHS standards) for the accreditation of hospitals and day surgery centers; however, there is no NSQHS standard for office-based facilities. The main legislative driver for compliance is access to reimbursement schemes for service delivery.

Conclusions: The legislative and accreditation framework creates a situation whereby healthcare facilities that provide services outside the various legal definitions of surgery and those not covered by a reimbursement scheme, can operate without licensing and accreditation oversight. This situation exposes patients to potential increased risk of harm when receiving treatment in such unregulated facilities.

Several studies have been carried out regarding different interventions in an effort to improve levels of physical activity in people with an intellectual disability (ID). Studies have been carried out regarding physical activity interventions (1), educational interventions (2), and multimodal interventions (3). New technologies are currently being used to carry out changes in daily habits and to promote health in different pathologies but not in people with an ID (4;5). Due to the poor maintenance of physical activity that we have found in people with ID, the objective of the present study is to determine the maintenance of levels of physical activity, quality of life, self-efficacy, and social support in this population through an intervention with an application in their smartphones.

Objectives: Multicomponent interventions (MCIs), consisting of at least two interventions, are common in rehabilitation and other healthcare fields. When the effectiveness of the MCI versus that of its single interventions is comparable or unknown, evidence of their expected incremental cost-effectiveness can be helpful in deciding which intervention to recommend. As such evidence often is unavailable this study proposes an approach to estimate what is more cost-effective; the MCI or the single intervention(s).

Methods: We reviewed the literature for potential methods. Of those identified, headroom analysis was selected as the most suitable basis for developing the approach, based on the criteria of being able to estimate the cost-effectiveness of the single interventions versus that of the MCI (a) within a limited time frame, (b) in the absence of full data, and (c) taking into account carry-over and interaction effects. We illustrated the approach with an MCI for cancer survivors.

Results: The approach starts with analyzing the costs of the MCI. Given a specific willingness-to-pay-value, it is analyzed how much effectiveness the MCI would need to generate to be considered cost-effective, and if this is likely to be attained. Finally, the cost-effectiveness of the single interventions relative to the potential of the MCI for being cost-effective can be compared.

Conclusions: A systematic approach using headroom analysis was developed for estimating whether an MCI is likely to be more cost effective than one (or more) of its single interventions.

Objectives: The methodological quality of an economic evaluation performed alongside a clinical trial can be underestimated if the paper does not report key methodological features. This study discusses methodological assessment issues on the example of a systematic review on cost-effectiveness of physiotherapy for knee osteoarthritis.

Methods: Six economic evaluation studies included in the systematic review and related clinical trials were assessed using the 10-question check-list by Drummond and the Physiotherapy Evidence Database (PEDro) scale.

Results: All economic evaluations were performed alongside a clinical trial but the studied interventions were too heterogeneous to be synthesized. Methodological quality of the economic evaluations reported in the papers was not free of drawbacks, and in some cases, it improved when information from the related clinical trial was taken into account.

Conclusions: Economic evaluation papers dedicate little space to methodological features of related clinical trials; therefore, the methodological quality can be underestimated if evaluated separately from the trials. Future economic evaluations should follow more strictly the recommendations about methodology and the authors should pay special attention to the quality of reporting.

Objectives: For medical devices, benefits other than direct clinical effects may have a large impact on the patients’ well-being, but a standardized method for measuring these benefits is unavailable. The objective was to explore potential patient benefits provided by medical devices, and to assess the relative preferences of these benefits in the general Swedish population.

Methods: To identify attributes of patient benefit, healthcare personnel within a wide range of disease areas were interviewed. The generalized attributes were then validated among healthcare personnel, patient organizations, and manufacturers; in two pilot studies in the general population; and in two rounds of cognitive interviews. The general population's preferences of the attributes were measured with a usability-tested questionnaire in a final responding sample of 3,802 individuals, representative of the Swedish population.

Results: Twenty attributes were identified, encompassing aspects of integrity, sense of security, social participation, and convenience. When measuring the relative preferences, the response rate was 37.0 percent, and the results showed that the attributes with the highest preferences concerned reliability, reduced need for assistance, and sense of control of the illness/disability.

Conclusions: A set of twenty attributes of patient benefit relevant to users of medical devices was identified and validated. A questionnaire for patient-reported assessment of the benefits provided by a medical device was developed, based on the attributes. The questionnaire, designated MedTech20, provides a generic measurement method for the evaluation of medical devices used in a wide range of diseases/disabilities.

Objectives: This study was designed to assess the sensitivity of three Ovid MEDLINE search filters developed to identify studies reporting health state utility values (HSUVs), to improve the performance of the best performing filter, and to validate resulting search filters.

Methods: Three quasi-gold standard sets (QGS1, QGS2, QGS3) of relevant studies were harvested from reviews of studies reporting HSUVs. The performance of three initial filters was assessed by measuring their relative recall of studies in QGS1. The best performing filter was then developed further using QGS2. This resulted in three final search filters (FSF1, FSF2, and FSF3), which were validated using QGS3.

Results: FSF1 (sensitivity maximizing) retrieved 132/139 records (sensitivity: 95 percent) in the QGS3 validation set. FSF1 had a number needed to read (NNR) of 842. FSF2 (balancing sensitivity and precision) retrieved 128/139 records (sensitivity: 92 percent) with a NNR of 502. FSF3 (precision maximizing) retrieved 123/139 records (sensitivity: 88 percent) with a NNR of 383.

Conclusions: We have developed and validated a search filter (FSF1) to identify studies reporting HSUVs with high sensitivity (95 percent) and two other search filters (FSF2 and FSF3) with reasonably high sensitivity (92 percent and 88 percent) but greater precision, resulting in a lower NNR. These seem to be the first validated filters available for HSUVs. The availability of filters with a range of sensitivity and precision options enables researchers to choose the filter which is most appropriate to the resources available for their specific research.

Objectives: The Priorities and Evaluation Committee (PEC) funding recommendations for new cancer drugs in British Columbia, Canada have been based on both clinical and economic evidence. The British Columbia Ministry of Health makes funding decisions. We assessed the association between cost-effectiveness of cancer drugs considered from 1998 to 2008 and the subsequent funding decisions.

Methods: All proposals submitted to the PEC between 1998 and 2008 were reviewed, and the association between cost-effectiveness and funding decisions was examined by (i) using logistic regression to test the hypothesis that interventions with higher incremental cost-effectiveness ratios (ICERs) have a lower probability of receiving a positive funding decision and (ii) using parametric and nonparametric tests to determine if a statistically significant difference exists between the mean cost-effectiveness of funded versus not funded proposals. A sub-analysis was conducted to determine if the findings varied across different outcome measures.

Results: Of the 149 proposals reviewed, 78 reported cost-effectiveness using various outcome measures. In the proposals that used life-years gained as the outcome (n = 22), a statistically significant difference of nearly $115,000 was observed between the mean ICERs for funded proposals ($42,006) and for unfunded proposals ($156,967). An odds ratio indicating higher ICERs have a lower probability of being funded was also found to be statistically significant (p < .05).

Conclusions: Economic evidence appears to play a role in British Columbia cancer funding decisions from 1998 to 2008; other decision-making criteria may also have an important role in recommendations and subsequent funding decisions.

Objectives: Technological development and the need for electronic health records management resulted in the need for a computer with dedicated, commercial software in daily dental practice. The alternative for commercial software may be open-source solutions. Therefore, this study reviewed the current literature on the availability and use of open-source software (OSS) in dentistry.

Methods: A comprehensive database search was performed on February 1, 2017. Only articles published in peer-reviewed journals with a focus on the use or description of OSS were retrieved. The level of evidence, according to Oxford EBM Centre Levels of Evidence Scale was classified for all studies. Experimental studies underwent additional quality reporting assessment.

Results: The screening and evaluation process resulted in twenty-one studies from 1,940 articles found, with 10 of them being experimental studies. None of the articles provided level 1 evidence, and only one study was considered high quality following quality assessment.

Twenty-six different OSS programs were described in the included studies of which ten were used for image visualization, five were used for healthcare records management, four were used for educations processes, one was used for remote consultation and simulation, and six were used for general purposes.

Conclusions: Our analysis revealed that the dental literature on OSS consists of scarce, incomplete, and methodologically low quality information.

Objectives: This study examines the cost-effectiveness of the OPTI-SCRIPT intervention on potentially inappropriate prescribing in primary care.

Methods: Economic evaluation, using incremental cost-effectiveness and cost utility analyses, conducted alongside a cluster randomized controlled trial of twenty-one general practices and 196 patients, to compare a multifaceted intervention with usual practice in primary care in Ireland. Potentially inappropriate prescriptions (PIPs) were determined by a pharmacist. Incremental costs, PIPs, and quality-adjusted life-years (QALYs) at 12-month follow-up were estimated using multilevel regression. Uncertainty was explored using cost-effectiveness acceptability curves.

Results: The intervention was associated with a nonsignificant mean cost increase of €407 (95 percent CIs, −357–1170), a significant mean reduction in PIPs of 0.379 (95 percent CI, 0.092–0.666), and a nonsignificant mean increase in QALYs of 0.013 (95 percent CIs, −0.016–0.042). The incremental cost per PIP avoided was €1,269 (95 percent CI, −1400–6302) and the incremental cost per QALY gained was €30,535 (95 percent CI, −334,846–289,498). The probability of the intervention being cost-effective was 0.602 at a threshold value of €45,000 per QALY gained and was at least 0.845 at threshold values of €2,500 per PIP avoided and higher.

Conclusions: While the OPTI-SCRIPT intervention was effective in reducing potentially inappropriate prescribing in primary care in Ireland, our findings highlight the uncertainty with respect to its cost-effectiveness. Further studies are required to explore the health and economic implications of interventions targeting potentially inappropriate prescribing.

Objectives: Tackling ethical dilemmas faced by reimbursement decision makers requires deeper understanding of values on which health technology assessment (HTA) agencies are founded and how trade-offs are made. This was explored in this study including the case of rare disease.

Methods: Representatives from eight HTA explored values on which institutions are founded using a narrative approach and reflective multicriteria (developed from EVIDEM, criteria derived from ethical imperatives of health care). Trade-offs between criteria and the impact of incorporating defined priorities (including for rare diseases) were explored through a quantitative values elicitation exercise.

Results: Participants reported a diversity of substantive and procedural values with a common emphasis on scientific excellence, stakeholder involvement, independence, and transparency. Examining the ethical imperatives behind EVIDEM criteria was found to be useful to further explore substantive values. Most criteria were deemed to reflect institutions’ values, while 70 percent of the criteria were reported by at least half of participants to be considered formally by their institutions. The quantitative values elicitation highlighted the difficulty to balance imperatives of “alleviating or preventing patient suffering,” “serving the whole population equitably,” “upholding healthcare system sustainability,” and “making decisions informed by evidence and context” but may help share the ethical reasoning behind decisions. Incorporating “Priorities” (including for rare diseases) helped reveal trade-offs from other criteria and their underlying ethical imperatives.

Conclusions: Reflective multicriteria are useful to explore substantive values of HTAs, reflect how these values and their ethical underpinnings can be operationalized into criteria, and explore the ethical reasoning at the heart of the healthcare debate.

Objectives: The aim of this study was to examine submissions made to the Pharmaceutical Benefits Advisory Committee (PBAC) and assess whether the predicted financial impact was associated with a recommendation. The second objective was to assess whether the financial and utilization estimates for listing the proposed medicine were reliable.

Methods: Data were extracted from public summary documents of major submissions considered by the PBAC from 2012 to 2014. Information collected included whether submissions were accepted, rejected, or deferred; estimated use; and financial impact. For those submissions that were recommended in 2012 and listed on the Pharmaceutical Benefits Scheme (PBS) by January 2014, a comparison was made between predicted and actual use and cost in 2014, based on PBS utilization.

Results: In 2012 to 2014, the PBAC considered 142 unique major submissions; of those, 65 were recommended for listing. A higher financial cost to the government was a statistically significant factor in predicting rejection (p = .004 for cost > AUD 30 million Australian dollars [20.7 million Euros] compared with cost-saving). Of the submissions that were recommended in 2012 and listed by 2014, the actual use was higher than predicted for 5/19 medications. The estimated cost was outside the predicted bracket of cost for 10/19 medications, with 8/19 medications having threefold underestimated expenditure, and 2/19 items having lower than predicted expenditure.

Conclusions: This study highlights that the predicted financial impact of a medication to the PBS budget is associated with a PBAC recommendation and also highlights that predicted use may not reflect actual prescribing practices.

Background: Hospitals with health technology assessment (HTA) programs have reported its positive effects on the management of resources and costs. This study aimed to identify the barriers faced by hospital-based HTA (HBHTA) in Iran by inductive content analysis of stakeholders’ and decision-makers’ points of view.

Methods: The key individuals and organizations that could provide rich, relevant, and diverse data in response to the research question were purposively selected for the interviews and focus group discussion.

Results: Twelve stakeholders from seven public hospitals participated in the interviews. Another eighteen stakeholders from twelve HBHTA-related organizations took part in the focus group discussion. Most of the hospitals’ senior management team did not feel the need for HBHTA and believed that in Iran a systematic process like HTA faces many challenges.

Conclusions: The stakeholders participating in this study highlighted the significance of certain points that needed to be addressed before establishing HBHTA in Iran.

Objectives: The aim of this study was to examine the evidence payers cited in their coverage policies for multi-gene panels and sequencing tests (panels), and to compare these findings with the evidence payers cited in their coverage policies for other types of medical interventions.

Methods: We used the University of California at San Francisco TRANSPERS Payer Coverage Registry to identify coverage policies for panels issued by five of the largest US private payers. We reviewed each policy and categorized the evidence cited within as: clinical studies, systematic reviews, technology assessments, cost-effectiveness analyses (CEAs), budget impact studies, and clinical guidelines. We compared the evidence cited in these coverage policies for panels with the evidence cited in policies for other intervention types (pharmaceuticals, medical devices, diagnostic tests and imaging, and surgical interventions) as reported in a previous study.

Results: Fifty-five coverage policies for panels were included. On average, payers cited clinical guidelines in 84 percent of their coverage policies (range, 73–100 percent), clinical studies in 69 percent (50–87 percent), technology assessments 47 percent (33–86 percent), systematic reviews or meta-analyses 31 percent (7–71 percent), and CEAs 5 percent (0–7 percent). No payers cited budget impact studies in their policies. Payers less often cited clinical studies, systematic reviews, technology assessments, and CEAs in their coverage policies for panels than in their policies for other intervention types. Payers cited clinical guidelines in a comparable proportion of policies for panels and other technology types.

Conclusions: Payers in our sample less often cited clinical studies and other evidence types in their coverage policies for panels than they did in their coverage policies for other types of medical interventions.