Objectives: As an initiative of the International Network of Agencies for Health Technology Assessment (INAHTA), a checklist for assessment reports was developed as a means of improving transparency and consistency in HTA.

Methods: Preparation of a summary of key elements in HTA reports, drawing on experience in preparation of such documents, information from guidelines for HTA, and individual assessments. Review by INAHTA agencies and modification of the summary to reflect the consensus.

Results: The resulting checklist includes 17 questions, with supporting detail. General areas covered include preliminary information, why and how the assessment has been prepared, the results of the assessment, implications of the results, and conclusions.

Conclusions: The checklist is intended to be considered by those preparing or using an HTA report. It reflects the views of INAHTA members and is seen as a mechanism to improve the standard of HTA reports, being complementary to the more detailed guidelines on how to conduct assessments.

Background and Purpose: Outcome in patients hospitalized for acute stroke varies considerably between populations. Within the framework of the GAIN International trial, a large multicenter trial of a neuroprotective agent (gavestinel, glycine antagonist), stroke outcome in relation to health care resource use has been compared in a large number of countries, allowing for differences in case mix.

Methods: This substudy includes 1,422 patients in 19 countries grouped into 10 regions. Data on prognostic variables on admission to hospital, resource use, and outcome were analyzed by regression models.

Results: All results were adjusted for differences in prognostic factors on admission (NIH Stroke Scale, age, comorbidity). There were threefold variations in the average number of days in hospital/institutional care (from 20 to 60 days). The proportion of patients who met with professional rehabilitation staff also varied greatly. Three-month case fatality ranged from 11% to 28%, and mean Barthel ADL score at three months varied between 64 and 73. There was no relationship between health care resource use and outcome in terms of survival and ADL function at three months. The proportion of patients living at home at three months did not show any relationship to ADL function across countries.

Conclusions: There are wide variations in health care resource use between countries, unexplained by differences in case mix. Across countries, there is no obvious relationship between resource use and clinical outcome after stroke. Differences in health care traditions (treatment pathways) and social context seem to be major determinants of resource use. In making comparisons between countries, great care should be exercised in using outcome variables as indicators of quality of stroke care.

Objectives: Setting funding priorities among research projects for complementary and alternative therapies is especially vulnerable to arbitrary, partisan criteria and opportunistic readings of controversial evidence. This study develops a procedural approach to characterizing, in a transparent and even-handed way, the available evidence on these treatments and demonstrates a simple analytical scheme for prioritizing competing, but typically incommensurable, research projects for public support.

Methods: A simple five-step scheme for categorizing therapies is developed and justified, based on a taxonomy of the study designs used to generate research evidence. Once identified, candidate therapies are assigned to ordered categories along these design criteria and effectively grouped into equivalence classes by type of evidence. Finally, a partial ordering on the therapies is formed within each class by means of secondary screening criteria.

Results: Twenty-five candidate therapies are assigned to equivalence classes. The intent, in effect, is to restrict comparisons to those therapies that fall within a particular class of similar study designs. Within-class orderings avoid the problem of having better-known or better-supported therapies crowd out lesser known ones when it comes to allocating dollars for more research. A set of criteria and procedure for prioritizing spending for further research is demonstrated.

Conclusions: Relying on an open, formal procedure for comparing unconventional therapies offers protection against prejudgment in setting funding priorities, especially when weak clinical evidence relates more to a low investment in research than lack of efficacy.Special thanks are due to Mary Ann Richardson, DrPH, formerly of the UT School of Public Health, currently with the National Center for Complementary and Alternative Medicine, National Institutes of Health, Bethesda, Maryland. Dr. Richardson introduced me to this problem and graciously provided information on several unconventional therapies. Partial funding was provided by UT's Center for Alternative Medicine through a contract with the Office of Alternative Medicine, U.S. National Institutes of Health.

Objectives: To analyze sources searched in Cochrane reviews, to determine the proportion of trials included in reviews that are indexed in major databases, and to compare the quality of these trials with those from other sources.

Methods: All new systematic reviews in the Cochrane Library, Issue1 2001, that were restricted to randomized controlled trials (RCTs) or quasi-RCTs were selected. The sources searched in the reviews were recorded, and the trials included were checked to see whether they were indexed in four major databases. Trials not indexed were checked to determine how they could be identified. The quality of trials found in major databases was compared with those found from other sources.

Results: The range in the number of databases searched per review ranged between one and twenty-seven. The proportion of the trials in the four databases were Cochrane Controlled Trials Register=78.5%, MEDLINE=68.8%, Embase=65.0%, and Science/Social Sciences Citation Index=60.7%. Searching another twenty-six databases after Cochrane Controlled Trials Register (CCTR), MEDLINE, and Embase only found 2.4% additional trials. There was no significant difference between trials found in the CCTR, MEDLINE, and Embase compared with other trials, with respect to adequate allocation concealment or sample size.

Conclusions: There was a large variation between reviews in the exhaustiveness of the literature searches. CCTR was the single best source of RCTs. Additional database searching retrieved only a small percentage of extra trials. Contacting authors and manufacturers to find unpublished trials appeared to be a more effective method of obtaining the additional better quality trials.

Objectives: To assess the cost-effectiveness of alternative initial strategies in the management of uninvestigated dyspepsia in adult patients presenting to a primary care physician in Canada.

Methods: Clinical decision-making was modeled based on outcomes data from the literature and expert opinion. Costs were obtained from official reimbursement lists, and a detailed microcosting analysis of a gastroscopy. Costs were analysed over a one-year period after initial presentation, including consideration of a single relapse of symptoms. The main outcome was a symptomatic cure. The seven management strategies based on different initial tests were endoscopy, double-contrast barium meal, empirical eradication therapy, empirical antisecretory treatment, urea breath test (UBT), laboratory serology, and sequential testing (laboratory serology followed, if Helicobacter pylori positive, by UBT).

Results: In patients under age 45, endoscopy, double-contrast barium meal, and sequential testing were not cost-effective approaches. UBT was the most effective and most costly strategy costing $8,238 per additional patient cure compared with laboratory serology. In patients over age 45, UBT was again the most effective and most costly strategy, but endoscopy was the most reliable in detecting gastric cancers. Clinical variables that impacted these findings were the probability of symptomatic relapse in patients with nonulcer dyspepsia (NUD) after successful versus failed H. pylori eradication, the probability of finding a duodenal ulcer (DU) in a young dyspeptic patient, the specificity of UBT, and the prevalence of H. pylori in patients with DU.

Conclusions: A “test-and-treat” approach was favored, with UBT being the most effective but most costly initial test in adult dyspeptics under age 45. The choice of the most cost-effective approach is dependent on the benefits of H. pylori eradication in patients with NUD. In patients over age 45, UBT is the most effective and most costly, but endoscopy results in the early detection of most gastric cancers.The authors are indebted to the other members of the UBTAN Group, including Drs. Phil Sherman, Alan Thomson, and Sander Veldhuyzen van Zanten. The authors also thank Paul Sinclair, Eileen Grace, and Lawrence Joseph for their much valued comments and opinions.AB and CAF are research scholars of the Fonds de la Recherche en Santé du Québec. This work was supported in part by an “at arms length” grant from AstraZeneca Canada.

Objectives: To explore one aspect of the external validity of the randomized controlled trial (RCT), specifically how being selected for inclusion in a trial and having participated has influenced the use and cost of asthma-related health services.

Methods: Services used by asthmatic users of inhaled corticosteroids (iCSTs) having previously participated in an RCT (TS, n = 46) were compared with individuals who had never participated (NS, n = 51).

Results: TS were more likely to use higher (≥400 μg) daily doses of iCSTs than NS (OR, 3.3; 95% CI, 1.1–8.3) but less likely to visit emergency departments (OR, 0.3; 95% CI, 0.1–0.7). Total asthma-related costs did not differ significantly.

Conclusions: Subject differences may impede generalizing from RCTs to real life.

Objectives: Guidelines for economic evaluation studies recommend that modeling be undertaken to estimate long-term, downstream costs. In this study, we conduct a review of a sample of studies that estimated the lifetime medical care costs for a variety of conditions.

Methods: We developed a categorization of the elements for a lifetime-costing study and based on these elements, we abstracted information from a sample of 33 papers in the following areas: study subject, purpose, scope, methods (including time profile, utilization, and cost), and results.

Results: We analyzed papers that were observational, models or that combined the two approaches. The time profiles were estimated from registry and published data. Utilization data were obtained from administrative data, chart reviews, and professional opinion. Costs were obtained from administrative and financial records and were estimated using all charges, allocated costs, and provider payments. We noted wide variations in methods and reporting practices.

Conclusions: Following current guidelines (CCOHTA), lifetime models can be more easily interpreted and applied if investigators are more clear in their study aims, if they incorporate assumptions that are based on current data, if they follow current methodological practices (such as deflation, discounting, and sensitivity analyses), and if reporting is more transparent.

Objectives: Health services research is a typical instance of applied research, meaning that its research findings should contribute to a more evidence-based health policy. A basic assumption for the usefulness of policy research is that it is of good scientific quality, but evaluation of health services research would be incomplete without an assessment of its societal relevance. However, there is no generally accepted framework in which criteria to assess societal relevance are defined.

Methods: In this study, we made an effort to specify criteria and indicators for policy relevance.

Results and Conclusions: In general, we concluded that it is important to define from the start of a research project, what kind of answers are being sought to the questions at hand, who the intended users of these answers will be, and how they can be reached. Furthermore, a productive dialogue between researchers and policy makers is indispensable for remaining in tune with policy thinking. In addition, we specified domains and a list of potential criteria for assessing policy relevance. For the purpose of quality assurance, these criteria have to be integrated into a quality improvement cycle. This means that the societal output of health services research should be related to the aims and missions at the level of projects, programs, and institutes. Furthermore, it is important that consensus is reached about (the weighting of) relevant indicators and that routines are developed for collecting information on these indicators.

Background: In 1994/95, two genes, BRCA1/2, associated with a predisposition to breast or ovarian cancer were identified. Genetic testing of deleterious BRCA1/2 mutations consequently can be proposed to individuals with a family history of breast or ovarian cancer to identify who is at risk. The granting of U.S. patents on BRCA1/2 to a privately owned company has led to the monopoly use of a unique technique (Direct Sequencing of the gene, DS) for BRCA1/2 testing in this country. Alternative strategies using prescreening techniques, however, have been experienced worldwide.

Methods: On the basis of data collected at three laboratories of French public hospitals, we carried out a cost-effectiveness study comparing DS to 19 alternative strategies with the number of deleterious BRCA1 mutations detected as the outcome.

Results: Results show that the DS strategy presents the highest average cost per mutation detected (9,882.5 °) and that there exist strategies using prescreening techniques that can reach similar effectiveness while reducing total costs. Moreover, other strategies can obtain a four- to sevenfold reduction in the average cost per mutation detected as soon as some rates of false negatives (2% to 13%) are deemed to be acceptable.

Conclusions: Results suggest that gene patents with a very broad scope, covering all potential medical applications, may prevent health care systems from identifying and adopting the most efficient genetic testing strategies due to the monopoly granted for the exploitation of the gene. Policy implications for regulatory authorities, in the current context of the extension of BRCA1/2 patents in other countries, are discussed.

Objectives: In an interrupted time series (ITS) design, data are collected at multiple instances over time before and after an intervention to detect whether the intervention has an effect significantly greater than the underlying secular trend. We critically reviewed the methodological quality of ITS designs using studies included in two systematic reviews (a review of mass media interventions and a review of guideline dissemination and implementation strategies).

Methods: Quality criteria were developed, and data were abstracted from each study. If the primary study analyzed the ITS design inappropriately, we reanalyzed the results by using time series regression.

Results: Twenty mass media studies and thirty-eight guideline studies were included. A total of 66% of ITS studies did not rule out the threat that another event could have occurred at the point of intervention. Thirty-three studies were reanalyzed, of which eight had significant preintervention trends. All of the studies were considered “effective” in the original report, but approximately half of the reanalyzed studies showed no statistically significant differences.

Conclusions: We demonstrated that ITS designs are often analyzed inappropriately, underpowered, and poorly reported in implementation research. We have illustrated a framework for appraising ITS designs, and more widespread adoption of this framework would strengthen reviews that use ITS designs.

Objectives: To examine quantitatively the effects of an increase in patient copayments from 10% to 20% on the demand for medical services in Japan.

Methods: The subjects of the study were the employees insured by the 1,797 health insurance societies, belonging to the National Federation of Health Insurance Societies, in 1996 and 1998. Indicators of medical service demands analyzed include the inpatient, outpatient, and dental case rates, the number of serviced days per case, the medical cost per day and the medical cost per insured.

Results: When the effects of an increase in patient copayments from 10% to 20% were evaluated, taking into account the average age, the average monthly salary, the total number, the gender (male-to-female) ratio and the dependent ratio of the insured, the estimated change in the case rate was −6.96% for inpatient, −4.79% for outpatient, and −5.77% for dental care. The estimated change in the number of serviced day per case was −4.66% for inpatient, −5.67% for outpatient, and −1.82% for dental care. The estimated change in the medical cost per day was −3.15% for inpatient, −13.00% for outpatient, and −11.48% for dental care. The estimated change in the medical cost per insured was −14.08% for inpatient, −21.54% for outpatient, and −18.11% for dental care.

Conclusions: The increase in patient copayments from 10% to 20% enabled insurers to substantially reduce medical costs by cost shifting from the insurer to the insured, with resultant changes in the case rate and the number of service days per case.We thank Ms. Danita Joell for administrative assistance. This study was founded by a grant-in-aid for scientific research (c) from the Japan Ministry of Education, Science, and Culture (grant number 11670373).

Objective: To determine if a pre-assessment can be used to establish whether cost-effectiveness results would meet the actual information needs of Dutch healthcare decision makers.

Methods: Two recent studies in rehabilitation medicine served as study material. Based on Wholey, a limited pre-assessment was performed in which the potential impact of cost-effectiveness analysis (CEA) results on intended users' decision making was assessed. Desk research and semi-structured interviews with several intended users of CEA results were performed. These included general practitioners, representatives of health insurance companies, the Health Care Insurance Board (CvZ), and medical guidelines committees.

Results: In day-to-day decision making of the interviewed decision makers, a cost-effectiveness criterion seemed to be of limited importance. Instead, results from clinical effectiveness studies and budget impact studies appeared to be sufficient. CvZ, however, preferred relative cost-effectiveness to be a criterion for inclusion in future reimbursement guidelines. In both cases the limited pre-assessments changed the expectations of the investigators regarding decision-making impact of an economic evaluation.

Conclusion: This study revealed that the use of CEA results for Dutch micro- and meso-level healthcare decision making is not self-evident. The main purpose of CEA results is to support health policy making and planning at a macroeconomic level. Pre-assessment can be a valuable tool in designing a CEA to support the actual information needs of the decision makers.

Objectives: This study reports the cost-effectiveness of interventions with nonsignificant differences in effect, and considers reporting of cost-effectiveness in situations where nonsignificant differences arise in some but not all end points.

Methods: Data on costs and effects associated with three end points (adequate assessment, risk factors, and life-years) were derived from a trial of methods to promote secondary prevention of coronary heart disease. Incremental cost per life-year gained figures were calculated, and the uncertainty around these was displayed on cost-effectiveness planes in the form of ellipses.

Results: There was a significant difference in one of the intermediate end points (adequate assessment) but nonsignificant differences in the other intermediate end point (risk factors) and the final end point (life-years). Estimation of cost per life-year figures revealed the cost-effectiveness of the interventions to be unfavorable.

Conclusions: Cost-effectiveness ratios based on final end points should be calculated even in situations where nonsignificant differences in life-years arise, to avoid publication bias and to provide decision makers with useful information. Uncertainty in the incremental cost-effectiveness ratios should be estimated and presented graphically.We thank Rebecca Turner for comments on an earlier draft; Professor Paul Fenn for permission to adapt an existing model; and Dr. Andy Briggs for advice on interpretation and presentation of results and comments on an earlier draft. In addition to the authors of the paper, the ASSIST trial collaborative group includes G. Fowler, E. Fullard, A. Fuller, M. Murphy, A. Neil, T. Schofield, R. Turner, S. Thompson, F. Wells, R. Wiles, and L. Youngman. This research was funded by a grant from the national research and development program grant of the NHS Executive.

Objectives: To examine the cost-effectiveness of Doppler ultrasonography in high-risk pregnancies.

Methods: An analysis was made of the cost-effectiveness of Doppler ultrasonography in high-risk pregnancies in relation to different organizational models. National costs of Doppler ultrasonography for singleton pregnancies with intrauterine growth retardation were estimated for three models. The cost-effectiveness analysis was based on results from a meta-analysis on clinical effects, patient costs, immediate health care costs, and costs per “saved” perinatal death.

Results: In the decentralized model (the current situation in Denmark), incremental health care costs were estimated to be 13.5 million DKK, with patient costs set at zero. In the regional and centralized models, the estimated costs were 9.3 million/0.9 million DKK and 3.4 million/2.6 million DKK, respectively. The incremental costs were more than outweighed by the savings made from significant reductions in obstetric interventions. The centralized model dominated the other two models in the cost-effective analysis. In the decentralized model, the costs of avoiding one perinatal death were estimated to be 1 million DKK. The sensitivity analysis suggested that the cost-effectiveness ratio differed considerably, depending on the assumptions used, although the rank order of the three models did not change.

Conclusions: The cost-effectiveness analysis showed that a centralized model with five obstetric centers offering Doppler ultrasonography dominates the regional and decentralized models. However, even with the decentralized model (which reflects the current situation in Denmark), the costs of avoiding one perinatal death would seem to be reasonable. In view of the paucity of available cost and effects data and the sensitivity of the results to changes in the assumptions made, more reliable information is needed before a decision can be made regarding the organization of Doppler ultrasonography for high-risk pregnancies.

We describe an evaluation model to guide public coverage of new predictive genetic tests in Ontario, Canada. The model confronts common “gray zones” in evaluation and coverage policy for challenging new technologies. Analysis addresses three domains of the evaluation picture. The first specifies evaluative criteria (purpose, effectiveness, additional effects, unit cost, demand, cost-effectiveness). The second induces or deduces acceptable cutoffs for each criterion. The third domain addresses the need to make decisions under uncertainty and to respond to “gray” evaluations with conditional-coverage decisions. The evaluation criteria should be applied within sound decision-making processes.

This study examines the parameters crucial to cost-effectiveness of universal hepatitis B immunization in India. An incremental cost-effectiveness analysis was done using a decision tree (Markov model) to follow up a hypothetical cohort of 100,000 newborns for the effects of hepatitis B acquired vertically at birth. The measure of effectiveness was disability-adjusted life-years gained. Uncertainty analysis and Scenario analysis were done using Latin hypercube sampling. Hepatitis B endemicity is the most important factor, followed by the cost of vaccine. Other factors of some influence are vaccination coverage, vaccine efficacy, HBeAg positivity, and vaccine wastage.

Objectives: To review the scientific evidence on the efficacy, effectiveness, and safety of intra-articular injections of Hylan G-F 20 for the treatment of knee osteoarthritis.

Methods: Systematic review of experimental and observational studies performed in humans up to December 1999. Qualitative and quantitative (meta-analytic) techniques were used for data synthesis.

Results: A single course of intra-articular Hylan G{-}F 20 provides a statistically significant and clinically relevant short-term decrease of the painful symptomatology of knee osteoarthritis and improves joint function. It also seems to delay the need for knee replacement, if results observed in noncontrolled studies are confirmed. Hylan G-F 20 has a comparable efficacy to that of oral NSAID, and a smaller risk of gastrointestinal adverse effects. It seems to be well tolerated and safe, but the short follow-up in most studies limits any extrapolation of the effectiveness and safety over the longer term. There is also scarce evidence on the effect of multiple courses of Hylan G-F 20, and the scientific rigor of both experimental and nonexperimental studies reviewed is somewhat limited.

Conclusions: Whereas there is good quality scientific evidence showing that Hylan G-F 20 is a safe and well-tolerated therapy providing a short-term decrease of the pain symptoms while improving joint function, the delay of the need for knee replacement as well as the durability of the effect over the longer term have only been demonstrated in noncontrolled clinical series. The available evidence is not sufficient to reach firm conclusions on the effect of multiple courses of intra-articular injections of Hylan G-F 20 on health outcomes.

Objectives: To access the cost-effectiveness of French recommendations for hepatitis C virus (HCV) screening and the extent to which earlier identification of carriers may or not improve the cost-effectiveness of therapeutic strategies.

Methods: Cost-effectiveness analysis were performed using decision-tree analysis and a Markov model. Four alternative strategies were compared: no screening and no treatment; initiation of HCV treatment after the diagnosis of cirrhosis; and two alternative strategies refer to the current French policies of HCV testing, i.e., two enzyme immunoblot assay (EIA) tests in series, or a polymerase chain reaction (PCR) analysis after the first positive EIA test. Costs were computed from the viewpoint of the health care system. The analysis has been applied to populations particularly at risk of infection, as well as the general population.

Results: The “wait and treat cirrhosis” strategy was more cost-effective in the general population and in transfusion recipients. The incremental cost-effectiveness ratio of this strategy compared with baseline strategy was 3,476 of euros and €15,300 in respective cohorts. Considering the HCV screening strategy, the additional cost would be of €4,933 and €240,250 per additional year of life saved, respectively. In the intravenous drug user (IDU) population, the “two EIA” screening strategy was the more cost-effective alternative, with an additional cost of €3,825 per additional year of life saved.

Conclusions: HCV screening would be discarded for transfusion recipients but should be encouraged for IDUs and also for the general population, in which the additional cost of screening is an order of magnitude more acceptable.

The pragmatism in health care has made health technology assessment (HTA) restrict its scope to a particular set of problems, c.q. methods. The “multidisciplinary and comprehensive nature” of HTA, as the concept is presented in certain definitions, is lacking. Health care is also dominated by a positivistic-rationalistic approach of evaluation. In contrast, social studies of evaluations learn that a major difference has to be made between scientific research on (potential) impact of a technology and valuing these effects. In this contribution, we will discuss how the positivistic scientific bias of current HTA practice can be made up with other research traditions. More specifically, we focus on the question of how social scientists and particularly how qualitative research can contribute to HTA, complementary to positivistic studies of evidence and efficacy.

Objectives: To introduce a health-related quality of life measure for home care and institutional long-term care settings based on the Minimum Data Set (MDS) and the Health Utilities Index Mark 2 (HUI2).

Methods: Health attributes of Health Related Quality of Life (HRQOL) were identified, and suitable constructs were determined. Items from the MDS were mapped to the HUI2. Scores for the Minimum Data Set Health Status Index (MDS-HSI) were calculated using the HUI2 scoring function. Measurement properties are examined and reported. HRQOL scores were compared across study populations and to an external reference population. Random samples were drawn from long-term care clients in private households (n=377), supportive housing apartments (n=80), two residential care facilities (n=166), and a chronic care hospital (n=274) in Ontario, Canada. All sampled residents were assessed for health-related items using the MDS.

Results: The MDS-HSI results provide preliminary evidence of good validity. Institutional populations had lower overall HRQOL scores than community populations. Comparisons to existing Canadian national data support construct validity.

Conclusions: The MDS-HSI provides a summary outcome measure and an indicator of health status in the six supporting attributes. Longitudinal research is required to assess the sensitivity of the measure to changes over time. Further research is also required to establish the consistency between the preference weights used in this application of the HUI2 and those that would be derived from a frail elderly population.This research was supported by the Seniors Independence Program (grant 4687-06-93-458) and the Providence Centre Foundation. Thanks to Copernicus Lodge, Providence Centre, and the Red Cross Homemaker Program for participating in this study. Also thanks to Roger Roberge and Dan Dekoker from Statistics Canada for their help in obtaining and analyzing the NPHS data. International copyright for the MDS-NH and MDS-HC is held by interRAI. International copyright for HUI2 and HUI3 materials are held by Health Utilities, Inc.