HOSPITAL MANAGERS’ NEED FOR INFORMATION ON HEALTH TECHNOLOGY INVESTMENTS

Objectives: There is growing interest in implementing hospital-based health technology assessment (HB-HTA) as a tool to facilitate decision making based on a systematic and multidisciplinary assessment of evidence. However, the decision-making process, including the informational needs of hospital decision makers, is not well described. The objective was to review empirical studies analysing the information that hospital decision makers need when deciding about health technology (HT) investments. Methods: A systematic review of empirical studies published in English or Danish from 2000 to 2012 was carried out. The literature was assessed by two reviewers working independently. The identified informational needs were assessed with regard to their agreement with the nine domains of EUnetHTA's Core Model. Results: A total of 2,689 articles were identified and assessed. The review process resulted in 14 relevant studies containing 74 types of information that hospital decision makers found relevant. In addition to information covered by the Core Model, other types of information dealing with political and strategic aspects were identified. The most frequently mentioned types of information in the literature related to clinical, economic and political/strategic aspects. Legal, social, and ethical aspects were seldom considered most important. Conclusions: Hospital decision makers are able to describe their information needs when deciding on HT investments. The different types of information were not of equal importance to hospital decision makers, however, and full agreement between EUnetHTA's Core Model and the hospital decision-makers’ informational needs was not observed. They also need information on political and strategic aspects not covered by the Core Model.

Hospitals are often the main entry level for new health technologies (HT), and they invest a considerable volume of resources in implementation of new HT. There is growing interest in hospital-based health technology assessment (HB-HTA) as a tool to facilitate hospital decision making that is based on a systematic and multidisciplinary assessment of the evidence for new HTs (1). The background knowledge and available scientific evidence that underpins HB-HTA is the same as that used for national and regional HTA. However, the information required for making decisions about the introduction of a new HT, the time frame and the relative importance given to the different types of information may differ according to the organizational level of the healthcare sector at which decisions are made (2).
HB-HTA links evidence-based clinical data with the unique organizational and economic implications of a new HT at the local hospital level, thus providing shorter and more timely HTA reports to hospital decision makers (2). The decisionmaking process at hospitals, including the informational needs of hospital and clinical managers when deciding whether or not to invest in health technologies, is generally not well described, however (3).
One of the most widely used set of guidelines on how to perform and report HTAs is the Core Model developed by EU-netHTA (the European network for Health Technology Assessment), from a collaboration of primarily national HTA institu-tions. The Core Model describes a large number of potential elements for assessment (topics and items) that are divided into nine domains (4). HB-HTA products should be aligned with the needs of the final decision makers at hospital level, but the level of agreement between the Core Model and the informational needs of hospital decision makers is currently unknown.
The primary objective of this study was to review empirical studies that analyze the information hospital decision makers want to have at their disposal before making decisions on HT investments. Furthermore, we wished to rank the importance of the requested information based on the number and frequency of mentions in the identified literature.
This systematic review is part of the European research project AdHopHTA (Adopting Hospital Based Health Technology Assessment in EU; http://www.adhophta.eu/) which aims to strengthen the use and impact of HTA in hospital settings. The results of this literature review will be used to develop qualitative interviews and a questionnaire survey among hospital decision makers in Europe.

Identification of Information Types
A systematic review of empirical studies was carried out to identify the informational needs of hospital and clinical managers when deciding about HT investments. We searched for empirical studies published in English or Danish from 2000 to 2012 (November) in the PubMed, Embase, Cochrane Library, and Web of Science databases. The subject of the literature search had three topics (decision maker [who], informational need [what], hospital setting [where]) and we limited the search to specific study designs. For each of the three topics, we defined the various queries (with similar search terms using both Medical Subject Headings [MeSH] and free text) and combined them into a final query. The search strategy was reviewed and refined by a senior research librarian. Supplementary Table 1 provides the full search histories.
After exclusion of 517 duplicates between the databases, the 2,689 articles that were identified as being potentially relevant were reviewed by two authors (K.K. and A.M.O.), working independently of each other. In case of disagreement, a third opinion (M.B.O.) was sought and in-or exclusion was resolved by discussion among the authors. Assessment of the literature was carried out in two phases-first by examining relevance by reading only the title and abstract, and then by reading the full text of those articles still deemed relevant. A kappa coefficient was calculated to measure the correlation between assessments made by the two independent reviewers. This coefficient takes into account the fact that part of the observed correlation between two assessments is due to chance (5).
When reviewing the literature, we looked specifically for evidence of which information that hospital decision makers needed when deciding on HT investments. This information was retrieved from empirical studies of hospital managers' attitudes and of decision-making processes in hospitals. Inclusion criteria were: (i) articles reporting on informational needs in a decision-making situation, (ii) in a hospital context, and (iii) based on an empirical study (not commentaries, letters, opinions, etc.). Systematic reviews of empirical studies were also accepted. The included articles were reviewed and the main types of information needed by hospital decision makers were listed.

Categorization of Information Types
The different types of information requested by hospital decision makers were categorized according to the nine domains of EUnetHTA's Core Model. The types of information were discussed and categorized by the authors jointly after reviewing the Core Model. The level of agreement between the identified types of information and the topics included under each domain in the Core Model was taken into consideration when categorizing the information.
The different types of information were seldom clearly defined in the literature, making it difficult to assess whether informational needs with very similar wording had the same meaning. Therefore, even when the identified types of information appeared to be very similar, the types were not merged unless their wording was exactly the same. The lack of clear definitions also meant that some types of information could be interpreted in different ways and could be categorized under two different domains.

Ranking of Importance of Information Types
The relative importance of the identified types of information was determined according to (i) the number of different information types within each domain, and (ii) the frequency with which the information types within each domain were mentioned in the literature. This second approach based on the frequency of mentions in the literature has been used in previous studies (6;7). The results were compared with the results of studies in which hospital decision makers were explicitly asked to assess the relative importance of different types of information.

Identification of Information Types
The combined search strategies identified 2,689 articles, from which 2,664 were excluded after review of the title and abstract. The remaining twenty-five full-text articles were reviewed and fourteen empirical studies or reviews of empirical studies were considered relevant and included in the analysis (Figure 1). Relevant characteristics of the included literature are presented in Table 1.
Reasons for exclusion were that (i) the aim of the study was not to identify informational needs of decision makers (wrong aim), (ii) the study did not deal with decision making by hospital or clinical managers (wrong population), (iii) the study did not take place in a hospital setting (wrong context), (iv) the article was not based on an empirical study or was based on an empirical study of low quality (wrong/poor study design), (v) the article was not in English or Danish (wrong language), or (vi) the article was not retrievable (n = 2). The kappa coefficient was 0.54, indicating moderate agreement between reviewers in the initial assessment of articles.
Half of the fourteen included studies were conducted in Canada, Spain, and Israel, and the rest were conducted in other European countries, United States, and Australia. The study methodologies varied, but typically included semi-structured interviews, questionnaire surveys and literature reviews. From the fourteen articles, we identified seventy-four different types of information requested by hospital and clinical managers.

Categorization of Informational Types
Most of the seventy-four types of information could be categorized within the nine domains of EUnetHTA's Core Model (Table 2). Due to ambiguity in wording, fifteen (20.3 percent) of the seventy-four information types were categorized under a maximum of two domains. Ten (13.5 percent) of the seventy-four information types did not fit easily into any of the domains of the Core Model and were categorized under a new, tenth domain entitled "political and strategic aspects" ( Table 2). Six of the fifteen (40.0 percent) information types that were not clearly defined were categorized under both the new, tenth domain and one of the other domains. Thus, a total of sixteen (21.6 percent) information types were categorized under the new domain covering "political and strategic aspects" of HT investments ( Table 2). Political aspects referred to, for example, the alignment between the decision to invest in a given technology and the local political climate or values. Strategic aspects referred to, for example, the fit between a given technology and the hospital's research strategy, as well as prestige and competition between hospitals in relation to a specific technology or health problem.
The third domain (D3) dealing with "clinical effectiveness," included information about clinical outcomes (e.g., quality of life) and effect sizes (e.g., patient impact) as well as characteristics of the evidence (e.g., quality of evidence).
The fifth domain (D5), dealing with "costs and economic evaluation," included information from traditional health economic evaluations with a broad societal perspective (e.g., costeffectiveness analysis) as well as narrower budget impact analyses with a hospital perspective (e.g., business cases).
The domains dealing with ethical (D6), safety (D4), and social (D8) aspects had the fewest number of information types (Table 2). Information on the patient perspective appeared only once in the literature (Domain 8 in Table 2).

Ranking of Importance of Information Types
Regardless of the approach used to rank the domains in terms of importance (i.e. number of different information types vs. frequency of mentions in the literature), the same five domains were ranked as the most important, albeit in differing order (  Table 2). These domains were those dealing with information about political and strategic aspects (D10), clinical aspects (D3), economic aspects (D5), organizational aspects (D7), and the health problem and current use of the technology (D1). Mainly information about organizational aspects changes from being the second most important type of information when looking at the number of different information types within each domain (column 2 in Table 2) to being the fourth most important type of information when looking at the frequency of mentions in the literature (column 4 in Table 2).
Three of the fourteen reviewed articles described studies in which hospital decision makers were directly asked to rank different types of information in order of their relative importance-one study from Spain (2) and two from Israel Semi-structured interviews and content analysis.
What constitutes the relative importance of evidence versus practical needs when determining clinical policy: 1) Practical in real world clinical setting, 2) Fit with values and local circumstances and context, 3) Resources, 4) Respond to political climate, 5) Meet compelling patient needs (volume, high-risk), 6) Fidelity (doing things just like in the trial implemented). 3. Ratcliffe J et al. (10).

UK
To describe the views of NHS decision makers concerning the concept of cost-effectiveness, equity and access, and to examine trade-offs between the importance of the concepts.
Health care decision makers in the NHS. n = 380.
Many respondents were prepared to trade between attributes related to cost-effectiveness, equity and access. Clinical decision-makers are more concerned about access and less about equity than decision makers at higher levels. 75% agree that cost-effectiveness is essential when allocating resources.

Hivon M et al. (11).
Canada To define more precisely how HTA is used by interviewees as well as the most significant barriers they encounter.
1) Health care administrator associations and governments, 2) Health care provider associations, 3) Patient associations. n = 42 Semi-structured interviews, qualitative analysis The vast majority recognize the usefulness of HTA. HTA use takes many forms: 1) Instrumental: implementation of services and programs, 2) Conceptual: framework to stimulate debate and orient government policies and 3) Symbolic: enforcing decisions already made. Significant barriers hinder the use of HTA: 1) Organizational: pure in-house communications, lack of long term planning, decision makers vested interests, 2) Scientific: absence of skilled staff and 3) Material: lack of time, human + material + financial resources. 5. Sampietro-Colom L et al. (2).

Spain
To develop and test a decision-support tool for prioritizing new competing HTs after their assessment using the mini-HTA approach.
28 decision-makers at national/regional level and hospital level.
A two-layer value/risk tool was developed based on the mini-HTA. The first layer included 12 mini-HTA variables classified in two dimensions-value and risk. Weights given to these were obtained from a questionnaire survey among decision-makers ( Delphi study with two rounds. Semi-structured questions. Public hospitals and primary healthcare centers preferred classic technology-centered HTA products such as HTA reports, systematic reviews, economic assessments and drug and medical/surgical procedure assessments, rapid reviews (micro perspective), whereas private hospitals and ministry representatives demanded, in addition to the requirement for classic and more innovative HTA products, more innovative HTA products such as organizational model and information system assessments, HTA research projects (macro perspective).

Finland
To review studies that empirically analyze macro and meso level decision making process for Including and/or excluding drugs in reimbursement lists and drug formularies in industrialized countries.

Review of 6 qualitative empirical studies
Clinical evidence on benefit and the quality of evidence were the main criteria. The costs of the drug emerged as the second major criteria. Formal pharmacoeconomic analyses had a minor role. Other criteria used by decision-makers were alternative treatments available, decision in other hospitals/systems, size of population affected, severity of disease and past decisions. External factors mentioned as influencing decision-making were patient demand, pharmaceutical company activities and clinicians' excitement. The criteria used varied between studies, and also between decisions. 12. Galani C et al. Participants found economic evaluations useful to inform policy decisions. The impact on policy s reported moderate in the majority of studies. Clinical aspects such as efficacy and effectiveness, and safety data were still considered the most influential arguments. Different aspects characterized each decision level: central = regulatory and political arguments, local = economics, physician = patient, disease and administrative burden. Top criteria for adoption: increased cost-effectiveness, increased efficacy, decreased complication rates. Top criteria against adoption: increased complication rates or side effects, decreased efficacy. It is optimal to use specialized medical journals, opinions of international leaders in a specific field, evidence from RCTs, large prospective studies and meta-analyses. Importance/frequency of criteria for/against adoption (in parentheses: authors' assessment of relevant

11
· Cost-effectiveness (5) · Incremental cost (1) · Net cost (1) · Investment effort (1) · Impact on hospital's profitability and other economic components (2) · Technology is associated with large capital investment (2) · Costs and budgetary constraints (3) · Change in hospital income (1) · Provisions of "out-of-pocket services" (1) · Resource implications (1) · Influence on hospital's exploitation (1) 19 D6: Ethical aspects * 3 · Ethical issues (2) · Equity (1) · Provisions of "out-of-pocket services" (1) 4 D7: Organizational aspects * 15 · Staff requirements (1) · Process of care impact (1) · Partnerships (1) · Interdependency (1) · Implementation issues (1) · Organization of services (1) · Training (1) · Organizational resources (1) · The need for personnel training (2) · Technology could be applied on an outpatient basis (1) · Physical space impact (1) · Fidelity (to the trial), (1) · Resource implications (1) · Influence on hospital's exploitation (1) · Fit with values and local circumstances and context (1)   (1) · Alignment with external directives (1) · Technology approved by the ministry of health (2) · National health strategy and directives (1) 8 D10: Political and strategic aspects 16 · Strategic fit (1) · Academic commitments (1) · Respond to political climate (1) · Local health priorities (1) · Prestige and competition among hospitals (2) · Pressures exerted inside and outside the hospitals (1) · Commercial pressures exerted by the industry (2) · Decisions in other hospitals (cover and use the technology), (2) · Pressures exerted by directors of medical wards or senior physicians (2) · Clinicians excitement (1) · Demand and pressures exerted by patients (2) · Past decisions (1) · Fit with values and local circumstances and context (1) · National health strategy and directives (1) · Alignment with external directives (1) · Technology approved by the ministry of health (2)   22 Note. For example, ten of the information types that were identified in the literature (Table 1) were categorized within Domain 1 (number), and they were mentioned a total of thirteen times (frequency). The shadowed cell indicates a new domain that is not covered by EUnetHTA's Core Model. (3;14). These studies used Likert scales, rank-order, or both to assign importance. Most of the information types that emerged from these studies as being important came under the same five domains that we identified as being important based on frequency of mentions in the literature. However, information about political and strategic aspects (D10) was rarely mentioned as important in the three direct measurement studies. Furthermore, a description of the technology and its technical characteristics (D2) and information about safety (D4) and legal aspects (D9) were ranked as important aspects in the direct measurement studies, but were not identified as important using our approaches. Of note, the social (D8) and ethical (D6) aspects of a new technology were rarely considered as important information for decision making.

DISCUSSION
This systematic review identified fourteen relevant empirical studies that contained seventy-four different types of information requested by hospital decision makers when deciding on HT investments. The EUnetHTA guidelines for performing HTA, that is, the nine domains of the Core Model (4), included most of these different types of information (4). However, we identified types of information that related to a new, tenth domain covering "political and strategic aspects." The additional domain dealing with the strategic and political aspects of investments in new health technologies is in line with McGregor's (19) assertion that investment decisions are dependent on political and social pressures and the opinion and values of the hospital decision makers. According to Gray (20) healthcare decisions are based on a combination of three factors: evidence, values, and resources. At present, some decisions may be driven principally by values and resources-a process Gray (20) describes as opinion-based decision making. The new domain dealing with "political and strategic aspects" (D10) cover information that by definition is something else than evidence. These types of information are, however, requested by hospital decision makers and, therefore, it may be worth considering including information on these more value-based aspects of HT investments as part of a basis for decision making in hospitals.
We found that information about political and strategic aspects of new technology (D10), together with clinical (D3), economic (D5), and organizational (D7) aspects were mentioned most frequently in the literature and were also those domains with the highest number of different information types. The importance of information about clinical effectiveness and economic aspects was confirmed in the three direct measurement studies, but not information about political and strategic aspects (D10). The reason may be that the hospital decision makers were not directly asked to consider these (new) aspects of HT investments in these studies. A recent systematic review of decision criteria for resource allocation and healthcare decision making showed that the most frequently cited category of criteria was "Overall context" including (among others) political aspects and stakeholders interests and pressures (7). It does seem then, that consideration of the political and strategic aspects are important in hospital decision making.
Our results suggested that hospital decision makers rarely focus on information about ethical (D6), safety (D4), social (D8), and legal (D9) aspects of new HT. This may be due to less familiarity with this type of information or with the way in which the information (especially about legal and ethical aspects) is collected. It may be that safety information was assumed to be included in the domain on clinical outcomes and effectiveness (D3). Furthermore, it was somewhat surprising that information on patient satisfaction and patient preferences was rarely directly requested, given that the patient perspective is one of four main categories in the Danish template for mini-HTA (21).
Future studies on the relative importance of different types of information to hospital decision makers should distinguish between information on clinical outcomes and effect sizes on the one hand, and information on the characteristics and quality of the evidence on the other. Both of these types of information were categorized here within the domain of clinical effectiveness (D3), but it is possible that hospital decision makers weight these types of information differently. Quality of evidence is a key determinant for the strength of recommendations for or against the adoption of a given HT (22) and this type of information was considered the most important in one of the three direct measurement studies (2).
It may also be relevant to examine more closely the relative importance of the broad societal perspective and the narrower hospital perspective in economic analyses of HT. Most of the information types identified from the literature related to the local hospital perspective, but investigating the relative importance of different perspectives when directly asking hospital decision makers to prioritize between information in future studies will be interesting. McGregor (19) noted that cost-effectiveness of a given technology does not determine investment decisions, but budget impact does. Similarly, Gallego et al. (15) found that information on the local budget impact of a new technology was more often requested by hospital decision makers than conventional economic evaluations with a broad societal perspective.
Likert scales and rank ordering exercises were used in the three direct measurement studies ( Table 1). The Spanish study used only a Likert scale, and the results showed that the decision makers mostly agreed that all the types of information were important to some extent (2). The use of Likert scales alone does not force decision makers to prioritize between different types of information. A combination of Likert scales and ranking (14) is, therefore, recommended in future studies.
A recent international questionnaire survey invited healthcare decision makers to report which criteria they consider when making decisions on healthcare interventions (22). Respondents were asked to indicate whether each decision criterion was "currently considered" or "should be considered" and its relative weights. The most relevant criteria were found to be: (i) clinical efficacy/effectiveness, (ii) safety, (iii) quality of evidence, (iv) disease severity, and (v) impact on healthcare costs. Organizational and skill requirements were frequently considered, but had relatively low weights, suggesting that their impact on the final decision might be fairly small. These results are largely consistent with the results of the three direct measurement studies in the current review.
We found that EUnetHTA's Core Model did not include all the types of information considered in hospital decision making. What are the implications of this for HB-HTA? Should guidelines for HB-HTA be adjusted and thereby differ from those for full HTA, for example, EUnetHTA's Core Model?
One possibility is that HB-HTA should focus exclusively on the clinical, economic, safety, organizational, strategic, and political issues associated with the introduction and use of a specific HT. McGregor (19) suggests that there is often a disconnection between the rigorous and careful collection of evidence as part of the HTA process, and the failure of this process to influence policy decisions. This is partly due to the HTA often being delivered too late for inclusion in the final basis for decisions. Results from a Polish study suggested that healthcare managers favor speed over accuracy of information in evidencebased decision making (13). A more focused and targeted approach to assessing HT, leaving out, for example, ethical and social aspects not valued highly by hospital decision makers, might allow a faster and less resource-consuming assessment at hospital level.
Further research is needed before concluding anything definitive about the practical implications for HB-HTA. We need more knowledge about what hospital decision makers understand by strategic and political aspects and what importance they place on the quality of clinical evidence and the different perspectives used in economic analyses of HT. These issues will be among those investigated in the interview study and the questionnaire survey among hospital decision makers in Europe that are part of the next steps in the AdHopHTA project.

Methodological Considerations
Several methodological considerations need to be taken into account when interpreting our results. First, the literature search was restricted to articles in English and Danish, and relevant literature in other languages could have been missed. In fact, no relevant literature in Danish was identified, so this language choice had no impact on the final results. In addition, we could not retrieve two potentially relevant articles.
Second, although all the reviewed studies included hospital managers and/or clinical managers, it was not always possible to isolate their results from those of other decision makers included in the study. Some studies were conducted at hospital level (micro), others at regional (meso) and national (macro) levels of decision making. Thus, we cannot be sure that our results are based solely on the informational needs of hospital and clinical managers.
Third, the included articles had very different purposes and research questions. Some articles investigated the relevance of a single criterion or element (e.g., the patients' perspectives) or particular product (e.g., mini-HTA or economic evaluation), while others focused on decision making in relation to a specific type of technology (e.g., expensive pharmaceuticals).The included literature also involved different study methodologies (typically systematic reviews, semi-structured interviews, and questionnaire surveys) and very different sample sizes, which is consistent with the methods used in the AdHopHTA project. This might, however, have influenced the importance ranking of the ten domains based on the number of information types or the frequency of their mentions in the literature, and these results should, therefore, be interpreted with caution. Notice, that the three direct measurement studies were among the fourteen relevant articles included in this review (Study no. 5, 8, and 14 in Table 1).
Fourth, the literature used a variety of concepts to describe the information that decision makers need when deciding whether or not to invest in HT, including, for example, "decision support," "information," and "decision criteria." We have not distinguished between these concepts, and this study thus concerns the need for "information" among hospital decision makers, which is a wider concept than the specific and measurable "criteria".
Fifth, the different types of information were discussed and categorized jointly by the authors after a thorough review of the Core Model. It would have been preferable for the authors to categorize the information independently before having a joint discussion. However, the result of the categorization was subsequently discussed with and validated by a group of HTA experts in the AdHopHTA project, which enhances the quality of the analysis (please see http://www.adhophta.eu/ for further details about the AdHopHTA project).
Because the different types of information were seldom clearly defined in the literature, we found fifteen of them to be sufficiently ambiguous that they had to be categorized under two domains. For example, information on the level of maturity of the technology ("Technology is still experimental") could be placed either in domain 1 (D1: Health problem and current use of the technology") or domain 2 (D2: Description and technical characteristics of the technology). Similarly, information about the effect of a specific technology on the exploitation of the hospital ("Influence on hospital's exploitation") could be categorized under domain 5 (D5: Costs and economic evaluation) or domain 7 (D7: Organizational aspects). Even when we forced each of these fifteen ambiguous types of information into one domain only, the results remained largely unchanged. Thus, the most important types of information were still within the same five domains. The only difference was that information on organizational aspects (D5) went from being the third most important to the second most important information together with information on clinical aspects (D3) when using the number of information types as a measure of the relative importance.
Finally, the included literature was based on research conducted in ten different countries. Results cannot be transferred uncritically from one context to another because of national differences in healthcare systems, decision-making processes, and attitudes toward the use of HTA.

CONCLUSION
The results of this systematic review suggest that hospital decision makers are able to describe their informational needs when deciding on HT investments. The domains of EUnetHTA's Core Model appeared to cover most of the informational needs of hospital and clinical managers, but full agreement was not observed. In addition to the domains of the Core Model, hospital decision makers also seek information on strategic and political aspects not covered by the model. Furthermore, the domains are not of equal importance to hospital decision makers. Clinical, economic, and strategic/political aspects are mentioned most frequently in the literature. The importance of clinical and economic aspects is confirmed in studies of relative importance of different types of information among hospital decision makers. Finally, this literature review also shows that the relative importance that hospital decision makers assign to different types of information has seldom been examined.
The results of this systematic review provide further knowledge about the types of information that hospital decision makers consider relevant when they decide on HT investments. This information will be useful for directing future empirical studies on this subject, including the interview study and questionnaire survey that will be conducted in the next phase of the AdHopHTA project.

CONFLICTS OF INTEREST
Dr. Ølholm reports grants from EC Seventh Framework Programme, during the conduct of the study. Drs. Kidholm, Birk-Olsen, and Buck Christensen have nothing to disclose.