Since the mid-1990s, all of the health systems of the OECD (Organisation for Economic Co-operation Development) and the majority of “middle-income countries” have incorporated health technology assessment (HTA) programs into their decision-making process. Despite certain national specificities, these tools have generally been institutionalized by HTA agencies, providing decision makers with scientific-based assessment of a wide set of medical technologies, whether pharmaceutical products, medical devices, or clinical interventions.
While recognizing the plurality of their practices, HTA experts and specialists also share many common standards, protocols, and applications. Consequently, their methods are resolutely focused on a form of “policy research that systematically examines the short and long-term consequences, in terms of health and resource use, as well as in terms of the application of health technologies” (Reference Henshall, Oortwijn and Stevens1). In this context, the scientific dimension of HTA provides these methods with conceptual unity. Indeed, many authors consider this to be as one of its key features: in each healthcare system where all or part of the decisions are made by means of HTA, professional researchers are at the root of the methodological support offered to decision makers. Hence, the metaphor of a bridge is regularly used to describe the relationship established by the HTA between the political and academic spheres (Reference Battista, Peckham and Smith2;Reference Maywald and Kirch3).
From this perspective, the history of HTA's professional societies (Reference Banta4) as well as its developments in various national contexts (Reference Banta5–Reference Drummond and Banta7) has been extensively narrated and studied. However, the sites on which such expansion was based, that is academic institutions, scientific journals and communities, have not yet received the same considerations. Given this situation, the goal of this empirical paper is to provide a comprehensive overview of HTA through the lens of its diffusion as a concept within the scientific literature, and this using a long-term perspective. The study thus examines institutions and individuals who have used this notion over the past 40 years. By describing different trends in this way, some appear to be closely connected to HTA's policy history.
Scientometric methods have been used to capture this dynamic. Previous research has demonstrated the appropriateness of such approaches to measure the factors that influence the use of scientific-based knowledge in public policies (Reference Benoit and Gorry8). In the health sector, the links between particular configurations of the scientific world and the areas, or arenas, of public policy is, furthermore, the focus of increasing interest (Reference Draborg and Gyrd-Hansen9–Reference Shen, Yao and Li13). With this in mind, we have undertaken a diachronic and synchronic study of the HTA concept to frame the channels through which it has been adopted. In so doing, we set out to examine the trends and institutions surrounding it to generate a panoramic view of the elements of academic research in this field. Considering the primacy of HTA's scientific dimension, we applied mapping techniques and co-publication network analysis to study a corpus, mainly drawn from scientific databases. In this study, we will use the metaphor of a “career of scientific concept” as a heuristic device to describe the process of institutionalization of HTA concept measured by historical bibliometrics. Thus, the term refers to the global context of its circulation.
METHODS
Scientometric methods were used to track the diffusion of the HTA concept in the scientific literature and in the press. We chose a database providing statistical functions, Scopus®, owned by Elsevier® which offers a better coverage of the humanities, social sciences, and public health than WoS® (Reference Mongeon and Paul-Hus14). As press releases taken up in the media, we searched within the Factiva® database, which gathers approximately 36,000 media sources. A search query was conducted across the title or keywords or abstracts in all types of records with the string of words “health technology assessment”, as well as the strings of words “medical technology assessment” and “health policy”.
We searched the Scopus® database for relevant articles published from the first one until December 31, 2014 and retrieved 2,865 documents referring to the HTA concept. In parallel, we retrieved 4,741 press releases from the Factiva® database. The corpus extracted from Scopus® was divided up between articles (52.67 percent), reviews (32.80 percent), conference papers (5.20 percent), editorials (2.35 percent), short surveys (2.30 percent), and 4.67 percent from other type of documents, including notes, letters, books, and book chapters. We chose to conserve and analyze these data for the insights they provided on the concept (Reference Gorry and Ragouet15).
To control the validity of the corpus, we tested the frequency of appearance of the keywords in the documents through an analysis conducted within Scopus® (Supplementary Table 1). Unsurprisingly, “technological assessment”, “health technological assessment”, “biomedical technological assessment” are the most recurrent terms. Alongside these notions, the majority of publications, display keywords such as “cost-effectiveness analysis”, “cost-benefit analysis”, “evidence-based medicine”, as well as “health care policy”, “decision making”, or “health care cost”. The publications are, therefore, closely related to Health Policy and Health Economics disciplines, and the dataset is representative of the HTA field.
HTA Scientific Journal Ranking
To pursue this analysis further, the corpus was extracted from the Scopus® database and imported, parsed, and disambiguated for author names with the help of the Intellixir® software to build an in-house database. First, the data obtained were used to capture the temporal dynamic of the HTA concept in the literature. Second, we isolated the premier specialized journals in which the publications appeared and ranked them by number of publications. Third, we focused our interest on the academic establishments that generated these publications, based on the authors’ affiliations mentioned in the publications, and drew a cloropeth map of institutions publishing on HTA with the help of Intellixir®. This cartographic analysis of spatial distribution of HTA publications enabled a comparison of geographical trends concerning the concept. Finally, we have ranked authors according to the number of articles they have published.
To gain a more exhaustive representation of the trajectory of HTA, the results were analytically reinforced by the measure of cumulated citations for HTA publications by institutions or authors and the combination of two metrological tests: the journal impact factor indicator taken from SCImago® (Reference Falagas, Kouranos, Arencibia-Jorge and Karageorgopoulos16), and the H-index (Reference Hirsch17) retrieved from Scopus® with the help of the unique author identifier. The total volume of articles published by academic journals or individuals does not directly reflect their influence; that is why citations are an important way to measure this influence.
While there is often a variation between the number of articles published per year, and the number of years’ journals have been publishing, the small number of main journals and their simultaneous publication tend to limit such an effect for our case-study (Reference Wagstaff and Culyer18). Author co-occurrence analysis in scientometrics is a leading approach for analyzing collaborative pattern, scholarly communication and the structure of science (Reference Newman19). The networks of the main authors co-publishing in the field of HTA were drawn with Intellixir® to evaluate the influence of academic network on the diffusion of the HTA concept. We established a filter to facilitate visualization of our corpus, setting a minimum threshold of two publications per individual, and two publications in collaboration with another person, to reveal the main contributors.
RESULTS
Emergence and Diffusion of the HTA Concept
Figure 1 shows the emergence and trends of the “health technology assessment” concept in scientific publications as well as of press media releases which mention the “health technology assessment” (HTA) concept. The first identified publication mentioning HTA is an article titled Health Technology Assessment and Impact, published by M.F. Kelty in 1978 (Reference Kelty20). Miriam Kelty was a behavioral scientist working at the National Institute of Health (Bethesda, MD) and was serving at that time on the United States (U.S.) congress commission on the Protection of Human Subjects of Biomedical and Behavioral Research.
HTA publications per year. Source: Scopus® and Factiva®.
However, the HTA concept really started to be used in the scientific literature from 1990 with more than ten publications per year, suggesting a peripheral status of the HTA concept in the literature until this turning point. Indeed, between the end of the 1970s and the beginning of the 1980s, we discovered two competing definitions of the concept: “medical technology assessment” (MTA) and “health technology assessment” (HTA). While the term MTA preceded the use of HTA by a year in the literature with two articles (Reference Ayers21;Reference McCarthy22), and was referenced at an early stage in the MeSH thesaurus within “technology assessment, biomedical” (year introduced: 1980) in the PubMed/MEDLINE's database, we found that the academic community never significantly adopted it (a total of 255 publications with a per year rate of 6.71).
By the year 1984, HTA had overtaken this concept: progressively, the term MTA disappeared from the scientific literature and has now fallen into disuse (Figure 1). Since that time, the growth of publications referring to the HTA concept in their title, abstract, or keywords has increased steadily, exceeding 100 in 2005 and reaching 388 publications in 2014 (average per year rate of 77.37 publications). Indeed, during this period, the increase of publications referring to the HTA concept was similar to the total number of publications mentioning “health policy” (Figure 1). To place the HTA in its proper context of diffusion, we reproduced our query using the Factiva® database.
Our aim was to compare the scientific dynamics of the concept with the media dissemination channel. The diffusion of the HTA concept can be traced in press releases back to 1991. But the number of press releases per year was lower (n = 10) than the annual rate of scientific publications (n = 43) until 2005. Next, newspapers, journals, and magazine started to transiently report HTA issues at a faster rate than the academic community, reflecting a new step in the institutionalization of HTA over the period 2005–2010 (Figure 1).
Journals as Vector of HTA Concept Diffusion
The vast majority of publications (and their growth) in reference to the HTA concept are supported by a relatively small number of journals. Table 1 shows the top 20 journals that have featured publications referring to the HTA concept in a title, abstract, or keywords. For each journal, the year of the first article mentioning the HTA concept, the total number of HTA publications, the percentage of these HTA publications out of the total publication by journal, and finally the journal impact factor, were documented.
As shown in Table 1, IJTAHC centralizes 16 percent of the total number of publications. This position explains the sensitivity of the curve to any change in the volume of articles generated by its special issues (Reference Jonsson and Reiser23). Publications in this field are also represented in other academic health journals such as Health Technology Assessment (6.04 percent), the journal of the U.K. HTA Programme, Health Policy (2.37 percent), or Value in Health (2.20 percent). Despite the weight of IJTAHC, its position in the field cannot be considered monopolistic. In fact, citations indices reveal a balance between different media. Papers citing the HTA concept published in IJTAHC are not the most cited (Supplementary Table 2), suggesting that the recognition of the concept by the academic community is not mechanically associated with this journal, and that the diffusion of the HTA concept might have suffered from being published in journals with lower impact factors. Proportionally, the recurrence of the HTA concept in other journals is thus greater. While we would expect these HTA most cited papers to be systematic reviews, analyses, policy reviews, or methods of HTA, many of them are not specifically on HTA, but are trials, studies, and guidelines.
During its recent history, the concept has moved from an area specialized in technology assessment in health care to other disciplines within the medical literature. Most recent articles using the term “health technology assessment” have thus been published in other fields of medicine, including Journal of Obstetrics and Gynaecology Canada, Journal of Pain and Palliative Care Pharmacotherapy, and the British Medical Journal. Consequently, outside IJTAHC, one can see that the concept remains a “minority” subject, but is also represented in almost all sub area of the healthcare literature, as well as being conveyed in a multitude of specialized journals, along with some general medical journals with a higher impact factor such as The Lancet (Supplementary Table 2).
Geography of Institutions Diffusing the HTA Concept
Supplementary Table 3 shows the top 3 institutions in terms of publications referring to the HTA concept as well as ranking the best-performing institution by country among the top 100 institutions. For each institution, the number of HTA publications, the total citations of HTA publications by institution, and the ratio of citations by publications is reported (HTA advocacy impact). The top ten universities in terms of publications are British (University of York, University of Birmingham, University of Sheffield) or Canadian (McMaster University), with the exception of Maastricht University in the Netherlands and Syddansk Universitet in Denmark.
However, the ranking of these organizations does not reflect any hegemonic position within the field. Indeed, the ranking can be nuanced regarding the index of citations. While an overrepresentation of British institutions is still apparent through the number of HTA publications, other universities (McMaster University, Canada; Harvard University, United States; Nasjonalt kunnskapssenter for helsetjenesten, Norway; UMIT Tirol, Austria) appear to have more impact as measured through the total number of citations or the ratio of citations by HTA publications. It is important to note that a pharmaceutical company appears in the top 100 ranking of institutions diffusing the HTA concept in the scientific literature.
Unsurprisingly in view of the distribution of HTA publications by country (Supplementary Figure 1), these outbreaks of publications are mainly concentrated in the United Kingdom, Canada, and the United States, which account for half of publications. However, a world map of geographical distributions of institutions referring to the HTA concept in their publications highlights contrasts between the different sites of HTA academic research (Figure 2): strongly developed and decentralized in the United Kingdom, it appears to be more fragmented in the United States. Conversely, only a small number of institutions in Canada cover the majority of publications. Finally, HTA publishing institutions are strongly implanted in Europe, mostly in Nordic countries and Germany, and are also very present in the emerging economies (Brazil, China, India).
World map of HTA publications by institutions. Source: Scopus®.
Networks of Scholars Responsible for HTA Concept Diffusion
Additionally, further analysis was thought necessary to look at the scholars responsible for the diffusion of the HTA concept. The distribution of HTA publications was, therefore, measured (data not shown). Surprisingly, the dataset does not follow Lotka's law (Reference Egghe24): articles with the HTA concept have been published by a large number of authors, with the top ten authors standing out from the rest of the corpus. They had generated only a small proportion of published papers (9.28 percent). Scientific production (h-index, publication, and citation) and the propensity to spread the HTA concept (date of the first HTA publication, number of HTA publications, and citation count) for this small group of individuals are shown in a Supplementary Table 4.
The Intellixir® software was used to draw a co-publications graph exploring scientific collaborations among the authors. Figure 3 is the mapping result with a minimum of two co-publications shared by pairs of authors. The overall network is made up of seven clusters centered around the most influential authors in diffusing the HTA concept; one is composed of D. Hailey with fourteen co-authors (cluster 1), a second is organized around D. Banta with three collaborators (cluster 2), the third one links C. Wild and F. Kristensen with twenty-three co-authors (cluster 3), the following one (cluster 4) is centered around M. Drummond and accounts for ten collaborators, one (cluster 5) is U. Siebert's group of collaborators (n = 20), another (cluster 6) is organized around R. Goree with sixteen collaborators, and the last one (cluster 7) is D. Menon's network (3 co-authors).
Collaborative network of main HTA authors. Source: Scopus®.
Over time the co-authors’ networks have evolved but the pioneering authors have remained. The nationality of the different authors represented in the network mirrors the findings in Figures 1 and 2. Indeed, D. Hailey is Australian, R. Goeree is Canadian, and C. Wild is Austrian. Several, such as D. Banta and U. Siebert, have worked on both sides of the Atlantic, explaining their position as smugglers within the network. By comparing the network analysis with previous results, we can identify the most prolific authors (M. Drummond, U. Siebert, and R Goeree), those who collaborate less (C. Wild), the pioneering authors (M. Drummond and D. Hailey), and those who are the most influential according to the different indicators (M. Drummond or D. Banta).
If we look at some authors in the corpus (M. Drummond, D. Banta, and P.J. Wiffen), many research profiles can be identified. M. Drummond publishes mostly in Health Economics reviews. His papers on HTA are related to this discipline. This more interdisciplinary positioning confers him with a “smuggling role” in Figure 3, linking together several small worlds (Reference Newman19). Almost at the other end of the spectrum, D. Banta is more of a “generalist” with publications concentrated in IJTAHC. P.J. Wiffen for his part has produced mainly reviews papers, in a wide range of journals.
DISCUSSION
Based on a historical perspective (Reference Banta4), our findings demonstrate the expansion of the HTA concept, expansion that appears to be continuing. Some key milestones in its diffusion have acted as generational turning points.
We can divide the growth of publications into three rounds: empowerment in the scientific field (1978–1987), legitimation in policy areas (1988–2002), and institutionalization and international development (2003–2013). This latter movement is made up of partially independent conditions, ranging from institutional (the emergence of demand led by newly created agencies) to individual (the biographic trajectories of influential scholars within scientific and bureaucratic fields).
During the 1980s, HTA became an autonomous field in medical science, and this situation can partly be attributed to the development of the “technology assessment” movement, which was started in the United States by the Office of Technology Assessment (Reference Burns25). This step is marked by the foundation of the International Journal of Technology Assessment in Health Care in 1985 and a significant rise in publications (Reference Banta4;Reference Jonsson and Reiser23). However, the occurrences of the HTA concept in the literature did not significantly progress with the foundation of IJTAHC or other journals (Health Policy, Value in Health).
The 1990s saw the emergence of the HTA concept in the literature concomitantly with the foundation of the English and Welsh National Institute for Health and Clinical Excellence (NICE, 1999). From then on, the concept very quickly became an issue discussed in nonacademic journals, as suggested by the growth in press releases on HTA and NICE as the institution with the largest media coverage with themes related to decision making and to the institutionalization of HTA (data not shown).
Some efforts from the scientific field to address practical issues began to bear fruits. In Nordic countries and in Commonwealth nations, different types of organizations (public bodies, independent agencies, hospitals) have increased demand in HTA expertise (Reference Banta5;Reference Drummond and Banta7). Existing networks between HTA experts in different countries have fostered the development of similar approaches to similar problems. The scientific and the policy dynamics of the concept have been mutually reinforcing: in this way, a conceptual structure has been progressively established. One can, therefore, refer to a legitimation cycle.
Finally, the growth in publications during the 2000s reveals two trends. The first is a greater implementation of HTA agencies in Europe (inducing a higher growth of the number of publications). This reflects the stabilized legitimacy of HTA in Europe, both from the viewpoint of policy makers and within other sub-areas of the medical field (as revealed by the use of the concept in most of its disciplinary areas, as well as its presence in major medical journals). The second trend is that of the diffusion of HTA in the emerging countries, such as Brazil and China. Thus, the scientific development of the field now seems likely to be gradually strengthened, thanks to encouragements received from these major sources of publications.
In linking these HTA publications to their production sites, two elements have stood out. On the one hand, the international dimension of the field is clear. There is a global craze for HTA that is reflected in the scope of its diffusion. On the other hand, important research centers stand out prominently as leading spaces, but the ranking of these organizations does not reflect any hegemonic position within the field. In addition to these general properties, it is important to emphasize that some countries, although poorly represented in publications productivity in absolute value, have nevertheless developed strong HTA research programs.
At a more micro level, the most influential authors have followed different publishing pathways. Although their position in the field is often seen as highly influential, in the same way as with research organization, we cannot confirm that it is hegemonic.
These milestones offer an empirical illustration of the “natural history” of HTA, as modeled by Battista and Hodge as a linear process involving three steps (emergence-consolidation-expansion). The “career” of the concept can be seen as the building and reinforcing of a conceptual structure at the crossroads of several geographical and social worlds, within which its legitimation in one has had causal ratchet effects which appear to have favored its legitimation in another (Reference Wagstaff and Culyer18).
In this study on the diffusion of the HTA policy concept, scientometric methods have provided perspectives that could and should be taken further. The links between HTA policies and other contextual elements, like the pharmaceutical industry and the regulation of drugs, could be for example be subjected to such investigation, and this over and beyond the question of the academic expertise in HTA (Reference Benoit and Gorry26).
SUPPLEMENTARY MATERIAL
Supplementary Table 1: https://doi.org/10.1017/S0266462317000186
Supplementary Table 2: https://doi.org/10.1017/S0266462317000186
Supplementary Table 3: https://doi.org/10.1017/S0266462317000186
Supplementary Figure 1: https://doi.org/10.1017/S0266462317000186
Supplementary Table 4: https://doi.org/10.1017/S0266462317000186
CONFLICTS OF INTEREST
No conflicts of interest.
Open access
