Hostname: page-component-6b989bf9dc-md2j5 Total loading time: 0 Render date: 2024-04-13T18:36:57.042Z Has data issue: false hasContentIssue false

Why do health technology assessment drug reimbursement recommendations differ between countries? A parallel convergent mixed methods study

Published online by Cambridge University Press:  06 September 2019

Elena Nicod*
Bocconi University, Centre for Research on Health and Social Care Management (CERGAS), Milano, Lombardia, Italy
Laia Maynou
Department of Health Policy, London School of Economics and Political Science (LSE), London, UK Centre for Research in Health and Economics (CRES), University of Pompeu Fabra, Barcelona, Spain
Erica Visintin
Department of Social Policy, London School of Economics and Political Science (LSE), London, UK
John Cairns
Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK
*Corresponding author. Email:


Using quantitative and qualitative research designs, respectively, two studies investigated why countries make different health technology assessment (HTA) drug reimbursement recommendations. Building on these, the objective of this study was to (a) develop a conceptual framework integrating the factors explaining these decisions, (b) explore their relationship and (c) assess if they are congruent, complementary or discrepant. A parallel convergent mixed methods design was used. Countries included in both previous studies were selected (England, Sweden, Scotland and France). A conceptual framework that integrated and organised the factors explaining the decisions from the two studies was developed. Relationships between factors were explored and illustrated through case studies. The framework distinguishes macro-level factors from micro-level ones. Only two of the factors common to both studies were congruent, while two others reached discrepant conclusions (stakeholder input and external review of the evidence processes). The remaining factors identified within one or both studies were complementary. Bringing together these findings contributed to generating a more complete picture of why countries make different HTA recommendations. Results were mostly complementary, explaining and enhancing each other. We conclude that differences often result from a combination of factors, with an important component relating to what occurs during the deliberative process.

Copyright © Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Banta, D (2003) The development of health technology assessment. Health Policy 63, 121132.CrossRefGoogle ScholarPubMed
Cerri, KH, Knapp, M and Fernandez, JL (2014) Decision making by NICE: examining the influences of evidence, process and context. Health Economics, Policy, and Law 9, 119141.CrossRefGoogle Scholar
Cerri, KH, Knapp, M and Fernandez, JL (2015) Untangling the complexity of funding recommendations: a comparative analysis of health technology assessment in four European countries. Pharmaceutical Medicine 29, 341359.CrossRefGoogle Scholar
Charokopou, M, Majer, IM, Raad, JD, et al. (2015) Which factors enhance positive drug reimbursement recommendations in Scotland? A retrospective analysis 2006–2013. Value in Health 18, 284291.CrossRefGoogle Scholar
Chevreil, K, Berg Brigham, K, Durand-Zaleski, I and Hernandez-Quevedo, C (2015) France: health system review. Health Systems in Transition 17, 1218.Google Scholar
Clement, FM, Harris, A, Li, JJ, Yong, K, Lee, KM, Manns, BJ, Charokopou, M, Majer, IM, Raad, Jd, Broekhuizen, S, Postma, M, Heeg, B, Dakin, H, Devlin, N, Feng, Y, Rice, N, O'Neill, P and Parkin, D (2009) Using effectiveness and cost-effectiveness to make drug coverage decisions, a comparison of Britain, Australia, and Canada. Journal of the American Medical Association 302, 14371443.CrossRefGoogle ScholarPubMed
Creswell, J and Plano Clark, V (2011 a) Chapter 6. Collecting data in mixed methods research. In Creswell, J and Plano Clark, V (eds), Designing and Conducting Mixed Methods Research. Thousand Oaks, California: SAGE Publications. pp. 171–202.Google Scholar
Creswell, J and Plano Clark, V (2011 b) Chapter 3. Choosing a mixed methods design. In Creswell, J and Plano Clark, V (eds), Designing and Conducting Mixed Methods Research. Thousand Oaks, California: Sage Publications. pp. 53–106.Google Scholar
Dakin, H, Devlin, NJ and Odeyemi, IAO (2006) “Yes, “No” or “Yes, but”? Multinomial modelling of NICE decision-making. Health Policy 77, 352367.CrossRefGoogle ScholarPubMed
Dakin, H, Devlin, N, Feng, Y, et al. (2015) The influence of cost-effectiveness and other factors on NICE decisions. Health Economics 24, 12561271.CrossRefGoogle ScholarPubMed
Devlin, N and Parkin, D (2004) Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Economics 13, 437452.CrossRefGoogle ScholarPubMed
Ferrario, A and Kanavos, P (2015) Dealing with uncertainty and high prices of new medicines: a comparative analysis of the use of managed entry agreements in Belgium, England, the Netherlands and Sweden. Social Science & Medicine (1982) 124, 3947.CrossRefGoogle ScholarPubMed
Fischer, KE (2012) A systematic review of coverage decision-making on health technologies-evidence from the real world. Health Policy 107, 218230.CrossRefGoogle ScholarPubMed
Fischer, KE, Rogowski, WH and Stollenwerk, B (2013) Transparency vs. closed-door policy: do process characteristics have an impact on the outcomes of coverage decisions? A statistical analysis. Health Policy 112, 187196.CrossRefGoogle ScholarPubMed
Garau, M and Mestre-Ferrandiz, J (2009) Access mechanisms for orphan drugs: a comparative study of selected European countries. OHE Briefing 52.Google Scholar
Gerring, J.What is a case study? The problem of definition. In Gerring, J (ed.), Case Study Research: Principles and Practices. Cambridge, United States: University Press, 2007. pp. 27–29.Google Scholar
HAS – Commission de Transparence, Haute Autorité de Santé (2014) Évaluation des médicaments en vue de leur remboursement. Available at (Accessed 1 December 2016).Google Scholar
Kanavos, P, Nicod, E, Pomedli, S and van den Aardweg, S (2010) The impact of health technology assessments: an international comparison. EuroObserver 12, 17.Google Scholar
Mason, AR and Drummond, M (2009) Public funding of new cancer drugs: is NICE getting nastier? European Journal of Cancer 45, 11881192.CrossRefGoogle ScholarPubMed
Maynou, L and Cairns, J (2015) Why do some countries approve a cancer drug and others don't? Journal of Cancer Policy 4, 2125.Google Scholar
Maynou, L and Cairns, J (2016) An empirical analysis of drug reimbursement decisions in 10 European countries. LSHTM-THETA Working Paper N°2.Google Scholar
Maynou, L and Cairns, J (2019) What is driving HTA decision-making? Evidence from cancer drug reimbursement decisions from 6 European countries. Health Policy 123, 130139.CrossRefGoogle ScholarPubMed
Mertens, D (2011) Publishing mixed methods research. Journal of Mixed Methods Research 5, 36.CrossRefGoogle Scholar
NICE – National Institute for Health and Care Excellence (2015) Single technology appraisal: User guide for company evidence submission template. Available at (Accessed 1 December 2016).Google Scholar
Nicod, E (2017) Why do HTA coverage recommendations for the same drugs differ across four European countries? Applying a mixed methods framework to a sample of orphan drug decisions. The European Journal of Health Economics 18, 715730.CrossRefGoogle Scholar
Nicod, E and Kanavos, P (2012) Commonalities and differences in HTA outcomes: a comparative analysis of five countries and implications for coverage decisions. Health Policy 108, 167177.CrossRefGoogle ScholarPubMed
Nicod, E and Kanavos, P (2016 a) Developing an evidence-based methodological framework to systematically compare HTA coverage decisions across countries: a mixed methods study. Health Policy 120, 3545.CrossRefGoogle ScholarPubMed
Nicod, E and Kanavos, P (2016 b) Scientific and social value judgments for orphan drugs in HTA. International Journal of Technology Assessment 32, 218232.CrossRefGoogle Scholar
Nicod, E, Berg Brigham, K and Durand-Zaleski, I (2017) Dealing with uncertainty and accounting for social value judgments in value assessments for orphan drugs: evidence from four European countries. Value in Health 20, 919926.CrossRefGoogle ScholarPubMed
Schwarzer, R and Siebert, U (2009) Methods, procedures, and contextual characteristics of health technology assessment and health policy decision making: comparison of health technology assessment agencies in Germany, United Kingdom, France, and Sweden. International Journal of Technology Assessment 25, 305314.CrossRefGoogle ScholarPubMed
SMC – Scottish Medicines Consortium (2016) New Product Assessment Form. Available at (Accessed 1 December 2016).Google Scholar
Svensson, M, Nilsson, FOL and Arnberg, K (2015) Reimbursement decisions for pharmaceuticals in Sweden: the impact of disease severity and cost effectiveness. Pharmacoeconomics 33, 12291236.CrossRefGoogle ScholarPubMed
TLV – The Dental and Pharmaceutical Benefits Agency (2016) Guide for companies when applying for subsidies and pricing for pharmaceutical products (Version 2.0). Available at (Accessed 1 December 2016).Google Scholar
Wijnands, T, Kreeftmeijer, J, Kooreman, PJ, Wagner, P and Van Engen, A (2016) Nice collateral effect: what is the influence of nice on other HTA bodies? Value in Health 19, A486. Scholar
Supplementary material: File

Nicod et al. supplementary material

Nicod et al. supplementary material 1

Download Nicod et al. supplementary material(File)
File 28.9 KB
Supplementary material: Image

Nicod et al. supplementary material

Nicod et al. supplementary material 2

Download Nicod et al. supplementary material(Image)
Image 75.6 KB