Skip to main content Accessibility help
Hostname: page-component-684899dbb8-xt4p2 Total loading time: 0.368 Render date: 2022-05-25T22:08:24.454Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true }


Published online by Cambridge University Press:  30 October 2017

David Tordrup
World Health Organization, Representation to the
Christos Chouaid
Respiratory Medicine Department, Centre Hospitalier Intercommunal Creteil
Pim Cuijpers
Department of Clinical, Neuro and Developmental Psychology, Vrije Universiteit Amsterdam
William Dab
French National Institute for Science, Technology and Management (Cnam), Chair of Hygiene and Safety
Johanna Maria van Dongen
Department of Health Sciences, Vrije Universiteit Amsterdam
Jaime Espin
Andalusian School of Public Health
Bengt Jönsson
Department of Economics, Stockholm School of Economics
Christian Léonard
Belgian Health Care Knowledge Centre
David McDaid
Personal Social Services Research Unit, London School of Economics and Political Science
Martin McKee
Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine
José Pereira Miguel
Instituto de Medicina Preventiva e Saúde Pública, Faculdade de Medicina de Lisboa
Anita Patel
Centre for Primary Care & Public Health, Queen Mary University of London
Jean-Yves Reginster
Department of Public Health Sciences, University of Liège
Walter Ricciardi
Institute of Hygiene, Preventive Medicine and Public Health, Catholic University of the Sacred Heart Rome
Maureen Rutten-van Molken
Institute for Medical Technology Assessment/Institute of Health Care Policy and Management, Erasmus University Rotterdam
Valentina Prevolnik Rupel
Institute for Economic Research
Tracey Sach
Norwich Medical School, University of East Anglia
Franco Sassi
Health Division, Organisation for Economic Co-operation and Development (OECD)
Norman Waugh
Warwick Medical School, University of Warwick
Roberto Bertollini
World Health Organization, Representation to the EU


Background: The importance of economic evaluation in decision making is growing with increasing budgetary pressures on health systems. Diverse economic evidence is available for a range of interventions across national contexts within Europe, but little attention has been given to identifying evidence gaps that, if filled, could contribute to more efficient allocation of resources. One objective of the Research Agenda for Health Economic Evaluation project is to determine the most important methodological evidence gaps for the ten highest burden conditions in the European Union (EU), and to suggest ways of filling these gaps.

Methods: The highest burden conditions in the EU by Disability Adjusted Life Years were determined using the Global Burden of Disease study. Clinical interventions were identified for each condition based on published guidelines, and economic evaluations indexed in MEDLINE were mapped to each intervention. A panel of public health and health economics experts discussed the evidence during a workshop and identified evidence gaps.

Results: The literature analysis contributed to identifying cross-cutting methodological and technical issues, which were considered by the expert panel to derive methodological research priorities.

Conclusions: The panel suggests a research agenda for health economics which incorporates the use of real-world evidence in the assessment of new and existing interventions; increased understanding of cost-effectiveness according to patient characteristics beyond the “-omics” approach to inform both investment and disinvestment decisions; methods for assessment of complex interventions; improved cross-talk between economic evaluations from health and other sectors; early health technology assessment; and standardized, transferable approaches to economic modeling.

Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


1. Thomson, S, Foubister, T, Mossialos, E. Financing health care in the European Union: Challenges and policy responses. Geneva: World Health Organization; 2009.Google Scholar
2. Busse, R, Blumel, M, Scheller-Kreinsen, D, Zentner, A. Tackling chronic disease in Europe - Strategies, interventions and challenges. Obs Stud Ser No 20. 2010. (accessed December 1, 2013).Google Scholar
3. European Commission and the Economic Policy Committee (AWG). Joint report on health systems. Occasional Papers 74. 2010. (accessed December 3, 2013).Google Scholar
4. OECD. OECD StatExtracts. Published 2015 (accessed December 3, 2013).Google Scholar
5. Banta, D. The development of health technology assessment. Health Policy (New York). 2003;63:121132. doi:10.1016/S0168-8510(02)00059-3.CrossRefGoogle ScholarPubMed
6. Elshaug, AG, Hiller, JE, Moss, JR. Exploring policy-makers’ perspectives on disinvestment from ineffective healthcare practices. Int J Technol Assess Health Care. 2008;24:19.CrossRefGoogle ScholarPubMed
7. Elshaug, AG, Hiller, JE, Tunis, SR, Moss, JR. Challenges in Australian policy processes for disinvestment from existing, ineffective health care practices. Aust New Zealand Health Policy. 2007;4:23. doi:10.1186/1743-8462-4-23.CrossRefGoogle ScholarPubMed
8. Aguiar-Ibáñez, R, Nixon, J, Glanville, J, et al. Economic evaluation databases as an aid to healthcare decision makers and researchers. Expert Rev Pharmacoecon Outcomes Res. 2005;5:721732. doi:10.1586/14737167.5.6.721.CrossRefGoogle ScholarPubMed
9. Buxton, MJ. Economic evaluation and decision making in the UK. Pharmacoeconomics. 2006;24:11331142. doi:10.2165/00019053-200624110-00009.CrossRefGoogle ScholarPubMed
10. Murray, CJL, Vos, T, Lozano, R, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:21972223. doi:10.1016/S0140-6736(12)61689-4.CrossRefGoogle ScholarPubMed
11. Koster J. PubReMiner. Published 2014 (accessed December 15, 2013).Google Scholar
12. Karyotaki, E, Tordrup, D, Buntrock, C, et al. Economic evidence for the clinical management of major depressive disorder: A systematic review and quality appraisal of economic evaluations alongside randomised controlled trials. Epidemiol Psychiatr Sci. 2016. [Epub ahead of print]. doi:10.1017/S2045796016000421.Google ScholarPubMed
13. Ali, S, Stone, MA, Peters, JL, Davies, MJ, Khunti, K. The prevalence of co-morbid depression in adults with Type 2 diabetes: A systematic review and meta-analysis. Diabet Med. 2006;23:11651173. doi:10.1111/j.1464-5491.2006.01943.x.CrossRefGoogle ScholarPubMed
14. Janssen-Heijnen, ML, Schipper, RM, Razenberg, PP, Crommelin, MA, Coebergh, J-WW. Prevalence of co-morbidity in lung cancer patients and its relationship with treatment: A population-based study. Lung Cancer. 1998;21:105113. doi:10.1016/S0169-5002(98)00039-7.CrossRefGoogle ScholarPubMed
15. Currie, SR, Wang, J. Chronic back pain and major depression in the general Canadian population. Pain. 2004;107:5460. doi:10.1016/j.pain.2003.09.015.CrossRefGoogle ScholarPubMed
16. Forster, A, Young, J. Incidence and consequences offalls due to stroke: A systematic inquiry. BMJ. 1995;311:8386. doi:10.1136/bmj.311.6997.83.CrossRefGoogle Scholar
17. Wolf, PA, D'Agostino, RB, Kannel, WB, Bonita, R, Belanger, AJ. Cigarette smoking as a risk factor for stroke: The Framingham Study. JAMA. 1988;259:10251029.CrossRefGoogle ScholarPubMed
18. Hecht, SS. Tobacco smoke carcinogens and lung cancer. J Natl Cancer Inst. 1999;91:11941210. doi:10.1093/jnci/91.14.1194.CrossRefGoogle ScholarPubMed
19. Mannino, DM, Buist, AS. Global burden of COPD: Risk factors, prevalence, and future trends. Lancet. 2007;370:765773.CrossRefGoogle ScholarPubMed
20. Yusuf, S, Reddy, S, Ounpuu, S, Anand, S. Global burden of cardiovascular diseases: Part I: General Considerations, the epidemiologic transition, risk factors, and impact of urbanization. Circulation. 2001;104:27462753. doi:10.1161/hc4601.099487.CrossRefGoogle ScholarPubMed
21. Feldman, DE, Rossignol, M, Shrier, I, Abenhaim, L. Smoking: A risk factor for development of low back pain in adolescents. Spine (Phila Pa 1976). 1999;24:2492.CrossRefGoogle ScholarPubMed
22. MacMahon, S, Peto, R, Collins, R, et al. Blood pressure, stroke, and coronary heart disease: Part 1, prolonged differences in blood pressure: Prospective observational studies corrected for the regression dilution bias. Lancet. 1990;335:765774.CrossRefGoogle ScholarPubMed
23. Kang, JG, Park, C-Y. Anti-obesity drugs: A review about their effects and safety. Diabetes Metab J. 2012;36:1325. doi:10.4093/dmj.2012.36.1.13.CrossRefGoogle ScholarPubMed
24. Manson, JE. A prospective study of exercise and incidence of diabetes among US male physicians. JAMA. 1992;268:63. doi:10.1001/jama.1992.03490010065031.CrossRefGoogle ScholarPubMed
25. Barnett, K, Mercer, SW, Norbury, M, Watt, G, Wyke, S, Guthrie, B. Epidemiology of multimorbidity and implications for health care, research, and medical education: A cross-sectional study. Lancet. 2012;380: 3743.CrossRefGoogle ScholarPubMed
26. Tordrup, D, Attwill, A, Crosby, L, Bertollini, R. Research agenda for health economic evaluation (forthcoming). August 1, 2015).Google Scholar
27. Britton, A, McKee, M, Black, N, McPherson, K, Sanderson, C, Bain, C. Threats to applicability of randomised trials: Exclusions and selective participation. J Health Serv Res Policy. 1999;4: 112121.CrossRefGoogle ScholarPubMed
28. Hägg, L, Johansson, C, Jansson, J-H, Johansson, L. External validity of the ARISTOTLE trial in real-life atrial fibrillation patients. Cardiovasc Ther. 2014;32:214218. doi:10.1111/1755-5922.12087.CrossRefGoogle ScholarPubMed
29. Simon, ST, Higginson, IJ, Harding, R, et al. Enhancing patient-reported outcome measurement in research and practice of palliative and end-of-life care. Support Care Cancer. 2012;20:15731578.CrossRefGoogle Scholar
30. Cohen, D. Attacks on publicly funded trials: What happens when industry does not want to know the answer. BMJ. 2015;350:h1701.CrossRefGoogle ScholarPubMed
31. May, P, Normand, C, Morrison, RS. Economic impact of hospital inpatient palliative care consultation: Review of current evidence and directions for future research. J Palliat Med. 2014;17:10541063.CrossRefGoogle ScholarPubMed
32. Longworth, L, Sculpher, MJ, Bojke, L, Tosh, JC. Bridging the gap between methods research and the needs of policy makers: A review of the research priorities of the National Institute for Health and Clinical Excellence. Int J Technol Assess Health Care. 2011;27:180187. doi:10.1017/S0266462311000043.CrossRefGoogle ScholarPubMed
33. Drummond, M, Marshall, D. IHE Methodology forum: Prioritizing methodological research in the evaluation of health technologies in Canada. Alberta, Canada: IHE; 2010.Google Scholar
34. Chalkidou, K, Whicher, D, Kary, W, Tunis, S. Comparative effectiveness research priorities: Identifying critical gaps in evidence for clinical and health policy decision making. Int J Technol Assess Health Care. 2009;25:241248. doi:10.1017/S0266462309990225.CrossRefGoogle ScholarPubMed
35. European Commission. Public consultation on strengthening EU cooperation on Health Technology Assessment (HTA). Published 2016 (accessed December 22, 2016).Google Scholar
36. George, B, Harris, AH, Mitchell, AJ. Cost effectiveness analysis and the consistency of decision making: Evidence from pharmaceutical reimbursement in Australia 1991–96. Australia: Centre for Health Program Evaluation Melbourne; 1999.Google Scholar
37. Devlin, N, Parkin, D. Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004;13:437452. doi:10.1002/hec.864.CrossRefGoogle ScholarPubMed
38. Claxton, K, Martin, S, Soares, M, et al. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold. Southampton (UK): NIHR Journals Library; 2015 Feb. (Health Technology Assessment, No. 19.14.) Appendix 1, Systematic review of the literature on the cost-effectiveness threshold. (accessed May 5, 2015)Google Scholar
39. Cleemput, I, Neyt, M, Thiry, N, De Laet, C, Leys, M. Threshold values for cost-effectiveness in health care. KCE Rep 100C. 2009. (accessed February 23, 2014).Google Scholar
40. Claxton, K, Sculpher, M, Palmer, S, Culyer, AJ. Causes for concern: Is NICE failing to uphold its responsibilities to all NHS patients? Health Econ. 2015;24:17. doi:10.1002/hec.3130.CrossRefGoogle ScholarPubMed
41. Claxton, K, Martin, S, Soares, M, et al. Methods for the estimation of the National Institute for Health and Care Excellence cost-effectiveness threshold. Health Technol Assess. 2015;19:1504. doi:10.3310/hta19140.CrossRefGoogle ScholarPubMed
42. Bobinac, A, van Exel, NJA, Rutten, FFH, Brouwer, WBF. Willingness to pay for a quality-adjusted life-year: The individual perspective. Value Health. 2010;13:10461055. doi:10.1111/j.1524-4733.2010.00781.x.CrossRefGoogle ScholarPubMed
43. Simoens, S. Pricing and reimbursement of orphan drugs: The need for more transparency. Orphanet J Rare Dis. 2011;6:11721176.CrossRefGoogle ScholarPubMed
44. Baltussen, R, Niessen, L. Priority setting of health interventions: The need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.CrossRefGoogle ScholarPubMed
45. Tony, M, Wagner, M, Khoury, H, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): Field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. 2011;11:329. doi:10.1186/1472-6963-11-329.CrossRefGoogle Scholar
46. Pearson, S, Littlejohns, P. Reallocating resources: How should the National Institute for Health and Clinical Excellence guide disinvestment efforts in the National Health Service? J Health Serv Res Policy. 2007;12:160165. doi:10.1258/135581907781542987.CrossRefGoogle ScholarPubMed
47. Hughes, DA, Ferner, RE. New drugs for old: Disinvestment and NICE. BMJ. 2010;340:c572. doi:10.1136/bmj.c572.CrossRefGoogle ScholarPubMed
48. Eddama, O, Coast, J. Use of economic evaluation in local health care decision-making in England: A qualitative investigation. Health Policy. 2009;89:261270. doi:10.1016/j.healthpol.2008.06.004.CrossRefGoogle ScholarPubMed
49. Wang, R, Lagakos, SW, Ware, JH, Hunter, DJ, Drazen, JM. Statistics in medicine — Reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357:21892194.CrossRefGoogle ScholarPubMed
50. Godwin, M, Ruhland, L, Casson, I, et al. Pragmatic controlled clinical trials in primary care: The struggle between external and internal validity. BMC Med Res Methodol. 2003;3:28. doi:10.1186/1471-2288-3-28.CrossRefGoogle ScholarPubMed
51. Kaiser, C, Brunner-La Rocca, HP, Buser, PT, et al. Incremental cost-effectiveness of drug-eluting stents compared with a third-generation bare-metal stent in a real-world setting: Randomised Basel Stent Kosten Effektivitäts Trial (BASKET). Lancet. 2005;366:921929. doi:10.1016/S0140-6736(05)67221-2.CrossRefGoogle Scholar
52. Pietri, G, Masoura, P. Market access and reimbursement: The increasing role of real-world evidence. Value Health. 2014;17:A450A451. doi:10.1016/j.jval.2014.08.1216.CrossRefGoogle ScholarPubMed
53. Davis, C, Lexchin, J, Jefferson, T, Gøtzsche, P, McKee, M. “Adaptive pathways” to drug authorisation: Adapting to industry? BMJ. 2016;354:i4437 CrossRefGoogle Scholar
54. New, JP, Bakerly, ND, Leather, D, Woodcock, A. Obtaining real-world evidence: The Salford Lung Study. Thorax. 2014;69:11521154. doi:10.1136/thoraxjnl-2014-205259.CrossRefGoogle ScholarPubMed
55. Tashkin, DP, Ferguson, GT. Combination bronchodilator therapy in the management of chronic obstructive pulmonary disease. Respir Res. 2013;14:49. doi:10.1186/1465-9921-14-49.CrossRefGoogle ScholarPubMed
56. Tosh, J, Stevenson, M, Akehurst, R. Health economic modelling of treatment sequences for rheumatoid arthritis: A systematic review. Curr Rheumatol Rep. 2014;16:447. doi:10.1007/s11926-014-0447-2.CrossRefGoogle ScholarPubMed
57. Brilleman, SL, Purdy, S, Salisbury, C, Windmeijer, F, Gravelle, H, Hollinghurst, S. Implications of comorbidity for primary care costs in the UK: A retrospective observational study. Br J Gen Pract. 2013;63:e274-82. doi:10.3399/bjgp13X665242.CrossRefGoogle ScholarPubMed
58. Tappenden, P, Chilcott, J, Brennan, A, Squires, H, Stevenson, M. Whole disease modeling to inform resource allocation decisions in cancer: A methodological framework. Value Health. 2012;15:11271136.CrossRefGoogle ScholarPubMed
59. Tappenden, P, Chilcott, J, Brennan, A, Squires, H, Glynne-Jones, R, Tappenden, J. Using whole disease modeling to inform resource allocation decisions: Economic evaluation of a clinical guideline for colorectal cancer using a single model. Value Health. 2013;16: 542553.CrossRefGoogle ScholarPubMed
60. Tosh, J, Kearns, B, Brennan, A, et al. Innovation in health economic modelling of service improvements for longer-term depression: Demonstration in a local health community. BMC Health Serv Res. 2013;13:150. doi:10.1186/1472-6963-13-150.CrossRefGoogle Scholar
61. Hunger, M, Thorand, B, Schunk, M, et al. Multimorbidity and health-related quality of life in the older population: Results from the German KORA-age study. Health Qual Life Outcomes. 2011;9:53. doi:10.1186/1477-7525-9-53.CrossRefGoogle ScholarPubMed
62. Logsdon, RG, Gibbons, LE, McCurry, SM, Teri, L. Assessing quality of life in older adults with cognitive impairment. Psychosom Med. 2002;64:510519.CrossRefGoogle ScholarPubMed
63. Bausewein, C, Simon, ST, Benalia, H, et al. Implementing patient reported outcome measures (PROMs) in palliative care-users’ cry for help. Heal Qual Life Outcomes. 2011;9:111.Google ScholarPubMed
64. Personal Social Services Research Unit (PSSRU). Adult Social Care Outcomes Toolkit - ASCOT. Published 2015 (accessed January 3, 2017).Google Scholar
65. Peña-Longobardo, LM, Oliva-Moreno, J, Hidalgo-Vega, Á, Miravitlles, M. Economic valuation and determinants of informal care to disabled people with Chronic Obstructive Pulmonary Disease (COPD). BMC Health Serv Res. 2015;15:101. doi:10.1186/s12913-015-0759-6.CrossRefGoogle Scholar
66. Ramsey, S, Willke, R, Briggs, A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: The ISPOR RCT-CEA Task Force report. Value Health. 2005;8:521533. doi:10.1111/j.1524-4733.2005.00045.x.CrossRefGoogle ScholarPubMed
67. Drummond, MF, Jefferson, TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ. 1996;313:275283. doi:10.1136/bmj.313.7052.275.CrossRefGoogle ScholarPubMed
68. IMS Core Diabetes Model. Published 2013 (accessed August 1, 2015).Google Scholar
69. Kristensen, FB, Lampe, K, Chase, DL, et al. Practical tools and methods for health technology assessment in Europe: Structures, methodologies, and tools developed by the European Network for Health Technology Assessment, EUnetHTA. Int J Technol Assess Health Care. 2009;25 (Suppl 2):18. doi:10.1017/S0266462309990626.CrossRefGoogle ScholarPubMed
70. NICE International. The Gates Reference Case - What it is, why it's important, and how to use it. 2014. (accessed July 11, 2017).Google Scholar
71. Husereau, D, Drummond, M, Petrou, S, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and elaboration: A report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. July 11, 2017).Google Scholar
72. World Health Organization. The economics of the social determinants of health and health inequalities: A resource book. ISBN 978 92 4 154862 5. Geneva: WHO; 2013.Google Scholar
73. Lozano, R, Naghavi, M, Foreman, K, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380 (9859):2095-128.CrossRefGoogle ScholarPubMed
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the or variations. ‘’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Available formats

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *