Skip to main content Accessibility help


  • Nataly Tanios (a1), Monika Wagner (a2), Michèle Tony (a1), Rob Baltussen (a3), Janine van Til (a4), Donna Rindress (a2), Paul Kind (a5), Mireille M. Goetghebeur (a6) and the International Task Force on Decision Criteria...


Objectives: The aim of this study was to gather qualitative and quantitative data on criteria considered by healthcare decision makers.

Methods: Using snowball sampling and an online questionnaire with forty-three criteria organized into ten clusters, decision makers were invited by an international task force to report which criteria they consider when making decisions on healthcare interventions in their context. Respondents reported whether each criterion is “currently considered,” “should be considered,” and its relative weight (scale 0–5). Differences in proportions of respondents were explored with inferential statistics across levels of decision (micro, meso, macro), decision maker perspectives, and world regions.

Results: A total of 140 decision makers (1/3 clinical, 2/3 policy) from 23 countries in five continents completed the survey. The most relevant criteria (top ranked for “Currently considered,” “Should be considered,” and weights) were Clinical efficacy/effectiveness, Safety, Quality of evidence, Disease severity, and Impact on healthcare costs. Organizational and skill requirements were frequently considered but had relatively low weights. For almost all criteria, a higher proportion of decision makers reported that they “Should be considered” than that they are “Currently considered” (p < .05). For more than 74 percent of criteria, there were no statistical differences in proportions across levels of decision, perspectives and world regions. Statistically significant differences across several comparisons were found for: Population priorities, Stakeholder pressure/interests, Capacity to stimulate research, Impact on partnership and collaboration, and Environmental impact.

Conclusions: Results suggest convergence among decision makers on the relevance of a core set of criteria and on the need to consider a wider range of criteria. Areas of divergence appear to be principally related to contextual factors.



Hide All
1.Baltussen, R, Niessen, L. Priority setting of health interventions: The need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14.
2.Daniels, N, Sabin, J. Limits to health care: Fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos Public Aff. 1997;26:303350.
3.Culyer, AJ, Bombard, Y. An equity framework for health technology assessments. Med Decis Making. 2012;32:428441.
4.Phillips, LD, Fasolo, B, Zafiropoulos, N, Beyer, A. Is quantitative benefit-risk modelling of drugs desirable or possible? Drug Discov Today Technol. 2011;8:e3e10.
5.Burls, A, Austin, D, Moore, D. Commissioning for rare diseases: View from the frontline. BMJ. 2005;331:10191021.
6.Bowen, S, Zwi, AB. Pathways to “evidence-informed” policy and practice: A framework for action. PLoS Med. 2005;2:e166.
7.Browman, GP, Manns, B, Hagen, N, et al.6-STEPPPs: A modular tool to facilitate clinician participation in fair decisions for funding new cancer drugs. J Oncol Pract. 2008;4:27.
8.Goetghebeur, M, Wagner, M, Khoury, H, et al.Evidence and value: Impact on DEcisionMaking - the EVIDEM framework and potential applications. BMC Health Serv Res. 2008;8:270.
9.Golan, OG, Hansen, P. Which health technologies should be funded? A prioritization framework based explicitly on value for money. Isr J Health Policy Res. 2012;1:44.
10.Honore, PA, Fos, PJ, Smith, T, Riley, M, Kramarz, K. Decision science: A scientific approach to enhance public health budgeting. J Public Health Manag Pract. 2010;16:98103.
11.Johnson, AP, Sikich, NJ, Evans, G, et al.Health technology assessment: A comprehensive framework for evidence-based recommendations in Ontario. Int J Technol Assess Health Care. 2009;25:141150.
12.Tannahill, A. Beyond evidence–to ethics: A decision-making framework for health promotion, public health and health improvement. Health Promot Int. 2008;23:380390.
13.Wilson, EC, Rees, J, Fordham, RJ. Developing a prioritisation framework in an English Primary Care Trust. Cost Eff Resour Alloc. 2006;4:3.
14.Goetghebeur, MM, Wagner, M, Khoury, H, et al.Combining multicriteria decision analysis, ethics and health technology assessment: Applying the EVIDEM decisionmaking framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8:418.
15.Guindo, LA, Wagner, M, Baltussen, R, et al.From efficacy to equity: Review of decision criteria used in resource allocation and healthcare decisionmaking. Cost Eff Resour Alloc. 2012;10:9.
16.Mirelman, A, Mentzakis, E, Kinter, E, et al.Decision-making criteria among national policymakers in five countries: A discrete choice experiment eliciting relative preferences for equity and efficiency. Value Health. 2012;15:534539.
17.Youngkong, S, Kapiriri, L, Baltussen, R. Setting priorities for health interventions in developing countries: A review of empirical studies. Trop Med Int Health. 2009;14:930939.
18.Golan, O, Hansen, P, Kaplan, G, Tal, O. Health technology prioritization: Which criteria for prioritizing new technologies and what are their relative weights? Health Policy. 2011;102:126135.
19.Gibson, JL, Martin, DK, Singer, PA. Evidence, economics and ethics: Resource allocation in health services organizations. Healthc Q. 2005;8:5059, 4.
20.Djulbegovic, B, Trikalinos, TA, Roback, J, Chen, R, Guyatt, G. Impact of quality of evidence on the strength of recommendations: An empirical study. BMC Health Serv Res. 2009;9:120.
21.Brozek, JL, Akl, EA, Alonso-Coello, P, et al.Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions. Allergy. 2009;64:669677.
22.Evers, S, Goossens, M, de Vet, H, van Tulder, M, Ament, A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005;21:240245.
23.Saarni, SI, Hofmann, B, Lampe, K, et al.Ethical analysis to improve decision-making on health technologies. Bull World Health Organ. 2008;86:617623.
24.Burls, A, Caron, L, Cleret de Langavant, G, et al.Tackling ethical issues in health technology assessment: A proposed framework. Int J Technol Assess Health Care. 2011;27:230237.
25.Peacock, SJ, Mitton, C, Ruta, D, et al.Priority setting in healthcare: Towards guidelines for the program budgeting and marginal analysis framework. Expert Rev Pharmacoecon Outcomes Res. 2010;10:539552.
26.Dubois, RW, Graff, JS. Setting priorities for comparative effectiveness research: From assessing public health benefits to being open with the public. Health Aff (Millwood). 2011;30:22352242.
27.Hyder, AA, Corluka, A, Winch, PJ, et al.National policy-makers speak out: Are researchers giving them what they need? Health Policy Plan. 2011;26:7382.



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed