Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-05-17T10:23:33.188Z Has data issue: false hasContentIssue false
  • English
  • Français

Beyond the numbers: a critique of quantitative multi-criteria decision analysis

Published online by Cambridge University Press:  01 July 2020

Michael J. DiStefano Open the ORCID record for Michael J. DiStefano [Opens in a new window]  and
Carleigh B. Krubiner
Michael J. DiStefano*
Affiliation:
Department of Health Policy & Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA Berman Institute of Bioethics, Johns Hopkins University, BaltimoreMD, USA
Carleigh B. Krubiner
Affiliation:
Berman Institute of Bioethics, Johns Hopkins University, BaltimoreMD, USA Center for Global Development, Washington, DC, USA
*
Corresponding author: Michael J. DiStefano, MBE, Berman Institute of Bioethics, 1809 Ashland Ave., Baltimore, MD21205, USA, mdistef1@jhu.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

When setting priorities for health, there is broad agreement that a range of social values and ethical principles beyond clinical and cost-effectiveness matter, but exactly how health technology assessment (HTA) should account for a broader set of criteria remains an area of ongoing debate. In light of this, we welcome a recent review paper by Baltussen et al. evaluating the potential of different multi-criteria decision analysis (MCDA) approaches to enable HTA agencies to incorporate a broader set of values in their appraisals. The authors describe three approaches to MCDA—qualitative MCDA, quantitative MCDA, and MCDA with decision rules—laying out their relative advantages and disadvantages and providing recommendations for how they can best be implemented. While we endorse many of the authors' assessments and conclusions, including the critical role of deliberation in any MCDA approach and the undertaking of qualitative MCDA at a minimum, we take a stronger position regarding the flaws of quantitative MCDA and strongly caution against it. We find quantitative MCDA antithetical to at least two of the ways MCDA is intended to improve HTA recommendations: (i) enhancing quality and (ii) promoting transparency. Quantitative MCDA may mask the complex tradeoffs that exist within and between decision criteria and remain generally inaccessible to those who are not well-versed in its technical methods of appraisal. We advocate for a predominantly qualitative approach to MCDA appraisal centered around deliberation and supplemented with decision aids to help account for health opportunity costs.

Keywords

Priority-settingDeliberationTransparencyLegitimacyEthics
Type
Article Commentary
Information
International Journal of Technology Assessment in Health Care , Volume 36 , Issue 4 , August 2020 , pp. 292 - 296
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Hofmann, B. Why not integrate ethics in HTA: Identification and assessment of the reasons. GMS Health Technol Assess. 2014;10. doi: 10.3205/hta000120.Google Scholar
Assasi, N, Schwartz, L, Tarride, JE, Campbell, K, Goeree, R. Methodological guidance documents for evaluation of ethical considerations in health technology assessment: A systematic review. Expert Rev Pharmacoecon Outcomes Res. 2014;14:203–20.10.1586/14737167.2014.894464CrossRefGoogle ScholarPubMed
Bellemare, CA, Dagenais, P, K-Bédard, S, Béland, J-P, Bernier, L, Daniel, C-E et al. Ethics in health technology assessment: A systematic review. Int J Technol Assess Health Care. 2018;34:447–57.10.1017/S0266462318000508CrossRefGoogle ScholarPubMed
Baltussen, R, Marsh, K, Thokala, P, Vakaramoko, D, Castro, H, Cleemput, I et al. Multi-criteria decision analysis to support HTA agencies – Benefits, limitations and the way forward. Value Health. 2019;22:1283–8.10.1016/j.jval.2019.06.014CrossRefGoogle Scholar
Devlin, N, Sussex, J. Incorporating multiple criteria in HTA: Methods and processes. Office Health Econ. 2011. https://www.ohe.org/publications/incorporating-multiple-criteria-hta-methods-and-processes.Google Scholar
Youngkong, S, Baltussen, R, Tantivess, S, Mohara, A, Teerawattananon, Y. Multicriteria decision analysis for including health interventions in the universal health coverage benefit package in Thailand. Value Health. 2012;15:961–70.CrossRefGoogle ScholarPubMed
Persad, G. Justice and public health. In: Mastroianni, A, Kahn, JP, Kass, NE, editors. The Oxford handbook of public health ethics. Oxford, UK: Oxford University Press; 2019. p. 33–43.Google Scholar
Endrei, D, Molics, B, Ágoston, I. Multicriteria decision analysis in the reimbursement of new medical technologies: Real-world experiences from Hungary. Value Health. 2014;17:487–9.CrossRefGoogle ScholarPubMed
Daniels, N, Sabin, J. Limits to health care: Fair procedures, democratic deliberation, and the legitimacy problem for insurers. Philos Public Aff. 1997;26:303–50.CrossRefGoogle ScholarPubMed
Gutmann, A, Thompson, DF. Why deliberative democracy? Princeton, NJ: Princeton University Press; 2004.CrossRefGoogle Scholar
Persad, G. Transparency trade-offs: Priority setting, scarcity, and health fairness. In: Cohen, IG, Evans, B, Lynch, H, Shachar, C, editors. Transparency in health and health care. New York: Cambridge University Press; 2019. p. 44–57.Google Scholar
Naurin, D. Deliberation behind closed doors: Transparency and lobbying in the European Union. Colchester, UK: ECPR Press; 2007.Google Scholar
Charlton, V. NICE and fair? Health technology assessment policy under the UK's National Institute for Health and Care Excellent, 1999–2018. Health Care Anal. 2019. https://doi.org/10.1007/s10728-019-00381-x.Google Scholar
Johnston, CD, Ballard, AO. Economists and public opinion: Expert consensus and economic policy judgments. J Polit. 2016;78:443–57.CrossRefGoogle Scholar
Lee, MK. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data Soc. 2018;5:1–16.CrossRefGoogle Scholar
Woodruff, A, Fox, SE, Rousso-Schindler, S, Warshaw, J. A qualitative exploration of perceptions of algorithmic fairness. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Montreal, QC, Canada; 2018.CrossRefGoogle Scholar
Johri, M, Norheim, OF. Can cost-effectiveness analysis integrate concerns for equity? Systematic review. Int J Technol Assess Health Care. 2012;28:125–32.CrossRefGoogle ScholarPubMed
Nussbaum, MC. The costs of tragedy: Some moral limits of cost-benefit analysis. J Legal Stud. 2000;29:1005–36.CrossRefGoogle Scholar
Goold, SD, Biddle, AK, Klipp, G, Hall, CN, Danis, M. Choosing Healthplans All Together: A deliberative exercise for allocating limited health care resources. J Health Polit Policy Law. 2005;30:563–602.10.1215/03616878-30-4-563CrossRefGoogle ScholarPubMed
Goetghebeur, MM, Wagner, M, Khoury, H, Rindress, D, Grégoire, J-P, Deal, C. Combining multicriteria decision analysis, ethics and health technology assessment: Applying the EVIDEM decision-making framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. 2010;8. doi: 10.1186/1478-7547-8-4.CrossRefGoogle ScholarPubMed
Rutten-van Mölken, M, Leijten, F, Hoedemakers, M, Tsiachristas, A, Verbeek, N, Karimi, M et al. Strengthening the evidence-based of integrated care for people with multi-morbidity in Europe using Multi-Criteria Decision Analysis (MCDA). BMC Health Serv Res. 2018;18. https://doi.org/10.1186/s12913-018-3367-4.CrossRefGoogle Scholar
Wagner, M, Khoury, H, Bennetts, L, Willet, J, Lister, J, Berto, P et al. Appraising the holistic value of lenvatinib for radio-iodine refractory differentiated thyroid cancer: A multi-country study applying pragmatic MCDA. BMC Cancer. 2017;17. doi: 10.1186/s12885-017-3258-9CrossRefGoogle ScholarPubMed
Youngkong, S, Teerawattananon, Y, Tantivess, S, Baltussen, R. Multi-criteria decision analysis for setting priorities on HIV/AIDS interventions in Thailand. Health Res Policy Syst. 2012;10. https://doi.org/10.1186/1478-4505-10-6.CrossRefGoogle ScholarPubMed
O'Neill, O. A question of trust: The BBC Reith Lectures. Cambridge, UK: Cambridge University Press; 2002.Google Scholar
Cookson, R, Mirelman, AJ, Griffin, S, Asaria, M, Dawkins, B, Norheim, OF et al. Using cost-effectiveness analysis to addresshealth equity concerns. Value Health. 2017;20:206–12.10.1016/j.jval.2016.11.027CrossRefGoogle ScholarPubMed
Baltussen, R, Jansen, MP, Mikkelsen, E, Tromp, N, Hontelez, J, Bijlmakers, L et al. Priority setting for universal health coverage: We need evidence-informed deliberative processes, not just more evidence on cost-effectiveness. Int J Health Policy Manag. 2016;5:615–18.CrossRefGoogle Scholar
Kapiriri, L, Baltussen, R, Oortwijn, W. Implementing evidence-informed deliberative processes in health technology assessment: A low income country perspective. Int J Technol Assess Health Care. 2020;36:1–5. https://doi.org/10.1017/S0266462319003398.CrossRefGoogle ScholarPubMed
Oortwijn, W, Klein, P. Addressing health system values in health technology assessment: The use of evidence-informed deliberative processes. Int J Technol Assess Health Care. 2019;35:82–84.10.1017/S0266462319000187CrossRefGoogle Scholar
Bond, K. Deliberative processes in health technology assessment: Prospects, problems, and policy proposals. Paper presented at HTAi Global Policy Forum 2020; New Orleans, LA; 2020. Available from: https://htai.org/wp-content/uploads/2020/02/HTAi_GPF-newOrleans_program_background-paper.pdf (Accessed May 16, 2020).Google Scholar