Skip to main content Accessibility help
×
Home
Hostname: page-component-99c86f546-md8df Total loading time: 0.325 Render date: 2021-11-28T12:42:17.593Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL

Published online by Cambridge University Press:  19 February 2018

Kei Long Cheung
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht University kl.cheung@maastrichtuniversity.nl
Mickaël Hiligsmann
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht University
Maximilian Präger
Affiliation:
Helmholtz Zentrum München (GmbH); German Research Center for Environmental Health (Institute of Health Economics and Health Care Management)
Teresa Jones
Affiliation:
Health Economics Research Group, Brunel University London
Judit Józwiak-Hagymásy
Affiliation:
Syreon Research Institute
Celia Muñoz
Affiliation:
Centre for Research in Economics and Health (CRES), University Pompeu Fabra
Adam Lester-George
Affiliation:
LeLan Solutions
Subhash Pokhrel
Affiliation:
Health Economics Research Group, Brunel University London
Ángel López-Nicolás
Affiliation:
Polytechnic University of Cartagena
Marta Trapero-Bertran
Affiliation:
Centre for Research in Economics and Health (CRES), University Pompeu Fabra
Silvia M.A.A. Evers
Affiliation:
Caphri School of Public Health and Primary Care, Health Services Research, Maastricht University; Trimbos Institute, Netherlands Institute of Mental Health and Addiction
Hein de Vries
Affiliation:
Caphri School of Public Health and Primary Care, Health Promotion, Maastricht University

Abstract

Objectives: Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study.

Methods: A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool.

Results: A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation.

Conclusions: Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

Type
Assessments
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Park, AL, McDaid, D, Weiser, P, et al. Examining the cost effectiveness of interventions to promote the physical health of people with mental health problems: A systematic review. BMC Public Health. 2013;13:787.CrossRefGoogle ScholarPubMed
2. Nicod, E, Kanavos, P. Commonalities and differences in HTA outcomes: A comparative analysis of five countries and implications for coverage decisions. Health Policy. 2012;108:167177.CrossRefGoogle ScholarPubMed
3. Drummond, MF, Schwartz, JS, Jönsson, B, et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. Int J Technol Assess Health Care. 2008;24: 244258.CrossRefGoogle ScholarPubMed
4. van Velden ME, Severens JL, Novak, A. Economic evaluations of healthcare programmes and decision making. Pharmacoeconomics. 2005;23:10751082.CrossRefGoogle ScholarPubMed
5. Oliver, K, Innvar, S, Lorenc, T, Woodman, J, Thomas, J. A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res. 2014;14:2.CrossRefGoogle ScholarPubMed
6. Macintyre, S, Chalmers, I, Horton, R, Smith, R. Using evidence to inform health policy: Case study. BMJ. 2001;322:222.CrossRefGoogle ScholarPubMed
7. Drummond, M. Economic evaluation in health care: Is it really useful or are we just kidding ourselves? Aust Econ Rev. 2004;37: 311.CrossRefGoogle Scholar
8. Garrido, MV. Health technology assessment and health policy-making in Europe: Current status, challenges and potential. Geneva: WHO Regional Office Europe; 2008.Google Scholar
10. Lim, SS, Vos, T, Flaxman, AD, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990–2010: A systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2013;380: 22242260.CrossRefGoogle Scholar
11. WHO. WHO Framework Convention on Tobacco Control. 2003. http://www.who.int/fctc/en/ (accessed December 17, 2015).Google Scholar
12. European Commission. “Public Health: Tobacco Policy.” 2012. http://ec.europa.eu/health/tobacco/policy/index_en.htm (accessed December 17, 2015).Google Scholar
13. Pokhrel, S, Evers, S, Leidl, R, et al. EQUIPT: Protocol of a comparative effectiveness research study evaluating cross-context transferability of economic evidence on tobacco control. BMJ Open. 2014;4:e006945.CrossRefGoogle ScholarPubMed
14. van Gemert-Pijnen, JE, Nijland, N, van Limburg, M, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. 2011;13:e111.CrossRefGoogle ScholarPubMed
15. Eysenbach, G. The law of attrition. J Med Internet Res. 2005;7:e11.CrossRefGoogle ScholarPubMed
16. Horsky, J, Schiff, GD, Johnston, D, Mercincavage, L, Bell, D, Middleton, B. Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012;45:12021216.CrossRefGoogle ScholarPubMed
17. ISO. 9241-11:1998. Ergonomic requirements for office work with visual display terminals (VDT)s – Part 11: Guidance on usability. 1998. https://www.iso.org/standard/16883.html (accessed January 16, 2018).Google Scholar
18. Bevan, N. Usability is quality of use. Adv Hum Factors Ergon. 1995;20:349.Google Scholar
19. Wichansky, AM. Usability testing in 2000 and beyond. Ergonomics. 2000;43:9981006.CrossRefGoogle ScholarPubMed
20. Cho, V, Cheng, TE, Lai, WJ. The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Comput Educ. 2009;53:216227.CrossRefGoogle Scholar
21. Becker, S, Mottay, FE. A global perspective on web site usability. IEEE Software. 2001;18:5461.CrossRefGoogle Scholar
22. Ahern, DK. Challenges and opportunities of eHealth research. Am J Prev Med. 2007;32:S75-S82.CrossRefGoogle ScholarPubMed
23. Nielsen, J. Usability inspection methods. New York: John Wiley and Sons; 1994;17:2562.Google Scholar
24. Voncken-Brewster, V, Moser, A, Van Der Weijden T, Nagykaldi Z, De Vries H, Tange H. Usability evaluation of an online, tailored self-management intervention for chronic obstructive pulmonary disease patients incorporating behavior change techniques. JMIR Res Protoc. 2013;2:e3.CrossRefGoogle ScholarPubMed
25. Cheung, KL, Evers, SM, Hiligsmann, M, et al. Understanding the stakeholders’ intention to use economic decision-support tools: A cross-sectional study with the Tobacco Return on Investment tool. Health Policy. 2016;120:4654.CrossRefGoogle ScholarPubMed
26. Centre for Disease Control and Prevention. 2011. Introduction to program evaluation for public health programs: A self-study guide. http://www.cdc.gov/eval/guide/CDCEvalManual.pdf (accessed January 21, 2018).Google Scholar
27. Nielsen, J, Landauer, TK, editors. A mathematical model of the finding of usability problems. Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems, Amsterdam, The Netherlands, 1993: ACM.Google Scholar
28. Jaspers, MW. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence. Int J Med Inform. 2009;78:340353.CrossRefGoogle ScholarPubMed
29. Brooke, J. SUS-A quick and dirty usability scale. Usability Eval Ind. 1996;189:47.Google Scholar
30. Jaspers, MW, Steen, T, van Den Bos, C, Geenen, M. The think aloud method: A guide to user interface design. Int J Med Inform. 2004;73:781795.CrossRefGoogle ScholarPubMed
31. Nielsen, J, editor. Reliability of severity estimates for usability problems found by heuristic evaluation. Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems, Monterey, California, 1992: ACM.Google Scholar
32. Brooke, J. SUS: A retrospective. J Usability Stud. 2013;8:2940.Google Scholar
33. Jeffries, R, Miller, JR, Wharton, C, Uyeda, K, editors. User interface evaluation in the real world: A comparison of four techniques. Proceedings of the SIGCHI conference on Human factors in computing systems, Montreal, Canada, 1991: ACM.Google Scholar
34. Lai, T-Y, Larson, EL, Rockoff, ML, Bakken, S. User Acceptance of HIV TIDES—tailored interventions for management of depressive symptoms in persons living with HIV/AIDS. JAMA. 2008;15:217226.Google ScholarPubMed
35. Bailey, RW, Allan, RW, Raiello, P, editors. Usability testing vs. heuristic evaluation: A head-to-head comparison. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1992: SAGE Publications.Google Scholar
36. Rogers, EM. Diffusion of innovations. New York: Simon and Schuster; 2010.Google Scholar

Cheung et al. supplementary material 1

Supplementary Video

Download Cheung et al. supplementary material 1(Video)
Video 144 MB
Supplementary material: File

Cheung et al. supplementary material 2

Cheung et al. supplementary material

Download Cheung et al. supplementary material 2(File)
File 24 KB
Supplementary material: File

Cheung et al. supplementary material 3

Cheung et al. supplementary material

Download Cheung et al. supplementary material 3(File)
File 46 KB
Supplementary material: File

Cheung et al. supplementary material 4

Cheung et al. supplementary material

Download Cheung et al. supplementary material 4(File)
File 31 KB
2
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *