Hostname: page-component-797576ffbb-k7d4m Total loading time: 0 Render date: 2023-12-02T05:24:08.168Z Has data issue: false Feature Flags: { "corePageComponentGetUserInfoFromSharedSession": true, "coreDisableEcommerce": false, "useRatesEcommerce": true } hasContentIssue false

Setting priorities in clinical and health services research: Properties of an adapted and updated method

Published online by Cambridge University Press:  15 April 2010

Silvina Berra
Catalan Agency for Health Technology Assessment and Research (CAHTA) and CIBERESP (CIBER en Epidemiología y Salud Pública), Spain
Emília Sánchez
Catalan Agency for Health Technology Assessment and Research (CAHTA) and CIBERESP (CIBER en Epidemiología y Salud Pública), Spain
Joan M. V. Pons
Department of Health, Catalonia; Catalan Agency for Health Technology Assessment and Research (CAHTA); and CIBERESP (CIBER en Epidemiología y Salud Pública), Spain
Cristian Tebé
Catalan Agency for Health Technology Assessment and Research (CAHTA) and CIBERESP (CIBER en Epidemiología y Salud Pública), Spain
Jordi Alonso
IMIM – Hospital del Mar; Pompeu Fabra University Barcelona; and CIBERESP (CIBER en Epidemiología y Salud Pública), Spain
Marta Aymerich
University of Girona


Objectives: The objectives of this study is to review the set of criteria of the Institute of Medicine (IOM) for priority-setting in research with addition of new criteria if necessary, and to develop and evaluate the reliability and validity of the final priority score.

Methods: Based on the evaluation of 199 research topics, forty-five experts identified additional criteria for priority-setting, rated their relevance, and ranked and weighted them in a three-round modified Delphi technique. A final priority score was developed and evaluated. Internal consistency, test–retest and inter-rater reliability were assessed. Correlation with experts’ overall qualitative topic ratings were assessed as an approximation to validity.

Results: All seven original IOM criteria were considered relevant and two new criteria were added (“potential for translation into practice”, and “need for knowledge”). Final ranks and relative weights differed from those of the original IOM criteria: “research impact on health outcomes” was considered the most important criterion (4.23), as opposed to “burden of disease” (3.92). Cronbach's alpha (0.75) and test–retest stability (interclass correlation coefficient = 0.66) for the final set of criteria were acceptable. The area under the receiver operating characteristic curve for overall assessment of priority was 0.66.

Conclusions: A reliable instrument for prioritizing topics in clinical and health services research has been developed. Further evaluation of its validity and impact on selecting research topics is required.

Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)



1. Asua Batarrita, J. Priorización de necesidades de evaluación en el País Vasco. Madrid: Universidad Complutense de Madrid, Instituto de Salud Carlos III; 1999.Google Scholar
2. Bernal-Delgado, E, Peiró, S, Sotoca, R. Prioridades de investigación en servicios sanitarios en el Sistema Nacional de Salud. Una aproximación por consenso de expertos. Gac Sanit. 2006;20:287294.Google Scholar
3. Black, N. A national strategy for research and development: Lessons from England. Annu Rev Public Health. 1997;18:485505.Google Scholar
4. Brown, P, Brunnhuber, K, Chalkidou, K, et al. How to formulate research recommendations. BMJ. 2006;333:804806.Google Scholar
5. Brown, K, Dyas, J, Chahal, P, et al. Discovering the research priorities of people with diabetes in a multicultural community: A focus group study. Br J Gen Pract. 2006;56:206213.Google Scholar
6. Buxton, M, Hanney, S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1:3543.Google Scholar
7. Caron-Flinterman, JF, Broerse, JE, Teerling, J, Bunders, JF. Patients’ priorities concerning health research: The case of asthma and COPD research in the Netherlands. Health Expect. 2005;8:253263.Google Scholar
8. Carson, N, Ansari, Z, Hart, W. Priority setting in public health and health services research. Aust Health Rev. 2000;23:4657.Google Scholar
9. Claxton, K, Ginnelly, L, Sculpher, M, Philips, Z, Palmer, S. A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme. Health Technol Assess. 2004;8:1103.Google Scholar
10. Dault, M, Lomas, J, Barer, M, on behalf of the Listening for Direction II partners. Listening for direction II. National consultation on health services and policy issues for 2004–2007. Ottawa, Ontario: Canadian Health Services Research Foundation and Institute of Health Services and Policy Research; 2004.Google Scholar
11. Department of Health. Best research for best health: A new national health research strategy. The NHS contribution to health research in England. London: HMSO; 2006.Google Scholar
12. Department of Health. Research for health. London: HMSO; 1992.Google Scholar
13. Donaldson, MS, Sox, HC, eds. Setting priorities for health technology assessment: A model process. Washington DC: National Academy Press; 1992.Google Scholar
14. Eden, J, Wheatley, B, McNeil, B, Sox, H, eds. Knowing what works in health care. Washington DC: National Academy Press; 2008.Google Scholar
15. Evidence-Based Care Resource Group. Evidence-based care: 1. Setting priorities: How important is this problem? Can Med Assoc J. 1994;150:12491254.Google Scholar
16. Fleurence, RL, Torgerson, DJ. Setting priorities for research. Health Policy. 2004;69:110.Google Scholar
17. Fulop, N, Allen, P, Clarke, A, Black, N. From health technology assessment to research on the organisation and delivery of health services: Addressing the balance. Health Policy. 2003;63:155165.Google Scholar
18. Gross, CP, Anderson, GF, Powe, NR. The relation between funding by the National Institutes of Health and the burden of disease. N Engl J Med. 1999;340:18811887.Google Scholar
19. Hanney, S, Mugford, M, Grant, J, Buxton, M. Assessing the benefits of health research: Lessons from research into the use of antenatal corticosteroids for the prevention of neonatal respiratory distress syndrome. Soc Sci Med. 2005;60:937947.Google Scholar
20. Henshall, C, Oortwijn, W, Stevens, A, Granados, A, Banta, D. Priority setting for health technology assessment. Int J Technol Assess Health Care. 1997;13:144185.Google Scholar
21. Institute of Medicine. Initial national priorities for comparative effectiveness research. Washington, DC: Institute of Medicine; 2009.Google Scholar
22. Jones, R, Lamont, T, Haines, A. Setting priorities for research and development in the NHS: A case study on the interface between primary and secondary care. BMJ. 1995;311:10761080.Google Scholar
23. Lionis, C, Stoffers, HE, Hummers-Pradier, , et al. Setting priorities and identifying barriers for general practice research in Europe. Results from an EGPRW meeting. Fam Pract. 2004;21:587593.Google Scholar
24. Meneu, R. Variabilidad de las decisiones médicas y su repercusión sobre las poblaciones. Barcelona: Masson; 2002.Google Scholar
25. Michaud, CM, Murray, CJ, Bloom, BR. Burden of disease: Implications for future research. JAMA. 2001;285:535539.Google Scholar
26. Moreno-Casbas, T, Martin-Arribas, C, Orts-Cortes, I, Comet-Cortes, P, Investen-ISCIII Co-ordination and Development of Nursing Research Centre. Identification of priorities for nursing research in Spain: A Delphi study. J Adv Nurs. 2001;35:857863.Google Scholar
27. Oortwijn, WJ, Vondeling, H, van Barneveld, T, van Vugt, C, Bouter, LM. Priority setting for health technology assessment in The Netherlands: Principles and practice. Health Policy. 2002;62:227242.Google Scholar
28. Phelps, CE, Parente, ST. Priority setting in medical technology and medical practice assessment. Med Care. 1990;28:703723.Google Scholar
29. Sánchez, E, Solans, M, Jaldon, C. VI call for clinical and health services research CAHTA 2006. CAHTA's Newsletter 2006. (accessed August 14, 2009)Google Scholar
30. Streiner, DL, Norman, GR. Health measurement scales. A practical guide to their development and use. 3rd ed. New York: Oxford University Press; 2003.Google Scholar
31. Swedish Council on Technology Assessment in Health Care. The ECHTA/ECAHI Project. 1999. (accessed August 14, 2009)Google Scholar
32. Tunis, SR, Stryer, DB, Clancy, CM. Practical clinical trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:16241632.Google Scholar
33. Welfare, MR, Colligan, J, Molyneux, S, Pearson, P, Barton, JR. The identification of topics for research that are important to people with ulcerative colitis. Eur J Gastroenterol Hepatol. 2006;18:939944.Google Scholar
34. Zerhouni, EA. US biomedical research: Basic, translational, and clinical sciences. JAMA. 2005;294:13521358.Google Scholar