Skip to main content


  • Jozien M. Bensing (a1), Wilma M. C. M. Caris-Verhallen (a2), Joost Dekker (a3), Diana M. J. Delnoij (a2) and Peter P. Groenewegen (a1)...

Objectives: Health services research is a typical instance of applied research, meaning that its research findings should contribute to a more evidence-based health policy. A basic assumption for the usefulness of policy research is that it is of good scientific quality, but evaluation of health services research would be incomplete without an assessment of its societal relevance. However, there is no generally accepted framework in which criteria to assess societal relevance are defined.

Methods: In this study, we made an effort to specify criteria and indicators for policy relevance.

Results and Conclusions: In general, we concluded that it is important to define from the start of a research project, what kind of answers are being sought to the questions at hand, who the intended users of these answers will be, and how they can be reached. Furthermore, a productive dialogue between researchers and policy makers is indispensable for remaining in tune with policy thinking. In addition, we specified domains and a list of potential criteria for assessing policy relevance. For the purpose of quality assurance, these criteria have to be integrated into a quality improvement cycle. This means that the societal output of health services research should be related to the aims and missions at the level of projects, programs, and institutes. Furthermore, it is important that consensus is reached about (the weighting of) relevant indicators and that routines are developed for collecting information on these indicators.

Hide All
Bouter LM. 2000. Evaluation of societal relevance of applied health research. Is there a role for bibliometric analysis of non-indexed journals? Amsterdam: KNAW;
Brown LD. 1991: Knowledge and power: Health services research as a political resource. In: Ginzberg E, ed. Health services research: Key to health policy. Cambridge, MA: Harvard University Press; 20 45.
Eisenberg JM. 2001; Putting research to work: Reporting and enhancing the impact of health services research. Health Serv Res. 36: x-xvii.
Groenewegen PP, Bensing JM. 1995; Maatschappelijke kwaliteit van gezondheidszorgonderzoek. [Societal relevance of health services research] TSG. 73: 245 249.
Hurst JW. 1991; Reforming health care in seven European nations. Health Affairs. 10: 7 21.
Janis I. 1972. Victims of Groupthink: Psychological study of foreign-policy decisions and fiascoes. 2nd ed. Boston: Houghton Mifflin;
Klasen EC. 2001 Societal impact in ex ante evaluations. Unpublished speech. KNAW-symposium 19th June 2001.
Koninklijke Nederlandse Academie van Wetenschappen (KNAW), subcommissie beoordeling GZO. The societal impact of applied health research: Towards a quality assessment system. Available at: Accessed: 13 March 2002.
Lilford R, Jecock R, Shaw H, et al. 1999; Commissioning health services research: An iterative method. J Health Serv Res Policy. 4: 164 167.
Mulder HP, Walraven G, de Groot A, et al. 1991; Gebruik van beleids-evaluatie onderzoek bij de Rijksoverheid. [Use of policy evaluation research by the national government] Beleidswetenschap. 5: 203 227.
Oortwijn WJ, Ament AJHA, Vondeling H. 1996; Use of societal criteria in evaluation of medical technology assessment research proposals in the Netherlands: Development and testing of a checklist. Z Gesundheitswissenschaft. 4: 5 19.
Oortwijn WJ, Vondeling H, Bouter L. 1998; Use of societal criteria in priority setting for health technology assessment in the Netherlands. Int J Technol Assess Health Care. 14: 226 236.
Tennison B. 1996; What do purchasers of health care want from health services research? J Health Serv Res Policy. 1: 126 127.
Tijssen IMJG. 1988. Kwaliteit noodt tot meer gebruik. Bruikbaarheid van sociaal-wetenschappelijke onderzoeksresultaten volgens arts-bestuurders [quality asks for more utilization; usefulness of social science research according to the board members of the Royal Dutch Medical Association] (dissertation). Nijmegen: KUN;
Walt G. 1994; How far does research influence policy. Eur J Public Health. 4: 233 235.
Wamelink FJM, Spaapen JB. 1999. The evaluation of university research. A method for the incorporation of the societal value of research. Maarssen-Amsterdam: Sci-Quest;
Weiss CH. 1999; The interface between evaluation and public policy. Evaluation. 5: 468 486.
World Health Organization (WHO). 2000. World Health Report 2000. Geneva: WHO;
Ziekenfondsraad [Health Insurance Council]. 1993. Advies inzake kosten-effectiviteitsevaluatie van bestaande verstrekkingen. [Advisements on cost effectiveness of health care services] Amstelveen: Health Insurance Council;
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

International Journal of Technology Assessment in Health Care
  • ISSN: 0266-4623
  • EISSN: 1471-6348
  • URL: /core/journals/international-journal-of-technology-assessment-in-health-care
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 3
Total number of PDF views: 36 *
Loading metrics...

Abstract views

Total abstract views: 119 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 24th November 2017. This data will be updated every 24 hours.