Hostname: page-component-5db58dd55d-qmkzp Total loading time: 0 Render date: 2026-07-09T19:45:26.250Z Has data issue: false hasContentIssue false
  • English
  • Français

Systematic Literature Reviews: An Introduction

Part of: Design Methods and Tools

Published online by Cambridge University Press:  26 July 2019

Guillaume Lame

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

Systematic literature reviews (SRs) are a way of synthesising scientific evidence to answer a particular research question in a way that is transparent and reproducible, while seeking to include all published evidence on the topic and appraising the quality of this evidence. SRs have become a major methodology in disciplines such as public policy research and health sciences. Some have advocated that design research should adopt the method. However, little guidance is available. This paper provides an overview of the SR method, based on the literature in health sciences. Then, the rationale for SRs in design research is explored, and four recent examples of SRs in design research are analysed to illustrate current practice. Foreseen challenges in taking forward the SR method in design research are highlighted, and directions for developing a SR method for design research are proposed. It is concluded that SRs hold potential for design research and could help us in addressing some important issues, but work is needed to define what review methods are appropriate for each type of research question in design research, and to adapt guidance to our own needs and specificities.

Keywords

Research methodologies and methodsEvaluationDesign methodologySystematic reviewsDesign Research

Information

Type
Article
Information
Proceedings of the Design Society: International Conference on Engineering Design , Volume 1 , Issue 1 , July 2019 , pp. 1633 - 1642
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s) 2019

References

Bastian, H., Glasziou, P. and Chalmers, I. (2010), “Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?”, PLOS Medicine, Vol. 7 No. 9, p. e1000326. http://doi.org/10.1371/journal.pmed.1000326Google Scholar
Blessing, L.T.M. and Chakrabarti, A. (2009), DRM, a Design Research Methodology. London: Springer London.Google Scholar
Bonvoisin, J., Halstenberg, F., Buchert, T. and Stark, R. (2016), “A systematic literature review on modular product design”, Journal of Engineering Design, Vol. 27 No. 7, pp. 488–514. http://doi.org/10.1080/09544828.2016.1166482Google Scholar
Borry, P., Schotsmans, P. and Dierickx, K. (2006), “Evidence-based medicine and its role in ethical decision-making”, Journal of Evaluation in Clinical Practice, Vol. 12 No. 3, pp. 306–311. http://doi.org/10.1111/j.1365-2753.2006.00548.xGoogle Scholar
Bramer, W.M., Giustini, D. and Kramer, B.M.R. (2016), “Comparing the coverage, recall, and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: a prospective study”, Systematic Reviews, Vol. 5 No. 1, p. 39. http://doi.org/10.1186/s13643-016-0215-7Google Scholar
Cantamessa, M. (2003), “An empirical perspective upon design research”, Journal of Engineering Design, Vol. 14 No. 1, pp. 1–15. http://doi.org/10.1080/0954482031000078126Google Scholar
Cash, P.J. (2018), “Developing theory-driven design research”, Design Studies, Vol. 56 No., pp. 84–119. http://doi.org/10.1016/j.destud.2018.03.002Google Scholar
Chalmers, I., Hedges, L.V. and Cooper, H. (2002), “A Brief History of Research Synthesis”, Evaluation & the Health Professions, Vol. 25 No. 1, pp. 12–37. http://doi.org/10.1177/0163278702025001003Google Scholar
Chassin, M.R., Kosecoff, J., Park, R.E. and et al. (1987), “Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures”, JAMA, Vol. 258 No. 18, pp. 2533–2537. http://doi.org/10.1001/jama.1987.03400180067028Google Scholar
Cook, D.J., Mulrow, C.D. and Haynes, R. (1997), “Systematic reviews: Synthesis of best evidence for clinical decisions”, Annals of Internal Medicine, Vol. 126 No. 5, pp. 376–380. http://doi.org/10.7326/0003-4819-126-5-199703010-00006Google Scholar
Crowe, M. and Sheppard, L. (2011), “A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed”, Journal of Clinical Epidemiology, Vol. 64 No. 1, pp. 79–89. http://doi.org/10.1016/j.jclinepi.2010.02.008Google Scholar
Dixon-Woods, M. (2010), “Systematic reviews and qualitative methods”, in Silverman, D. (ed.) Qualitative research: theory, method and practice. 3rd ed. London: Sage, pp. 331–346.Google Scholar
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B. and Sutton, A. (2005), “Synthesising qualitative and quantitative evidence: A review of possible methods”, Journal of Health Services Research & Policy, Vol. 10 No. 1, pp. 45–53. http://doi.org/10.1177/135581960501000110Google Scholar
Eckert, C.M. and Summers, J.D. (2013), “Interviewing as a method for data gathering in engineering design research”.Google Scholar
Egger, M., Smith, G. D. and Altman, D. G. (eds.) (2008), Systematic reviews in health care : meta-analysis in context. 2nd edn. London: BMJ Books.Google Scholar
Every-Palmer, S. and Howick, J. (2014), “How evidence-based medicine is failing due to biased trials and selective publication”, Journal of Evaluation in Clinical Practice, Vol. 20 No. 6, pp. 908–914. http://doi.org/10.1111/jep.12147Google Scholar
Fanelli, D. (2012), “Negative results are disappearing from most disciplines and countries”, Scientometrics, Vol. 90 No. 3, pp. 891–904. http://doi.org/10.1007/s11192-011-0494-7Google Scholar
Frey, D.D. and Dym, C.L. (2006), “Validation of design methods: lessons from medicine”, Research in Engineering Design, Vol. 17 No. 1, pp. 45–57. http://doi.org/10.1007/s00163-006-0016-4Google Scholar
Gericke, K., Eckert, C.M. and Stacey, M. “What do we need to say about a design method?”. ICED 2017, 2017/08/21/25. Vancouver, BC, Canada: The Design Society.Google Scholar
Gough, D., Thomas, J. and Oliver, S. (2012), “Clarifying differences between review designs and methods”, Systematic Reviews, Vol. 1 No. 1, p. 28. http://doi.org/10.1186/2046-4053-1-28Google Scholar
Grant, M.J. and Booth, A. (2009), “A typology of reviews: an analysis of 14 review types and associated methodologies”, Health Information & Libraries Journal, Vol. 26 No. 2, pp. 91–108. http://doi.org/10.1111/j.1471-1842.2009.00848.xGoogle Scholar
Greenhalgh, T. and Peacock, R. (2005), “Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources”, BMJ, Vol. 331 No. 7524, pp. 1064–1065. http://doi.org/10.1136/bmj.38636.593461.68Google Scholar
Hay, L., Duffy, A.H.B., McTeague, C., Pidgeon, L.M., Vuletic, T. and Grealy, M. (2017), “A systematic review of protocol studies on conceptual design cognition: Design as search and exploration”, Design Science, Vol. 3 No., p. e10. http://doi.org/10.1017/dsj.2017.11Google Scholar
Horváth, I. (2004), “A treatise on order in engineering design research”, Research in Engineering Design, Vol. 15 No. 3, pp. 155–181. http://doi.org/10.1007/s00163-004-0052-xGoogle Scholar
Jinha, A.E. (2010), “Article 50 million: an estimate of the number of scholarly articles in existence”, Learned Publishing, Vol. 23 No. 3, pp. 258–263. http://doi.org/10.1087/20100308Google Scholar
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J. and Linkman, S. (2009), “Systematic literature reviews in software engineering – A systematic literature review”, Information and Software Technology, Vol. 51 No. 1, pp. 7–15. http://doi.org/10.1016/j.infsof.2008.09.009Google Scholar
Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., Clarke, M., Devereaux, P.J., Kleijnen, J. and Moher, D. (2009), “The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration”, PLOS Medicine, Vol. 6 No. 7, p. e1000100, http://doi.org/10.1371/journal.pmed.1000100Google Scholar
McAlister, F.A., Clark, H.D., van Walraven, C., Straus, S.E., Lawson, F.M.E., Moher, D. and Mulrow, C.D. (1999), “The medical review article revisited: Has the science improved?”, Annals of Internal Medicine, Vol. 131 No. 12, pp. 947–951. http://doi.org/10.7326/0003-4819-131-12-199912210-00007Google Scholar
McMahon, C.A. (2012), “Reflections on diversity in design research”, Journal of Engineering Design, Vol. 23 No. 8, pp. 563–576. http://doi.org/10.1080/09544828.2012.676634Google Scholar
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G. and The, P.G. (2009), “Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement”, PLOS Medicine, Vol. 6 No. 7, p. e1000097, http://doi.org/10.1371/journal.pmed.1000097Google Scholar
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L.A. and Group, P.-P. (2015), “Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement”, Systematic Reviews, Vol. 4 No. 1. http://doi.org/10.1186/2046-4053-4-1Google Scholar
Mulrow, C.D. (1987), “The medical review article: State of the science”, Annals of Internal Medicine, Vol. 106 No. 3, pp. 485–488. http://doi.org/10.7326/0003-4819-106-3-485Google Scholar
Munn, Z., Stern, C., Aromataris, E., Lockwood, C. and Jordan, Z. (2018), “What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences”, BMC Medical Research Methodology, Vol. 18 No. 1, p. 5. http://doi.org/10.1186/s12874-017-0468-4Google Scholar
Murad, M.H., Asi, N., Alsawas, M. and Alahdab, F. (2016), “New evidence pyramid”, Evidence-Based Medicine, Vol. 21 No. 4, pp. 125–127. http://doi.org/10.1136/ebmed-2016-110401Google Scholar
Petticrew, M. (2001), “Systematic reviews from astronomy to zoology: myths and misconceptions”, BMJ, Vol. 322 No. 7278, pp. 98–101. http://doi.org/10.1136/bmj.322.7278.98Google Scholar
Petticrew, M. (2003), “Why certain systematic reviews reach uncertain conclusions”, BMJ, Vol. 326 No. 7392, pp. 756–758. http://doi.org/10.1136/bmj.326.7392.756Google Scholar
Rethlefsen, M.L., Farrell, A.M., Osterhaus Trzasko, L.C. and Brigham, T.J. (2015), “Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews”, Journal of Clinical Epidemiology, Vol. 68 No. 6, pp. 617–626. http://doi.org/10.1016/j.jclinepi.2014.11.025Google Scholar
Sackett, D.L., Rosenberg, W.M.C., Gray, J.A.M., Haynes, R.B. and Richardson, W.S. (1996), “Evidence based medicine: what it is and what it isn't”, BMJ, Vol. 312 No. 7023, pp. 71–72. http://doi.org/10.1136/bmj.312.7023.71Google Scholar
Saidani, M., Yannou, B., Leroy, Y., Cluzel, F. and Kendall, A. (2019), “A taxonomy of circular economy indicators”, Journal of Cleaner Production, Vol. 207 No., pp. 542–559, http://doi.org/10.1016/j.jclepro.2018.10.014Google Scholar
Seepersad, C.C., Pedersen, K., Emblemsvåg, J., Bailey, R., Allen, J.K. and Mistree, F. (2006), “The Validation Square: How Does One Verify and Validate a Design Method?”, in Lewis, K.E., Chen, W. and Schmidt, L.C. (eds.) Decision Making in Engineering Design. New York, NY: ASME.Google Scholar
Shepperd, S., Lewin, S., Straus, S., Clarke, M., Eccles, M.P., Fitzpatrick, R., Wong, G. and Sheikh, A. (2009), “Can We Systematically Review Studies That Evaluate Complex Interventions?”, PLOS Medicine, Vol. 6 No. 8, p. e1000086, http://doi.org/10.1371/journal.pmed.1000086Google Scholar
Shojania, K.G., Sampson, M., Ansari, M.T., Ji, J., Doucette, S. and Moher, D. (2007), “How quickly do systematic reviews go out of date? a survival analysis”, Annals of Internal Medicine, Vol. 147 No. 4, pp. 224–233. http://doi.org/10.7326/0003-4819-147-4-200708210-00179Google Scholar
Sio, U.N., Kotovsky, K. and Cagan, J. (2015), “Fixation or inspiration? A meta-analytic review of the role of examples on design processes”, Design Studies, Vol. 39 No., pp. 70–99. http://doi.org/10.1016/j.destud.2015.04.004Google Scholar
Tranfield, D., Denyer, D. and Smart, P. (2003), “Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review”, British Journal of Management, Vol. 14 No. 3, pp. 207–222. http://doi.org/10.1111/1467-8551.00375Google Scholar
van Aken, J.E. and Romme, G. (2009), “Reinventing the future: adding design science to the repertoire of organization and management studies”, Organization Management Journal, Vol. 6 No. 1, pp. 5–12. http://doi.org/10.1057/omj.2009.1Google Scholar