Hostname: page-component-848d4c4894-p2v8j Total loading time: 0.001 Render date: 2024-05-14T11:10:54.024Z Has data issue: false hasContentIssue false
  • English
  • Français

The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps

Part of: Special Collection - Methods

Published online by Cambridge University Press:  27 October 2020

Lynda Ayiku Open the ORCID record for Lynda Ayiku [Opens in a new window] ,
Thomas Hudson ,
Sarah Glover ,
Nicola Walsh ,
Rachel Adams ,
Jemma Deane  and
Amy Finnegan
Lynda Ayiku*
Affiliation:
National Institute for Health and Care Excellence (NICE), Level 1a, City Tower, Piccadilly Plaza, Manchester, M1 4BT, UK
Thomas Hudson
Affiliation:
National Institute for Health and Care Excellence (NICE), Manchester, UK
Sarah Glover
Affiliation:
National Institute for Health and Care Excellence (NICE), Manchester, UK
Nicola Walsh
Affiliation:
National Institute for Health and Care Excellence (NICE), Manchester, UK
Rachel Adams
Affiliation:
National Institute for Health and Care Excellence (NICE), Manchester, UK
Jemma Deane
Affiliation:
National Institute for Health and Care Excellence (NICE), London, UK
Amy Finnegan
Affiliation:
National Institute for Health and Care Excellence (NICE), Manchester, UK
*
Author for correspondence: Lynda Ayiku, E-mail: lynda.ayiku@nice.org.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives

Health apps are software programs that are designed to prevent, diagnose, monitor, or manage conditions. Inconsistent terminology for apps is used in research literature and bibliographic database subject headings. It can therefore be challenging to retrieve evidence about them in literature searches. Information specialists at the United Kingdom's National Institute for Health and Care Excellence (NICE) have developed novel validated search filters to retrieve evidence about apps from MEDLINE and Embase (Ovid).

Methods

A selection of medical informatics journals was hand searched to identify a “gold standard” (GS) set of references about apps. The GS set was divided into a development and validation set. The filters’ search terms were derived from and tested against the development set. An external development set containing app references from published NICE products was also used to inform the development of the filters. The filters were then validated using the validation set. Target recall was >90 percent. The filters’ overall recall, specificity, and precision were calculated using all the references identified from the hand search.

Results

Both filters achieved 98.6 percent recall against their validation sets. Overall, the MEDLINE filter had 98.8 percent recall, 71.3 percent specificity, and 22.6 percent precision. The Embase filter had 98.6 percent recall, 74.9 percent specificity, and 24.5 percent precision.

Conclusions

The NICE health apps search filters retrieve evidence about apps from MEDLINE and Embase with high recall. They can be applied to literature searches to retrieve evidence about the interventions by information professionals, researchers, and clinicians.

Keywords

Bibliographiccomputer-assisteddatabasesinformation scienceinformation storage and retrievalmobile applicationstherapy
Type
Method
Information
International Journal of Technology Assessment in Health Care , Volume 37 , Issue 1 , 2021 , e16
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

National Institute for Health and Care Excellence [Internet] Evidence standards framework for digital health technologies: user guide. 2019 [cited 2020 Sept 14]. Available from: https://www.nice.org.uk/Media/Default/About/what-we-do/our-programmes/evidence-standards-framework/user-guide.pdfGoogle Scholar
Dillon, B, Powell, J, Campbell, M, Salmon, M. OP123 Health technology assessment In digital health: A rapid approach To assess health apps. Int J Technol Assess Health Care. [Internet]. 2018 Jan [cited 2020 Sept 14];33:56. Available from: https://www.cambridge.org/core/journals/international-journal-of-technology-assessment-in-health-care/article/op123-health-technology-assessment-in-digital-health-a-rapid-approach-to-assess-health-apps/2C09299F0A716800F65F1D1432813EB8CrossRefGoogle Scholar
Medicines and Healthcare products Regulatory Agency [Internet] Medical devices: software applications (apps). 2018 [cited 2020 Sept 14]. Available from: https://www.gov.uk/government/publications/medical-devices-software-applications-appsGoogle Scholar
Barnes, CM, Miller, JA, Bostock, S. Helping employees sleep well: Effects of cognitive behavioral therapy for insomnia on work outcomes. J Appl Psychol. [Internet]. 2017 Jan [cited 2020 Sept 14];102:104–13. Available from: https://content.apa.org/record/2016-47187-001CrossRefGoogle ScholarPubMed
McGrath, ER, Espie, CA, Power, A, Murphy, AW, Newell, J, Kelly, C et al. Sleep to lower elevated blood pressure: A randomized controlled trial (slept). Am J Hypertens. [Internet]. 2017 Jan [cited 2020 Sept 14];30: 319–27. Available from: https://academic.oup.com/ajh/article/30/3/319/2927604CrossRefGoogle Scholar
Bourne, S, DeVos, R, North, M, Chauran, A, Green, B, Brown, T et al. Online versus face-to-face pulmonary rehabilitation for patients with chronic obstructive pulmonary disease: Randomised controlled trial. BMJ Open. [Internet]. 2017 Jul [cited 2020 Sept 14];7: no pagination. Available from: https://bmjopen.bmj.com/content/7/7/e014580CrossRefGoogle ScholarPubMed
Glanville, JM, Bayliss, S, Booth, A, Dundar, Y, Fernandes, H, Fleeman, ND et al. So many filters, so little time: The development of a search filter appraisal checklist. J Med Libr Assoc. [Internet]. 2018 Oct [cited 2020 Sept 14];96: 356–61. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2568852/CrossRefGoogle Scholar
Jenkins, M. Evaluation of methodological search filters: A review. Health Info Libr J. [Internet]. 2004 Aug [cited 2020 Sept 14];21:148–63. Available from: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1471-1842.2004.00511.xCrossRefGoogle ScholarPubMed
ISSG Search Filter Resource [Internet]. The InterTASC Information Specialists’ Sub-Group Search Filter Resource. 2020 [cited 2020 Sept 14] Available from: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/homeGoogle Scholar
Beynon, R, Leeflang, MM, McDonald, S, Eisinga, A, Mitchell, RL, Whiting, P et al. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev. [Internet]. 2013 Sept [cited 2020 Sept 14];(9):Mr000022. Available from: doi:10.1002/14651858.MR000022.pub3Google ScholarPubMed
Lefebvre, C, Glanville, J, Briscoe, S, Littlewood, A, Marshall, C, Metzendorf, M et al. [Internet]. Chapter 4: Searching for and selecting studies. In Higgins, JPT, Thomas, J, Chandler, J et al. . (eds), Cochrane handbook for systematic reviews of interventions, Second Edition. 2019, London: Cochrane [cited 2020 Sept 14]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1002/9781119536604.ch4Google Scholar
National Center for Biotechnology Information, U.S. National Library of Medicine [Internet] NLM Catalog. 2020 [cited 2020 Sept 14]. Available from: https://www.ncbi.nlm.nih.gov/nlmcatalog?term=Medical+Informaticspercent5Bstpercent5DGoogle Scholar
Randomness and Integrity Services Ltd [Internet] RANDOM.ORG. 2020 [cited 2020 Sept 14]. Available from: https://www.random.org/Google Scholar
Sampson, M, Zhang, L, Morrison, A, Barrowman, NJ, Clifford, TJ, Platt, RW et al. An alternative to the hand searching gold standard: Validating methodological search filters using relative recall. BMC Med Res Methodol. [Internet]. 2006 Jul [cited 2020 Sept 14];6: 33. Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-6-33CrossRefGoogle ScholarPubMed
DataBasic.io [Internet] DataBasic Word Counter. 2020 [cited 2020 Sept 14]. Available from: https://DataBasic.io/en/wordcounter/Google Scholar
Lowres, N, Neubeck, L, Salkeld, G, Krass, I, McLachlan, AJ, Redfern, J et al. Feasibility and cost-effectiveness of stroke prevention through community screening for atrial fibrillation using iPhone ECG in pharmacies: The SEARCH-AF study. Thromb Haemost. [Internet]. 2014 [cited 2020 Sept 14];111:1167–76. Available from: https://www.thieme-connect.de/products/ejournals/abstract/10.1160/TH14-03-0231CrossRefGoogle ScholarPubMed
Marks, IM, Kenwright, M, McDonough, M, Whittaker, M. Saving clinicians’ time by delegating routine aspects of therapy to a computer: A randomized controlled trial in phobia/panic disorder. Psychol Med. [Internet]. 2004 Jan [cited 2020 Sept 14];34:9–17. Available from: https://www.cambridge.org/core/journals/psychological-medicine/article/saving-clinicians-time-by-delegating-routine-aspects-of-therapy-to-a-computer-a-randomized-controlled-trial-in-phobiapanic-disorder/C41FCAFD73639A4E217CC1983ADE4561CrossRefGoogle ScholarPubMed
Lefebvre, C, Glanville, J, Beale, S, Boachie, C, Duffy, S, Fraser, C et al. Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: Five literature reviews and a qualitative study. Health Technol Assess. [Internet]. 2017 Nov [cited 2020 Sept 14];21:1–148. Available from: https://www.journalslibrary.nihr.ac.uk/hta/hta21690#/full-reportCrossRefGoogle ScholarPubMed
Supplementary material: File

Ayiku et al. supplementary material

Ayiku et al. supplementary material 1
Download Ayiku et al. supplementary material(File)
File 29.3 KB
Supplementary material: File

Ayiku et al. supplementary material

Ayiku et al. supplementary material 2
Download Ayiku et al. supplementary material(File)
File 24.4 KB
Supplementary material: File

Ayiku et al. supplementary material

Ayiku et al. supplementary material 3
Download Ayiku et al. supplementary material(File)
File 29.4 KB
Supplementary material: File

Ayiku et al. supplementary material

Ayiku et al. supplementary material 4
Download Ayiku et al. supplementary material(File)
File 25.9 KB
Supplementary material: File

Ayiku et al. supplementary material

Ayiku et al. supplementary material 5
Download Ayiku et al. supplementary material(File)
File 23.3 KB