Skip to main content Accessibility help
×
Home

WHICH DATABASES SHOULD BE USED TO IDENTIFY STUDIES FOR SYSTEMATIC REVIEWS OF ECONOMIC EVALUATIONS?

  • Mick Arber (a1), Julie Glanville (a2), Jaana Isojarvi (a2), Erin Baragula (a2), Mary Edwards (a2), Alison Shaw (a2) and Hannah Wood (a2)...

Abstract

Objectives:

This study investigated which databases and which combinations of databases should be used to identify economic evaluations (EEs) to inform systematic reviews. It also investigated the characteristics of studies not identified in database searches and evaluated the success of MEDLINE search strategies used within typical reviews in retrieving EEs in MEDLINE.

Methods:

A quasi-gold standard (QGS) set of EEs was collected from reviews of EEs. The number of QGS records found in nine databases was calculated and the most efficient combination of databases was determined. The number and characteristics of QGS records not retrieved from the databases were collected. Reproducible MEDLINE strategies from the reviews were rerun to calculate the sensitivity and precision for each strategy in finding QGS records.

Results:

The QGS comprised 351 records. Across all databases, 337/351 (96 percent) QGS records were identified. Embase yielded the most records (314; 89 percent). Four databases were needed to retrieve all 337 references: Embase + Health Technology Assessment database + (MEDLINE or PubMed) + Scopus. Four percent (14/351) of records could not be found in any database. Twenty-nine of forty-one (71 percent) reviews reported a reproducible MEDLINE strategy. Ten of twenty-nine (34.5 percent) of the strategies missed at least one QGS record in MEDLINE. Across all twenty-nine MEDLINE searches, 25/143 records were missed (17.5 percent). Mean sensitivity was 89 percent and mean precision was 1.6 percent.

Conclusions:

Searching beyond key databases for published EEs may be inefficient, providing the search strategies in those key databases are adequately sensitive. Additional search approaches should be used to identify unpublished evidence (grey literature).

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      WHICH DATABASES SHOULD BE USED TO IDENTIFY STUDIES FOR SYSTEMATIC REVIEWS OF ECONOMIC EVALUATIONS?
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      WHICH DATABASES SHOULD BE USED TO IDENTIFY STUDIES FOR SYSTEMATIC REVIEWS OF ECONOMIC EVALUATIONS?
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      WHICH DATABASES SHOULD BE USED TO IDENTIFY STUDIES FOR SYSTEMATIC REVIEWS OF ECONOMIC EVALUATIONS?
      Available formats
      ×

Copyright

Footnotes

Hide All

This research received no specific grant from any funding agency or other body. YHEC is a consultancy company conducting systematic reviews of the effectiveness and cost-effectiveness of interventions and provides search services and training in search conduct.

Footnotes

References

Hide All
1.York York Health Economics Consortium. Economic evaluation [online]. 2016. http://www.yhec.co.uk/glossary/economic-evaluation/ (accessed September 30, 2018).
2.Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. 4th ed. 2017. https://www.cadth.ca/dv/guidelines-economic-evaluation-health-technologies-canada-4th-edition (accessed September 30, 2018).
3.National Institute for Health and Care Excellence. Methods for the development of NICE public health guidance. Process and methods [PMG4]. 3rd ed. 2012. https://www.nice.org.uk/process/pmg4/chapter/introduction (accessed September 30, 2018).
4.National Institute for Health and Care Excellence. Developing NICE guidelines: The manual. Process and methods [PMG20]. 2014. https://www.nice.org.uk/process/pmg20/chapter/introduction-and-overview (accessed September 30, 2018).
5.National Institute for Health and Care Excellence. Guide to the processes of technology appraisal. 2018. https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-guidance/NICE-technology-appraisals/technology-appraisal-processes-guide-apr-2018.pdf (accessed September 30, 2018).
6.Scottish Medicines Consortium. How we make our decisions. 2018. https://www.scottishmedicines.org.uk/how-we-decide/ (accessed September 30, 2018).
7.Borah, R, Brown, AW, Capers, PL, Kaiser, KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7:e012545.
8.Alton, V, Eckerlund, I, Norlund, A. Health economic evaluations: How to find them. Int J Technol Assess Health Care. 2006;22:512-517.
9.Pitt, C, Goodman, C, Hanson, K. Economic evaluation in global perspective: A bibliometric analysis of the recent literature. Health Econ. 2016;25(Suppl 1):9-28.
10.Royle, P, Waugh, N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess. 2003;7(34):iii, ix-x, 1-51.
11.Sassi, F, Archard, L, McDaid, D. Searching literature databases for health care economic evaluations: How systematic can we afford to be? Med Care. 2002;40:387-394.
12.Centre for Reviews and Dissemination. Undertaking the review. In: Centre for Reviews and Dissemination, editor. Systematic reviews: CRD's guidance for undertaking reviews in health care. York. University of York. 2009. https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/1_3_UNDERTAKING_THE_REVIEW.htm (accessed September 30, 2018).
13.Shemilt, I, Mugford, M, Byford, S, et al. Incorporating economics evidence. In: Higgins, JPT, Green, S, eds. Cochrane handbook for systematic reviews of interventions version 5.1.0. The Cochrane Collaboration. 2011. 449-80 www.handbook.cochrane.org (accessed September 30, 2018).
14.Wood, H, Arber, M, Glanville, JM. Systematic reviews of economic evaluations: How extensive are their searches? Int J Technol Assess Health Care. 2017;33:25-31.
15.Briscoe, S, Cooper, C, Glanville, J, Lefebvre, C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8:256-257.
16.Thielen, FW, Van Mastrigt, G, Burgers, LT, et al. How to prepare a systematic review of economic evaluations for clinical practice guidelines: Database selection and search strategy development (part 2/3). Expert Rev Pharmacoecon Outcomes Res. 2016;16:705-721.
17.Kaunelis, D, Glanville, J. Costs and economic evaluation. Summarized research in information retrieval for HTA. 2018. http://vortal.htai.org/?q=sure-info (accessed September 30, 2018).
18.Kaunelis, D, Farrah, K. Appraisal of: Pitt, C, Goodman, C, Hanson, K. Economic evaluation in global perspective: A bibliometric analysis of the recent literature. Health Econ. 2016;25(Suppl 1):9-28.
19.Sampson, M, Zhang, L, Morrison, A, et al. An alternative to the hand searching gold standard: Validating methodological search filters using relative recall. BMC Med Res Methodol. 2006;6:33.
20.Aagaard, T, Lund, H, Juhl, C. Optimizing literature search in systematic reviews - Are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16:161.
21.Ayiku, L, Levay, P, Hudson, T, et al. The medline UK filter: Development and validation of a geographic search filter to retrieve research about the UK from OVID medline. Health Info Libr J. 2017. doi:10.1111/hir.12187.
22.Selva, A, Sola, I, Zhang, Y, et al. Development and use of a content search strategy for retrieving studies on patients’ views and preferences. Health Qual Life Outcomes. 2017;15:126.
23.National Institute for Health and Care Excellence. Guide to the methods of technology appraisal [PMG9]. London: National Institute for Health and Care Excellence; 2013. https://www.nice.org.uk/process/pmg9/chapter/foreword (accessed September 30, 2018).
24.London School of Hygiene & Tropical Medicine, Public Health England, University College London. Evidence review & economic analysis of excess winter deaths for the National Institute for Health and Care Excellence (NICE). Review 2: Interventions and economic studies. Undated. https://www.nice.org.uk/guidance/NG6/documents/excess-winter-deaths-and-illnesses-review-22 (accessed September 30, 2018).
25.Lam, MT, De Longhi, C, Turnbull, J, Lam, HR, Besa, R. Has Embase replaced MEDLINE since coverage expansion? J Med Libr Assoc. 2018;106:227-234.
26.Lefebvre, C, Manheimer, E, Glanville, JM. Chapter 6: Searching for studies. In: Higgins, JPT, Green, S, eds. Cochrane handbook for systematic reviews of interventions [Version 5.1.0] [updated March 2011]. The Cochrane Collaboration. 2011. http://handbook-5-1.cochrane.org/.
27.Duffy, S, de Kock, S, Misso, K, et al. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE In-Process searches via Ovid. J Med Libr Assoc. 2016;104:309-312.
28.University of York, Centre for Reviews and Dissemination. Search strategies: NHS EED. http://www.crd.york.ac.uk/crdweb/searchstrategies.asp#nhseedmedline (accessed September 30, 2018).
29.McGowan, J, Sampson, M, Salzwedel, DM, et al. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40-6.
30.Meert, D, Torabi, N, Costella, J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. J Med Libr Assoc. 2016;104:267-277.
31.Koffel, JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: A cross-sectional survey of recent authors. PLoS One. 2015;10:e0125931.
32.Rethlefsen, ML, Farrell, AM, Osterhaus Trzasko, LC, Brigham, TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68:617-626.
33.Effective Practice and Organisation of Care (EPOC) Cochrane Group. A collection of databases, web sites and journals relevant to Low-and Middle-Income Countries (LMICs). 2013. http://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/LMIC%20Databases%20August%202013.pdf (accessed September 30, 2018).

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed