Skip to main content Accessibility help

An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care

  • C. E. Adams (a1), A. Power (a1), K. Frederick (a1) and C. Lefebvre (a1)


Valid reviews of the effects of mental health care depend on identifying as high a proportion as possible of relevant randomized controlled trials (RCTs). To investigate the sensitivity and precision both of MEDLINE and of hand-searching for RCTs in mental health, 12 journals specializing in mental health and indexed by the National Library of Medicine (NLM) for MEDLINE were searched for the years 1971, 1976, 1981, 1986 and 1991. The sensitivity of the hand-search was 94% (95% Confidence Interval (CI) 93–95%), but it had a precision of only 7% (CI 6–8%). The optimal MEDLINE search had a sensitivity of only 52% (CI 48–56%) and precision of 59% (CI 55–63%). Of the reports of RCTs identified by the hand-search, 9% (CI 7–11 %) were not included in MEDLINE at all. Authors had included methodological descriptions in 84% (CI 80–88%) of RCTs found by the hand-search but missed by the MEDLINE search. Systematic reviews of mental health care which are based solely on MEDLINE searches of the literature will miss a large proportion of the relevant RCTs, and are thus liable to random error and bias. A register of mental health RCTs is urgently required.


Corresponding author

1Address for correspondence: Dr Clive E. Adams, University of Oxford, Department of Psychiatry, Warneford Hospital, Oxford OX3 7JX


Hide All
Adams, C. E. & Gelder, M. G. G. (1994). The case for establishing a register of randomized controlled trials of mental health care. British Journal of Psychiatry 164, 433436.
Adams, C. E., Lefebvre, C. & Chalmers, I. (1992). Difficulty with MEDLINE searches for randomised controlled trials. Lancet 340, 915916.
Antman, E. M., Lau, J., Kupelnick, B., Mosteller, F. & Chalmers, T. C. (1992). A comparison of results of meta-analysis of randomized controlled trials and recommendations of clinical experts. Treatment for myocardial infarction. Journal of the American Medical Association 268, 240248.
Balestrieri, M., Williams, P. & Wilkinson, G. (1988). Specialist mental health treatment in general practice: a meta-analysis. Psychological Medicine 18, 711717.
Bareta, J. C., Larson, D. B., Lyons, J. S. & Zorc, J. J. (1990). A comparison of manual and MEDLARS reviews of the literature in consultation-liaison psychiatry. American Journal of Psychiatry 147, 10401042.
Bernstein, F. (1988). The retrieval of randomized controlled trials in liver diseases from the medical literature: manual versus MEDLARS searches. Controlled Clinical Trials 9, 2331.
Chalmers, I., Hetherington, J., Newdick, M., Mutch, L., Grant, A., Enkin, M., Enkin, E. & Dickersin, K. (1986). The Oxford Register of Perinatal Trials: developing a register of published reports of controlled trials. Controlled Clinical Trials 7, 306324.
Chalmers, I., Enkin, M. & Keirse, M. J. N. C. (eds.) (1989). Effective Care in Pregnancy and Childbirth. Oxford University Press: Oxford.
Chalmers, I., Adams, M., Dickersin, K., Hetherington, J., Tarnow-Mordi, W., Meinert, C., Tonascia, S. & Chalmers, T. C. (1990). A cohort study of summary reports of controlled trials. Journal of the American Medical Association 263, 14011405.
Chalmers, I., Dickersin, K. & Chalmers, T. C. (1992). Getting to grips with Archie Cochrane's agenda. British Medical Journal 305, 786788.
Cochrane Pregnancy and Childbirth Database (1994). Update Software, Oxford.
Collins, R., Peto, R., MacMahon, S., Hebert, P., Feibach, N. H., Eberlin, K. A., Goodwin, J., Qizilbash, N., Taylor, J. O. & Hennekens, C. H. (1990). Blood pressure, stroke and CHD. Part 2. Overview of randomized drug trials in the epidemiological context. Lancet 335, 827838.
Depression Guideline Panel (1993). Depression in Primary Care: Volume 2, Treatment of Major Depression. Clinical Practice Guideline, No. 5. US Department of Health and Human Services. Public Health Service. Agency for Health Care Policy and Research. AHCPR Publication No. 93–0551: Rockville, MD.
Dickersin, K. & Min, Y. I. (1993). NIH clinical trials and publication bias [article]. Online Journal of Current Clinical Trials [serial online] Apr 28; 1993 (Doc. No. 50).
Dickersin, K., Hewitt, P., Mutch, L., Chalmers, I. & Chalmers, T. C. (1985). Comparison of MEDLINE searching with a perinatal trial database. Controlled Clinical Trials 6, 306317.
Dickersin, K., Min, Y. I. & Meinert, C. L. (1992). Factors influencing publication of research results: follow-up of applications submitted to two substantial review boards. Journal of the American Medical Association 267, 374378.
Dickersin, K., Scherer, R. & Lefebvre, C. (1994). Identification of relevant studies for systematic reviews. British Medical Journal (in the press).
Drug and Therapeutics Bulletin (1992). Systematic overviews of controlled trials help clarify treatment effects. 30, 2527.
Easterbrook, P. J., Berlin, J. A., Gopalan, R. & Matthews, D. R. (1991). Publication bias in clinical research. Lancet 337, 867872.
Gøtzsche, P. C. & Lange, B. (1991). Comparison of search strategies for recalling double-blind trials from MEDLINE. Danish Medical Bulletin 38, 476479.
Hofmans, E. A. (1990). The results of a MEDLINE search. The accessibility of research on the effectiveness of acupuncture, II. Huisarts Wet 33, 103106.
House of Commons Health Committee (1992). Maternity Services. (2nd report) vol. 1. p. xlix. HMSO: London.
Jadad, A. R. & McQuay, H. J. (1993 a). A high-yield strategy to identify randomized controlled trials for systematic reviews. Online Journal of Current Clinical Trials [serial online], Doc. No. 33.
Jadad, A. R. & McQuay, H. J. (1993 b). Be systematic in your searching. British Medical Journal 307, 66.
Kirpalani, H., Schmidt, B., McKibbon, K. A., Haynes, B. & Sinclair, J. (1989). Searching MEDLINE for randomised controlled trials involving care of the newborn. Paediatrics 83, 543546.
Kleijnen, J. & Knipschild, P. (1992). The comprehensiveness of MEDLINE and EMBASE computer searches. Pharmaceutisch Weekblad Scientific Edition 14, 316320.
Lancet (1991). Making clinical trialists register. Lancet 338, 244245.
Meinert, C. L. (1988). Toward prospective registration of clinical trials. Controlled Clinical Trials 9, 15.
Poynard, T. & Conn, H. O. (1985). The retrieval of randomised clinical trials in liver disease from medical literature. Controlled Clinical Trials 6, 271279.
Scherer, R. & Dickersin, K. (1992). Publication of randomized clinical trials in vision research submitted as abstracts to national ophthalmology meetings. Controlled Clinical Trials 13, 421.
Schuyler, P. L., Dickersin, K., Scherer, R. & Wright, N. D. (1993). Identification of randomized clinical trials using MEDLINE.Presented at the 2nd International Congress on Peer Review,10 September, 1993.
Silagy, C. (1993). Developing a register of randomized controlled trials in primary care. British Medical Journal 306, 897900.
Simes, R. J. (1986). Publication bias: the case for an international registry of clinical trials. Journal of Clinical Oncology 4, 15291541.
WHO Scientific Group on Treatment of Psychiatric Disorders (1991). Evaluation of Methods for the Treatment of Mental Disorders. WHO: Geneva.

Related content

Powered by UNSILO

An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care

  • C. E. Adams (a1), A. Power (a1), K. Frederick (a1) and C. Lefebvre (a1)


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.