Skip to main content Accessibility help

Assessing the Quality of Randomized Controlled Trials: Current Issues and Future Directions

  • David Moher (a1), Alejandro R. Jadad (a2) and Peter Tugwell (a1)

Assessing the quality of randomized controlled trials is a relatively new and important development. Three approaches have been developed: component, checklist, and scale assessment. Component approaches evaluate selected aspects of trials, such as masking. Checklists and scales involve lists of items thought to be integral to study quality. Scales, unlike the other methods, provide a summary numeric score of quality, which can be formally incorporated into a systematic review. Most scales to date have not been developed with sufficient rigor, however. Empirical evidence indicates that differences in scale development can lead to important differences in quality assessment. Several methods for including quality scores in systematic reviews have been proposed, but since little empirical evidence supports any given method, results must be interpreted cautiously. Future efforts may be best focused on gathering more empirical evidence to identify trial characteristics directly related to bias in the estimates of intervention effects and on improving the way in which trials are reported.

Hide All
1.Ad Hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Annals of Internal Medicine, 1987, 106, 598604.
2.Andrew, E.Method for assessment of the reporting standard of clinical trials with Roentgen contrast media. Acta Radiologica Diagnosis, 1984, 25, 5558.
3.Andrew, E., Eide, H., Fuglerud, P., et al. Publications on clinical trials with x-ray contrast media: Differences in quality between journals and decades. European Journal of Radiology, 1990, 10, 9297.
4.Badgley, R. F.An assessment of research methods reported in 103 scientific articles from two Canadian medical journals. Canadian Medical Association Journal, 1961, 85, 246–50.
5.Beckerman, H., de Bie, R. A., Bouter, L. M., et al. The efficacy of laser therapy for musculoskeletal and skin disorders. In Beckerman, H. & Bouter, L. (eds.), Effectiviteit van fysiotherapie: Een literatuuronderzoek. Maastricht: Rijksuniversiteit Limburg, 1990, xxxx.
6.Bland, J. M., Jones, D. R., Bennett, S., et al. Is the clinical trial evidence about new drugs statistically adequate? British Journal of Clinical Pharmacology, 1985, 19, 155–60.
7.Brown, S. A.Measurement of quality of primary studies for meta-analysis. Nursing Research, 1991, 40, 352–55.
8.Brown, A. S.Meta-analysis of diabetes patient education research: Variations in intervention effects across studies. Research in Nursing & Health, 1992, 15, 409–19.
9.Chalmers, I.Preparing, maintaining, and disseminating systematic review of the effects of health care. Annals of the New York Academy of Sciences, 1993, 703, 156–65.
10.Chalmers, I., Adams, M., Dickersin, K., et al. A cohort study of summary reports of controlled trials. Journal of the American Medical Association, 1990, 263, 1401–05.
11.Chalmers, I., Hetherington, J., Elbourne, D., et al. Materials and methods used in synthesizing evidence to evaluate the effects of care during pregnancy and childbirth. In Chalmers, I., Enkin, M., & Keirse, M. J.n.c. (eds.), Effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989, xxxx.
12.Chalmers, T. C., Adams, M., Dickersin, K., et al. A method for assessing the quality of randomized control trial. Journal of the American Medical Association, 1981, 2, 3149.
13.Chalmers, T. C., Celano, P., Sacks, H. S., & Smith, H.Bias in treatment assignment in controlled clinical trials. New England Journal of Medicine, 1983, 309, 1358–61.
14.Chalmers, T. C., & Lau, J.Meta-analytic stimulus for changes in clinical trails. Statistical Methods in Medical Research, 1993, 2, 161–72.
15.Cho, M. K., & Bero, L. A.Instruments for reassessing the quality of drug studies published in the medical literature. Journal of the American Medical Association, 1994, 272, 101–04.
16.Colditz, G. A., Miller, J. N., & Mosteller, F.How study design affects outcomes in comparison of therapy. Medical Statistics in Medicine, 1989, 8, 441–54.
17.DerSimonian, R., Charette, L. J., McPeek, B., & Mosteller, F.Reporting on methods in clinical trials. New England Journal of Medicine, 1982, 306, 1332–37.
18.Detsky, A. S., Naylor, C. D., O’Rourke, K., et al. Incorporating variations in the quality of individual randomized trials into meta-analysis. Journal of Clinical Epidemiology, 1992, 45, 225–65.
19.Evans, M., & Pollack, A. V.A score system for evaluating random control clinical trials of prophylaxis of abdominal surgical wound infection. British Journal of Surgery, 1985, 72, 256–60.
20.Feinstein, A. R.Clinicmetrics. New Haven, CT: Yale University Press, 1987.
21.Fleiss, J. L., & Gross, A. J.Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: A critique. Journal of Clinical Epidemiology, 1991, 44, 127–39.
22.Gardner, M. J., Machin, D., & Campbell, M. J. Use of checklists in assessing the statistical content of medical studies. In Statistics with confidence: Confidence intervals and statistical guidelines. London: BMJ Publications, 1989, 101–08.
23.Goodman, S. N., Berlin, J., Fletcher, R. H., & Fletcher, S. W.Manuscript quality before and after peer review and editing. Annals of Internal Medicine, 1994, 121, 1121.
24.Gøtzsche, P.Methodology and overt and hidden bias: Reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis. Controlled Clinical Trials, 1989, 10, 3156 (erratum:356).
25.Grant, A.Reporting controlled trials. British Journal of Obstetrics and Gynaecology, 1989, 96, 397400.
26.Grégoire, G., Derderian, F., & Le Lorier, J.Selecting the language of the publications included in a meta-analysis: Is there a tower of babel bias? Journal of Clinical Epidemiology, 1995, 48, 158–63.
27.Haynes, R. B., Mulrow, C. D., Huth, E. J., et al. More informative abstracts revisited. Annals of Internal Medicine, 1990, 113 6976.
28.Imperiale, T. F., & McCulloughm, A. J.Do corticosteroids reduce mortality from alcoholic hepatitis? A meta-analysis of the randomised trials. Annals of Internal Medicine, 1990, 113, 299307.
29.Jadad-Bechara, A. R., Moore, R. A., & Carrol, D.Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials, 1996, 17, 112.
30.Jenicek, M.Meta-analysis in medicine: Where we are and where we want to go. Journal of Clinical Epidemiology, 1989, 42, 3544.
31.Kleijnen, J., Knipschild, P., & ter Riet, G.Clinical trials of homeopathy. British Medical Journal, 1991, 302, 316–23.
32.Koes, B. W., Assendelft, W. J. J., van der Heijden, G. J. M. G., et al. Spinal manipulation and mobilisation for back and neck pain: A blinded review. British Medical Journal, 1991, 303, 1298–303.
33.Levine, J.Trial assessment procedure scale (TAPS). Bethesda, MD: Department of Health and Human Services, National Institute of Mental Health, 1980.
34.Lionel, N. D. W., & Herxheimer, A.Assessing reports of therapeutic trials. British Medical Journal, 1970, 3, 637–40.
35.Mahon, W. A., & Daniel, E. E.A method for the assessment of reports of drug trials. Canadian Medical Association Journal, 1964, 90, 565–69.
36.McDowell, I., & Newell, C.Measuring health: A guide to rating scales and questionnaires. New York: Oxford University Press, 1987.
37.Meinert, C. L.Clinical trials, design, conduct, and analysis. New York: Oxford University Press, 1986.
38.Moher, D., Jadad, A. R., Nichol, G., et al. Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, in press.
39.Nurmohame, M. T., Rosendaal, F. R., Buller, H. R., et al. Low-molecular-weight heparin versus standard heparin in general and orthopaedic surgery: A meta-analysis. Lancet, 1992, 340, 152–56.
40.Onghenia, P., & Van Houdenhove, B.Antidepressants-induced analgesia in chronic nonmalignant pain: A meta-analysis of 39 placebo-controlled studies. Pain, 1992, 49, 205–19.
41.Pocock, S. J.Clinical trials: A practical approach. Chichester, UK: John Wiley & Sons, 1983.
42.Pocock, S. J., Hughes, M. D., & Lee, R. J.Statistical problems in the reporting of clinical trials: A survey of three medical journals. New England Journal of Medicine, 1987, 317, 426–32.
43.Powe, N. R., Kinnison, M. L., & Steinberg, E. P.Quality assessment of randomized controlled trials of contrast media. Radiology, 1989, 170, 377–80.
44.Poynard, T.Evaluation de la qualité méthodologique des essais therapeutiques randomisés. La Presse Medicate, 1988, 17, 315–18.
45.Reisch, J. S., Tyson, J. E., & Mize, S. G.Aid to the evaluation of therapeutic studies. Pediatrics, 1989, 84, 815–27.
46.Sandercock, P. A. G., van den Belt, A. G. M., Lindley, R. I., & Slattery, J.Antithrombotic therapy in acute ischaemic stroke: An overview of the completed randomized trials. Journal of Neurology, Neurosurgery, and Psychiatry, 1993, 56, 1725.
47.Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G.Failure to conceal intervention allocation schedules in trials influenced estimates of treatment effects. Controlled Clinical Trials, 1994, 15, 63S.
48.Smith, K., Cook, D., Guyatt, G. H., et al. Respiratory muscle training in chronic airflow limitation: A meta-analysis. American Review of Respiratory Disease, 1992, 145, 533–39.
49.Spitzer, W. O., Lawrence, V., Dales, R., et al. Links between passive smoking and disease: A best evidence synthesis. Clinical and Investigative Medicine, 1990, 13, 1742.
50.Standards of Reporting Trials Group. A proposal for structured reporting of randomized controlled trials. Journal of the American Medical Association, 1994, 272, 1926–31.
51.Steiner, D. L., & Norman, G. R.Health measurement scales: A practical guide to their development and use. Oxford: Oxford University Press, 1989.
52.Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment of pulmonary tuberculosis: A Medical Research Council investigation. British Medical Journal, 1948, 11, 769–82.
53.Ter Riet, G., Kleijnen, J., & Knipschild, P.Acupuncture and chronic pain: A criteria-based meta-analysis. Journal of Clinical Epidemiology, 1990, 43, 1191–99.
54.Thomson, M. E., & Kramer, M. S.Methodologic standards for controlled clinical trials of early contact and maternal-infant behavior. Pediatrics, 1984, 73, 294–200.
55.Weintraub, M.How to critically assess clinical drug trials. Drug Therapy, 1982, 12, 131–48.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

International Journal of Technology Assessment in Health Care
  • ISSN: 0266-4623
  • EISSN: 1471-6348
  • URL: /core/journals/international-journal-of-technology-assessment-in-health-care
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed