Clinical practice should be evidence based, but the hard choices arise when trying to decide which evidence to use. Evidence is often gathered from hospital protocols, textbooks and national guidance and these provide suggestions to improve practice, but are often not based upon hard evidence. It can be disconcerting for practitioners to discover that, in order to find evidence of an effective practice, they may need to plough through tens, hundreds or thousands of reports of primary studies, and at the end of this process the evidence they have found may have contradictory results.
A systematic review aims to collate all of the available evidence and produce a clear and more consistent answer. If available, the best source of evidence to base clinical practice upon is the systematic review. This is because systematic reviews are a better source of evidence compared with other (non-systematic or selective) reviews, since the methods used to construct them aim to reduce both conscious and unconscious bias (usually in favour of the active or new intervention being tested). It is well documented that researchers and health professionals remember (and cite) studies where an effect was seen (i.e. that ‘worked’), but studies where no effect was seen (i.e. that did not ‘work’) are harder to sourceReference Glasziou, Irwig, Bain and G1. It is worth highlighting that systematic reviews and meta-analyses are not always the same thing; a systematic review can be carried out without performing a meta-analysis, and equally one can perform a meta-analysis using the results from studies which have been selected using a methodology which is not systematic.
The Centre for Review and Dissemination2 (based at the University of York) defines a systematic review as ‘a review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review.’ The unit of analysis is the primary study, and the same scientific principles and rigour apply to systematic reviewing as are applied to good quality primary research. If a review does not state clearly whether and how all relevant studies were identified and synthesised it is not a systematic review, and an excellent way to determine if a review is a ‘systematic review’ or not, is whether it has an explicit methods sectionReference Summerbell3. Without such a section a systematic review is unlikely to be publishedReference Summerbell3. In addition, the question considered by the systematic review must be clinically relevant.
The majority of systematic reviews on topics related to the effectiveness of health care are conducted under the auspices of the Cochrane Collaboration4, 5 and published in the Cochrane Library6, which is consequently the best source of good-quality systematic reviews. The mission of the Cochrane Collaboration is to prepare, maintain and disseminate systematic reviews of the effects of health care. The Cochrane Library comprises several individual databases including the Cochrane Database of Systematic Reviews (CDSR), the CRD Database of Abstracts of Reviews of Effectiveness (DARE), the Cochrane Controlled Trials Registers (CCTR) and the Cochrane Review Methodology database. Systematic reviews (both completed and planned) can be found in CDSR and DARE, good but unsystematic reviews can be found in DARE, and details of randomised controlled trials (identified through hand searching) can be found in CCTR. Systematic reviews can also be found elsewhere; particularly searching Medline and applying a ‘review’ filter or more quickly (but less precisely) using ‘meta-analysis’ as a MeSH term and a publication type or even ‘googling’ using the topic and the term ‘systematic review’.
What are the steps involved in writing a systematic review?
In terms of writing a good quality systematic review, there are protocols and guides available, but a good starting point is to look at how Cochrane reviews are carried out as these are of an extremely high quality. The steps involved are as follows:
1 Definition of the review question; this directs all of the work that the author(s) carry out, including the writing of the search strategy, the collection of studies and the process of examining studies for eligibility.
2 Writing, and approval, of the protocol; including the exact search strategy that will be used, definition of the inclusion and exclusion criteria, and specification of the outcome measures to be considered.
3 Operation of search strategy; the searching should take place in, at the very least, the following places: academic databases, the foreign language literature, ‘grey literature’, reference lists from the primary papers, other unpublished studies, and raw data sources known to experts in the field.
4 Study assessment; all of the papers considered for a systematic review are assessed using the same inclusion/exclusion criteria, and once included a data extraction form is completed for each included study.
5 Creation of meta-analyses; if the data collected for the review are of sufficient quality and similarity, they can be summarised statistically in a meta-analysis, which can provide a more powerful estimate of clinical effect than the individual results from individual studies.
6 Putting results of review into context; when all of the calculations have been completed, it is important to have the members of the review team who are active in the field of study to make sure firstly that the data have been interpreted correctly, secondly that the conclusions reached should be recommended to be implemented, and thirdly that these conclusions can be put into context alongside current thinking.
The British Journal of Nutrition is now publishing systematic reviews
It is ironic that, although the first published systematic review in the area of nutrition was in 1953Reference Stewart and Guthrie7, some nutritional recommendations and dietetic practices are still not underpinned by systematically reviewed evidence. In 1992, Garrow wrote an editorialReference Garrow8 on the subject of meta-analyses in clinical nutrition, and at that time, very few systematic reviews relating to nutrition and dietetics had been conducted. By 2005, the number had substantially increased, and systematic reviews around dietetics are beginning to appear in the Cochrane LibraryReference Summerbell, Waters, Edmunds, Kelly, Brown and Campbell9–Reference Moore, Summerbell, Hooper, Ashton and Kopelman11. However, there is clear need for many more systematic reviews in order to strengthen the evidence base for practice. Furthermore, as scientific fields advance and new studies emerge, there is a need to update the evidence base and so areas previously subject to systematic review will need to be reviewed again periodically, depending upon the rate of emergence of new primary studies or changing clinical contexts. In recognition of the important role of systematic reviews, the British Journal of Nutrition has now begun publishing themReference Kamphuis, Giskes, de Bruijn, Wendel-Vos, Brug and van Lenthe12–Reference Rennie and Livingstone15 and one of the authors (C. D. S.) has been appointed as Systematic Reviews Editor.
