Background
Rationale
A systematic review is a form of synthesis research that collates evidence pertaining to a specific research question, and uses standardized methods to identify, analyze, and report research that is relevant to that question (European Food Safety Authority, 2010). Results from independent studies included in a systematic review can be analyzed in several ways, including through meta-analysis, which involves the use of statistical methods to generate a summary effect measure or to investigate heterogeneity between studies (European Food Safety Authority, 2010; O'Connor et al., Reference O'Connor, Sargeant and Wang2014). Systematic reviews and meta-analyses have become a standard form of high-level evidence synthesis for evidence-based decision-making in the field of human healthcare (Higgins and Green, Reference Higgins and Green2011; Khan et al., Reference Khan, Kunz, Kleijnen and Antes2011). These methods generate a concise synthesis of quality-assessed scientific literature that relates to a particular clinical question or problem, allowing clinicians access to a substantial amount of relevant data in a single source (Khan et al., Reference Khan, Kunz, Kleijnen and Antes2011; Sargeant and O'Connor, Reference Sargeant and O'Connor2014). The use of these methods in the areas of veterinary medicine, livestock performance, and food safety has been growing, but the full extent of their application is unknown (Sargeant et al., Reference Sargeant, Rajić, Read and Ohlsson2006; Sargeant and O'Connor, Reference Sargeant and O'Connor2014).
Scoping reviews are an increasingly common method of reviewing and collating information about a field of research (Levac et al., Reference Levac, Colquhoun and O'Brien2010; Daudt et al., Reference Daudt, van Mossel and Scott2013; Pham et al., Reference Pham, Rajić, Greig, Sargeant, Papadopoulos and McEwen2014). A scoping review differs from a systematic review in that the former aims to provide an overview of a potentially broad body of literature without critically appraising or extracting specific results from any individual study, whereas the goal of the latter is to synthesize and evaluate evidence pertaining to a more focused research question (Arksey and O'Malley, Reference Arksey and O'Malley2005; Pham et al., Reference Pham, Rajić, Greig, Sargeant, Papadopoulos and McEwen2014). A statistical synthesis technique known as a meta-analysis can sometimes be applied to the data extracted through a systematic review to provide an overall summary effect measure and the precision of the estimate (O'Connor et al., Reference O'Connor, Sargeant and Wang2014). A scoping review approach was adopted to provide such a description and mapping of the use of systematic reviews and meta-analyses in the fields of animal health, performance, and on-farm food safety.
A scoping study is designed to ‘map the literature on a particular topic or research area and provide an opportunity to identify key concepts; gaps in the research; and types and sources of evidence to inform practice, policymaking, and research’ (Daudt et al., Reference Daudt, van Mossel and Scott2013; Pham et al., Reference Pham, Rajić, Greig, Sargeant, Papadopoulos and McEwen2014). Scoping studies may be conducted to: (1) describe the extent and nature of research in a particular field; (2) to determine the value and feasibility of undertaking a systematic review; (3) to summarize and present research results; or (4) to identify gaps in the existing literature (Arksey and O'Malley, Reference Arksey and O'Malley2005).
Objectives
The first and fourth of the aforementioned goals align most closely with the purpose of the present study. Systematic reviews and meta-analyses are important tools readily available in the human health literature, but their use is more recent and increasingly common in animal agriculture and veterinary medicine (Sargeant and O'Connor, Reference Sargeant and O'Connor2014). It is therefore important to explore the nature and availability of these forms of evidence for animal health decision-making. To our knowledge this has not yet been done in a rigorous and systematic manner. Our objective evolved out of a desire to understand the topics of systematic reviews and meta-analyses in veterinary medicine and animal agriculture over the past 25 years. The purpose of this scoping study was thus to inventory existing systematic reviews related to animal health, and to identify areas where such reviews do not yet exist. The scoping question was, ‘What are the characteristics and focus of systematic reviews and meta-analyses published in the fields of animal health, animal performance, and on-farm or harvest-level food safety over the past 25 years?.’
Methods
In 2005, Arksey and O'Malley (Arksey and O'Malley, Reference Arksey and O'Malley2005) published the first comprehensive framework for conducting scoping reviews. The present review followed that framework with minor modifications, including those suggested by Levac et al. (Reference Levac, Colquhoun and O'Brien2010). The reporting of this study is in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews (PRISMA-ScR) (Tricco et al., Reference Tricco, Lillie, Zarin, O'Brien, Colquhoun, Levac, Moher, Peters, Horsley, Weeks, Hempel, Akl, Chang, McGowan, Stewart, Hartling, Aldcroft, Wilson, Garritty, Lewin, Godfrey, Macdonald, Langlois, Soares-Weiser, Moriarty, Clifford, Tunçalp and Straus2018).
Arksey and O'Malley (Arksey and O'Malley, Reference Arksey and O'Malley2005) outlined five steps to conducting a scoping review: first, the research question is established; then eligibility criteria for study inclusion are identified; next, relevant studies are selected; then the data derived from those studies are charted and mapped; and finally the results are summarized and reported. Arksey and O'Malley (Arksey and O'Malley, Reference Arksey and O'Malley2005) add an optional sixth step, a consultation with outside experts and stakeholders, which some researchers advocate should be a mandatory practice (Daudt, van Mossel and Scott, Reference Daudt, van Mossel and Scott2013; Levac et al., Reference Levac, Colquhoun and O'Brien2010). Despite the potential benefits of such a consultation exercise (Arksey and O'Malley, Reference Arksey and O'Malley2005), due to time and resource limitations no such consultation was conducted for this study.
A research team with experience in evidence synthesis and epidemiology was established to identify the research question and the project purpose. This team advised on the development of the search strategy, the eligibility criteria for review inclusion, and the data to be extracted and characterized. Members of the team conducted the search, screened the identified papers, characterized the reviews, and discussed the results obtained from the included reviews.
Protocol and registration
A scoping review protocol was not established at the beginning of the study, but the project objectives and eligibility criteria were decided a priori.
Eligibility criteria
Articles were eligible for inclusion in the review if they included the terms ‘systematic review’ or ‘meta-analysis’ in the title or abstract. Articles published between 1 January 1993 and the search date were eligible for inclusion, capturing reviews published over the past 25 years. Reviews must have examined at least one outcome related to animal health, animal performance, or on-farm or harvest-level food safety. Reviews must have targeted domestic livestock, companion animals, or wildlife, either at the herd or animal level; or in the case of an animal product, at the farm level or at slaughter prior to processing. Additionally, reviews were included if they considered animal health-related pathogens in samples taken from live animals, on-farm animal products, or at slaughter. No restrictions were placed on language or publication type in the search, although all search terms were entered in English. Systematic reviews and/or meta-analyses that examined only animal models of human disease(s) were not eligible, as the primary outcome of these reviews concerned human rather than animal health. Similarly, reviews of animal-related environmental outcomes (e.g. methane emissions) were excluded, as these predominantly impact climate change and human health.
Information sources
Three electronic databases were searched for potentially relevant articles: MEDLINE via PubMed, CAB Direct, and AGRICOLA (via ProQuest). This review focused on the existing published literature, and so no grey literature sources were searched.
Search
The literature search was conducted on 28 May and 29 May 2018. We searched for systematic reviews and/or meta-analyses together with a list of terms related to livestock, companion animals, and wildlife. Table 1 shows the full search string, which was modified appropriately for each database including MeSH terms where appropriate.
Search string to identify systematic reviews or meta-analyses in the animal health, performance, or food safety literature
Citations identified by the search were imported into EndNote X7.8 (Thomson Reuters, Toronto, ON) for duplicate removal, and then imported into DistillerSR (Evidence Partners Incorporated, Ottawa, ON), a web-based systematic review program, for eligibility screening and data characterization. Additional duplicate citations identified in DistillerSR were also removed from the list of abstracts subject to screening.
Data charting process
Relevance screening was conducted in two stages: an initial relevance screening of the title and abstract, and a second, more in-depth screening and characterization. In the first screening round, two independent reviewers examined each title and abstract and answered the following questions: 1. Does the title or abstract contain the term systematic review and/or meta-analysis? 2. Was the target species an animal for companionship, food production or wildlife, or a pathogen impacting those species? If it was unclear whether the article met both of these criteria, the article was moved to the second round of screening. Each reviewer conducted a pre-test screening of 100 references, and any questions or conflicts were discussed before the remainder of the references was screened. Thereafter, two independent reviewers screened each title and abstract, with any disagreements resolved by consensus.
Reviews of animal models of human disease(s), narrative reviews, and protocols for future systematic reviews not yet conducted were eliminated at this stage. When authors claimed to have used systematic review or meta-analytic methods, we assumed that they did so; we did not attempt to evaluate whether the review methods met the generally accepted definitions of those terms.
All reviews that met the inclusion criteria in the first screening round were then subjected to a second check for relevance followed by review characterization. The research team developed a standard form to confirm the eligibility of reviews and to identify information from each review deemed relevant to the research question. This form was pre-screened by each reviewer using 10 references to ensure that the extracted data could adequately address the research question and to resolve any reviewer questions before the formal charting process began.
Two independent reviewers used this form to extract data from the citation and abstract of each review. Reviewers met frequently to resolve any conflicts regarding eligibility and/or characterization, and to discuss any uncertainty in the charting process. Only the citations and abstracts were examined because it was decided that the data items of interest were likely to be contained in review titles and abstracts, and given the time and labor costs necessary for retrieving the full text articles, the titles and abstracts were likely to provide an adequate overview of the topic area. As a post hoc exercise, 100 citations were randomly selected using a random number generator, and the full texts for these citations were acquired. A single reviewer (JMS or EV) blindly coded the full texts as one of the following: systematic review only, meta-analysis only, systematic review and meta-analysis, or other. The results of the exercise were compared to the results of the title-abstract charting.
Data items
Data extracted from each citation and abstract, where available, included: year of publication; journal; country of first author; target commodity group; synthesis method(s) used as stated by the authors; the level of interest of the review (e.g. animal/herd level, infectious disease agent, and animal product); the type of systematic review question (e.g. PICO/PECO, PO, PIT (defined below), ecological parameter synthesis, genomic meta-analysis, or synthesis related to genetics/breeding values); type(s) of interventions (if applicable); and the focus area of reported outcomes (animal health, animal performance, or food safety). Information on the country of the first author was extracted from the citation record where available, and a Google Scholar search was conducted by one of the reviewers to locate this information for the remaining papers.
‘PICO’ is an acronym that represents the key components of a well-structured research question related to the efficacy of an intervention, namely Population or Participants (P), Intervention(s) (I), Comparison group(s) (C), and Outcome(s) (O); PECO questions are similar but examine exposures/risk factors (E) rather than interventions (Higgins and Green, Reference Higgins and Green2011; European Food Safety Authority, 2010). These categories were combined for the present study, as distinguishing between the two was not always possible given the level of detail available in review abstracts. PO studies in the animal health literature typically examine the prevalence or incidence of a disease or condition in the target animal population. The acronym ‘PO’ identifies the key elements of this type of question: the Population (P) and Outcome (O) of interest (European Food Safety Authority, 2010). Other reviews investigate diagnostic test accuracy by employing a PIT-style question. The elements of this type of research question are the Population of interest (P), the Index test(s) (I), and the Target condition of the test (T) (European Food Safety Authority, 2010).
For systematic reviews investigating the effect of specific interventions or exposures, reviewers selected one or more options from a list of broad intervention types (e.g. vaccinations or antibiotics); this list expanded as the review progressed and additional interventions and exposures were identified.
Outcomes that measured animal health, such as morbidity or mortality, were recorded in an open text field; following data collection, these descriptions were categorized according to the type of health status being reviewed (e.g. mortality, infectious diseases, or non-infectious diseases or conditions). Outcome data were then sorted into these categories by a single reviewer and subsequently validated by a second reviewer. Performance outcomes, such as average daily gain or milk production, were selected from a list of possible outcomes, to which reviewers could add categories as the review progressed. For food safety outcomes (e.g. prevalence of E. coli) reviewers selected the level at which the outcome was measured from a list: in the live animal; in an animal product; at slaughter; or not stated. The final data charting form is available in Appendix 1.
Synthesis of results
Data from the reviews included in the characterization stage were compiled in a spreadsheet and exported to Microsoft Excel 2011 for Mac (Microsoft Corporation, Redmond, WA). Descriptive statistics, frequencies, and percentages were calculated to summarize the data.
Results
Selection of sources of evidence
The search returned 9374 references. Following duplicate removal, 6278 reviews were included in the screening round. During the screening stage, 4491 articles were considered as not relevant and were excluded. A total of 1787 papers were included in the scoping review, the titles and abstracts of which underwent data characterization. Figure 1 shows the flow of articles through the review process.
PRISMA flow chart. Chart illustrates the process of study inclusion in a scoping review of systematic reviews and meta-analyses in the animal health literature.
Synthesis of results
Results from all included reviews
The application of systematic review and meta-analytical methods in the fields of animal health and on-farm food safety is international in scope. Country of the first author remained unavailable for 4.8% (n = 86/1787) of the articles. Studies were identified from all continents. The majority were based in Europe (40.0%, n = 681/1701) or North America (33.5%, n = 569/1701). Other continents were represented as follows: South America (8.64%, n = 147/1701), Oceania (7.23%, n = 123/1701), Asia (6.35%, n = 108/1701), the Middle East (2.18%, n = 37/1701), and Africa (2.12%, n = 36/1701). Some reviews employed multinational research teams, and so these data likely underrepresent the true global extent of systematic reviews and meta-analyses in veterinary science.
The papers in this scoping review focused on a wide range of animals. Figure 2 shows that dairy cattle (13.3%, n = 302), fish (11.1%, n = 251), and pigs (10.6%, n = 240) represented the most common target species or commodity group.
Number of review studies targeting each livestock, companion animal, and/or wildlife species. The number of reviews may sum to more than the total number of included papers as some reviews examined multiple species. Reviews that investigated non-specific wildlife or other animal species were classified under ‘Only said “animals”.’ Reviews were classified as ‘Other’ if they examined animals other than or in addition to species of interest to the review.
The majority of the identified reviews contained the word ‘meta-analysis’ in the title or abstract but not the words ‘systematic review.’ Articles with titles and abstracts that described a systematic review but not a corresponding meta-analysis accounted for 20.4% (n = 365) of the papers, and 11.5% (n = 205) of the included reviews stated in the title or abstract that both methods were employed. Few network meta-analyses and individual patient data meta-analyses were identified (n = 6 and n = 3, respectively). No overviews or systematic reviews of other pre-existing systematic reviews were found. In the post hoc analysis of 100 randomly selected full-text articles, the classification of the review method was the same for 98 of the references. For the two reviews where the results differed between the citation evaluation and the full-text evaluation, one abstract indicated that meta-analysis was intended, but in the full text the authors stated that the data did not support meta-analysis; the second abstract stated that meta-analysis was intended, but the data ultimately did not support this method (i.e. the citation was miscoded in the title/abstract evaluation).
Figure 3 shows the number of articles employing systematic reviews, meta-analyses, or both methods over time. The use of all of the methods increased over time, with very few systematic reviews published before the year 2000. The number of meta-analytical studies began to increase in the early 2000s, and then increased dramatically after 2006. The use of systematic reviews and combined methods did not begin in earnest until the mid-2000s, and continues to lag behind the publication of meta-analyses.
Synthesis methods used in the included review studies (N = 1674). Time frame spans the duration of the scoping study period (1993–2017). Only complete years were included; reviews published in 2018 were excluded (n = 113).
Figure 4 shows the number of reviews for each review question type. Most reviews investigated the impact(s) of an intervention or exposure using a PICO or PECO question (59.9%, n = 1114). A smaller number of PO (n = 162) and PIT (n = 54) reviews were identified. Some additional review question forms were also identified in the data charting phase. A large proportion of the identified articles (11.5%, n = 214) explored ecological parameters such as species diversity, abundance, and wildlife responses to natural phenomena. These research questions were categorized as ‘Estimating ecological parameters.’ A number of reviews (6.8%, n = 127) were identified that examined specific genomic characteristics of animals or of various pathogens affecting animal health. In addition, 2.2% (n = 40) of the included reviews assessed breeding values or heritability traits in the target animal commodity group. The specific form of the review question was indeterminable for 4.7% (n = 87) of the included articles. Finally, despite containing the words ‘systematic review’ or ‘meta-analysis’ in the title or abstract, 3.4% (n = 64) of the reviews appeared to be scoping studies that explored the extent of the published literature on a topic rather than appearing to address a specific researchable question.
Types of review questions and the number of review studies employing each type of question (N = 1787).
Review questions of interest
In the fields of animal health and food and feed safety, certain forms of research questions are considered suitable for systematic review applications (European Food Safety Authority, 2010; O'Connor and Sargeant, Reference O'Connor and Sargeant2015). These questions are defined by the key elements, which are summarized by the acronyms PICO/PECO, PO, and PIT (European Food Safety Authority, 2010). To focus the results of this review on papers most relevant to the research question, the remainder of the Results section will include only reviews with a PICO/PECO, PO, or PIT question (n = 1300). If an article explored ecological or genetic parameters in addition to a research question in one of these formats, then it was retained. Figure 5 shows how the number of reviews employing each of these relevant review questions changed over time. PICO/PECO studies remained the most prominent question format across the review period, increasing steadily from the mid-1990s until a large jump in the number of studies occurred in the mid-2000s. The number of published PO reviews remained close to zero and only began increasing steadily after 2011. Only a small number of PIT reviews were captured, and these were mostly published after 2009.
Types of review question used in the included review studies (N = 1213). Time frame spans the duration of the scoping study period (1993–2017). Only complete years were included; reviews published in 2018 were excluded (n = 87).
Health, performance, and food safety outcomes differ both in their nature and in their broader policy and research implications. To aid in the interpretation of the results pertaining to each type of outcome, the PICO/PECO/PO/PIT reviews were further sub-divided according to the focus area of interest – animal health, animal performance, or on-farm food safety. A total of 418/1300 reviews focused exclusively on animal health, 701/1300 reviews included only performance outcomes, 62/1300 reviews investigated only on-farm food safety outcomes, 101/1300 reviews included both health and performance outcomes, 14/1300 reviews included both health and food safety outcomes but did not investigate performance, 2/1300 reviews examined both performance and food safety but not health, and finally 2/1300 reviews included all three outcome types.
Table 2 shows the methods employed in the reviews that fell under each of the areas of focus (health, performance, and food safety). Of those reviews investigating health outcomes, the numbers using systematic reviews or meta-analyses were similar (36.1 versus. 40.8%, n = 193/535 versus. n = 218/535). Just over 20% (n = 116/535) of the reviews reported incorporating both systematic review and meta-analysis components. All of the network meta-analyses captured in this scoping review (n = 6) had animal health outcomes.
Synthesis methods employed for each focus area (N = 1300)
a No reviews of previous systematic reviews were identified in the scoping review.
b Includes all reviews with at least one health outcome, with a health outcome only, with both health and performance outcomes, with both health and food safety outcomes, or with health, performance, and food safety outcomes.
c Percentage of reviews for each focus area (e.g. percentage of all health outcome reviews).
d Includes all reviews with performance outcomes only.
e Includes all reviews with food safety outcomes only, and with both food safety and performance outcomes but without a health outcome.
Most of the performance reviews were based exclusively on meta-analyses (87.6%, n = 614/701), without the authors describing a corresponding systematic review component in the abstract. Small but similar numbers of reviews employed only systematic review methods or both systematic reviews and meta-analyses (6.3%, n = 44/701; 6.0%, n = 42/701, respectively). Only one individual patient data meta-analysis was conducted for a performance outcome, and no network or mixed-treatment meta-analyses were found.
For the food safety reviews, over half incorporated both systematic review and meta-analysis components (51.6%, n = 33/64), and the remaining reviews were divided almost equally between systematic review methods only and meta-analysis only (23.4%, n = 15/64; and 25.0%, n = 16/64, respectively).
Similarly, Table 3 indicates the level of interest for each of the included reviews by focus area. Most articles with an animal health outcome were focused at the individual animal or herd level (71.8%, n = 384/535). In addition, one quarter (n = 134/535) of animal health reviews focused on a specific infectious disease agent rather than an animal, tissue, or product. The level of interest for performance reviews was almost exclusively at the animal or herd level (88.4%, n = 620/701). Animal products were the next most common level of focus, but to a much lesser degree than the animal or herd level (7.1%, n = 50/701). The most common level of interest for food safety reviews was infectious disease agents (48.4%, n = 31/64), followed by the animal or herd level (34.4%, n = 22/64).
Level of interest for each focus area (N = 1300)
a Includes viruses, bacteria, parasites, and any other disease-causing pathogens.
b For example, ovaries, liver, etc. Does not include animal products for human consumption.
c Products at the farm level that are ultimately intended for human consumption, such as unpasteurized milk or meat prior to processing.
d Includes all reviews with at least one health outcome, with a health outcome only, with both health and performance outcomes, with both health and food safety outcomes, or with health, performance, and food safety outcomes.
e Percentage of reviews for each focus area (e.g. percentage of all health outcome reviews).
f Includes all reviews with performance outcomes only.
g Includes all reviews with food safety outcomes only, and with both food safety and performance outcomes but without a health outcome.
Reviews reporting health outcomes
In total, 535 articles examined one or more animal health outcomes with 546 outcomes examined in total. Of these articles, 101 also reported a performance outcome, 14 reported a food safety outcome, and 2 reported both a performance and a food safety outcome. Table 4 summarizes the health outcomes from all of these reviews according to the health categories under investigation. The most common health-related focus was infectious diseases (50.2%, n = 274/546), followed by non-infectious diseases or conditions (23.1%, n = 126/546) and mortality (9.5%, n = 52/546). The most common target animal populations were cattle (not further specified), dogs, dairy cattle, and pigs.
Categorization of reviews with animal health outcomes according to the nature of the health status under investigation (N = 535)
a Antimicrobial resistance.
b Includes reviews that only described general morbidity, a non-specific symptom (e.g. ‘inflammation’), or a non-specific disease or disease state (e.g. any disease affecting cattle).
c May sum to more than the total number of health outcome articles if reviews examined multiple types of health outcomes.
For those animal health reviews adopting a PICO or PECO research question, the particular intervention or exposure under review was captured in the data characterization phase. Twenty different intervention types were identified; because of the large volume of data generated, Table 5 shows only the five most commonly identified interventions against the health outcome category of interest. The table also includes the health outcome categorization for PO and PIT reviews, in which specific interventions were not indicated. The full list of intervention types can be found in Appendix 2. From Table 5, most vaccination reviews targeted infectious diseases, as did most reviews of antibiotic interventions. Different management practices were reviewed regarding their ability to manage primarily morbidity due to infectious causes, and to lesser degree morbidity due to non-infectious causes, or mortality. The risk factors that were studied for different health outcomes varied, ranging from the impact of animal age and sex to temperature variables; these tended to be exposure factors not amenable to manipulation by researchers or animal managers. Consequently, the categories of health outcomes for which risk factors were assessed also varied, although infectious diseases remained the most commonly studied category. Non-antibiotic drug treatments were most often reviewed for non-infectious diseases, but also targeted infectious diseases. PO reviews almost exclusively examined the incidence or prevalence of infectious diseases. Finally, the focus of most PIT reviews was diagnosing infectious diseases, followed by various non-infectious diseases or conditions.
Health outcome categories by review question type (N = 535)
a The number of reviews may sum to more than the total number of articles if reviews examined the impact of multiple types of interventions on a single outcome.
b The number of outcomes may sum to more than the total number of articles if reviews investigated a single intervention with multiple outcomes.
c Antimicrobial resistance.
d Includes studies that only described general morbidity, a non-specific symptom (e.g. ‘inflammation’), or a non-specific disease or disease state (e.g. any disease affecting cattle).
e Other than vaccinations.
f PO and PIT questions may also be combined with a PICO/PECO question; these are captured separately.
Reviews reporting performance outcomes
Of the captured articles, 701 reported solely on animal performance outcomes. These outcomes included changes in feed intake, production outcomes, carcass and product quality, physiologic changes (e.g. heart rate), and reproductive outcomes. Dairy cattle, pigs, fish, and beef cattle represented the most commonly studied animals.
There were 18 performance outcome categories and 20 interventions captured in the data characterization phase. Table 6 shows the five most common performance outcome categories plotted against the five most common interventions, as well as PO and PIT reviews. The full table of performance outcome categories and interventions can be found in Appendix 2. Production variables (muscle, milk, or egg production, animal growth rates, and feed efficiency) were the most common outcome category, representing 32.6% (n = 471) of all performance outcomes. The most frequent interventions implemented to impact production outcomes were changes in diet (n = 162) and non-antibiotic feed additives (n = 131). As with production outcomes, interventions targeting dietary changes and feed additives were among the most common interventions investigated for both of these outcome categories. Broader risk factors for the outcome variables were also frequently studied with regard to all three of the most popular outcome categories.
Performance outcome categories by review question type (N = 701)
a The number of reviews may sum to more than the total number of articles if reviews examined the impact of multiple types of interventions on a single outcome.
b The number of outcomes may sum to more than the total number of articles if reviews investigated a single intervention with multiple outcomes.
c Includes outcomes related to growth rates, average daily gain (ADG), and feed efficiency including feed conversion ratios.
d Other than vaccinations.
e PO and PIT questions may also be combined with a PICO/PECO question; these are captured separately.
The number of PO and PIT reviews were limited for performance outcomes (n = 12 and n = 6, respectively), and the outcome categories of interest in these studies did not necessarily align with the five most common categories. PO reviews examined production, reproduction, and physiology, as well as economic impacts, physiology, and microbiomes, among others. PIT reviews focused on animal welfare, reproduction, measures related to pain, physiology, and species classification.
Reviews reporting food safety outcomes
A total of 62 captured articles focused exclusively on on-farm food safety outcomes, and an additional two reviews incorporated both food safety and performance outcomes. Pigs, broiler poultry, and beef cattle represented the most common target commodity group for these reviews.
Most of the food safety reviews incorporated a PICO or PECO question (61.2%, n = 41), although PO questions were also common (35.8%, n = 24). PIT questions were much less common (3.0%, n = 1), and one review also incorporated a genomic meta-analysis research question. Three of the reviews contained more than one type of review question.
For those reviews examining a specific intervention (i.e. PICO/PECO questions), the most commonly analyzed interventions were management practices (32.0%, n = 19) and other risk factors for the outcome variable (20.3%, n = 12). The impact of vaccines and non-antibiotic feed additives were also investigated in multiple articles (each 13.6%, n = 8), as were antibiotics and toxins (each 5.1%, n = 3). Much less commonly studied were the impacts of dietary factors (3.4%, n = 2), genetics, disease(s) as a risk factor, deliberate disease challenges, and methods for challenging animals in a disease trial (each 1.7%, n = 1).
Almost half (49.3%, n = 34) of the included reviews investigated food safety outcomes in a live animal, 33.3% (n = 23) were concerned with food safety at the point of slaughter, and 8.7% (n = 6) looked at on-farm animal products such as raw milk. Several reviews (n = 5) considered food safety outcomes at multiple stages prior to processing. The point of sample collection from the articles included in the review was unknown for 8.7% (n = 6) of the food safety outcome papers.
Two reviews also included a performance outcome in addition to a food safety outcome. The first, conducted in pigs, looked at the impact of management practices on both carcass quality and food safety parameters at slaughter; the other examined the impact of genetics and other risk factors on both the physiologic characteristics of poultry as well as a food safety outcome in the live animals.
Discussion
Summary of evidence
Systematic reviews represent a transparent, scientifically defensible method for collecting and synthesizing diverse evidence from a body of research to aid in evidence-based decision-making (European Food Safety Authority, 2010; Khan et al., Reference Khan, Kunz, Kleijnen and Antes2011; Sargeant and O'Connor, Reference Sargeant and O'Connor2014). Combined with a systematic review, a statistical meta-analysis can provide a comprehensive summary measure of the effect of an intervention on the outcome of interest (O'Connor et al., Reference O'Connor, Sargeant and Wang2014). The results of this scoping review indicate that the use of the systematic review and meta-analytic methods are increasingly being applied in the fields of animal health, animal performance, and on-farm food safety. The reviews identified in this study covered a wide breadth of interventions and outcomes, with publication occurring around the world. However, to fully understand the potential of these methods to aid in decision-making in these fields, there are several important gaps that must be addressed.
Reviews targeting cattle, pigs, and fish were relatively prevalent, but few reviews examined the poultry industry (layers, broilers, and turkeys). The paucity of reviews targeting the poultry sector may be a reflection of under-reviewing; however, it is also possible that controlled management strategies coupled with the highly integrated nature of the global poultry industry may have contributed to an absence of publicly reported reviews, or an absence of publicly reported primary poultry data for reviewing. In animal agriculture, it is possible that the nature and structure of specific industries affect the quantity and type of publicly available research. Future studies seeking to identify gaps in the existing animal health primary or review literature should consider the impact of industry-specific factors on research format, conduct, and publication.
In synthesis papers that examined animal health outcomes, some topics were reviewed less frequently than others. For example, summaries of the incidence, prevalence, or diagnostic testing of non-infectious diseases were limited. Again, this may reflect under-reviewing, a lack of available primary data or the dominant place of infectious diseases in livestock species. An investigation may be warranted to determine the reason why few reviews on these topics exist. Because synthesis research methods such as systematic reviews provide clinicians and researchers with a quality-assessed portrait of the existing evidence on a topic, the absence of reviews on these topics could hinder understanding of the effects of health interventions on animal mortality or non-infectious disease control.
In the human medical literature, systematic review outcomes should be those most relevant to patients and healthcare decision-makers (Higgins and Green, Reference Higgins and Green2011). In livestock species, performance impacts are critical in animal health decision-making. Indeed, performance outcomes represented the largest category of included reviews, and the focus of these reviews was primarily on dietary treatments, either in the form of direct dietary manipulation or through feed additives. However, only 19.3% (n = 103/535) of reviews reporting a health outcome also investigated performance outcomes. Because decision-making in animal agriculture is multifaceted, future systematic reviews in animal health should consider whether there is also a relevant performance outcome.
Another critical issue, most notable in the animal performance reviews, was the abundance of meta-analyses conducted without a supporting systematic review. In order to conduct a comprehensive and transparent meta-analysis, a systematic review is required to populate the meta-analysis database (O'Connor et al., Reference O'Connor, Sargeant and Wang2014; Sargeant et al., Reference Sargeant, Kelton and O'Connor2014; Sargeant and O'Connor, Reference Sargeant and O'Connor2016). When the goal of a meta-analysis is to pool effect sizes from multiple studies to determine a summary effect size, a systematic review is essential to ensure that as many relevant studies as possible are incorporated, reducing the risk of selection bias in the results of the meta-analysis. Further, the risk of bias assessment that is an essential component of a systematic review allows a consideration of study quality in the interpretation of the results of the meta-analysis (Sargeant and O'Connor, Reference Sargeant and O'Connor2016). Meta-analyses therefore should incorporate a systematic review component to improve the accuracy and reduce the bias of the summary measures. However, it is important to note that we included and categorized review types based on whether the authors used the terms ‘systematic review’ and/or ‘meta-analysis’ in the title or abstract of their reviews. If authors conducted one or the other but did not use any of the specific terms, the paper would not have been included in our review. It is also possible that researchers conducted both a systematic review and a meta-analysis but only used one term in their abstract; in such cases the information captured in our review would be an incomplete reflection of the authors' methods. This speaks to the importance of comprehensive reporting of systematic reviews and meta-analyses; indeed, the PRISMA guidelines for the reporting of systematic reviews recommend that the title or abstract should contain the words ‘systematic review’ and ‘meta-analysis’ when these techniques are used (Liberati et al., Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche, Ioannidis, Clarke, Devereaux, Kleijnen and Moher2009; Moher et al., Reference Moher, Liberati, Tetzlaff and Altman2009).
Applications of the systematic review and meta-analytical methods were the most limited in the field of on-farm food safety; only a very small proportion of the PICO/PECO/PO/PIT reviews focused on this area (4.8%, n = 62). This may reflect the smaller number of pathogens that impact food safety relative to those affecting animal health or a lack of primary data; nonetheless, these reviews can play an important role in food safety-related decision-making. There may also exist additional reviews targeting food safety pathogens after slaughter (e.g. in processing or at the retail level), but capturing these articles was beyond the scope of the present review.
By contrast, some research areas appear to be well reviewed. For example, multiple reviews examined the impact of various interventions on infectious disease health outcomes, on animal production variables or feed intake levels, and on the effect of management practices on food safety outcomes. A compendium or repository of these reviews could provide further summation of existing syntheses, making the conclusions from these reviews more accessible to interested parties. Such a repository does exist (VetSRev; Centre for Evidence-Based Veterinary Medicine, 2013) and is an excellent resource, although it is not restricted entirely to systematic reviews and/or meta-analyses.
Limitations
There are several potential limitations to the present review. First, we did not verify that the actual methods used in each review were consistent with what authors claimed that they did in their titles or abstracts. It is therefore possible that some of the articles included in our review were not true systematic reviews or meta-analyses.
The characterization of the included reviews was based only on the available abstracts due to the large volume of relevant citations. For many scoping reviews, data characterization is conducted using full-text articles (Arksey and O'Malley, Reference Arksey and O'Malley2005; Daudt et al., Reference Daudt, van Mossel and Scott2013; Pham et al., Reference Pham, Rajić, Greig, Sargeant, Papadopoulos and McEwen2014), and so it is possible that had we obtained the full text we might have reached different conclusions. Some abstracts did not present the relevant information on the review subject, method, and outcome in an accessible and coherent manner. However, given the nature of the data items that were extracted in this study, it was anticipated that the necessary information should be contained in the title or abstract. As a post hoc analysis, we evaluated the reporting of review methods in titles and abstracts using the full texts of 100 randomly selected articles. We found that agreement was high, with one article miscoded and only one other article containing information in the full text that changed the characterization of the study type, suggesting that information on the study method was consistently reported in the title or abstract. Poor reporting of systematic review abstracts and systematic reviews more broadly reduces the value of these reports as a basis for decision-making. The PRISMA guidelines provide a framework for comprehensive, consistent, and transparent reporting of systematic reviews and meta-analyses (Liberati et al., Reference Liberati, Altman, Tetzlaff, Mulrow, Gøtzsche, Ioannidis, Clarke, Devereaux, Kleijnen and Moher2009; Moher et al., Reference Moher, Liberati, Tetzlaff and Altman2009). An extension of the PRISMA guidelines also exists for the abstracts of systematic reviews (Beller et al., Reference Beller, Glasziou, Altman, Hopewell, Bastian, Chalmers, Gøtzsche, Lasserson and Tovey2013). Given the importance of high-quality literature syntheses to evidence-based decision-making, closer adherence to these guidelines for future reviews is imperative.
Although the search strategy in this review was designed to be broad to capture all relevant articles, time and resource constraints limited the breadth of the search. The use of additional databases or other search methods, such as hand-searching journals, may have identified additional citations. However, the large number of articles that were captured by the search suggests broad coverage of the existing literature, and it is unlikely that additional searches would have uncovered articles that substantively altered the themes and results reported above.
In addition to excluding potentially relevant articles, the literature search was conducted using English terms only, and if the title and abstract of a paper were not in English then it would have been missed by the search. Subsequent studies should explore systematic reviews and meta-analyses in other languages to determine the true global extent of these forms of evidence in the animal health literature.
The lack of critical appraisal of studies included in a scoping review is a common criticism of the approach (Levac et al., Reference Levac, Colquhoun and O'Brien2010; Daudt et al., Reference Daudt, van Mossel and Scott2013). Arksey and O'Malley (Reference Arksey and O'Malley2005) concede that identifying research gaps through a scoping study may be limited because the method fails to account for gaps resulting from poor quality research. Daudt et al. (Reference Daudt, van Mossel and Scott2013) advocate for an assessment of included study quality using validated instruments as an additional component of the scoping review framework. However, Pham et al. (Reference Pham, Rajić, Greig, Sargeant, Papadopoulos and McEwen2014) argue that scoping reviews should include all potentially relevant studies regardless of methodological vigor in order to provide the most complete possible overview of the existing literature on a topic; this is the stance adopted in this paper.
Conclusions
Systematic reviews and meta-analyses represent an important and increasingly popular technique for summarizing, evaluating, and interpreting evidence in the fields of animal health, animal performance, and on-farm food safety. The results of this scoping study may not present a complete picture of review studies in the fields of animal health and veterinary medicine, but this study does provide some crucial insight into the state of the field at this moment. In general, well-reported review abstracts should contain the data items of interest for this scoping study.
At this point in time the application of review methods is varied across commodity groups, interventions, focus areas, and outcomes. In under-studied areas, investigations into the reasons for the paucity of reviews on those topics may be useful. In those areas, additional primary and/or synthesis studies could provide crucial insight to answer clinical questions and direct future research. In areas that have attracted more research attention, evidence repositories could be considered to collate the results of previous reviews. Future animal health reviews should consider potentially relevant performance outcomes, and researchers should strongly consider conducting a systematic review before undertaking any meta-analysis. Further, and because of the importance of this research, adherence to reporting guidelines such as the PRISMA guidelines for systematic reviews is critical to ensure research quality.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1466252319000197.
Author contributions
JMS, CBW, and AOC conceived of the project and developed the research question, objectives, and eligibility criteria. JMS, CBW, and AOC developed the search strategy and performed the scoping search. RV, JMS, EV, and MR performed the data screening and characterization. RV analyzed and interpreted the data with verification provided by JMS. RV drafted the initial manuscript. JMS, EV, MR, CBW, and AOC provided comments on this draft. All authors read and approved the final manuscript.
Financial support
This study received no external funding.
Conflict of interest
None of the authors has conflicts to declare.
Open access
