Introduction
Rationale
The majority of antimicrobial use in the dairy industry is for the treatment and prevention of intramammary infections (IMI); in the Netherlands, approximately 60% of all antimicrobial use in dairy is for this purpose, with two-thirds being dry cow therapy (Lam et al., Reference Lam, van Engelen, Scherpenzeel and Hage2012). In the United States, over 90% of dairy cows receive dry cow therapy after every lactation (USDA-APHIS, 2016), with the goal of treating or preventing IMI during the dry period. Prepartum IMI are strongly associated with the risk of development of clinical mastitis in the first 2 weeks post-calving, which represents the highest risk period for this disease (Green et al., Reference Green, Green, Medley, Schukken and Bradley2002). In the United States, clinical mastitis represents the most common disease treated with antimicrobials in adult dairy cows, with approximately 16% of cows reported as having been treated in 2007, with cephalosporins the most commonly used drug class (United States Department of Agriculture, 2008). To reduce IMI during the dry period, blanket dry cow therapy (intramammary antimicrobial administration to all quarters of all cows after the last milking of the lactation) has been recommended for decades (Neave et al., Reference Neave, Dodd, Kingwill and Westgarth1969), and has been widely adopted in North America and the United Kingdom (Ruegg, Reference Ruegg2017). However, choosing ineffective antimicrobials, or using antimicrobial when not warranted, unnecessarily contributes to use while having little impact on controlling disease, which has substantial bearing to both profitability and animal welfare (Leslie and Petersson-Wolfe, Reference Leslie and Petersson-Wolfe2012). There is a need for evidence-based antimicrobial use protocols surrounding udder health (Ruegg, Reference Ruegg2017). Systematic reviews of randomized controlled trials yield the highest level of evidence for the efficacy of treatment under field conditions (Sargeant et al., Reference Sargeant and O'Connor2014a). If sufficient numbers of primary studies on a given comparison are available, a pairwise meta-analysis provides the relative efficacy of the two treatments. However, pairwise comparisons often rely on trials with non-treated controls as the comparison group, and direct comparisons of potentially comparable interventions may be limited (Roy and Keefe, Reference Roy and Keefe2012). Previous systematic reviews have typically used pairwise meta-analysis to evaluate the efficacy of antimicrobial and non-antimicrobial interventions for dairy cattle at dry-off, including teat sealants (Halasa et al., Reference Halasa, Osteras, Hogeveen, van Werven and Nielen2009; Rabiee and Lean, Reference Rabiee and Lean2013; Naqvi et al., Reference Naqvi, De Buck, Dufour and Barkema2018), antimicrobials (Robert et al., Reference Robert, Seegers and Bareille2006; Halasa et al., Reference Halasa, Osteras, Hogeveen, van Werven and Nielen2009), and dry period length (van Knegsel et al., Reference van Knegsel, van der Drift, Cermakova and Kemp2013). For intramammary treatments of cattle at dry-off, numerous interventions are available, including teat sealants used with or without one of several different dry-cow antimicrobial products. In these cases, pairwise meta-analyses only provide information about a single comparison, and do not provide a summary of evidence across multiple interventions (Cipriani et al., Reference Cipriani, Higgins, Geddes and Salanti2013). Network meta-analysis provides a method of assessing relative efficacy among many treatments by the use of both direct (studies which compare given treatments) and indirect (studies which share common comparators) evidence, and is a commonly used approach in human medicine (Caldwell et al., Reference Caldwell, Ades and Higgins2005; Cipriani et al., Reference Cipriani, Higgins, Geddes and Salanti2013).
Establishing the efficacy of cow-level antimicrobial therapy for the prevention of IMI and clinical mastitis will serve to improve decision makers' ability to engage in effective stewardship of antimicrobials through the strategic use of these products with knowledge of implications for animal health and welfare.
This systematic review is conducted based on the guidance from the Cochrane Collaboration (Higgins and Green, Reference Higgins, Thomas, Chandler, Cumpston, Li, Page and Welch2011) and recommendations for conducting systematic reviews in animal agriculture and veterinary medicine (O'Connor et al., Reference O'Connor, Anderson, Goodell and Sargeant2014a, Reference O'Connor, Sargeant and Wang2014b; Sargeant and O'Connor, Reference Sargeant and O'Connor2014a, Reference Sargeant and O'Connor2014b; Sargeant et al., Reference Sargeant, Kelton and O'Connor2014a, Reference Sargeant, Kelton and O'Connor2014b). It is reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions (PRISMA-NMA) (Hutton et al., Reference Hutton, Salanti, Caldwell, Chaimani, Schmid, Cameron, Ioannidis, Straus, Thorlund, Jansen, Mulrow, Catala-Lopez, Gotzsche, Dickersin, Boutron, Altman and Moher2015).
Objective
The objective of this review was to determine the relative efficacy of antimicrobial administration at dry-off to prevent new IMI over the dry-period or clinical mastitis during early lactation.
Methods
Protocol
A review protocol, established in advance and reported in accordance with PRISMA-P guidelines (Moher et al., Reference Moher, Shamseer, Clarke, Ghersi, Liberati, Petticrew, Shekelle and Stewart2015), was published at the University of Guelph's institutional repository (https://atrium.lib.uoguelph.ca/xmlui/handle/10214/10046) on 25 June 2018. The protocol is also available through Systematic Reviews for Animals and Food (SYREAF) (http://www.syreaf.org/contact/).
Eligibility criteria
Primary research studies, both refereed and non-refereed (grey literature), available in English were eligible for inclusion. Studies must have enrolled dairy cows after their first (or greater) lactation, without IMI at the cessation of milking (i.e. at dry-off) (for IMI outcomes), or without clinical mastitis at dry-off (for the clinical mastitis outcome). Studies must have included at least one treatment arm with an intramammary antimicrobial, with or without another concurrent dry cow treatment, compared to no treatment, placebo, or an alternative treatment (such as an internal or external teat sealant). To be eligible, studies must have included at least one outcome. Outcomes included (i) the incidence of IMI (using the author's definition of IMI) during the pre-calving period following the intervention, (ii) the incidence of IMI during the first 30 days of the subsequent lactation and (iii) the incidence of clinical mastitis during the first 30 days of the subsequent lactation. Controlled trials with natural disease exposure were the only eligible study design, although challenge trials and analytical observational studies were documented during the full-text screening stage.
Information sources
Databases searched were: Agricola (via ProQuest, 1970 to current), CAB Abstracts and Global Health (via Web of Science, 1910 to current), Epub ahead of print, In-process & other non-indexed citations, Ovid MEDLINE®(R) Daily, and Ovid MEDLINE® (R) (via Ovid, 1946 to current), Conference Proceedings Citation Index – Science (via Web of Science, 1990 to current), and Science Citation Index (via Web of Science, 1900 to current). A reviewer hand-searched the table of contents of the following relevant conferences from 1997 to 2018: Proceedings of the American Association of Bovine Practitioners, World Association for Buiatrics, and the National Mastitis Council Proceedings. The Food and Drug Administration (FDA) website containing the Freedom of Information New Animal Drug Approvals (NADA) summaries was also searched.
Search
The search strategy initially was developed for the Science Citation Index (Web of Science) interface, and employed a multi-stranded approach to maximize the sensitivity (Table 1). The conceptual structure combined the concepts of ‘dairy cows’ AND ‘dry off’ AND ‘antimicrobials’; or ‘dry cow’ AND ‘antimicrobials’; or ‘dairy cows’ AND ‘prophylaxis’ AND ‘intra-mammary infections’. An additional precise search line to identify phrases such as ‘dry cow therapy’ and ‘dry cow management’ was also included in order to retrieve any records missed by the previous two combinations. Database searches were conducted on 28 June 2018. Search results were uploaded to EndNoteX7 (Clarivate Analytics, Philadelphia, PA, USA) and duplicate results documented and removed. Records were then uploaded to DistillerSR (Evidence Partners Inc., Ottawa, ON, USA) and additionally de-duplicated. If the same study and data were available as a conference abstract and as a full publication, the abstract was removed.
Table 1. Full electronic search strategy used to identify studies of antimicrobial treatments during the dry-off period in dairy cattle in Science Citation Index (Web of Science) conducted on 28 June 2018
TS, topic field search (includes the title, abstract, author keywords, and keywords plus fields); *, unlimited right-hand truncation symbol; NEAR/N, retrieves records that contain terms (in any order) within a specified number (N) of words of each other.
Validation of the search was done by identifying all articles included in the qualitative syntheses of reviews in the area of dry cow management as identified from the following papers: Robert et al., Reference Robert, Seegers and Bareille2006; Halasa et al., Reference Halasa, Osteras, Hogeveen, van Werven and Nielen2009; Pereira et al., Reference Pereira, Oliveira, Mesquita, Costa and Pereira2011; van Knegsel et al., Reference van Knegsel, van der Drift, Cermakova and Kemp2013; Enger et al., Reference Enger, White, Nickerson and Fox2016. All relevant articles identified in these reviews were found in the search.
Study selection
DistillerSR was used for all rounds of screening and data extraction. Title and abstracts were initially screened for eligibility. Two reviewers independently evaluated each citation, and all reviewers were trained by CBW and JMS on a pre-test of the first 250 titles and abstracts to ensure the clarity of understanding and consistency of application. The following questions were used to assess the eligibility:
(1) Does the study involve antimicrobial-containing dry-cow treatments in dairy cattle at the individual level or an evaluation of group-level strategies for administering antimicrobial-containing dry-cow treatments (such as selective treatment versus blanket treatment)? YES (neutral), NO (exclude), UNCLEAR (neutral)
(2) Is there a concurrent comparison group (i.e. controlled trial with natural or deliberate disease exposure, or analytical observational study)? YES (neutral), NO (exclude), UNCLEAR (neutral)
(3) Is the full text available in English? YES (include for full-text screening), NO (exclude), UNCLEAR (include for full-text screening)
Citations were excluded if both reviewers responded ‘NO’ to any of the questions; agreement was at the level of the form. Disagreements were resolved by consensus with mediation by JMS or CBW if an agreement could not be reached. Secondary screening was conducted on the full text of remaining studies independently by two reviewers, using the first 10 citations as a pre-test by all reviewers. This level of screening used the initial three questions with only YES (neutral) or NO (exclude) options, and additionally:
(4) Does the study evaluate any of the following outcomes: incidence of clinical or subclinical mastitis at 30 days in milk (DIM), or incidence of IMI or subclinical mastitis at calving? YES (neutral) NO (exclude)
(5) What is the study design? Experimental – natural disease exposure (neutral), experimental – deliberate disease exposure (exclude), analytical observational study (exclude)
(6) Does the study evaluate a group-level strategy for administering dry-cow treatments (such as selective treatment versus blanket treatment)? YES (exclude from this review; included in a separate review), NO (include)
The term ‘subclinical mastitis’ was included as authors may have referred to this instead of IMI, but reflects the same disease. Agreement was at the question level, with conflicts resolved by consensus or with mediation by JMS or CBW if an agreement could not be reached.
Data collection
Data from citations meeting the full-text screening inclusion criteria were independently extracted by two reviewers using a standardized form, which was piloted on the first five citations by all reviewers to ensure consistency. Discrepancies in data extraction were resolved by consensus, with mediation by JMS and CBW if an agreement could not be reached. Hierarchical forms were used in DistillerSR for data extraction, with forms nested as: (Study Characteristics (Outcome (Arm, Contrast, Risk of bias))). A PDF version of the full data extraction tool is available as Supplemental File S1.
Data items
Study characteristics
Study-level data included study design, country of conduct, year and months of study conduct, setting (research or commercial herd), breed of cattle, number of herds enrolled, inclusion criteria at the cow and herd level, and parity of enrolled animals.
Interventions and comparators
Details on the interventions, including antimicrobial(s) used, route of administration, frequency of administration, dose, dry period length, level of treatment allocation, and level of analysis were recorded. Baseline characteristics and loss to follow-up were captured. Case definitions and times at which the outcomes were recorded, including which methods were used to identify IMI. Following data extraction, interventions were identified and labeled on a treatment map (Table 2). To provide strength to the network, interventions in the same antimicrobial family (World Organisation for Animal Health, 2007) were considered the same treatment protocol.
Table 2. Description of treatment groups as labeled in subsequent figures and tables
While results of all comparisons in the network were included in the analysis, only treatment arms with an intramammary antimicrobial therapy, or non-treated control groups, are presented with relative efficacy rankings (i.e. teat sealants alone, or non-intramammary antimicrobial therapies were not ranked, but information captured on these comparator arms provided evidence to the network).
Eligible outcomes
Outcomes eligible for inclusion in the meta-analysis were:
• Incidence of clinical mastitis in the first 30 days of lactation,
• Incidence of IMI between treatment and calving, and
• Incidence of IMI in the first 30 days of lactation.
Prioritization of these outcomes for meta-analysis was determined during protocol development in consultation with content experts based on the frequency of use in the primary literature and being proxies to reflect the effects of infection during the dry period. Data reported for clinical mastitis were considered as incidence; cows were assumed to be free of clinical mastitis at dry-off unless otherwise reported in the study. For IMI incidence, cows were not assumed to be free of IMI at dry-off (according to the authors' definition), and studies had to report results separately for ‘new’ infections to proceed to data extraction. What constituted a ‘new’ infection was recorded: no pathogen growth initially followed by any pathogen growth; a new pathogen isolated on the follow-up sample; or if this information was not reported.
For included studies, information on other outcomes was extracted to describe their use in the literature, but data were not extracted for synthesis. These secondary outcomes were: total antimicrobial use during the first 30 days of lactation, total milk production over the next lactation, somatic cell count at the first milk recording test of the next lactation, average somatic cell count of the first three milk recording tests of the next lactation, and the risk of culling over the next lactation.
For outcomes for which data were extracted, the prioritized outcome measure was an adjusted summary effect (adjusted odds ratio (OR) or relative risk or risk ratio (RR) for dichotomous outcomes, or adjusted least square mean differences for continuous outcome). Variables included in adjustment and the corresponding precision estimate were recorded. If an adjusted measure was not reported, unadjusted summary effect size (second priority) or treatment arm-level (raw) data (third priority) were recorded, with an applicable variance measure. Continuous data presented without variance measures, and for which a measure of variance could not be calculated, were not extracted.
For multi-farm studies where clustering at the farm level was not adjusted for (i.e. those reporting raw data for multiple farms), if raw data were available by the farm, these were extracted as unique studies.
Geometry of the network
We visually evaluated the geometry of the network to determine if some pairwise comparisons dominated and to determine the network structure. We evaluated if there were intervention comparisons that were not linked to the network (i.e. did not have an intervention in common with one or more other published studies).
Risk of bias in individual studies
Risk of bias was assessed by outcome for all three outcomes extracted, using the Cochrane Risk of Bias 2.0 algorithm (Higgins et al., Reference Higgins, Sterne, Savovic, Page, Hróbjartsson and Boutron2016), with signaling questions modified to be specific to the topic of the review. This tool assesses the potential for bias arising from five areas or domains: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of the outcome, and bias in the selection of the reported results. In commodity groups such as swine or poultry, individual animal value is likely to be unknown or equal at the time of treatment allocation; for these livestock groups, the question on allocation sequence concealment may not be considered in the bias assessment for the domain related to the randomization process (Moura et al., Reference Moura, Totton, Sargeant, O'Sullivan, Linhares and O'Connor2019). In the case of dairy cattle, a decision was made to include the question on allocation concealment in the risk-of-bias assessment, as individual animal value is likely unequal and known at the time of treatment allocation in most (or all) studies. As well, an additional answer option was provided for the question on random allocation sequence, for studies using the word ‘random’ to describe the allocation sequence but not providing details on the method used to generate the random sequence.
Risk of bias was assessed independently in duplicate, with disagreement resolved by consensus and mediation by JMS or CBW if needed. The risk-of-bias tool is available as Supplementary File S2. For studies with each outcome, risk of bias for all studies is presented by outcome by domain of bias.
Summary measures
After extracting the outcomes, the analysis was conducted on the log OR for the analysis. For presentation purposes, the log OR was back-transformed to the RR using the baseline risk from the model data. The posterior mean and standard deviation of the baseline risk mean were −1.3610 and 1.0947. The posterior mean and standard deviation of the baseline risk standard deviation were 1.0588 and 0.1864.
Network meta-analysis
Planned method of statistical analysis
A network meta-analysis was conducted for the outcome of IMI at calving. The method has been previously described in detail elsewhere (Dias et al., Reference Dias, Welton, Caldwell and Ades2010; O'Connor et al., Reference O’Connor, Coetzee, da Silva and Wang2013). Raw data or ORs were converted to a log OR, and RRs were converted to a log OR using the risk of disease in the control group. If probabilities were reported, the values were back converted to a log OR, using a process described by Hu et al. (Reference Hu, Wang and O'Connor2019).
Selection of prior distributions in Bayesian analysis
The prior distributions were originally based on the approach reported previously (Dias et al., Reference Dias, Welton, Sutton and Ades2011). For the model, 
$\sigma $ ~ U (0,2) and 
$\sigma $ ~ U (0,5) were assessed, and the analysis suggested 
$\sigma $ ~ U (0,5) was preferred, so this prior was retained in the model.
Implementation and output
All posterior samples were generated using Markov Chain Monte Carlo (MCMC) simulation implemented using Just Another Gibbs Sampler (JAGS) software (version 3.4.0) (Plummer, Reference Plummer2015). All statistical analyses were performed using R software (version 3.2.1) (RCore, 2015) in a Linux system. The model was fit by calling JAGS from R through the RJAGS package (Plummer, Reference Plummer2015). Three chains were simulated and the convergence was assessed using Gelman–Rubin diagnostics. A total of 5000 ‘burn-in’ iterations were discarded, and based the inferences on a further 10,000 iterations. The model output included all possible pairwise comparisons using log ORs (for inconsistency assessment), RRs (used for comparative efficacy reporting), the rankings (for comparative efficacy reporting), and the probability of being the worst treatment option (for comparative efficacy reporting).
Assessment of model fit
The fit of the model was assessed based on the log OR, by examining the residual deviance between the predicted values from the mixed-treatment comparison model and the observed value for each study (Dias et al., Reference Dias, Welton, Caldwell and Ades2010).
Assessment of inconsistency
Inconsistency was assessed by examining the consistency between direct and indirect evidence for all pairwise comparisons, using the method described by Dias et al. (Reference Dias, Welton, Caldwell and Ades2010). Means and standard deviations of log OR of treatment effects were calculated using direct (head-to-head) evidence only, indirect evidence only, and the combined evidence. We compared the estimates from the direct and indirect models and considered the standard deviation of each estimate, rather than relying on the P-values.
Risk of bias in the overall network
Risk of bias in the overall network of evidence was assessed using the Confidence In Network Meta-Analysis (CINeMA) platform (http://cinema.ispm.ch), which uses a frequentist approach through the ‘metafor’ package (Viechtbauer, Reference Viechtbauer2010) to determine the basis for the contribution matrix for the risk of bias. CINeMA evaluates within-study bias, across-studies bias, indirectness, imprecision, heterogeneity, and incoherence. As opposed to presenting an overall assessment of bias and of indirectness, we reported the contribution of studies based on an approach to allocation to groups and blinding, as there is evidence in animal health that failure to include these design elements is associated with exaggerated treatment effects (Wellman and O'Connor, Reference Wellman and O'Connor2007; Burns and O'Connor, Reference Burns and O'Connor2008; Sargeant et al., Reference Sargeant, Elgie, Valcour, Saint-Onge, Thompson, Marcynuk and Snedeker2009a, Reference Sargeant, Saint-Onge, Valcour, Thompson, Elgie, Snedeker and Marcynuk2009b). Risk of bias due to randomization was assessed as ‘low’ if the authors reported randomization and details of the method used to generate the sequence, ‘some concerns’ if random allocation was reported but no details on how the random sequence was generated were reported, and ‘high’ if no information on allocation was provided or if a non-random method was used. Risk of bias due to blinding was assessed as ‘low’ if both caregivers and outcomes assessors were blind to the treatment group, ‘unclear’ if caregivers or outcome assessors were blinded but not both, and ‘high’ if neither caregivers nor outcome assessors were blinded.
Indirectness (how closely the populations in the included studies resembled the target populations for the intervention) was not considered to be an issue due to the eligibility criteria for the review, and therefore the risk of bias was considered ‘low’ for all studies. Bias due to imprecision was assessed using 0.8 as a clinically important OR. Similarly, a 0.8 OR was used to assess heterogeneity. Incoherence (inconsistency) analysis was not reported from CINeMA as this was conducted using Bayesian analysis.
The process recommended to assess across-studies bias in an NMA is not well developed. Further, no pairwise comparisons in this review included more than 10 trials, which is the number typically believed to be necessary for an accurate across-studies bias assessment (Sterne et al., Reference Sterne, Gavaghan and Egger2000). Therefore, across-studies bias was not evaluated.
Results
Study selection
Results of the search and flow of studies through the screening process are presented in Fig. 1, including reasons for full-text exclusions. Full details on all searches are available as Supplemental File S3.
Fig. 1. Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) study flow diagram (Moher et al., Reference Moher, Shamseer, Clarke, Ghersi, Liberati, Petticrew, Shekelle and Stewart2015) for the systematic review of trials examining the efficacy of antimicrobials given at dry-off.
From an initial 3480 articles screened by title and abstract, 756 full texts were reviewed, with 697 not meeting full-text eligibility criteria, and 59 studies including 75 trials included after full-text screening. Of these 75 clinical trials, 35 had data that were not usable (e.g. data not presented, no variance measure provided for continuous outcomes, data presented in graphs or figures only, etc.). Therefore, data were extracted for one or more outcomes from 40 trials.
Study characteristics
Full details on study characteristics of the 40 trials with data extracted for one or more outcomes are included as Supplemental File S4. Studies were conducted in 12 countries, most frequently in the United States (n = 12), New Zealand (n = 4), and the United Kingdom (n = 3). The country of conduct was not reported in 30% of studies (n = 12). Study setting was most commonly a commercial dairy (28/40; 70%), with a small number of studies conducted at a research facility (7/40; 18%), or a combination of a research facility and commercial dairies (2/40; 5%). In three studies, the setting was not reported. The majority of studies did not report year of conduct (28/40; 70%), with eight studies (20%) conducted since 2000, and four studies (10%) conducted prior to 2000. Breed was reported in 21 (53%) studies, with Holstein/Friesian (n = 13; 33%) and cross-bred or multiple breeds (n = 8) being reported. Sixteen studies were conducted in a single herd (40%), and the number of herds ranged from 1 to 75. The number of herds was reported in all but one study.
Outcomes
Of the 40 included trials, IMI at calving was the most commonly reported outcome (n = 39), with three trials reporting the incidence of clinical mastitis in the first 30 DIM, and one reporting the incidence of IMI in the first 30 DIM. For additional outcomes in these included trials, two reported linear score (LS) or SCC at first test after calving, and one reported milk production over the subsequent lactation.
A new IMI was most commonly defined as the growth of a new pathogen on the follow-up sample (28/39; 72%), while eight trials (21%) defined new IMI as initially no pathogen growth on the dry-off sample, followed by growth of one or two pathogens on follow-up sampling. Three trials did not report how a new IMI was defined. Follow-up sampling was done at calving in 17 trials (44%), while the remaining trials measured from 1 to 15 DIM.
Risk of bias – IMI at calving
The results of the risk-of-bias assessment for the 36 trials included in the network meta-analysis are presented in Fig. 2, showing risk in the five evaluated domains for each outcome assessed in the network meta-analysis of IMI at calving. All trials were rated with an overall risk of bias as either ‘some concerns’ or ‘high’ (the trial's highest risk of bias in any one domain).
Fig. 2. Risk of bias by domain for trials included in the network meta-analysis assessing the efficacy of antimicrobials given at dry-off to prevent intramammary infections (IMI) at calving (n = 36). Risk of bias was assessed according to the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) (Higgins et al., Reference Higgins, Sterne, Savovic, Page, Hróbjartsson and Boutron2016).
For bias arising from the randomization process, all studies were assessed as ‘some concerns’. This was primarily driven by a lack of reporting, as only one trial reported if the allocation sequence was concealed when cows were assigned to intervention groups, and random allocation of treatment with information on the method used to generate the random sequence was reported in 4/36 trials (11%). An additional 15 trials reported random assignment of cows or quarter to treatment, but did not provide evidence of randomization, eight reported a non-random process (such as even- and odd-numbered ear tags), and nine did not provide sufficient information to assess this area.
Bias due to deviations from intended interventions in many studies was assessed as ‘some concerns’ (28/36; 78%), as blinding of caregivers and study personnel was uncommonly done. As well, details regarding deviations from intended interventions were not often reported, although animals were commonly housed in mixed groups where differential care would be implausible. Bias due to missing outcome data was generally assessed as low risk (19/36; 53%), with 16 studies rated as some concerns, and one with a high risk of bias. ‘Some concerns’ resulted from a lack of reported information on loss to follow-up, and a ‘high’ risk of bias was due to a high level of missing data that was non-random or unequal between groups where results likely were not robust to the presence of missing data.
Bias due to measurement of the outcome was considered to be low in all trials, as although blinding of outcome assessors was rarely done (5/36; 14%), laboratory diagnoses were often used and considered relatively objective.
For bias arising from the selection of the reported results, information regarding a priori intentions of outcome measurements and analyses were not available for any studies; this domain generally requires the examination of a trial protocol or statistical analysis plan documented ahead of the trial if there are multiple ways an outcome could be measured or analyzed. As a result, all trials were assessed as ‘some concerns’ in this area.
Results of individual studies
Studies with data extracted but not included in the meta-analysis were a result of treatments being collapsed to a single arm per study (two trials), zero cells in event columns (one trial), or if the trial contained no treatment arms which linked to the network (zero trials). Of the 36 included trials, three reported adjusted data and 33 reported raw data. Thirty trials reported results at the quarter level, but only three trials controlled for clustering within the cow. Eighteen trials enrolled cows on multiple farms; two presented data adjusted for lack of independence within herd.
Quantitative summary
A network meta-analysis was conducted for trials examining the incidence of IMI at calving. No other analyses were conducted as very few studies were found that examined the incidence of IMI or clinical mastitis in the first 30 DIM for an informative network meta-analysis.
Network meta-analysis – incidence of intramammary infection at calving
The full network plot for IMI at calving is shown in Fig. 3; all treatments identified for this outcome were connected through one or more common trial arms. The network of evidence used in the meta-analysis is shown in Fig. 4, and represents 79 intervention arms from 36 trials, including 28 two-arm trials, six three-arm trials, one four-arm trial, and one five-arm trial. A full description of treatment acronyms used in Figs. 3 and 4 is given in Table 2.
Fig. 4. Treatment arm network for the examination of the relative efficacy of antimicrobials given at dry-off to prevent intramammary infections (IMI) at calving. The size of the circle indicates the relative number of arms and the width of the lines indicates the relative number of direct comparisons. The number in brackets is the number of arms involving the product. Full treatment arm descriptions are presented in Table 2.
Assessment of consistency
The consistency assessment for all direct and indirect comparisons is shown in Table 3. Means and standard deviations of log OR of treatment effects are shown using direct (head-to-head) evidence only, indirect evidence only, and the combined evidence. The inconsistency estimate and standard deviation are presented, and there was no evidence of significant inconsistency between direct and indirect estimates. The contribution of studies to estimates based on randomization status is presented in Fig. 5, and the contribution of studies to estimates based on blinding in Fig. 6. Most pairwise comparisons (34/45) included a majority contribution from studies that did not report random allocation or reported a non-random method, while 9/45 had a majority contribution from studies describing random allocation with no supporting evidence. A small proportion of the contribution in some pairwise comparisons came from studies reporting random allocation to treatment with supporting evidence, but this was not the majority contribution for a single pairwise comparison. For contributions of studies to estimates based on blinding (Fig. 6), in most pairwise comparisons, there was only a very small, or no, contribution from studies reporting blinding of either caregiver or outcome assessor, and a smaller yet contribution from those reporting blinding of both. The majority contribution in 43/45 pairwise comparisons was from studies not reporting blinding of either caregivers or outcome assessors. Table 4 summarizes the majority contribution for each pairwise comparison for randomization and blinding, imprecision, and heterogeneity.
Fig. 5. The contribution of studies to the point estimate based on the description of allocation approach for studies contributing to the network meta-analysis examining the relative efficacy of antimicrobial treatments given at dry-off to prevent intramammary infections (IMI) at calving (n = 36). Green indicates studies that randomly allocated to treatment and provided evidence of random sequence generation, yellow indicates studies that reported random allocation but did not provide supporting evidence, and red indicates studies that did not report allocation approach or reported a non-random method. White vertical lines indicate the percentage contribution of separate studies.
Fig. 6. The contribution of studies to the point estimate based on the description of blinding for studies contributing to the network meta-analysis examining the relative efficacy of antimicrobial treatments given at dry-off to prevent intramammary infections (IMI) at calving (n = 36). Green indicates studies that reported both caregivers and outcome assessors were blinded to treatments, yellow indicates studies that reported caregivers or outcome assessors were blinded to treatment (but not both), and red indicates studies where blinding was not used, or not reported, for both caregivers and outcome assessors. White vertical lines indicate the percentage contribution of separate studies.
Table 3. Direct (dir) and indirect (rest) comparisons for the consistency assumption of pairwise comparisons within the network of studies examining the efficacy of antimicrobials given at dry-off to prevent new intramammary infections (IMI) at calving
The inconsistency estimate (ωXY) and standard deviation (SDωXY) are shown. Posterior means (d) and standard deviation (SD) of the log-odds ratio of intervention effects calculated for direct (head-to-head) evidence only (dir), indirect evidence only (rest), and a combination of all evidence (MTC). The first treatment listed is the referent (denominator) and the second listed is the comparator (numerator).
Table 4. Summary of the overall quality of evidence of the network of studies examining the efficacy of antimicrobial treatments given at dry-off to prevent new intramammary infections (IMI) at calving, using the Confidence In Network Meta-Analysis (CINeMA) platform (http://cinema.ispm.ch), with a modified approach, to determine the risk of bias due to approach to randomization, blinding, imprecision, and heterogeneity
Imprecision and heterogeneity were determined using a clinically important odds ratio of 0.8.
Rankings and distribution probability of IMI at calving
RRs from the network meta-analysis comparing all treatments are shown in Table 5. The RR is the risk of the event (IMI at calving) in the column header (numerator), divided by the risk of the event in the row header (denominator). For example, the estimated risk of IMI at calving was 2.68 times greater in non-active controls (NAC) compared to those given cephalosporin (CEPH) at dry-off. The corresponding confidence interval is located in the lower left-hand section of the table, with rows and columns reversed (95% CI 1.53–4.32).
Table 5. Risk ratio comparison of all interventions assessed in the network meta-analysis for the outcome of IMI at calving
The upper right-hand section of the table represents the risk ratio between the numerator (upper left treatment) and denominator (lower right treatment). The lower left section of the table represents the 95% credibility interval for the comparison, with the rows and columns reversed. For example, the risk ratio for IMI at calving for a non-treated control (NAC) compared to cephalosporin (CEPH) is 2.68 (95% CI 1.53–4.32).
Mean rankings and 95% credibility intervals are presented as a forest plot (Fig. 7), and in Table 6 where rankings at the 2.5, 50, and 97.5% points of the distribution are shown. The distribution of the probability of treatment failure (probability of an IMI event at calving) is presented for each treatment in the network meta-analysis in Fig. 8a–c.
Fig. 7. Forest plot of mean rank and 95% credibility interval for the network meta-analysis examining the relative efficacy of antimicrobial treatments given at dry-off to prevent intramammary infections (IMI) at calving. Full treatment arm descriptions are presented in Table 2.
Fig. 8. (a–c) The distribution of the probability of treatment failure in the 5000 simulations in the network meta-analysis examining the relative efficacy of antimicrobials given at dry-off to prevent intramammary infections (IMI) at calving.
Table 6. Mean rank, standard deviation, and rankings at the 2.5, 50, and 97.5% points of the distribution for treatments in the network meta-analysis examining the relative efficacy of antimicrobial treatments given at dry-off to prevent intramammary infections (IMI) at calving
Full treatment arm descriptions are presented in Table 2.
Risk of a new IMI at calving was higher for non-treated controls compared to cloxacillin (RR = 1.83, 95% CI 1.26–2.60), cephalosporins (RR = 2.68, 95% CI 1.53–4.32), and penicillin with aminoglycosides (RR = 2.36, 95% CI 1.38, 3.88). However, 95% credibility intervals had rankings that overlapped for non-treated controls, cloxacillin and penicillin with aminoglycosides. Between antimicrobial protocols, due to imprecision of estimation, differences in the RR of IMI at calving between antimicrobials were not observable.
Discussion
Multiple intervention options exist for cows at dry-off to prevent IMI and clinical mastitis. Relative efficacy is an important component of decision making, as rarely do producers or veterinarians only wish to know the efficacy of a product compared to a non-treated control, or to an incomplete set of comparators. While clinical perceptions of relative efficacy may be based on observations or anecdote, network meta-analysis provides an evidence-based instrument to afford decision makers with information regarding relative efficacy. In addition to relative efficacy, treatment decisions may be driven by multiple additional factors, including availability, cost (e.g. direct costs, discarded milk, residue risk, etc.), and importance to human health. With these in mind, relative efficacy can help inform decision making; for example, if two treatments are not different in efficacy, one with a lower cost, or lower importance to human health, can be selected. Similarly, the use of apparently ineffective products can be avoided to decrease unnecessary antimicrobial use.
Summary of evidence
Based on the evidence presented here, the use of a cephalosporin, cloxacillin, or penicillin with aminoglycoside appeared to be more effective than no treatment at preventing new IMI at calving, when given to cows without pre-existing IMI at dry off. However, the definition of a ‘new IMI’ varied, and may contribute to differences between studies.
For the comparison of non-treated controls to cephalosporin, cloxacillin, or penicillin with aminoglycoside, imprecision was assessed as ‘no concerns’, which indicates that the 95% CI around the point estimate does not include values that would lead to different clinical decisions, based on a clinically significant OR of 0.8. However, some concerns were noted due to heterogeneity (cephalosporin, penicillin with aminoglycoside) and major concerns in the case of cloxacillin. This is a result of the 95% credibility interval not agreeing in relation to the predetermined clinically important effect, meaning the interval spans values which would lead to different clinical decisions. This indicates there are some between-study variations within these comparisons, which could be due (in part) to different study populations or definitions of the outcome. Examining the pairwise comparisons between antimicrobials, the majority had major concerns with regard to imprecision, meaning the 95% CI extends into the estimated ORs favoring either treatment (‘major concerns’). This may be driven by the small number of studies included for each unique treatment (Fig. 4). With such large confidence intervals, it is not possible to compare to the predicative interval to assess heterogeneity (all would be ranked as ‘no concerns’ simply based on the wide 95% CI).
All treatments found in the studies meeting criteria for data extraction were connected by one or more intervention arms, which allowed for estimates of relative efficacy for all interventions extracted. When treatment arms are not common to multiple trials, the utility of the original research is impaired.
Blinding of caregivers and outcome assessors was uncommonly reported for studies evaluating the incidence of IMI at calving (Fig. 6); however, as this outcome is objective, this resulted in a low overall risk of bias due to the assessment of the outcome (Fig. 2). However, bias arising from missing outcome data was observed in some trials, which in some cases was due to a lack of reporting of the number of study units analyzed. The Reporting guidElines For randomized controL trials in livEstoCk and food safTey (REFLECT) statement recommends that authors report the flow of study units through each stage of the study, including the number allocated, receiving the intervention, completing the protocol, and analyzed for each outcome, with the use of a diagram recommended (O'Connor et al., Reference O'Connor, Sargeant, Gardner, Dickson, Torrence, Dewey, Dohoo, Evans, Gray, Greiner, Keefe, Lefebvre, Morley, Ramirez, Sischo, Smith, Snedeker, Sofos, Ward and Wills2010; Sargeant et al., Reference Sargeant, O'Connor, Gardner, Dickson and Torrence2010).
Randomization was done in some (4/36) trials, but non-random allocation, such as assignment by even or odd ear tag number, was conducted in several, and many did not report the method of allocation. There is evidence that reporting of randomization has improved since the publication of reporting guidelines such as the REFLECT statement (Totton et al., Reference Totton, Cullen, Sargeant and O'Connor2018). However, reporting specific to dairy science revealed that although 104 of a sample of 137 trials published in 2017 reported random allocation to study group, only seven reported the method of randomization (Winder et al., Reference Winder, Churchill, Sargeant, LeBlanc, O'Connor and Renaud2019). Assumptions for many statistical methods rely on interchangeable groups, and failure to randomize has been shown to be associated with exaggerated treatment effects (Burns and O'Connor, Reference Burns and O'Connor2008; Sargeant et al., Reference Sargeant, Elgie, Valcour, Saint-Onge, Thompson, Marcynuk and Snedeker2009a; Brace et al., Reference Brace, Taylor and O'Connor2010). Even in trials of genetically identical mice, failure to randomize has shown similar associations (Egan et al., Reference Egan, Vesterinen, Beglopoulos, Sena and Macleod2016).
Limitations of the body of literature
Despite a large number of trials in this area, there was a limited number of studies eligible to be combined in the meta-analysis (Fig. 1). Lack of comparable outcomes and inadequate presentation of required data were the most common reasons for the exclusion of trials from the network. However, these limitations of a sparse body of comparable work pertain to any research synthesis approach.
Case definition varied within the single outcome of IMI at calving. The exact role of existing minor pathogen IMI on the risk of new major pathogen IMI is unclear, as a protective effect has been reported in challenge trials, but not observational studies, and there is a large amount of heterogeneity in these meta-analyses (Reyher et al., Reference Reyher, Haine, Dohoo and Revie2012). If the existing infection does influence the risk of a new infection, then it is important that primary research consider this and ensure adequate reporting of the case definition. Risk period was also variable among studies, which, assuming this has influence on outcomes, limits the ability to further utilize this body of research. Standardized outcomes with biological meaning for a given intervention would strengthen the value of primary research. In human health, efforts to standardize outcome measures exist in multiple research areas (Williamson et al., Reference Williamson, Altman, Blazeby, Clarke, Devane, Gargon and Peter2012; Macefield et al., Reference Macefield, Jacobs, Korfage, Nicklin, Whistance, Brookes, Sprangers and Blazeby2014).
Confidence intervals surrounding the more commonly replicated interventions were also quite wide. This highlights the need for replication, in order to derive more precise estimates of efficacy and appropriately rank treatments, for interventions of interest to end users.
Limitations of the review
A large number of studies were excluded at full-text screening as they were not available in English, and as a result, our conclusions may not reflect the entirety of literature assessing the efficacy of dry-cow antimicrobial therapy on the prevention of IMI and CM. Although it is unlikely that language would be associated with different estimates of effect, additional studies would have increased the precision of estimation.
Additionally, the outcome assessed in the network (IMI at calving) likely reflects a variety of pathogens, which may differ between study populations. The efficacy of each antimicrobial for prevention may differ by an agent based on the differences in pharmacology, and this may have accounted for some of the heterogeneity seen across studies. Treatments were grouped based on OIE antimicrobial category, and therefore there may be differential effects of specific antimicrobials (e.g. product, dose) within a collapsed category. However, assigning each product and dose to a unique treatment would have resulted in an increasingly sparse network, and we attempted to be transparent with how these data were grouped for analysis.
Conclusions
From the network of evidence produced by this analysis, it was apparent that the use of cephalosporins, cloxacillin, or penicillin with aminoglycoside given to cows without existing IMI at dry-off provided a significantly protective effect for the development of new IMI at calving, compared to non-treated controls. There were no apparent differences among these antimicrobials. However, the precision of the estimates of the comparisons among antimicrobials was of major concern due to wide confidence intervals on the estimated rankings, meaning it is possible the true effects of some of these treatments are not equivalent. Synthesis of the primary research revealed challenges with comparable outcomes, replication and connection of interventions, and quality of reporting of study conduct in order to assess the potential risk of bias in the reported results. Consideration of the use of reporting guidelines by journals and authors, and standardized outcomes would increase the value of primary research in this area.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S1466252319000239.
Author contributions
CBW assisted with the development of the review protocol, co-coordinated the research team, assisted with data screening, extraction, and risk of bias assessment, interpreted the results, and wrote the manuscript drafts. JMS developed the review protocol, co-coordinated the research team, interpreted the results, commented on manuscript drafts, and approved the final manuscript. DH conducted the data analysis, provided guidance for the interpretation of results, commented on manuscript drafts, and approved the final manuscript. CW assisted with the development of the review protocol, provided guidance on the conduct of the analysis and interpretation of results, and approved the final manuscript. JG and HW developed the search strings, conducted all searches, commented on manuscript drafts, and approved the final manuscript. KJC, MdB, KD, SM, BD, MR, and CM conducted relevance screening, extracted data, conducted the risk-of-bias assessments, commented on manuscript drafts, and approved the final manuscript version. AMOC, DFK, SJL, and TFD co-developed the review protocol, provided guidance on the interpretation of results, commented on manuscript drafts, and approved the final manuscript.
Financial support
Support for this project was provided by The Pew Charitable Trusts.
Conflict of interest
None of the authors has conflicts to declare.
Open access
