Introduction
The newborn calf is highly dependent on colostrum intake to achieve an optimal immune protection. The level of transfer of passive immunity is most often assessed by the serum immunoglobulin G concentration (IgG) measured during the first week of life (Weaver et al., Reference Weaver, Tyler, VanMetre, Hostetler and Barrington2000). The IgG type 1 (IgG1) is actively secreted by transcytosis in the mammary gland and represents the largest fraction of IgG in bovine colostrum (compared with low concentration of IgG2) (Baumrucker and Bruckmaier, Reference Baumrucker and Bruckmaier2014). Failure of passive transfer (FPT) is classically diagnosed when serum IgG concentration is <10 g l−1 (Besser et al., Reference Besser, Gay and Pritchett1991; Furman-Fratczak et al., Reference Furman-Fratczak, Rzasa and Stefaniak2011) but cut-offs of 12 g l−1 (Robison et al., Reference Robison, Stott and DeNise1988) and 16 g l−1 (Wittum and Perino, Reference Wittum and Perino1995) or 27 g l−1 (Dewell et al., Reference Dewell, Hungerford, Keen, Laegreid, Griffin, Rupp and Grotelueschen2006) in beef calves have also been reported. In a recent US study, 19.2% of dairy calves presented evidence of partial or complete FPT, and 40% of the 394 participating dairy farms had at least one calf with FPT (Beam et al., Reference Beam, Lombard, Kopral, Garber, Winter, Hicks and Schlater2009). Partial or total FPT has been associated with increased risk of morbidity and mortality (Godden, Reference Godden2008; Windeyer et al., Reference Windeyer, Leslie, Godden, Hodgins, Lissemore and LeBlanc2014).
Determination of transfer of passive immunity can be performed by different methods that are correlated with serum IgG concentrations. The IgG concentration can be directly measured by different tests (radial immunodiffusion assay (RID) or turbidimetric immunoassay (TIA)). The RID is the reference method to assess IgG (Weaver et al., Reference Weaver, Tyler, VanMetre, Hostetler and Barrington2000). The TIA has also been used as valuable and reliable to assess IgG concentrations and suggested as a reference method (McVicker et al., Reference McVicker, Rouse, Fowler, Perry, Miller and Johnson2002). Both methods are highly reliable to quantify IgG concentrations with high correlation between both methods (R 2 = 0.98) (Alley et al., Reference Alley, Haines and Smith2012). However, they cannot be used easily on the farms because the test results are classically obtained 24–36 h after sampling and the techniques need to be performed in a laboratory. For these reasons, several more practical methods that are correlated with serum IgG concentrations have been investigated (Tyler et al., Reference Tyler, Hancock, Parish, Rea, Besser, Sanders and Wilson1996; Deelen et al., Reference Deelen, Ollivett, Haines and Leslie2014). Among these methods, optical or digital refractometry (REF) and Brix refractometry (BRIX) have been reported as valuable techniques to assess IgG concentration (Chigerwe et al., Reference Chigerwe, Tyler, Middleton, Spain, Dill and Steevens2008; Elsohaby et al., Reference Elsohaby, McClure and Keefe2015). Both tests used serum samples and the result given is a continuous number (in g l−1 for the refractometer or % in Brix refractometry). Specific cut-off points have been used to define acceptable level to achieve adequate transfer of passive immunity (5.0, 5.2 or 5.5 g l−1 with refractometry (Weaver et al., Reference Weaver, Tyler, VanMetre, Hostetler and Barrington2000; McGuirk and Collins, Reference McGuirk and Collins2004; Godden, Reference Godden2008)).
The diagnostic accuracy of REF and BRIX has been evaluated in various studies estimating test sensitivity (Sensitivity (Se): proportion of calves with FPT with a positive test) and specificity (Specificity (Sp): proportion of calves without FPT with a negative test). However, direct comparison between studies may be hazardous without a structured approach acknowledging for within and between study variability. The evidence-based approach is now more frequently used in veterinary medicine to have a structured approach to assess the available evidence (Vandeweerd et al., Reference Vandeweerd, Gustin and Buczinski2012). Diagnostic test accuracy can also be assessed using an evidence-based approach to identify eventual sources of heterogeneity (O'Connor and Evans, Reference O'Connor and Evans2007; Leeflang et al., Reference Leeflang, Deeks, Takwoingi and Macaskill2013).
Rationale
The diagnostic accuracy (as assessed by Se and Sp) of REF and BRIX should be precisely reported in order to provide informed advice regarding the probability of FPT in the tested calves. Potential limitations impacting test accuracy at the individual and population level should be acknowledged as well. Improving FPT diagnosis would improve calf management as well as contributing in the reduction of occurrence of infectious diseases and therefore antimicrobial use. The aim of this paper is to provide a protocol for a systematic review following the preferred reporting items for systematic review and meta-analysis (PRISMA) statement (Moher et al., Reference Moher, Shamseer, Clarke, Ghersi, Liberati, Petticrew, Shekelle and Stewart2015).
Objectives
The objective of this study is to perform a systematic review and meta-analysis of the two index tests REF and BRIX in diagnosing FPT based on IgG concentration determination (<versus≥ cut-off). The specific question we are trying to answer is: ‘What is the overall diagnostic accuracy of the index tests REF or BRIX for to diagnose FPT using RID or TIA as reference standards in neonatal calves?’ We will use a classification model including for each selected study the classification of the calves according to their reference standard (pass or fail depending on the cut-off) and index test results (<versus≥ cut-off).
Methods
The following approaches will be used to answer the previous question: (i) determination of the global accuracy of these tests for diagnosing FPT using the hierarchical summary operating characteristic (HSROC) curve; (ii) determination of summary estimates of Se/Sp for specific REF and BRIX cut-offs (if five or more studies on the same cut-offs are available), 95% confidence intervals (CI) and prediction intervals (PI); and (iii) assessing the impact of study covariate, study quality or reference standard (RID or TIA) in BRIX and REF accuracy.
Eligibility criteria
Target conditions being diagnosed
The target condition is FPT defined as a serum concentration of IgG (total IgG (or IgG1 if specific IgG1 test used)) <10; <12; <16 or <27 g l−1 depending on the cut-off presented in the study. This IgG measurement should have been performed in calves aged from 1 to 8 days.
Reference standards
The reference standards used to diagnose the target condition will be the RID and TIA to determine IgG (or IgG1) concentration.
Index tests
The index tests assessed are: (i) refractometer using a g l−1 scale and (ii) refractometer readings on a Brix scale (%).
Clinical pathway to use the index tests
The BRIX and REF tests are routinely performed on a daily basis by veterinarians to replace the direct determination of IgG which is time consuming, fastidious and expensive.
Criteria for considering studies for this review
The studies that will be considered to be included for this review will be any study one-gate (cohort) or two-gate (case–control) study (Rutjes et al., Reference Rutjes, Reitsma, Vandenbroucke, Glas and Bossuyt2005) reporting reference standard and one of the two index tests for assessing FPT status in calves. The studies will be screened based on a specific literature search and reference lists screening of retrieved articles. Studies reporting coefficient of determination (R 2) between the index test and reference standards will also be kept as a measure of association in the different studies.
Participants
The participants will be dairy or beef calves that would have been blood sampled between 1 and 8 days of life. Only studies reporting calves that had received cows’ colostrum would be included. The calves receiving colostrum replacer products will be excluded from the present study due to the variability in IgG concentration of the products (Godden, Reference Godden2008).
Search strategy for identification of studies
The identification of potential studies will be performed using the electronic databases Pubmed and CAB Abstracts from 1986 to 2016. Research abstracts will be searched in the Searchable Proceeding of Animal Conferences (S-PAC) and thesis will be assessed using the Pro-Quest Dissertation and Thesis Global database. The gray literature will also be assessed using Google-Scholar search. The exact research strategy is indicated in the Appendix. The sensitivity analysis was performed by one author (S.B.) for Pubmed and CABAbstract searching for eight studies previously identified as relevant articles to be found.
The articles written in English, French or Spanish will potentially be included. The screening, eligibility and inclusion of the relevant articles to be included will be performed by two independent reviewers. In case of any discrepancy between the two reviewers, a third reviewer will be consulted and a consensus will be reached.
Data collection and analysis
The data collection and analysis will be performed using RevMan 5.3 (Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014) and SAS software (version 9.3, SAS Institute, Cary, NC, USA).
Selection of studies
The studies will be included in the systematic review if the specific information for retrieving a 2 × 2 table (i.e. cross-classification of calves according to one REF or BRIX cut-offs and FPT status defined by the reference standard test) is available. The studies reporting association between both tests (Pearson R 2: coefficient of determination) will also be collected to be described although R 2 cannot be used as a measure of diagnostic accuracy or agreement between two tests (Kottner et al., Reference Kottner, Audige, Brorson, Donner, Gajewski, Hrobjartsson, Roberts, Shoukri and Streiner2011). The proportion of studies only reporting these parameter versus accuracy measurements will be described.
Data extraction and management
For each study the data extracted will be the first author and date of publication, country where the study was performed, the study design (one-gate versus two-gate (Rutjes et al., Reference Rutjes, Reitsma, Vandenbroucke, Glas and Bossuyt2005)), peer-reviewed versus non-peer-reviewed study, the fact that diagnostic accuracy was the main objective of the study, the type of calves used (beef, dairy or veal calves), the age range at sampling, the reference standard test used (RID or TIA), IgG or IgG1 cut-off for FPT definition, the use of an optical versus digital refractometer, fresh or refrigerated versus frozen serum samples, as well as the prevalence of FPT in the study as defined by the reference standard:
$$\eqalign{P_{\rm FPT} &= n_{{\rm Calves}\;{\rm with}\;{\rm IgG \lt cut - off}\;{\rm (RID}\;{\rm or}\;{\rm TIA)}}/ \cr &\quad \left(n_{{\rm Calves}\;{\rm with}\;{\rm IgG \lt cut - off}\;{\rm (RID}\;{\rm or}\;{\rm TIA)}} \right. \cr &\quad \left. + n_{{\rm Calves}\;{\rm with}\;{\rm IgG^{3}cut \ge off (RID}\;{\rm or}\;{\rm TIA)}} \right).} $$
This prevalence will then be used to estimate study with low versus high prevalence of FPT using the median prevalence of selected studies to dichotomize the data. The specific information concerning the 2 × 2 accuracy table will be obtained (with true positive (TP): number of calves with FPT that are index test positive, true negative (TN): number of calves without FPT that are index test negative, false positive (FP): number of calves without FPT that are index test positive and false negative (FN): number of calves with FPT that are test negative). This information will be retrieved directly or after calculation from the Se, Sp and predictive values of the test or prevalence of FPT based on the total of animals tested in the study using RevMan software. For studies reporting multiple cut-offs with accuracy parameters, all the possible 2 × 2 table information will be retrieved. These data will be completed by two independent reviewers and any discrepancy between the reviewers will automatically be submitted to a third reviewer until a consensus would be obtained.
The specific information concerning the suggested cut-offs in every study will be used for modeling HSROC model of test accuracy. If no specific cut-off was proposed by the authors, one will be randomly assigned from the different proposed cut-offs.
Assessment of methodological quality
The methodological quality of the studies will be assessed using the QUADAS-2 tool (Whiting et al., Reference Whiting, Rutjes, Westwood, Mallett, Deeks, Reitsma, Leeflang, Sterne and Bossuyt2011). This tool assesses the risk of bias and applicability concerns of the study. The QUADAS-2 is also the assessment tool available in RevMan for diagnostic test accuracy systematic review. The QUADAS-2 results will be obtained from two different reviewers and disagreement will be resolved by a third reviewer.
Statistical analysis and data synthesis
The forest plots of Se and Sp of BRIX and REF for FPT diagnosis will be reported from RevMan outputs. A hierarchical summary receiver operating characteristic (HSROC) curve will be obtained using RevMan and SAS METADAS macro output since multiple cut-offs are likely to occur depending of the studies and a between and within study variation should be taken into account (Rutter and Gatsonis, Reference Rutter and Gatsonis2001; Takwoingi and Deeks, Reference Takwoingi and Deeks2008). For specific BRIX and REF cut-offs reported in five or more studies (which is the minimal number we consider to do any additional analysis), a specific summary estimate of Se and Sp as well as 95% CI and 95% PI will be obtained. These estimates will be obtained taking into account within and between study variability for each cut-off. We anticipate a cut-off effect with higher cut-offs having higher Sp and lower Se than lower cut-offs. The comparative diagnostic accuracy of BRIX versus REF will be calculated if possible from studies where both tests were performed simultaneously.
Sensitivity analysis
A pre-specified sensitivity analysis will be performed excluding the studies with higher risk of bias or peer-reviewed studies. Sensitivity analysis does not replace heterogeneity investigation but it helps the reader to assess if any arbitrary decision taken during the review process had an impact on the review findings (for instance, Can inclusion or removing studies where only an abstract was available, or low-quality studies based on QUADAS-2 tool have an impact on the test accuracy?).
Investigations of heterogeneity
The heterogeneity will be assessed for specific study or quality (based on QUADAS-2 assessment) covariates that present at least five studies per class (for instance, peer-reviewed versus non-peer-reviewed, low versus high prevalence of FPT, digital versus optical refractometer). The impact of these covariates will be assessed in HSROC modeling as previously recommended (Takwoingi and Deeks, Reference Takwoingi and Deeks2008).
Assessment of reporting bias
The assessment of publication bias (i.e. probability that studies with lower accuracy findings were not reported) would be obtained based on the retrieved studies for each index test and using the specific data from each suggested cut-off. The Deek's test plotting the 1/(root(effective sample size)) = f(ln DOR) (where ‘DOR’ is diagnostic odds ratio of the test: TP*TN/(FP*FN)) will be used to assess publication bias as recommended for meta-analysis of diagnostic accuracy study (van Enst et al., Reference van Enst, Ochodo, Scholten, Hooft and Leeflang2014). A P < 0.10 will be considered as an indicator of publication bias (Deeks et al., Reference Deeks, Macaskill and Irwig2005).
Comparison of BRIX and REF accuracy for predicting FPT
A specific comparison of HSROC curves will be performed for studies reporting both BRIX and REF accuracy if there are at least five studies using these two tests in the same population.
Practical applications of systematic review findings using stochastic modeling
One of the main challenges of any systematic review of diagnostic test accuracy is to be able to translate the main findings in a way that can be easily understood by the end-users of the tests (Leeflang et al., Reference Leeflang, Deeks, Takwoingi and Macaskill2013; Zhelev et al., Reference Zhelev, Garside and Hyde2013); in this specific case, farmers and veterinary practitioners. Based on the Se/Sp (95% CI) estimates obtained for each specific cut-off, as well as on the ranges of FPT prevalence from the different studies included in the review, a stochastic modeling with Monte-Carlo simulation will be performed. This model will be used to assess the added value of the test for detecting FPT (ModelRisk, Vose Software BVBA, Gent, Belgium). This model will be based on the Bayes theorem (Gardner, Reference Gardner2002):
$${\rm LR}{^\ast}{\,p_{{\rm pre - test}}}{\rm /}\left( {1 - {\,p_{{\rm pre - test}}}} \right) = {\,p_{{\rm post - test}}}/\left( {1 - {\,p_{{\rm post - test}}}} \right),$$
where LR is the likelihood ratio of a positive (LR+ = Se/(1 − Sp)) or negative test (LR− = (1 Se)/Sp); p pre-test is the pre-test probability of FPT and p post-test is the post-test probability upgraded by the test result. Using this stochastic approach will help to determine the practical added value of the test determining the increase or decrease of post-test probability of presenting FPT if the animal is positive or negative.
Conclusions
We aim to be able to make a comprehensive review and recommendations of the best test and cut-offs to use for assessing FPT status in calves. This study will therefore be helpful for veterinary practitioners, producers and researchers to improve the management of passive transfer immunity in dairy and beef farms. Improving the management of passive transfer immunity for the calves will help to improve calf health and to decrease infectious diseases and consequently antimicrobial use during the pre-weaning period.
Author contribution
S.B. is the main author and guarantor of the study. He provided the first draft of the protocol. All authors have been significantly involved in the preparation, reviewing and suggestions concerning the current systematic review protocol.
Financial support
This review received no specific source of funding. None of the authors had any specific conflict of interest regarding the current study.
APPENDIX SEARCH STRATEGY
Research strategy Pubmed
-
1. (((((refractometry[MeSH Terms]) OR index, refractive[MeSH Terms]) OR indices, refractive[MeSH Terms]) OR error, refractive[MeSH Terms]) OR errors, refractive[MeSH Terms]) OR serum proteins[MeSH Terms]
-
2. (((((((((((((((allotype, immunoglobulin[MeSH Terms]) OR allotype, immunoglobulin gm[MeSH Terms]) OR allotypes, immunoglobulin gm[MeSH Terms]) OR allotype, ig[MeSH Terms]) OR allotypes, ig[MeSH Terms]) OR deficiencies, igg[MeSH Terms]) OR deficiency, igg[MeSH Terms]) OR antibody deficiency syndrome[MeSH Terms]) OR antibody deficiency syndromes[MeSH Terms]) OR passive transfer tests[MeSH Terms]) OR passive transfer test[MeSH Terms]) OR passive transfer of immunity[MeSH Terms]) OR test, passive transfer[MeSH Terms]) OR tests, passive transfer[MeSH Terms]) OR immunodiffusion[MeSH Terms]) OR immunodiffusions[MeSH Terms]
-
3. cattle[MeSH Terms]
-
4. AND/1–3 NOT human
-
5. ((((((((brix) OR refractometry) OR refractometer) OR gravity) OR total solids) OR total solid) OR total protein) OR total proteins) OR refractive index
-
6. ((((((((immunoglobulin) OR immunoglobulins) OR gamma globulin) OR antibody) OR antibodies) OR immunodiffusion) OR passive transfer) OR spectrometry) OR IgG
-
7. ((calf) OR calves)
-
8. AND/5–7
-
9. 4 OR 8
Search strategy CAB Abstract: (1986-now)
-
10. (specific gravity or protein or refractometry or analytical methods or refractive index).sh.
-
11. (immunoglobulins or immunity or colostrum or colostral immunity or passive immunization or antibodies or passive immunity).sh.
-
12. calves/or veal calves/
-
13. AND/1–3
-
14. Brix.mp
-
15. Refractometer.mp
-
16. Refractometry.mp
-
17. OR/5–7
-
18. Passive transfer.mp
-
19. Antibody.mp
-
20. Antibodies.mp
-
21. Immunoglobulin.mp
-
22. IgG.mp
-
23. OR/9–13
-
24. Calf.mp
-
25. Calves.mp
-
26. OR/15–16
-
27. 8 AND 14 AND 17
-
28. 4 OR 18
Search strategy in Searchable proceeding of animal conferences (SPAC)
Refractometer ’ |’ refractometry ‘|’ Brix ‘&’ calves
Search strategy in Google Scholar
Refractometry and ‘passive transfer’
Appendix: Signalling questions
Title selection:
The title has no relation with the topic of interest (calf health): if yes the study should be discarded, if no the abstract should be read.
Abstract selection:
Key signaling question concerning studies at the abstract level
Is the study describing REF or BRIX for determining FPT in calves or accuracy parameter (Se, Sp, predictive values or likelihood ratio or R 2) of one of these two index tests?
All abstracts where this question is answered by ‘yes or maybe’ should go to the second step. If no, the study will not go to the next step.
Manuscript selection:
Key signaling question at the study level:
Is the manuscript reporting REF or BRIX accuracy (correlation) versus (with) IgG concentration determination using the reference standard test (RID or TIA) in calves?
Yes/No (if no the article will not be included)
If yes: is the article reporting any Se/Sp or predictive values and prevalence of FPT? If yes which IgG cut-off is used?
If no: is there any correlation measure between REF or BRIX and IgG ? (R 2)
If none of the above applies, the manuscript will not be included.
For all selected articles specific data extraction will be performed using a pre-specified spreadsheet (see main text). The qualitative assessment will be performed using the QUADAS-2 tool by two independent reviewers.
Other causes of exclusions will include language, species or age restrictions.
