Laboratory-based evaluation of a simplified point-of-care test intended to support treatment decisions in non-severe bovine clinical mastitis

Abstract To limit the use of antimicrobials in dairy cattle, farmers are increasingly encouraged to adopt targeted treatment decisions based on knowledge of the pathogens causing clinical mastitis (CM), whereby treatment of non-severe CM is generally recommended for gram-positive mastitis but not for gram-negative or culture-negative mastitis. The objectives of this study were to conduct a laboratory-based evaluation of the performance of a simplified slide test as a tool to differentiate gram-positive CM from other cases of CM, and to compare its performance against a commercially available on-farm test that is commonly used in our area (VétoRapid). Test outcomes after 24–48 h incubation were compared to results from bacteriological culture and matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-ToF MS). Milk samples (n = 156) were obtained from cases of severe and non-severe CM on seven farms and collected by farm personnel. After removal of small numbers of contaminated samples and organisms with unknown species identity, the simplified slide test showed high sensitivity and accuracy (>80%), similar to the comparator test. For most outcomes of interest (culture positive, Escherichia coli, or gram-positive growth), the specificity of the slide test was higher than the specificity of the comparator test. When considering non-severe cases of CM only, and interpreting detection of gram-positive organisms as indicative of the need for antimicrobial treatment, the simplified test had higher specificity (77.4% v. 60.4%) and higher positive predictive value (79.7% v. 70.0%) than the comparator test and similar sensitivity (83.9% v. 87.5%). The proportion of sampled CM cases, contaminated samples and gram-positive mastitis cases – which affects the positive and negative predictive value, the economic value of diagnostic testing and its potential to reduce antimicrobial use – differed between farms. The simplicity and accuracy of the slide test could make it an attractive tool for farmers to target antimicrobial treatment of non-severe clinical mastitis.


Abstract
To limit the use of antimicrobials in dairy cattle, farmers are increasingly encouraged to adopt targeted treatment decisions based on knowledge of the pathogens causing clinical mastitis (CM), whereby treatment of non-severe CM is generally recommended for gram-positive mastitis but not for gram-negative or culture-negative mastitis. The objectives of this study were to conduct a laboratory-based evaluation of the performance of a simplified slide test as a tool to differentiate gram-positive CM from other cases of CM, and to compare its performance against a commercially available on-farm test that is commonly used in our area (VétoRapid). Test outcomes after 24-48 h incubation were compared to results from bacteriological culture and matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-ToF MS). Milk samples (n = 156) were obtained from cases of severe and non-severe CM on seven farms and collected by farm personnel. After removal of small numbers of contaminated samples and organisms with unknown species identity, the simplified slide test showed high sensitivity and accuracy (>80%), similar to the comparator test. For most outcomes of interest (culture positive, Escherichia coli, or gram-positive growth), the specificity of the slide test was higher than the specificity of the comparator test. When considering non-severe cases of CM only, and interpreting detection of gram-positive organisms as indicative of the need for antimicrobial treatment, the simplified test had higher specificity (77.4% v. 60.4%) and higher positive predictive value (79.7% v. 70.0%) than the comparator test and similar sensitivity (83.9% v. 87.5%). The proportion of sampled CM cases, contaminated samples and gram-positive mastitis caseswhich affects the positive and negative predictive value, the economic value of diagnostic testing and its potential to reduce antimicrobial usediffered between farms. The simplicity and accuracy of the slide test could make it an attractive tool for farmers to target antimicrobial treatment of non-severe clinical mastitis.
Mastitis control is the most frequent reason for antibiotic use in lactating and non-lactating dairy cows (Pol and Ruegg, 2007). Because of concerns that antibiotic usage may lead to antimicrobial resistance (WHO, 2015), strategies are needed to promote and ensure prudent use of antimicrobials for mastitis control. Selective treatment for CM in lactating cows, based on differentiation between gram-positive pathogens and other causes of mastitis, has the potential to reduce antibiotic use significantly without negative impact on udder health, production or culling (Lago et al., 2011a(Lago et al., , 2011bMansion-de Vries et al., 2014). This has led to development of an array of diagnostic tools for on-farm classification of mastitis pathogens to support selective treatment.
Numerous culture-based detection kits for classification of mastitis pathogens have been reviewed, and new tests are becoming commercially available (Malcata et al., 2020). There are diagnostic tests based on Petrifilm, agar plates, or tube-test based systems. Some identify bacteria as gram-positive or gram-negative, whereas other tests identify bacteria to genus or species-level (Malcata et al., 2020). Some assays also include antibiotic susceptibility testing (Jones et al., 2019). All tests are more reliable when used for diagnosis of broad categories, such as growth, gram-positive and gram-negative species, rather than at genus or species level (Lago and Godden, 2018). The performance of diagnostic assays can be evaluated using scientific characteristics such as sensitivity, specificity and accuracy, and convenience aspects such as cost, ease of use and turn-around time. For example, most Petrifilm or agarbased tests are cheap but require considerable user training whereas Mastatest (Jones et al., 2019) costs more but provides automated reading to increase ease of use.
Although many tests were designed to identify pathogens to genus or species level, farmers are more interested in advice on antibiotic use than identification of causative agents of CM (Griffioen et al., 2016). A simplified test to differentiate grampositive organisms from other causes of mastitis could be sufficient to decide whether antimicrobial treatment of non-severe CM is needed. In a different context, namely bacteriuria in pregnant women, a similar need for reliable and simple testing to differentiate gram-positive, gram-negative and culturenegatives samples led to development of the Uricult dip-slide (Van Dorsten and Bannister, 1986). The dip-slide is a plastic paddle with two selective media that can be dipped in a liquid sample such as urine or milk, allowing for growth of either gram-positive or gram-negative organisms.
The aim of our study was to evaluate the laboratory performance of a simplified slide test for bovine mastitis, to determine whether it differentiates gram-positive from other forms of mastitis with similar accuracy as a commercially available comparator test commonly used in our practice.

Sample collection
Seven dairy farms in Scotland were selected based on herd size, location, and willingness to cooperate in the study (online Supplementary Table S1). Farm staff, including milkers and herd managers, were trained to identify CM and to classify cases as mild (abnormal milk, e.g. clots, flakes or serous milk), moderate (abnormal milk and signs of udder inflammation: hardness, swelling, redness, heat or pain) or severe (presence of additional systemic signs of disease, e.g. fever, tachycardia, tachypnoea, dehydration, or anorexia) (Pinzón-Sánchez and Ruegg, 2011). They were taught how to collect milk samples aseptically according to National Mastitis Council recommendations (NMC, 2017). CM cases were sampled regardless of mastitis severity. If multiple quarters of a cow were affected simultaneously, each affected quarter was sampled. Any CM episode in a quarter occurring >14 d after the previous episode, or caused by a different aetiological agent, was considered a new CM case. Animals were eligible for inclusion in the first week after calving but no animals included were within 14 d of administration of antimicrobial products.
Samples were collected from January to May 2018. They were stored on farm at −20°C, and transported once a week to Glasgow University's Veterinary Diagnostic Services laboratory where they were stored at −20°C until processing. All samples were cultured within 4 weeks from CM detection.

Reference test
Samples were thawed at ambient temperature for up to 8 h and processed simultaneously using the reference test, the simplified slide test, and the commercially available plate-based comparator test as described in the following two sections. For consistency, all media were inoculated and read by the first author, starting with the slide test. Bacteriological culture (NMC, 2017) with subsequent determination of species identity using MALDI-ToF MS was used as the reference test as detailed in the supplementary file.

Slide test
Media of the simplified slide test (VétoSlide, Vétoquinol, Lure, France) were inoculated by applying milk directly to each side using cotton wool swabs (approximate volume 0.1 ml) to moisten the entire surface of the media, as per manufacturer's instructions. The inoculated slides were incubated aerobically at 37°C and examined after 24-48 h. When at least one colony was visible, the sample was considered positive (Dohoo et al., 2011). Based on the manufacturer's guidelines, any growth on the green media was considered to indicate presence of gram-negative bacteria and red colonies on the green media were considered Escherichia coli. Growth on the red media was considered to indicate presence of gram-positive bacteria (online Supplementary Figure S1). When there was growth on both media, it was considered to indicate mixed infection with gram-positive and gramnegative bacteria. Guidelines to identify contaminated samples were not given, so samples were never classed as contaminated based on the slide test.

Comparator test
Plates for the comparator test (VétoRapid, Vétoquinol, Lure, France) were chosen for benchmarking because it is the most commonly used on-farm test in the dairy community of the authors and was previously evaluated in similar study settings (Viora et al., 2014). These plates were inoculated with 0.01 ml of milk per sector using disposable sterile calibrated plastic loops, incubated aerobically at 37°C and examined after 24-48 h, as detailed in the supplementary file. Results were summarized as gram-positive, gram-negative, E. coli and no growth for comparison with the reference and slide tests results. Samples not yielding visible colonies on the comparator test were considered negative for mastitis-associated pathogens. As for the slide test, a contaminated category was not specified by the manufacturer.

Data analysis
Samples that were contaminated or contained non-identifiable isolates by MALDI-ToF MS were excluded from evaluation of diagnostic test performance. All other culture-positive and culture-negative samples (n = 130) were used to calculate sensitivity, specificity, accuracy, positive predictive values (PPV) and negative predictive values (NPV) for growth, gram-positives, gram-negatives and E. coli. The reference test was used to classify results from the slide test and comparator test as correct or incorrect. To evaluate the potential of the test kits as treatment decision support tools, the calculations were repeated using a subset of the 130 samples, namely those from non-severe CM cases (n = 109), and the outcome was expressed as 'treatment'. This outcome is equivalent to gram-positive growth or no gram-positive growth. The latter category includes gram-negative bacteria, non-bacterial growth, and culture-negative results.
Statistical analysis was performed in Excel (Microsoft Corp., Redmond, USA) using tabular methods, and in R. If the 95% confidence interval for the difference between tests excluded zero, test performance was considered significantly different. Full details are provided in the supplementary file.

Reference test
Between 5 and 58 samples were collected from each farm (online Supplementary Table S1). Of 156 samples, 23 (14.7%) were contaminated. Among 133 non-contaminated samples, 14 (10.5%) showed no growth, and 116 (87.2%) showed growth of one or two colony types that could be identified by the reference method (Table 1). Three samples with growth of organisms that could not be identified by the reference method were excluded from further analysis. Within each farm's sample set, gram-positive and gramnegative isolates were identified, with a preponderance of gram-positive results for some farms (Farms 3, 4 and 5), mostly gram-negative results for others (Farms 2 and 6) and an even balance for the remainder (Farms 1 and 7; Figure 1). The proportion of contaminated samples per farm ranged from 0 to 33%, indicating considerable differences in sample quality. The most common species were E. coli and Streptococcus uberis, followed by other major mastitis pathogens, including Streptococcus dysgalactiae, Staphylococcus aureus and Klebsiella spp. (Table 1).

Slide test
After excluding contaminated samples and those with unidentified organisms, 130 samples were used to evaluate the performance of the slide test. A milk sample could be culture negative or culture positive, contain a single colony type or two colony types (two gram-positive morphotypes, two gram-negative morphotypes, or mixed gram-positive and gram-negative growth). The latter were considered gram-positive in the gram-positive analysis and gram-negative in the gram-negative analysis. The proportion of culture negative results was considerably higher for the slide test (20%) than for the reference test (10.8%). Of 26 culture-negative samples in the slide test, 12 (46.2%) were correctly classified. Of 14 false negative slide test results, seven were Other gram-positive 13 (10.0) n/a 1 (0.8) Prototheca zopfii 2 (1.5) n/a n/a n/a, not applicable. a The total number of species/genera listed exceeds the number of samples because more than one species/genus was detected in some samples that were not contaminated based on the NMC standard definition of 3 or more colony types (the percentage shown is related to the proportion of samples). from samples with gram-positive growth in the reference test and seven from samples with gram-negative growth. Mixed gram-positive and gram-negative growth was more common in the slide test (12.3%) than in the reference test (5.4%) ( Table 1). The sensitivity of the slide test was similar for grampositive and gram-negative organisms, but specificity was higher for the latter, resulting in higher accuracy for gram-negative organisms (89.2%) or E. coli (92.3%) than for gram-positive organisms (79.2%: Table 2). For the samples from non-severe CM (n = 109), the potential of the slide test to be used as treatment decision support tool was evaluated. Sensitivity and specificity for this subset were similar to those for all CM cases (Table 2).
Mean PPV of the slide test was high (between 92.6 and 98.1%) for growth, gram-negative results, and E. coli and moderate (between 75 and 80%) for gram-positive results (all based on 130 samples) and treatment (based on 109 samples). The mean NPV was less than 50% for growth and between 82 and 90.6% for all other outcomes (Table 2).

Benchmarking against the comparator test
Despite using a higher inoculum, the slide test gave fewer false positive results for growth than the comparator test, resulting in significantly higher specificity. Low specificity of the comparator test was partly due to moderate specificity in the detection of Staph. aureus and Strep. uberis (70.7 and 82.6%, respectively; online Supplementary Table S2), which was accompanied by high sensitivity for the same pathogens (100 and 90.5%, respectively). For sensitivity, there were no statistically significant differences among tests for any of the outcomes (Table 2). Despite the greater specificity of the slide test, overall accuracy of the two tests was not significantly different. In our study population, the slide test had significantly greater PPV than the comparator test for growth, gram-positive, E. coli and treatment, whereas their NPVs were similar for all outcomes.

Discussion
We evaluated the laboratory performance of a simplified culturebased slide test, VétoSlide, which was developed as a potential point-of-care tool to support farmers' CM treatment decisions.  (Leimbach and Krömker, 2018), although comparisons between studies are complicated by differences in study design, populations, and methods of analysis. Such differences make comparison of predictive values problematic because they are highly dependent on pathogen prevalence, which is farm-specific.
Benchmarking of the new slide test against a commercially available comparator in a single study allowed us to compare predictive values, which are more important in practice than sensitivity, specificity, or accuracy. A high PPV means that unnecessary treatment is minimized whereas a high NPV means that treatment is withheld only when cows truly do not need it. The slide test outperformed the comparator test in PPV and had similar NPV, meaning that the reduced risk of over-treating was not accompanied by an increased risk of under-treating. Whether positive or negative predictive value is considered more important in informing treatment decisions differs between regions. In some areas, such as southern Europe, it is generally assumed that antimicrobial treatment of mastitis is needed until proven otherwise (Busani et al., 2004). Conversely, in northern Europe, it is assumed that treatment is not needed until proven otherwise (Persson Waller et al., 2016). Within countries, this balance may shift over time, as illustrated by work from The Netherlands on selective dry cow treatment (DCT). Two split-udder trials conducted two decades apart (Schukken et al., 1993;Scherpenzeel et al., 2014) in the same country both showed that blanket DCT prevents CM when compared to selective DCT. However, the first study concluded that blanket DCT should be used to prevent CM despite the need to 'eliminate unnecessary use of antibiotics', whereas the second study emphasized the reduction in antimicrobial use that could be achieved by abandoning blanket DCT.
Whether the price of the diagnostic test is worth paying in terms of financial benefit is a matter of debate. On farms with a single dominant pathogen, the value of information may be limited (Cha et al., 2016), but our data showed that several farms did not have clear predominance of gram-positive or gram-negative mastitis over other types of mastitis. Some authors argue that even with just 20% of gram-positive mastitis, use of on-farm diagnostics would not be cost-effective (Down et al., 2017). Hence, the value of information would be farm-specific and no blanket statements around cost-benefit or reductions in antimicrobial use can be made based on our results. It is clear, however, that some farms will need further training in sample collection and handling to reduce the number of contaminated samples and to make investment of time and money into diagnostic testing better value for money. Moreover, before uptake of the slide test can be recommended on-farm, evaluation under on-farm conditions will be needed, as our laboratory-based analysis included freezing and thawing of milk, which would not be part of its on-farm use. Reading of plates at 24 and 48 h, as done here to allow for comparison with VétoRapid results, would cause considerable delay in treatment decisions, and shorter incubation times would need to be considered, with growth of gram-negative organisms often visible well within 24 h (data not shown).
The reference test used in our study included species identification by MALDI-ToF MS and revealed the presence of several species that are not recognized as typical mastitis pathogens, e.g. Bacillus and Lysinibacillus species. Although both genera are gram-positive, it is debatable whether they should be targeted with antimicrobial treatment because little is known about their role as pathogenic agents or their response to treatment. None of the currently available point-of-care tests for mastitis have the ability to differentiate such organisms from recognized mastitis pathogens. When information at species or subspecies level is required for advanced investigations or decision making at herd or animal level, laboratory-based microbial diagnostics continue to be important (Mansion-de Vries et al., 2014). For on-farm treatment decision making, however, the simplified slide test appears to have the potential to be an affordable, accurate, and user-friendly option.
In conclusion, using laboratory-based evaluation of farmer collected milk samples we demonstrated that a simplified slide test performs similar to the commercially available on-farm test that was used for benchmarking in terms of sensitivity or accuracy, whilst performing better in terms of specificity. The simplicity of the slide test can make it an attractive tool for farmers to target antimicrobial treatment of non-severe CM cases caused by grampositive organisms with good diagnostic accuracy. Further evaluation of user-friendliness and test accuracy in on-farm settings is needed, followed by assessment of uptake, economic impact, and reduction in antimicrobial use. In addition, users' willingness and ability to collect high quality milk samples needs to be understood and supported.