Hostname: page-component-76fb5796d-9pm4c Total loading time: 0 Render date: 2024-04-29T23:48:20.424Z Has data issue: false hasContentIssue false
  • English
  • Français

Inter-observer agreement, diagnostic sensitivity and specificity of animal-based indicators of young lamb welfare

Published online by Cambridge University Press:  08 April 2013

C. J. Phythian ,
N. Toft ,
P. J. Cripps ,
E. Michalopoulou ,
A. C. Winter ,
P. H. Jones ,
D. Grove-White  and
J. S. Duncan
C. J. Phythian*
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
N. Toft
Affiliation:
Department of Large Animal Sciences, Faculty of Life Sciences, University of Copenhagen, Grønnegaardsvej 8, DK-1870 Frederiksberg C, Denmark
P. J. Cripps
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
E. Michalopoulou
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
A. C. Winter
Affiliation:
School of Veterinary Science, University of Liverpool, Leahurst, Neston CH64 7TE, UK
P. H. Jones
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
D. Grove-White
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
J. S. Duncan
Affiliation:
Department of Epidemiology and Population Health, Institute of Global Health and Infection, University of Liverpool, Leahurst, Neston CH64 7TE, UK
*
E-mail: C.J.Phythian@bristol.ac.uk
Get access

Abstract

A scientific literature review and consensus of expert opinion used the welfare definitions provided by the Farm Animal Welfare Council (FAWC) Five Freedoms as the framework for selecting a set of animal-based indicators that were sensitive to the current on-farm welfare issues of young lambs (aged ⩽6 weeks). Ten animal-based indicators assessed by observation – demeanour, response to stimulation, shivering, standing ability, posture, abdominal fill, body condition, lameness, eye condition and salivation were tested as part of the objective of developing valid, reliable and feasible animal-based measures of lamb welfare The indicators were independently tested on 966 young lambs from 17 sheep flocks across Northwest England and Wales during December 2008 to April 2009 by four trained observers. Inter-observer reliability was assessed using Fleiss's kappa (κ), and the pair-wise agreement with an experienced, observer designated as the ‘test standard observer’ (TSO) was examined using Cohen's κ. Latent class analysis (LCA) estimated the sensitivity (Se) and specificity (Sp) of each observer without assuming a gold standard and predicted the Se and Sp of randomly selected observers who may apply the indicators in the future. Overall, good levels of inter-observer reliability, and high levels of Sp were identified for demeanour (κ = 0.54, Se ⩾ 0.70, Sp ⩾ 0.98), stimulation (κ = 0.57, Se = 0.30 to 0.77, Sp ⩾ 0.98), shivering (κ = 0.55, Se = 0.37 to 0.85, Sp ⩾ 0.99), standing ability (0.54, Se ⩾ 0.80, Sp ⩾ 0.99), posture (κ = 0.45, Se ⩾ 0.56, Sp = 0.99), abdominal fill (κ = 0.44, Se = 0.39 to 0.98, Sp = 0.99), body condition (κ = 0.72, Se ⩾ 0.38 to 0.90, Sp = 0.99), lameness (κ = 0.68, Se > 0.73, Sp = 1.00), and eye condition (κ = 0.72, Se ⩾ 0.86, Sp = 0.99). LCA predicted that randomly selected observers had Se > 0.77 (acceptable), and Sp ⩾ 0.98 (high) for assessments of demeanour, lameness, abdominal fill posture, body condition and eye condition. The diagnostic performance of some indicators was influenced by the composition of the study population, and it would be useful to test the indicators on lambs with a greater level of outcomes associated with poor welfare. The findings presented in this paper could be applied in the selection of valid, reliable and feasible indicators used for the purposes of on-farm assessments of lamb welfare.

Keywords

animal welfareindicatorsheepobserver agreementsensitivityspecificity
Type
Behaviour, welfare and health
Information
animal , Volume 7 , Issue 7 , July 2013 , pp. 1182 - 1190
Copyright
Copyright © The Animal Consortium 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Amon, T, Amon, B, Ofner, E, Boxberger, J 2001. Precision of assessment of animal welfare by the ‘TGI 35 L’ Austrian needs index. Acta Agriculturae Scandinavica Section A – Animal Science 51, 114–117.Google Scholar
Angus, K 1991. Arthritis in lambs and sheep. In Practice 13, 204–207.Google Scholar
Anzuino, K, Bell, NJ, Bazeley, KJ, Nicol, CJ 2010. Assessment of welfare on 24 commercial UK dairy goat farms based on direct observations. Veterinary Record 167, 774–780.Google Scholar
Baadsgaard, NP, Jørgensen, E 2003. A Bayesian approach to the accuracy of clinical observations. Preventive Veterinary Medicine 59, 189–206.Google Scholar
Bertrand, P, Benichou, J, Grenier, P, Chastang, C 2005. Hui and Walter's latent-class reference-free approach may be more useful in assessing agreement than diagnostic performance. Journal of Clinical Epidemiology 58, 688–700.Google Scholar
Bonde, M, Toft, N, Thomsen, PT, Sorensen, J 2010. Evaluation of sensitivity and specificity of routine meat inspection of Danish slaughter pigs using latent class analysis. Preventive Veterinary Medicine 94, 165–169.Google Scholar
Burn, CC, Pritchard, JC, Whay, HR 2009. Observer reliability for working equine welfare assessment: problems with high prevalences of certain results. Animal Welfare 18, 177–187.Google Scholar
Channon, AJ, Walker, AM, Pfau, T, Sheldon, IM, Wilson, AM 2009. Variability of manson and leaver locomotion scores assigned to dairy cows by different observers. Veterinary Record 164, 388–392.Google Scholar
Cohen, J 1960. A coefficient of agreement for nominal scales. Education and Psychological Measurement 20, 37–46.Google Scholar
Dwyer, CM 2008. The welfare of the neonatal lamb. Small Ruminant Research 76, 31–41.Google Scholar
Dwyer, CM 2009. Welfare of sheep: providing for welfare in an extensive environment. Small Ruminant Research 86, 14–21.Google Scholar
Farm Animal Welfare Council (FAWC) 1994. Report on the welfare of sheep. PB 1755. Farm Animal Welfare Council (FAWC) publication, London, UK.Google Scholar
Fleiss, LL 1981. Statistical methods for rates and proportions. John Wiley and Sons, New York, USA.Google Scholar
Foddai, A, Green, LE, Mason, SA, Kaler, J 2012. Evaluating observer agreement of scoring systems for foot integrity and footrot lesions in sheep. BMC Veterinary Research 8, 65.CrossRefGoogle ScholarPubMed
Greiner, M, Gardner, IA 2000. Epidemiologic issues in the validation of veterinary diagnostic tests. Preventive Veterinary Medicine 45, 3–22.Google Scholar
Hoehler, FK 2000. Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. Journal of Clinical Epidemiology 53, 499–503.Google Scholar
Hui, SL, Walter, SD 1980. Estimating the error rates of diagnostic tests. Biometrics 36, 167–171.Google Scholar
Kaler, J, Wassink, GJ, Green, LE 2009. The inter- and intra-observer reliability of a locomotion scoring scale for sheep. The Veterinary Journal 180, 189–194.Google Scholar
Knierim, U, Winckler, C 2009. On-farm welfare assessment in cattle: validity, reliability and feasibility issues and future perspectives with special regard to the Welfare Quality® approach. Animal Welfare 18, 451–458.Google Scholar
Kristensen, E, Dueholm, L, Vink, D, Andersen, JE, Jakobsen, EB, Illum-Nielsen, S, Petersen, FA, Enevoldsen, C, 2006. Within- and across-person uniformity of body condition scoring in Danish Holstein cattle. Journal of Dairy Science 89, 3721–3728.Google Scholar
Lucas, NP, Macaskill, P, Irwig, L, Bogduk, N 2010. The development of a quality appraisal tool for studies of diagnostic reliability (QAREL). Journal of Clinical Epidemiology 63, 854–861.Google Scholar
Lunn, D, Spiegelhalter, D, Thomas, A, Best, N 2009. The Bugs project: evolution, critique and future directions. Statistics in Medicine 28, 3049–3067.Google Scholar
Main, DCJ, Kent, JP, Wemelsfelder, F, Ofner, E, Tuyttens, FAM 2003. Applications for methods of on-farm welfare assessment. Animal Welfare 12, 523–528.Google Scholar
Matheson, SM, Rooke, JA, McIlvaney, K, Jack, M, Ison, S, Bünger, L, Dwyer, CM 2011. Development and validation of on-farm behavioural scoring systems to assess birth assistance and lamb vigour. Animal 5, 776–783.Google Scholar
Mullan, S, Edwards, SA, Butterworth, A, Whay, HRA, Main, DCJ 2011. Inter-observer reliability testing of pig welfare outcome measures proposed for inclusion within farm assurance schemes. Veterinary Journal 190, 100–109.Google Scholar
Nérette, P, Stryhn, H, Dohoo, I, Hammell, L 2008. Using pseudogold standards and latent-class analysis in combination to evaluate the accuracy of three diagnostic tests. Preventive Veterinary Medicine 85, 207–225.Google Scholar
Nielsen, LR, Toft, N, Ersboll, AK 2004. Evaluation of an indirect serum ELISA and a bacteriological faecal culture test for diagnosis of Salmonella serotype Dublin in cattle using latent class models. Journal of Applied Microbiology 96, 311–319.Google Scholar
Petersen, HH, Enoe, C, Nielsen, EO 2004. Observer agreement on pen level prevalence of clinical signs in finishing pigs. Preventive Veterinary Medicine 64, 147–156.CrossRefGoogle ScholarPubMed
Phythian, CJ, Michalopoulou, E, Jones, PH, Winter, AC, Clarkson, MJ, Stubbings, LA, Grove-White, D, Cripps, PJ, Duncan, JS 2011a. Validating indicators of sheep welfare through a consensus of expert opinion. Animal 5, 943–952.Google Scholar
Phythian, CJ, Wemelsfelder, F, Michalopoulou, E, Duncan, JS 2011b. Qualitative behaviour assessment in sheep: consistency across time and association with health indicators. In Proceedings of the 5th international workshop on the assessment of animal welfare at farm and group level (ed. T Widowski, P Lawlis and K Sheppard), pp. 14. Wageningen Academic Publishers, Wageningen, The Netherlands.Google Scholar
Toft, N, Akerstedt, J, Tharaldsen, J, Hopp, P 2007a. Evaluation of three serological tests for diagnosis of Maedi-Visna virus infection using latent class analysis. Veterinary Microbiology 120, 77–86.CrossRefGoogle ScholarPubMed
Toft, N, Innocent, GT, Gettinby, G, Reid, SWJ 2007b. Assessing the convergence of Markov Chain Monte Carlo methods: an example from evaluation of diagnostic tests in absence of a gold standard. Preventive Veterinary Medicine 79, 244–256.CrossRefGoogle ScholarPubMed
Ulvund, MJ 2012. Important sheep flock health issues in Scandinavia/northern Europe. Small Ruminant Research 106, 6–10.Google Scholar
Veissier, I, Butterworth, A, Bock, B, Roe, E 2008. European approaches to ensure good animal welfare. Applied Animal Behaviour Science 113, 279–297.CrossRefGoogle Scholar
Walter, SD, Eliasziw, M, Donner, A 1998. Sample size and optimal designs for reliability studies. Statistics in Medicine 17, 101–110.Google Scholar
Wemelsfelder, F, Lawrence, AB 2001. Qualitative assessment of animal behaviour as an on-farm welfare-monitoring tool. Acta Agriculturae Scandinavica Section A – Animal Science 30, 21–25.Google Scholar
Wemelsfelder, F, Hunter, AE, Paul, ES, Lawrence, AB 2012. Assessing pig body language: agreement and consistency between pig farmers, veterinarians, and animal activists. Journal of Animal Science 90, 3652–3665.Google Scholar
Whay, HR, Main, DCJ, Green, LE, Webster, AJF 2003a. Animal based measures for the assessment of welfare state of dairy cattle, pigs, and laying hens: consensus of expert opinion. Animal Welfare 12, 205–217.Google Scholar
Whay, HR, Main, DCJ, Green, LE, Webster, AJF 2003b. An animal-based welfare assessment of group-housed calves on UK dairy farms. Animal Welfare 12, 611–617.Google Scholar
Wickham, SL, Collins, T, Barnes, AL, Miller, DW, Beatty, DT, Stockman, C, Blache, D, Wemelsfelder, F, Fleming, PA 2012. Qualitative behavioral assessment of transport-naïve and transport-habituated sheep. Journal of Animal Science 90, 4523–4535.Google Scholar
Supplementary material: File

Phythian supplementary material

Phythian supplementary material

Download Phythian supplementary material(File)
File 37.9 KB