Skip to main content
    • Aa
    • Aa
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 2
  • Cited by
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Artstein, Ron and Poesio, Massimo 2008. Inter-Coder Agreement for Computational Linguistics. Computational Linguistics, Vol. 34, Issue. 4, p. 555.

    SAMPSON, GEOFFREY and BABARCZY, ANNA 2008. Definitional and human constraints on structural annotation of English. Natural Language Engineering, Vol. 14, Issue. 04,


Definitional, personal, and mechanical constraints on part of speech annotation performance

  • DOI:
  • Published online: 06 December 2005

For one aspect of grammatical annotation, part-of-speech tagging, we investigate experimentally whether the ceiling on accuracy stems from limits to the precision of tag definition or limits to analysts' ability to apply precise definitions, and we examine how analysts' performance is affected by alternative types of semi-automatic support. We find that, even for analysts very well-versed in a part-of-speech tagging scheme, human ability to conform to the scheme is a more serious constraint than precision of scheme definition. We also find that although semi-automatic techniques can greatly increase speed relative to manual tagging, they have little effect on accuracy, either positively (by suggesting valid candidate tags) or negatively (by lending an appearance of authority to incorrect tag assignments). On the other hand, it emerges that there are large differences between individual analysts with respect to usability of particular types of semi-automatic support.

Hide All
A version of this paper was presented orally at the workshop “Empirical methods in the new millennium: Linguistically Interpreted Corpora” (LINC-01), at the 34th Meeting of the Societas Linguistica Europaea, Leuven, Belgium, 28 Aug–1 Sep 2001. The research was supported by the Economic and Social Research Council (UK) under award no. R00023 8146.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *