Skip to main content

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
Cancel
×
×
Home
  • Home
Article
  • Cited by 9
  • Cited by
    Crossref Citations
    Crossref logo
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.
    Gutiérrez, Yoan Vázquez, Sonia and Montoyo, Andrés 2017. Spreading semantic information by Word Sense Disambiguation. Knowledge-Based Systems, Vol. 132, Issue. , p. 47.
    Tekli, Joe 2016. An Overview on XML Semantic Disambiguation from Unstructured Text to Semi-Structured Data: Background, Applications, and Ongoing Challenges. IEEE Transactions on Knowledge and Data Engineering, Vol. 28, Issue. 6, p. 1383.
    Dinh, Duy and Tamine, Lynda 2015. Identification of concept domains and its application in biomedical information retrieval. Information Systems and e-Business Management, Vol. 13, Issue. 4, p. 647.
    Agirre, Eneko López de Lacalle, Oier and Soroa, Aitor 2014. Random Walks for Knowledge-Based Word Sense Disambiguation. Computational Linguistics, Vol. 40, Issue. 1, p. 57.
    Pilehvar, Mohammad Taher and Navigli, Roberto 2014. A Large-Scale Pseudoword-Based Evaluation Framework for State-of-the-Art Word Sense Disambiguation. Computational Linguistics, Vol. 40, Issue. 4, p. 837.
    Yazdani, Majid and Popescu-Belis, Andrei 2013. Computing text semantic relatedness using the contents and links of a hypertext encyclopedia. Artificial Intelligence, Vol. 194, Issue. , p. 176.
    Jusoh, Shaidah and Alfawareh, Hejab M. 2011. Automated Translation Machines: Challenges and a Proposed Solution. p. 77.
    Mandreoli, Federica and Martoglia, Riccardo 2011. Knowledge-based sense disambiguation (almost) for all structures. Information Systems, Vol. 36, Issue. 2, p. 406.
    Chen, Junpeng Liu, Juan Yu, Wei and Wu, Peng 2009. Combining Lexical Stability and Improved Lexical Chain for Unsupervised Word Sense Disambiguation. p. 430.
    ×
  • Get access
    Add to cart USD35.00
    Check if you have access via personal or institutional login
    Log in Register Recommend to librarian

A structural approach to the automatic adjudication of word sense disagreements

  • ROBERTO NAVIGLI (a1)
Abstract

The semantic annotation of texts with senses from a computational lexicon is a complex and often subjective task. As a matter of fact, the fine granularity of the WordNet sense inventory [Fellbaum, Christiane (ed.). 1998. WordNet: An Electronic Lexical Database MIT Press], a de facto standard within the research community, is one of the main causes of a low inter-tagger agreement ranging between 70% and 80% and the disappointing performance of automated fine-grained disambiguation systems (around 65% state of the art in the Senseval-3 English all-words task). In order to improve the performance of both manual and automated sense taggers, either we change the sense inventory (e.g. adopting a new dictionary or clustering WordNet senses) or we aim at resolving the disagreements between annotators by dealing with the fineness of sense distinctions. The former approach is not viable in the short term, as wide-coverage resources are not publicly available and no large-scale reliable clustering of WordNet senses has been released to date. The latter approach requires the ability to distinguish between subtle or misleading sense distinctions. In this paper, we propose the use of structural semantic interconnections – a specific kind of lexical chains – for the adjudication of disagreed sense assignments to words in context. The approach relies on the exploitation of the lexicon structure as a support to smooth possible divergencies between sense annotators and foster coherent choices. We perform a twofold experimental evaluation of the approach applied to manual annotations from the SemCor corpus, and automatic annotations from the Senseval-3 English all-words competition. Both sets of experiments and results are entirely novel: structural adjudication allows to improve the state-of-the-art performance in all-words disambiguation by 3.3 points (achieving a 68.5% F1-score) and attains figures around 80% precision and 60% recall in the adjudication of disagreements from human annotators.

Copyright
COPYRIGHT: © Cambridge University Press 2008
References
Hide All
Agirre, Eneko and de Lacalle, Oier López. 2003. Clustering wordnet word senses. In Proceedings of Conference on Recent Advances on Natural Language (RANLP), Borovets, Bulgary, pp. 121–30.
Agirre, Eneko, Martínez, David, de Lacalle, Oier López, and Soroa, Aitor. 2006. Two graph-based algorithms for state-of-the-art wsd. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, pp. 585–93.
Barzilay, Regina and Elhadad, Michael. 1997. Using lexical chains for text summarization. In Proceedings of the ACL Workshop on Intelligent Scalable Text Summarization, Madrid, Spain, pp. 10–17.
Bentivogli, Luisa, Forner, Pamela, and Pianta, Emanuele. 2004. Evaluating cross-language annotation transfer in the multisemcor corpus. In Proceedings of the 20th International Conference on Computational Linguistics (COLING 2004), Geneva, Switzerland, pp. 364–70.
Berners-Lee, Tim. 1999. Weaving the Web. Harper, San Francisco, CA, USA.
Brody, Samuel, Navigli, Roberto, and Lapata, Mirella. 2006. Ensemble methods for unsupervised WSD. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics joint with the 21st International Conference on Computational Linguistics (COLING-ACL 2006), Sydney, Australia, pp. 97–104.
Chklovski, Tim and Mihalcea, Rada. 2002. Building a sense tagged corpus with open mind word expert. In Proceedings of ACL 2002 Workshop on WSD: Recent Successes and Future Directions, Philadelphia, PA.
Chklovski, Tim and Rada, Mihalcea. 2003. Exploiting agreement and disagreement of human annotators for word sense disambiguation. In Proceedings of Recent Advances in NLP (RANLP 2003), Borovetz, Bulgaria.
Cohen, Jacob A. 1960. A coefficient of agreement of nominal scales. Educational and Psychological Measurement 20 (1): 37–46.
Cuadros, Montse and German, Rigau. 2006. Quality assessment of large scale knowledge resources. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, pp. 534–41.
Decadt, Bart, Hoste, Véronique, Daelemans, Walter, and Antal, van den Bosch. 2004. Gambl, genetic algorithm optimization of memory-based wsd. In Proceedings of ACL 2004 SENSEVAL-3 Workshop. Barcelona, Spain, pp. 108–12.
Dolan, William B. 1994. Word sense ambiguation: clustering related senses. In Proceedings of 15th Conference on Computational Linguistics (COLING), Kyoto, Japan, pp. 712–16.
Edmonds, Philip and Adam, Kilgarriff. 2002. Introduction to the special issue on evaluating word sense disambiguation systems. Journal of Natural Language Engineering 8 (4): 279–91.
Fellbaum, Christiane (ed.) 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA, USA.
Fellbaum, Christiane, Joachim, Grabowski, and Shari, Landes. 1998. Performance and confidence in a semantic annotation task. In Fellbaum, Christiane (ed.) WordNet: an Electronic Lexical Database, pp. 217–37, MIT Press, Cambridge, MA, USA.
Florian, Radu, Cucerzan, Silviu, Schafer, Charles, and Yarowsky, David. 2002. Combining classifiers for word sense disambiguation. Journal of Natural Language Engineering 8 (4): 1–14.
Galley, Michel and McKeown, Kathleen. 2003. Improving word sense disambiguation in lexical chaining. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI), Acapulco, Mexico, pp. 1486–8.
Hanks, Patrick. 2000. Do word meanings exist? Computers and the Humanities 34 (1–2): 205–15.
Harabagiu, Sanda, Miller, George, and Moldovan, Dan. 1999. Wordnet 2 – a morphologically and semantically enhanced resource. In Proceedings of SIGLEX-99, University of Maryland, USA, pp. 1–8.
Hirst, Graeme and St-Onge, David. 1998. Lexical chains as representations of context for the detection and correction of malapropisms. In Fellbaum, Christiane (ed.) WordNet: An electronic lexical database, pp. 305–32, MIT Press.
Hovy, Eduard H., Marcus, Mitchell P., Palmer, Martha, Ramshaw, Lance A., and Weischedel, Ralph M.. 2006. Ontonotes: the 90% solution. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, New York, USA.
Jiang, Jay J. and Conrath, David W.. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of International Conference Research on Computational Linguistics (ROCLING X), Taiwan, pp. 19–33.
Kilgarriff, Adam. 1997. I don't believe in word senses. Computers and the Humanities 31 (2): 91–113.
Klein, Dan, Toutanova, Kristina, Ilhan, H. Tolga, Kamvar, Sepandar D., and Manning, Christopher D.. 2002. Combining heterogeneous classifiers for word-sense disambiguation. In Proceedings of the ACL-02 workshop on Word sense disambiguation: recent successes and future directions, Morristown, NJ, pp. 74–80.
Kohomban, Upali Sathyajith and Lee, Wee Sun. 2007. Optimizing classifier performance in word sense disambiguation by redefining sense classes. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI), Hyderabad, India, pp. 1635–40.
Lea, Diana (ed.) 2002. Oxford Collocations. Oxford University Press, USA.
Leacock, Claudia, Chodorow, Martin, and Miller, George. 1998. Using corpus statistics and wordnet relations for sense identification. Computational Linguistics 24 (1): 147–65.
Litkowski, Ken. 2004. Senseval-3 task: word-sense disambiguation of wordnet glosses. In Proceedings of ACL 2004 SENSEVAL-3 Workshop, Barcelona, Spain, pp. 13–16.
Longman, (ed.) 2003. Longman Language Activator. Pearson Education, Harlaw, Essex, UK.
Magnini, Bernardo and Cavaglià, Gabriela. 2000. Integrating subject field codes into wordnet. In Proceedings of the 2nd Conference on Language Resources and Evaluation (LREC), Athens, Greece, pp. 1413–18.
Mihalcea, Rada and Faruque, Ehsanul. 2004. Senselearner: minimally supervised word sense disambiguation for all words in open text. In Proceedings of ACL 2004 SENSEVAL-3 Workshop, Barcelona, Spain, pp. 155–8.
Mihalcea, Rada, Tarau, Paul, and Figa, Elizabeth. 2004. Pagerank on semantic networks, with application to word sense disambiguation. In Proceedings of the 20th COLING 2004, Geneva, Switzerland, pp. 1126–32.
Miller, George A., Leacock, Claudia, Tengi, Randee, and Bunker, Ross T.. 1993. A semantic concordance. In Proceedings of the ARPA Workshop on Human Language Technology, Princeton, NJ, USA, pp. 303–8.
Miller, Irwin and Miller, Marylees (eds.) 2003. John E. Freund's Mathematical Statistics with Applications, 7th Edition. Prentice Hall, NJ, USA.
Morris, Jane and Hirst, Graeme. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics 17 (1): 21–43.
Navigli, Roberto. 2005. Semi-automatic extension of large-scale linguistic knowledge bases. In Proceedings of the 18th FLAIRS, Clearwater Beach, USA, pp. 548–53.
Navigli, Roberto. 2006a. Consistent validation of manual and automatic sense annotations with the aid of semantic graphs. Computational Linguistics 32 (2): 273–81.
Navigli, Roberto. 2006b. Experiments on the validation of sense annotations assisted by lexical chains. In Proceedings of the European Chapter of the Annual Meeting of the Association for Computational Linguistics (EACL), Trento, Italy, pp. 129–36.
Navigli, Roberto. 2006c. Meaningful clustering of senses helps boost word sense disambiguation performance. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics joint with the 21st International Conference on Computational Linguistics (COLING-ACL 2006), Sydney, Australia, pp. 105–12.
Navigli, Roberto and Velardi, Paola. 2005. Structural semantic interconnections: a knowledge-based approach to word sense disambiguation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 27 (7): 1075–88.
Navigli, Roberto, Litkowski, Kenneth C., and Hargraves, Orin. 2007. Semeval-2007 task 07: coarse-grained english all-words task. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), Czech Republic, pp. 30–5, Prague, Association for Computational Linguistics.
Ng, Hwee T., Lim, Chung Y., and Foo, Shou K.. 1999. A case study on the inter-annotator agreement for word sense disambiguation. In Proceedings of ACL Workshop: Standardizing Lexical Resources, College Park, MD, pp. 9–13.
Palmer, Martha. 2000. Consistent criteria for sense distinctions. Computers and the Humanities 34 (1–2): 217–22.
Palmer, Martha, Dang, Hoa, and Fellbaum, Christiane. 2007. Making fine-grained and coarse-grained sense distinctions, both manually and automatically. Journal of Natural Language Engineering 13 (2): 137–63.
Peters, Wim, Peters, Ivonne, and Vossen, Piek. 1998. Automatic sense clustering in eurowordnet. In Proceedings of the 1st Conference on Language Resources and Evaluation (LREC), Granada, Spain.
Pianta, Emanuele, Bentivogli, Luisa, and Girardi, Christian. 2002. Multiwordnet: developing an aligned multilingual database. In Proceedings of the First International Conference on Global WordNet, Mysore, India, pp. 21–5.
Pustejovsky, James. 1995. The Generative Lexicon. Cambridge, MA, MIT Press.
Rigau, German, Atserias, Jordi, and Agirre, Eneko. 1997. Combining unsupervised lexical knowledge methods for word sense disambiguation. In Proceedings of 35th Annual Meeting of the Association for Computational Linguistics joint with 8th Conference of the European Chapter of the Association for Computational Linguistics (ACL/EACL'97), Madrid, Spain, pp. 48–55.
Snyder, Benjamin and Palmer, Martha. 2004. The english all-words task. In Proceedings of ACL 2004 SENSEVAL-3 Workshop, Barcelona, Spain, pp. 41–43.
Soanes, Catherine and Stevenson, Angus (ed.) 2003. Oxford Dictionary of English. Oxford University Press.
Stevenson, Mark and Wilks, Yorick. 2001. The interaction of knowledge sources in word sense disambiguation. Computational Linguistics 27 (3): 321–49.
Véronis, Jean. 2001. Sense tagging: does it make sense? In Corpus Linguistics 2001 Conference, Lancaster, UK.
Véronis, Jean. 2004. Hyperlex: lexical cartography for information retrieval. Computer, Speech and Language 18 (3): 223–52.
Yuret, Deniz. 2004. Some experiments with a naive bayes wsd system. In Proceedings of ACL 2004 SENSEVAL-3 Workshop, Barcelona, Spain, pp. 265–68.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *

CAPTCHA *

Skip to the audio challenge
×

Metrics

Full text views

Total number of HTML views: 1
Total number of PDF views: 16 *
Loading metrics...

Abstract views

Total abstract views: 96 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 12th June 2018. This data will be updated every 24 hours.

Cancel
Confirm
×