Skip to main content
×
Home
    • Aa
    • Aa

BLANC: Implementing the Rand index for coreference evaluation

  • M. RECASENS (a1) and E. HOVY (a2)
Abstract
Abstract

This paper addresses the current state of coreference resolution evaluation, in which different measures (notably, MUC, B3, CEAF, and ACE-value) are applied in different studies. None of them is fully adequate, and their measures are not commensurate. We enumerate the desiderata for a coreference scoring measure, discuss the strong and weak points of the existing measures, and propose the BiLateral Assessment of Noun-Phrase Coreference, a variation of the Rand index created to suit the coreference task. The BiLateral Assessment of Noun-Phrase Coreference rewards both coreference and non-coreference links by averaging the F-scores of the two types, does not ignore singletons – the main problem with the MUC score – and does not inflate the score in their presence – a problem with the B3 and CEAF scores. In addition, its fine granularity is consistent over the whole range of scores and affords better discrimination between systems.

Copyright
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

E. Bengtson , and D. Roth 2008. Understanding the value of features for coreference resolution. In Proceedings of EMNLP, Honolulu, HI, pp. 294303.

A. Haghighi , and D. Klein 2009. Simple coreference resolution with rich syntactic and semantic features. In Proceedings of EMNLP, Suntec, Singapore, pp. 11521161.

H. Poon , and P. Domingos 2008. Joint unsupervised coreference resolution with Markov logic. In Proceedings of EMNLP, Honolulu, HI, pp. 650659.

A. Rahman , and V. Ng 2009. Supervised models for coreference resolution. In Proceedings of EMNLP, Suntec, Singapore, pp. 968977.

V. Stoyanov , N. Gilbert , C. Cardie , and E. Riloff 2009. Conundrums in noun phrase coreference resolution: making sense of the state-of-the-art. In Proceedings of ACL-IJCNLP, Suntec, Singapore, pp. 656664.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 3
Total number of PDF views: 13 *
Loading metrics...

Abstract views

Total abstract views: 186 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 25th March 2017. This data will be updated every 24 hours.