Skip to main content
×
×
Home

One, no one and one hundred thousand events: Defining and processing events in an inter-disciplinary perspective*

  • R. SPRUGNOLI (a1) (a2) and S. TONELLI (a1)
Abstract
Abstract

We present an overview of event definition and processing spanning 25 years of research in NLP. We first provide linguistic background to the notion of event, and then present past attempts to formalize this concept in annotation standards to foster the development of benchmarks for event extraction systems. This ranges from MUC-3 in 1991 to the Time and Space Track challenge at SemEval 2015. Besides, we shed light on other disciplines in which the notion of event plays a crucial role, with a focus on the historical domain. Our goal is to provide a comprehensive study on event definitions and investigate which potential past efforts in the NLP community may have in a different research domain. We present the results of a questionnaire, where the notion of event for historians is put in relation to the NLP perspective.

Copyright
Footnotes
Hide All
*

The title is inspired by the novel ‘One, No One and One Hundred Thousand’ written by the Nobel prize winner Luigi Pirandello.

Footnotes
References
Hide All
Aguilar J., Beller C., McNamee P., Van Durme B., Strassel S., Song Z., and Ellis J., 2014. A comparison of the events and relations across ACE, ERE, TAC-KBP, and FrameNet annotation standards. In Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Baltimore, Maryland, USA: ACL, pp. 4553.
Allen J. F., 1984. Towards a general theory of action and time. Artificial Intelligence 23 (2): 123–54.
Aone C., Halverson L., Hampton T., and Ramos-Santacruz M. 1998. SRA: description of the IE2 system used for MUC-7. In Proceedings of the 7th Message Understanding Conferences (MUC-7). Fairfax, VA.
Appelt D. E., Hobbs J. R., Bear J., Israel D., and Tyson M., 1993. FASTUS: a finite-state processor for information extraction from real-world text. In Proceedings of IJCAI, Chambéry, France, vol. 93, pp. 11721178.
Archer D. 2014. Exploring verbal aggression in English historical texts using USAS. In Taavitsainen I., Jucker A. H. and Tuominen J. (eds.), Diachronic Corpus Pragmatics (Pragmatics & Beyond New Series), pp. 273302. University of Helsinki/University of Zurich: John Benjamins Publishing Company.
Asahara M., Yasuda S., Konishi H., Imada M., and Maekawa K., 2013. BCCWJ-TimeBank: temporal and event information Annotation on Japanese text. In Proceedings of PACLIC 27, Taipei, Taiwan, pp. 206214.
Ashish N., Appelt D., Freitag D., and Zelenko D. 2006. Papers from the AAAI workshop on event extraction and synthesis. Technical Report WS-06-07, American Association for Artificial Intelligence.
Bach E. 1986. The algebra of events. Linguistics and Philosophy, 9 (1): 516, D. Reidel Publishing Company.
Baker C. F., Fillmore C. F., and Lowe J. B., 1998. The Berkeley FrameNet project. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Montréal, Quebec, Canada: ACL, pp. 8690.
Berr H., 1922. L’histoire traditionnelle et la synthse historique. Revue belge de philologie et d’histoire 1 (3): 556556.
Bethard S., Derczynski L., Savova G., Pustejovsky J., and Verhagen M. 2015. SemEval-2015 Task 6: clinical TempEval. In Proceedings of SemEval 2015, Denver, Colorado, USA: ACL.
Bittar A., Amsili P., Denis P., and Danlos L., 2011. French TimeBank: an ISO-TimeML annotated reference corpus. In Proceedings of ACL, Portland, Oregon, USA, ACL, pp. 130134.
Bjrne J., and Salakoski T. 2013. TEES 2.1: automated annotation scheme learning in the BioNLP 2013 Shared Task. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.
Bui Q. C., Campos D., van Mulligen E. M., and Kors J. A. 2013. A fast rule-based approach for biomedical event extraction. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.
Caselli T., Bartalesi Lenzi V., Sprugnoli R., Pianta E., and Prodanof I. 2011. Annotating events, temporal expressions and relations in Italian: the It-TimeML experience for the Ita-TimeBank. In Proceedings of the 5th Linguistic Annotation Workshop, Portland, Oregon: ACL.
Caselli T., Sprugnoli R., Speranza M., and Monachini M. 2014. EVENTI EValuation of events and temporal INformation at Evalita 2014. In Proceedings of the 4th International Workshop EVALITA 2014, Pisa, Italy, Pisa University Press.
Chinchor N. A. 1998. Overview of MUC-7/MET-2. In Proceedings of the 7th Message Understanding Conference (MUC-7). Fairfax, VA.
Cybulska A., and Vossen P., 2011. Historical event extraction from text. In Proceedings of the 5th ACL-HLT LaTeCH Workshop, Portland, Oregon: ACL, pp. 3943.
Cybulska A., and Vossen P., 2014. Using a sledgehammer to crack a nut? Lexical diversity and event coreference resolution. In Proceedings of LREC 2014, pp. 26–31, Reykjavik, Iceland, European Language Resources Association (ELRA), pp. 2631.
Doddington G. R., Mitchell A., Przybocki M. A., Ramshaw L. A., Strassel S., and Weischedel R. M. 2004. The automatic content extraction (ACE) program-tasks, data, and evaluation. In Proceedings of LREC 2004, Lisbon, Portugal, European Language Resources Association (ELRA).
Farzindar A., and Khreich W., 2015. A survey of techniques for event detection in Twitter. Computational Intelligence 31.1 (2015): 132164.
Febvre L. P. V. 1953. Combats pour l’histoire. Paris: Armand Colin.
Fellbaum C. 1998. WordNet. Blackwell Publishing Ltd. Cambridge: MIT Press.
Filatova E., and Hovy E. 2001. Assigning time-stamps to event-clauses. In Proceedings of the ACL-EACL 2001 Workshop for Temporal and Spatial Information Processing, Toulouse, France: ACL.
Forascu C., and Tufi D. 2012. Romanian TimeBank: an annotated parallel corpus for temporal information. In Proceedings of LREC 2012, Istanbul, Turkey. European Language Resources Association (ELRA).
Grishman R. 2010. The impact of task and corpus on event extraction systems. In Proceedings of LREC 2010, Valletta, Malta, European Language Resources Association (ELRA).
Guldi J., and Armitage D., 2014. The History Manifesto. Cambridge: Cambridge University Press.
Hagège C. 1996. L’homme de Paroles: Contribution Linguistique aux Sciences Humaines. Fayard, Paris.
Hakala K., Van Landeghem S., Salakoski T., Van de Peer Y., and Ginter P. 2013. EVEX in ST13: application of a large-scale text mining resource to event extraction and network construction. In Proceedings of the BioNLP Shared Task 2013 Workshop. Sofia, Bulgaria.
Hatzivassiloglou V., and Filatova E., 2003. Domain-independent detection, extraction, and labeling of atomic events. In Proceedings of RANLP, Borovetz, Bulgaria, pp. 145152.
Hovy E., Mitamura T., Verdejo F., Araki J., and Philpot A. 2013. Events are not simple: identity, non-identity, and quasi-identity. In Proceedings of the The 1st Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Atlanta, Georgia, USA: ACL.
Humphreys K., Gaizauskas R., Azzam S., Huyck C., Mitchell B., Cunningham H., and Wilks Y. 1998. University of Sheffield: description of the LaSIE-II system as used for MUC-7. In Proceedings of the 7th Message Understanding Conferences (MUC-7). Fairfax, VA.
Ide N., and Woolner D. 2004. Exploiting semantic web technologies for intelligent access to historical documents. In Proceedings of LREC 2004, Reykjavik, Iceland, European Language Resources Association (ELRA).
Ikuta R., Styler W. F. IV, Hamang M., O’Gorman T., and Palmer M. 2014. Challenges of adding causation to Richer Event Descriptions. In Proceedings of the The 2nd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Baltimore, Maryland, USA: ACL.
Im S., You H., Jang H., Nam S., and Shin H., 2009. Ktimeml: specification of temporal and event expressions in korean text. In Proceedings of the 7th Workshop on Asian Language Resources, Suntec, Singapore: ACL, pp. 115122.
Jung H., and Stent A., 2013. Att1: temporal annotation using big windows and rich syntactic and semantic features. In Proceedings of * SEM, Atlanta, Georgia, USA, vol. 2, pp. 2024.
Kay C., Roberts J., Samuels M., and Wotherspoon I. 2009. Unlocking the OED: the story of the historical thesaurus of the OED. In Historical Thesaurus of the Oxford English Dictionary: With Additional Material from a Thesaurus of Old English. Oxford: Oxford University Press.
Katz G., and Arosio F. 2001. The annotation of temporal information in natural language sentences. In Proceedings of the ACL 2001 Workshop on Temporal and Spatial Information Processing, Toulouse, France: ACL.
Kim J. D., Ohta T., Tateisi Y., and Tsujii J. 2006. GENIA corpus manual. Technical Report, Citeseer.
Kim J. D., Ohta T., and Tsujii J., 2008. Corpus annotation for mining biomedical events from literature. BMC Bioinformatics 9 (1): 10.
Kolomiyets O., and Moens M. F., 2013. KUL: a data-driven approach to temporal parsing of documents. In Proceedings of SemEval 2013, Atlanta, Georgia, USA, pp. 8387.
Lee H., Recasens M., Chang A., Surdeanu M., and Jurafsky D., 2012. Joint entity and event coreference resolution across documents. In Proceedings of EMNLP-2012, Jeju, South Korea, pp. 489500.
Linguistic Data Consortium. 2005. ACE (automatic content extraction) english annotation guidelines for events, version 5.4.3 2005.07.01.
Llorens H., Chambers N., UzZaman N., Mostafazadeh N., Allen J., and Pustejovsky J. 2015. SemEval-2015 task 5: QA TEMPEVAL-Evaluating temporal information understanding with question answering. In Proceedings of SemEval 2015, Denver, Colorado: ACL.
Llorens H., Saquete E., and Navarro B., 2010. TIPSem (English and Spanish): evaluating CRFs and semantic roles in TempEval-2. In Proceedings of SemEval-2010, Uppsala, Sweden: ACL, pp. 284291.
Marrou H.-I. 1954. De la connaissance historique. Paris: Seuil.
Meroño-Peñuela A., Ashkpour A., van Erp M., Mandemakers K., Breure L., Scharnhorst A., Schlobach S., and van Harmelen F. 2015. Semantic technologies for historical research: a survey. Semantic Web Journal, 6 (6): 539–64. IOS Press.
Metzler D., Cai C., and Hovy E. 2012. Structured event retrieval over microblog archives. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, Canada: ACL.
Mikheev A., Grover C., and Moens M. 1998. Description of the LTG system used for MUC-7. In Proceedings of 7th Message Understanding Conference (MUC-7), Fairfax, VA.
Minard A. L., Speranza M., Agirre E., Aldabe I., van Erp M., Magnini B., Rigau G., and Urizar R. 2015. Semeval-2015 task 4: timeline: cross-document event ordering. In Proceedings of SemEval 2015, ACL. Denver, Colorado, USA: Association for Computational Linguistics.
Minard A. L., Speranza M., Urizar R., Altuna B., van Erp M., Schoen A., and van Son C. 2016. MEANTIME, the NewsReader multilingual event and time corpus. In Proceedings of LREC 2016, Portoro, Slovenia, European Language Resources Association (ELRA).
Mirza P., and Minard A. L. 2014. FBK-HLT-time: a complete Italian temporal processing system for EVENTI-Evalita 2014. In Proceedings of the 4th International Workshop EVALITA 2014, Pisa, Italy: Pisa University Press.
Mirza P., and Tonelli S., 2014. An analysis of causality between events and its relation to temporal information. In Proceedings of COLING 2014, Dublin, Ireland: Dublin City University and ACL, pp. 20972106.
Mitamura T., Yamakawa Y., Holm S., Song Z., Bies A., Kulick S., and Strassel St., 2015. Event nugget annotation: processes and issues. In Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Denver, Colorado, USA: ACL, pp. 6676.
Mott J., Bies A., Song Z., and Strassel S. 2016. Parallel Chinese-English entities, relations and events corpora. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), Portoro, Slovenia, European Language Resources Association (ELRA).
Nédellec C., Bossy R., Kim J. D., Kim J. J., Ohta T., Pyysalo S., and Zweigenbaum P., 2013. Overview of BioNLP shared task 2013. In Proceedings of the BioNLP Shared Task 2013 Workshop, Sofia, Bulgaria, pp. 17.
Nieto M. G., Saurí R., and Bernabé Poveda M. A., 2011. ModeS TimeBank: a modern spanish TimeBank corpus. Procesamiento del lenguaje natural 47 (2011): 259267.
Palmer M., Gildea D., and Kingsbury P. 2005. The proposition bank: an annotated corpus of semantic roles. Computational Linguistics, 31 (1): 71106.
Petrović S., Osborne M., and Lavrenko V., 2010. Streaming first story detection with application to twitter. In Proceedings of NAACL 2010, Los Angeles, California, USA, pp. 181189.
Pustejovsky J., 1991. The syntax of event structure. Cognition 41 (1–3): 4781.
Pustejovsky J. 2005. A survey of dot objects. Author’s weblog. Retrieved from URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208.7525&rep=rep1&type=pdf
Pustejovsky J., Castaño J. M., Ingria R., Saurí R. G., Setzer A. and Katz G. 2003. TimeML: robust specification of event and temporal expressions in text. In Proceedings of IWCS-5. Tilburg, The Netherlands.
Pustejovsky J., Hanks P., Saurí R., See A., Gaizauskas R., Setzer A., Radev D., Sundheim B., Day D., Ferro L., and Lazo M., 2003. The TIMEBANK Corpus. In Proceedings of Corpus Linguistics 2003, Lancaster, UK, pp. 647656.
Pustejovsky J., Littman J., and Saurì R. 2007. Arguments in TimeML: events and entities. In Schilder F., Katz G., & Pustejovsky J. (eds), Annotating, Extracting and Reasoning about Time and Events: International Seminar Revised Papers (Lecture Notes in Computer Science), pp. 107126. Berlin: Springer Berlin Heidelberg.
Rayson P., Archer D., Piao S., and McEnery A. M., 2004. The UCREL semantic analysis system. In Proceedings of the Beyond Named Entity Recognition Semantic Labelling for NLP Tasks Workshop, Lisbon, Portugal, pp. 712.
Rayson P., Baron A., Piao S., and Wattam S. 2015. Large-scale time-sensitive semantic analysis of historical corpora. In Proceedings of the 36th Meeting of ICAME, Trier, Germany.
Ritter A., Mausam E. O., and Clark S., 2012. Open domain event extraction from Twitter. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Beijing, China: ACM, pp. 11041112.
Sasse H. J., 2002. Recent activity in the theory of aspect: accomplishments, achievements, or just non-progressive state. Linguistic Typology 6 (2): 199271.
Saurı R. 2010. Annotating temporal relations in catalan and spanish TimeML annotation guidelines. Technical Report BM 2010-04, Barcelona Media.
Saurí R., and Pustejovsky J., 2009. Factbank: a corpus annotated with event factuality. Language Resources and Evaluation 43 (3): 227–68.
Schilder F., and Habel C. 2001. From temporal expressions to temporal information: semantic tagging of news messages. In Proceedings of the ACL-EACL 2001 Workshop for Temporal and Spatial Information Processing. Toulouse, France: ACL.
Schilder F., and Habel C. 2003. Temporal information extraction for temporal question answering. In New Directions in Question Answering, AAAI, pp. 3544.
SemAf/Time Working Group ISO, 2008. ISO DIS 24617-1: 2008 Language Resource Management - Semantic Annotation Framework - Part 1: Time and Events. ISO Central Secretariat, Geneva.
Setzer A. 2001. Temporal Information in Newswire Articles: An Annotation Scheme and Corpus Study. PhD Thesis, University of Sheffield.
Simiand F. 1960. Méthode historique et science sociale. In Annales. Histoire, Sciences Sociales 15 (1): 83119. France: EHESS.
Song Z., Bies A., Strassel S., Riese T., Mott J., Ellis J., Wright J., Kulick S., Ryant N., and Ma X., 2015. From light to rich ERE: annotation of entities, relations, and events. In Proceedings of the The 3rd Workshop on EVENTS: Definition, Detection, Coreference, and Representation, Denver, Colorado, USA: ACL, pp. 8998.
Song Z., Biesa A., Strassel S., Ellis J., Mitamura T., Dang H., Yamakawa Y., and Holm S. 2016. Event nugget and event coreference annotation. In Proceedings of the 4th Workshop on EVENTS: Definition, Detection, Coreference, and Representation, NAACL HLT 2016.
Styler IV W. F., Bethard S., Finan S., Palmer M., Pradhan S., de Groen P.C., Erickson B., Miller T., Lin C., Savova G., and Pustejovsky J., 2014. Temporal annotation in the clinical domain. Transactions of the Association for Computational Linguistics 2 (2014): 143–54.
Sun W., Rumshisky A., and Uzuner O. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association. 20 (5): 806–13. Oxford University Press.
UzZaman N., Llorens H., Derczynski L., Allen J., Verhagen M., and Pustejovsky J., 2013. Semeval-2013 task 1: Tempeval-3: Evaluating time expressions, events, and temporal relations. In Proceedings of SemEval 2013, Atlanta, Georgia, USA: ACL, pp. 19.
Van Den Akker C., Aroyo L., Cybulska A., Van Erp M., Gorgels P., Hollink L., Jager C., Legene S., van der Meij L., Oomen J., van Ossenbruggen J., Schreiber G., Segers R., Vossen P., and Wielinga B. 2010. Historical event-based access to museum collections. In Proceedings of EVENTS 2010, Athens, Greece: ACL.
van Son C., van Erp M., Fokkens A., and Vossen P., 2014. Hope and fear: interpreting perspectives by integrating sentiment and event factuality. In Proceedings of LREC 2014, Reykjavik, Iceland, European Language Resources Association (ELRA), pp. 2631.
Vendler Z. 1967. Verbs and times. In Linguistics and Philosophy, pp. 97121. Ithaca, NY: Cornell University Press.
Verhagen M., Gaizauskas R., Schilder F., Hepple M., Katz G., and Pustejovsky J., 2007. Semeval-2007 task 15: Tempeval temporal relation identification. In Proceedings of SemEval-2007, Prague, Czech Republic: ACL, pp. 7580.
Verhagen M., Sauri R., Caselli T., and Pustejovsky J., 2010. Semeval-2010 task 13: Tempeval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden: ACL, pp. 5762.
Vossen P., Rigau G., Serafini L., Stouten P., Irving F., and Van Hage W. 2014. NewsReader: recording history from daily news streams. In Proceedings of LREC 2014, Reykjavik, Iceland, European Language Resources Association (ELRA).
Yaghoobzadeh J., Ghassem-Sani G., Mirroshandel S. A., and Eshaghzadeh M., 2012. ISO-TimeML event extraction in Persian text. In Proceedings of COLING 2012, Mumbai, India, pp. 29312944.
Zavarella V., and Tanev H., 2013. FSS-TimEx for TempEval-3: extracting temporal information from text. In Proceedings of SemEval 2013, Atlanta, Georgia, USA, ACL, pp. 5863.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Natural Language Engineering
  • ISSN: 1351-3249
  • EISSN: 1469-8110
  • URL: /core/journals/natural-language-engineering
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 13
Total number of PDF views: 98 *
Loading metrics...

Abstract views

Total abstract views: 604 *
Loading metrics...

* Views captured on Cambridge Core between 25th October 2016 - 15th December 2017. This data will be updated every 24 hours.