1. Introduction
Negation is a complex linguistic phenomenon present in all human languages, allowing for the uniquely human capacities of denial, contradiction, misrepresentation, lying, and irony (Horn Reference Horn1989). Broadly speaking, negation “relates an expression e to another expression with a meaning that is in some way opposed to the meaning of e” (Horn and Wansing Reference Horn, Wansing and Zalta2017). The key challenge to understanding negation is thus to figure out the meaning that is in some way opposed to e—a semantic and highly ambiguous undertaking that comes naturally to humans in everyday communication. As described by Lawler (Reference Lawler and Hogan2010), cognitively, negation “involves some comparison between a real situation lacking some particular element and an imaginal situation that does not lack it.”
From a linguistic perspective, it is generally understood that negation has scope and focus. Scope is “the part of the meaning that is negated” and focus “the part of the scope that is most prominently or explicitly negated” (Huddleston and Pullum Reference Huddleston and Pullum2002). Consider statement (1). The falsity of any of the propositions in (1a–1d) makes statement (1) true, thus the scope of the negation includes all the elements indicated in the right column in (1a–1d):
-
1. This camera doesn’t take good pictures in low light
1a. Something takes something in some condition. [verb] takes
1b. This camera takes something in some condition. [agent of taking, This camera]
1c. Something takes good pictures in some condition. [theme of taking, good pictures]
1d. Something takes something in low light. [manner of knowing, in low light]
Determining the focus consists in pinpointing which parts of the scope are intended to be interpreted as true and false given the original statement. Without further context, one may consider condition (1c) or condition (1d) to be interpreted as false, and conditions (1a, 1b, and 1d) or conditions (1a–1c) to be interpreted as true, respectively. The first option yields the affirmative interpretation This camera takes bad pictures in low light, and the second option yields This camera takes good pictures when there is enough light.
Propositions where a negation is present are generally understood to carry positive meaning, or in other words, to suggest an affirmative alternative. For example, John didn’t leave the house implicates that John stayed inside the house. Hasson and Glucksberg (Reference Hasson and Glucksberg2006) show that comprehending negation involves considering the representation of affirmative alternatives. While not fully understood, there is evidence that negation involves reduced access to the affirmative mental representation (Djokic et al. Reference Djokic, Maillard, Bulat and Shutova2019). Orenes, Beltrán, and Santamaría (Reference Orenes, Beltrán and Santamaría2014) provide evidence that humans switch to the affirmative alternative in binary scenarios (e.g., from not red to green when processing The figure could be red or green. The figure is not red). In such multi-scenarios, however, humans keep the negated representation unless the affirmative interpretation is obvious from context (e.g., humans keep not red when processing The figure is red, green, yellow or blue. The figure is not red.).
Previous work in computational linguistics has pointed out that processing negation is beneficial for some applications. In particular, sentiment analysis has a long tradition of incorporating components to process negation (Wiegand et al. Reference Wiegand, Balahur, Roth, Klakow and Montoyo2010), ranging from handcrafted rules to sophisticated neural network architectures. For example, like generally carries positive sentiment, but not when modified by a negation cue (e.g., don’t like). Wilson, Wiebe, and Hoffmann (Reference Wilson, Wiebe and Hoffmann2005) introduce the idea of contextual polarity, and note that negation may intensify rather than change polarity (e.g., not good vs. not only good but amazing). Jia, Yu, and Meng (Reference Jia, Yu and Meng2009) present a set of heuristic rules to determine sentiment when negation is present, and Councill, McDonald, and Velikovich (Reference Councill, McDonald and Velikovich2010) show that information about the scope of negation is beneficial to predict sentiment. More recently, Socher et al. (Reference Socher, Perelygin, Wu, Chuang, Manning, Ng and Potts2013) present a recursive neural network for sentiment analysis that captures sentiment polarity changes when negation is present. Reitan et al. (Reference Reitan, Faret, Gambäck and Bungum2015) show that a state-of-the-art Twitter sentiment analysis benefits from incorporating information about the scope of negation.
Outside sentiment analysis, researchers have pointed out that negation poses unsolved challenges for, among others, machine translation and natural language inference. Fancellu and Webber (Reference Fancellu and Webber2015) present a manual error analysis translating negation from Chinese to English, and Bentivogli et al. (Reference Bentivogli, Bisazza, Cettolo and Federico2016) point out that neural machine translation struggles as much as statistical machine translation when it comes to translating negation. The task of natural language inference—determining whether a text entails, contradicts, or is neutral with respect to a hypothesis—and in particular existing benchmarks (e.g., SNLI (Bowman et al. Reference Bowman, Angeli, Potts and Manning2015), MNLI (Williams, Nangia, and Bowman Reference Williams, Nangia and Bowman2018)), do not properly account for inference in the presence of negation. A couple of insights from the literature support this claim. First, Gururangan et al. (Reference Gururangan, Swayamdipta, Levy, Schwartz, Bowman and Smith2018) point out that negation cues (nobody, never, no, nothing, etc.) are a strong indicator of contradictions in MNLI. Second, creating adversarial examples by adding straightforward negations make existing systems underperform. For example, Naik et al. (Reference Naik, Ravichander, Sadeh, Rose and Neubig2018) concatenate the tautology “and false is not true” at the end of hypotheses and conclude that models are not robust to negation. Similarly, Wallace et al. (Reference Wallace, Feng, Kandpal, Gardner and Singh2019) introduce universal triggers, that is, sequences of tokens that fool models when concatenated to any input. They show that concatenating negation cues such as nobody, never, nothing, and none to hypotheses decreases accuracy to almost zero when the gold label is entailment or neutral.
Processing negation is crucial when extracting information from clinical and biomedical texts, since negation is very frequent and missing negated observations can lead to misleading conclusions that have impact in health care and biomedical research. For example, the medical relevance of the sentence “shows no symptoms of COVID-19” is quite different from the sentence “shows symptoms of COVID-19”. Negation detection systems, such as the NegEx algorithm (Chapman et al. Reference Chapman, Bridewell, Hanbury, Cooper and Buchanan2001) and its new version (Mehrabi et al. Reference Mehrabi, Krishnan, Sohn, Roch, Schmidt, Kesterson, Beesley, Dexter, Schmidt, Liu and Palakal2015), have shown good performance in clinical text. However, as Wu et al. (Reference Wu, Miller, Masanz, Coarr, Halgrim and Clark2014) indicate, negation is not solved because “current solutions are easily optimizable but not necessarily generalizable. Negation detection is still a challenge when considered from a practical, multi-corpus perspective, that is, one in which an algorithm is deployed in many clinical institutions and on many sources of text.” The authors conclude that in order to improve the performance in negation detection, more manually annotated data are needed in corpora that reflect the distributions of negation in different domains and genres. Cohen et al. (Reference Cohen, Goss, Zweigenbaum and Hunter2017) find out that there are differences in the distribution of negation between clinical reports and biomedical articles. For example, there is more affixal negation in the clinical texts, whereas affixal level is more frequent in journal articles.
This special issue is devoted to advances in processing negation from a computational perspective. Section 2 provides information about existing corpora where negation is annotated and shared tasks. Section 3 presents previous work on processing negation, mainly on scope and focus identification.
2. Annotated corpora and shared tasks
That the interest in processing negation has grown in the last years is reflected in the number of corpora that have been published. Fancellu et al. (Reference Fancellu, Reddy, Lopez and Webber2017b) group corpora that contain some representation of negation into two types. The first-type corpora represent negation in a logical form, using quantifiers, predicates, and relations. One of them is the Groningen Meaning Bank (Basile et al. Reference Basile, Bos, Evang and Venhuizen2012),Footnote a a collection of semantically annotated English texts with formal meaning representations. It is composed of newswire texts from Voice of America, country descriptions from the CIA Factbook, a collection of texts from the open ANC, and Aesop’s Fables. Another corpus is the DeepBank corpus (Flickinger, Zhang, and Kordoni Reference Flickinger, Zhang and Kordoni2012),Footnote b which contains rich syntactic and semantic annotations for the 25 Wall Street Journal sections in the Penn Treebank (Taylor, Marcus, and Santorini Reference Taylor, Marcus and Santorini2003). The second-type corpora use string-level annotations, where negation information (cues, scope, negated event, and focus) are annotated on spans of text (e.g., BioScope (Vincze et al. Reference Vincze, Szarvas, Farkas, Móra and Csirik2008), ConanDoyle-neg (Morante and Daelemans Reference Morante and Daelemans2012)). Jiménez-Zafra et al. (Reference Jiménez-Zafra, Morante, Martín-Valdivia and Ureña-López2020) recently surveyed corpora annotated with negation focusing on corpora that contain string-level negation. They describe the main features of the corpora based on the following criteria: the language, year of publication, domain, the availability, size, types of negation taken into account (syntactic and/or lexical and/or morphological), negation elements annotated (cue and/or scope and/or negated event and/or focus), and the way in which each corpus was tokenized, the annotation guidelines, and annotation scheme used. Information is provided about existing corpora for the following languages: English, Spanish, Swedish, Dutch, Japanese, Chinese, German, and Italian.
Corpora annotated with negation information usually include at least negation cues and either scope or focus. Some corpora also include event-level annotations indicating whether an event is negated. We note, however, that negation is ubiquitous in text. Jiménez-Zafra et al. (Reference Jiménez-Zafra, Morante, Martín-Valdivia and Ureña-López2020) report that the number of sentences with negation in the English corpora is between 9.37% and 32.16%, while in Spanish corpora it is between 10.67% and 34.22%, depending on the domain. Thus, virtually all corpora include examples of negation. For example, parallel corpora for machine translation, product reviews annotated with sentiment, and corpora annotated with coreference chains include sentences with negation even though the negations themselves are not explicitly annotated.
Early work on annotating negation targeted the biomedical domain, for example, explicit negation of biological relationships (Pyysalo et al. Reference Pyysalo, Ginter, Heimonen, Björne, Boberg, Järvinen and Salakoski2007), negated events (Kim, Ohta, and Tsujii Reference Kim, Ohta and Tsujii2008), and negation cues and their scope (Vincze et al. Reference Vincze, Szarvas, Farkas, Móra and Csirik2008). Negation cues and scope have also been annotated in product reviews (Councill et al. Reference Councill, McDonald and Velikovich2010; Konstantinova et al. Reference Konstantinova, de Sousa, Cruz, Maña, Taboada and Mitkov2012), short stories (Morante and Daelemans Reference Morante and Daelemans2012), and dialogs within intelligent tutoring systems (Banjade and Rus Reference Banjade, Rus, Chair, Choukri, Declerck, Goggi, Grobelnik, Maegaard, Mariani, Mazo, Moreno, Odijk and Piperidis2016). Liu, Fancellu, and Webber (Reference Liu, Fancellu, Webber, Chair, Choukri, Cieri, Declerck, Goggi, Hasida, Isahara, Maegaard, Mariani, Mazo, Moreno, Odijk, Piperidis and Tokunaga2018) present a parallel corpus of English and Chinese annotated with negation. For Spanish, Sandoval and Salazar (Reference Sandoval and Salazar2013) present annotations of negation cues and scope in news articles. Cruz et al. (Reference Cruz, Morante, Maña López, Mata Vázquez and Parra Calderón2017) present similar annotations in the medical domain and include syntactic, morphological, and lexical negation. The latest and largest corpus (9455 sentences) is the SFU corpus by Jiménez-Zafra et al. (Reference Jiménez-Zafra, Taulé, Martín-Valdivia, Ureña-López and Martí2018b), who annotate product reviews for a variety of product categories with, among others, negation cues and scopes. Negation markers and either events or scopes have also been annotated in Swedish (Dalianis and Velupillai Reference Dalianis and Velupillai2010), Dutch (Afzal et al. Reference Afzal, Pons, Kang, Sturkenboom, Schuemie and Kors2014), Chinese (Zou, Zhou, and Zhu Reference Zou, Zhou and Zhu2016), German (Cotik et al. Reference Cotik, Roller, Xu, Uszkoreit, Budde and Schmidt2016a), and Italian (Altuna, Minard, and Speranza Reference Altuna, Minard and Speranza2017).
Regarding focus of negation, existing corpora work primarily with English texts. PB-FOC (Blanco and Moldovan Reference Blanco and Moldovan2011; Morante and Blanco Reference Morante and Blanco2012) was the first corpus to annotate the focus of negation and targets the negations marked with argm-neg semantic role in PropBank (Palmer, Gildea, and Kingsbury Reference Palmer, Gildea and Kingsbury2005). All negations annotated in PB-FOC are verbal negations, that is, the negation cues (e.g., not, never) are grammatically associated with a verb. The authors choose as focus the semantic role of the verbal negation that is most prominently negated. This decision simplifies the annotation task since usually a few semantic roles span many tokens. Anand and Martell (Reference Anand and Martell2012) reannotate PB-FOC and argue that (a) PB-FOC overannotates foci, and (b) alternatives arising from scalar implicatures and neg-raising predicates should be separated from those arising from focus detection. Other efforts working with English texts also refine the annotations in PB-FOC. Blanco and Moldovan (Reference Blanco and Moldovan2012) define fine-grained focus as the tokens that are intended to be negated, in contrast to the coarse-grained foci annotated in PB-FOC, which are always all tokens belonging to a semantic role. They reannotate the foci in PB-FOC whose syntactic constituents are a PP, ADVP, or SBAR and discover that 68% of them can be refined. Blanco and Sarabi (Reference Blanco and Sarabi2016) present an alternative annotation schema. Instead of choosing as focus the semantic role that is most prominently negated, they consider all roles and rank the likelihood of the underlying positive interpretations with a score ranging from 0 to 5. Sarabi and Blanco (Reference Sarabi and Blanco2016) move away from semantic roles and work with syntactic dependencies. By selecting subtrees in dependency trees, they target foci of verbal negations—and the underlying positive interpretations—in a continuum of granularity. Finally, Sarabi and Blanco (Reference Sarabi and Blanco2017) follow a similar approach manipulating syntactic dependencies, but target negation cues that modify nouns or adjectives instead of verbs.
In addition to PB-FOC and the refinements aforementioned, there are two corpora with focus annotations. Banjade and Rus (Reference Banjade, Rus, Chair, Choukri, Declerck, Goggi, Grobelnik, Maegaard, Mariani, Mazo, Moreno, Odijk and Piperidis2016) work with dialogs retrieved from interactions between students and an intelligent tutoring system. Their corpus, DT-neg, contains annotations of negation cues as well as their scope and focus. The authors note that the scope and focus are often in a previous dialog turn. Kolhatkar et al. (Reference Kolhatkar, Wu, Cavasso, Francis, Shukla and Taboada2019) work with a collection of news articles and user-submitted comments online. Their corpus, the SFU Opinion and Comments Corpus (SOCC), contains annotations of negation cues as well as their scope and focus, among other annotations. To our knowledge, these two corpora are the only ones that work with user-generated content in English. Finally, to our knowledge, there is only one corpus annotating the focus of negation in languages other than English (Matsuyoshi, Otsuki, and Fukumoto Reference Matsuyoshi, Otsuki and Fukumoto2014). This corpus includes news articles and user reviews written in Japanese and targets negation cues and their foci.
Several shared tasks have addressed negation processing in English: in the BioNLP09 Shared Task 3 (Kim et al. Reference Kim, Ohta, Pyysalo, Kano and Tsujii2009) and in the the i2b2 NLP Challenge (Uzuner et al. Reference Uzuner, South, Shen and DuVall2011), systems had to detect negated events. The *SEM 2012 Shared Task (Morante and Blanco Reference Morante and Blanco2012) was devoted to processing the scope and focus of negation, and in the ShARe/CLEF eHealth Evaluation Lab 2014 Task 2 (Mowery et al. Reference Mowery, Velupillai, South, Christensen, Martinez, Kelly, Goeuriot, Elhadad, Pradhan, Savova and Chapman2014) participants had to detect whether a disorder was negated. For Spanish, there have been two editions of the NEGES task (Jiménez-Zafra et al. Reference Jiménez-Zafra, Díaz, Morante and Martín-Valdivia2018a; Reference Jiménez-Zafra, Cruz Díaz, Morante and Martín-Valdivia2019), where systems had to detect negation cues and evaluate the impact of negation in sentiment analysis.
3. Processing negation
The first attempts to process negation in English were mostly rule-based and focused on finding negated terms in clinical texts domain (Chapman et al. Reference Chapman, Bridewell, Hanbury, Cooper and Buchanan2001; Mutalik, Deshpande, and Nadkarni Reference Mutalik, Deshpande and Nadkarni2001; Goldin and Chapman Reference Goldin and Chapman2003; Auerbuch et al. Reference Auerbuch, Karson, Ben-Ami, Maimon and Rokach2004; Elkin et al. Reference Elkin, Brown, Bauer, Husser, Carruth, Bergstrom and Wahner-Roedler2005; Boytcheva et al. Reference Boytcheva, Strupchanska, Paskaleva, Tcharaktchiev and Str2005; Goryachev et al. Reference Goryachev, Sordo, Zeng and Ngo2006; Sanchez-Graillet and Poesio Reference Sanchez-Graillet and Poesio2007; Huang and Lowe Reference Huang and Lowe2007; Rokach, Romano, and Maimon Reference Rokach, Romano and Maimon2008). Some rule-based systems were developed based on lists of negations and stop words (Mitchell et al. Reference Mitchell, Becich, Berman, Chapman, Gilbertson, Gupta, Harrison, Legowski and Crowley2004; Mykowiecka, Marciniak, and Kupść Reference Mykowiecka, Marciniak and Kupść2009; Harkema et al. Reference Harkema, Dowling, Thornblade and Chapman2009; Uzuner, Zhang, and Sibanda Reference Uzuner, Zhang and Sibanda2009; Sohn, Wu, and Chute Reference Sohn, Wu and Chute2012). With the surge of opinion mining, negation was studied as a marker of polarity change (Das and Chen Reference Das and Chen2001; 2005b; Polanyi and Zaenen Reference Polanyi and Zaenen2006; Taboada et al. Reference Taboada, Brooke, Tofiloski, Voll and Stede2011; Jiménez-Zafra et al. Reference Jiménez-Zafra, Valdivia, Cámara and Ureña-López2017). Only with the release of the BioScope corpus (Vincze et al. Reference Vincze, Szarvas, Farkas, Móra and Csirik2008) did the work on negation receive a boost. But even so, despite the existence of several publications that focus on negation, it is difficult to find a negation processor for languages other than English. For English, some systems are available for processing clinical documents: NegEx (Chapman et al. Reference Chapman, Bridewell, Hanbury, Cooper and Buchanan2001), ConText (Harkema et al. Reference Harkema, Dowling, Thornblade and Chapman2009), Deepen (Mehrabi et al. Reference Mehrabi, Krishnan, Sohn, Roch, Schmidt, Kesterson, Beesley, Dexter, Schmidt, Liu and Palakal2015), NegBio Peng et al. (Reference Peng, Wang, Lu, Bagheri, Summers and Lu2018), and NegTool (Enger, Velldal, and Øvrelid Reference Enger, Velldal and Øvrelid2017).
3.1 Scope of negation identification
Compared to focus identification, scope identification has received substantially more attention, due to the existence of several corpora annotated with scopes. Inspired by the annotation of scopes in the BioScope corpus, the task of detecting negation scopes was introduced as a machine learning sequence labeling task (Morante, Liekens, and Daelemans Reference Morante, Liekens and Daelemans2008; Morante and Daelemans Reference Morante and Daelemans2009). Subsequently, three main types of approaches have been applied to processing negation:
-
• Rule-based systems, in an attempt to improve the NegEx algorithm, such as ConText (Harkema et al. Reference Harkema, Dowling, Thornblade and Chapman2009), DEEPEN (Mehrabi et al. Reference Mehrabi, Krishnan, Sohn, Roch, Schmidt, Kesterson, Beesley, Dexter, Schmidt, Liu and Palakal2015), and NegMiner (Elazhary Reference Elazhary2017). Packard et al. (Reference Packard, Bender, Read, Oepen and Dridan2014) present an approach that works over explicit and formal representations of propositional semantics, deriving the scope from the structure of logical-form meaning representations.
-
• Machine learning techniques (Agarwal and Yu Reference Agarwal and Yu2010; Li et al. Reference Li, Zhou, Wang and Zhu2010; Cotik et al. Reference Cotik, Stricker, Vivaldi and Rodríguez Hontoria2016b). Read et al. (Reference Read, Velldal, Øvrelid and Oepen2012) proposed an SVM-based ranking of syntactic constituents to identify the scope. Velldal et al. (Reference Velldal, Øvrelid, Read and Oepen2012) explore a combination of deep and shallow approaches to the problem of resolving the scope of speculation and negation. Reitan et al. (Reference Reitan, Faret, Gambäck and Bungum2015) apply a negation scope detection system to Twitter data.
-
• Deep learning approaches (Qian et al. Reference Qian, Li, Zhu, Zhou, Luo and Luo2016; Ren, Fei, and Peng Reference Ren, Fei and Peng2018; Lazib et al. Reference Lazib, Qin, Zhao, Zhang and Liu2018). Fancellu, Lopez, and Webber (Reference Fancellu, Lopez and Webber2016) present neural networks for this task, and Fancellu et al. (Reference Fancellu, Lopez, Webber and He2017a) present an error analysis showing that scope is much easier to identify when delimited by punctuation.
3.2 Focus of negation identification
Automated systems to predict the focus of negation as annotated in PB-FOC are built using traditional machine learning algorithms as well as sophisticated neural networks. Blanco and Moldovan (Reference Blanco and Moldovan2011) build a decision tree with a feature set emphasizing semantic role information. Rosenberg and Bergler (Reference Rosenberg and Bergler2012) present a rule-based approach implementing several heuristics. Zou, Zhou, and Zhu (Reference Zou, Zhou and Zhu2014) introduce a graph model for focus identification that takes into account inter-sentential information in addition to intra-sentential information. Doing so allows them to consider context from a lexical and topical perspective. Sarabi and Blanco (Reference Sarabi and Blanco2019) present a simpler approach to include contextual information: they train an SVM with features derived from an off-the-shelf rhetorical structure theory (Mann and Thompson Reference Mann and Thompson1987) discourse parser (Surdeanu, Hicks, and Valenzuela-Escárcega Reference Surdeanu, Hicks and Valenzuela-Escárcega2015). To date, the best results published are presented by Shen et al. (Reference Shen, Zou, Hong, Zhou, Zhu and Aw2019), who obtain 70.51 accuracy. They present a BiLSTM with a CRF layer and attention mechanisms defined at the word and topic level.
There are less efforts experimenting with the other corpora annotating focus of negation. Most use an SVM trained with lexical, syntactic, and semantic features (Blanco and Moldovan Reference Blanco and Moldovan2012; Blanco and Sarabi Reference Blanco and Sarabi2016; Sarabi and Blanco Reference Sarabi and Blanco2016; Sarabi and Blanco Reference Sarabi and Blanco2017). Matsuyoshi et al. (Reference Matsuyoshi, Otsuki and Fukumoto2014) present 16 heuristic rules to detect the focus in Japanese. Finally, van Son et al. (Reference van Son, Morante, Aroyo and Vossen2018) present a replication study of the corpus by Blanco and Moldovan (Reference Blanco and Moldovan2012). They present results modeling the scoring class as regression and classification tasks and suggest that three labels (true, false, and uncertain) may be a better choice than a score ranging from 0 to 5.
This material is based upon work supported partly by the National Science Foundation under Grant No. 1845757. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Roser Morante was supported by the Netherlands Organization for Scientific Research (NWO) via the Spinoza prize awarded to Piek Vossen (SPI 30-673, 2014-2019).
Open access
