3.2.1 Non-finite adverbial subordination

There exist finite and non-finite adverbial clauses in Turkish, where the non-finite forms are more numerous and more widely used (Göksel and Kerslake Reference Göksel and Kerslake2004). The subordinate verb forms in non-finite adverbial clauses are called converbs, and converbial suffixes form these by transforming verbs into adverbs. We map these suffixes to proper AMR relations to indicate the relationships between sentence constituents. Table 1 provides the mapping of some such suffixes to AMR relations. However, we should point out that the meanings of these suffixes may differ within different contexts. Therefore during the annotations, they should be mapped to proper relations accordingly. All such relations in English AMR start with the prep-X prefix, which holds for prepositions. One should note that these are rather postpositions in Turkish carrying the same meaning with X. Korean AMR studies (Choe et al. Reference Choe, Han, Park, Oh, Park and Kim2019b, Reference Choe, Han, Park, Oh and Kim2020) also discuss adverbial subordination in general. As opposed to these, we prefer to use the relation names as they are, to be in parallel with English AMR rather than renaming them as postp-X. We believe these prefixes are syntactic issues rather than semantic and should be removed in a universal schema.

Table 1. Some suffixes forming converbs and their corresponding AMR relations

In some cases, the needed relation type may not exist in AMR predefined relation list. For example, in Table 1, we define new relations :prep-while and :prep-after because of the absence of any relationship covering the meaning of these suffixes.

3.2.2 Headless relative constructions

Headless relative constructions are relative clauses without an explicit noun head implicitly inferred most of the time. Chinese AMR studies (Li et al. Reference Li, Wen, Qu, Bu and Xue2016, Reference Li, Wen, Song, Qu and Xue2019) also investigate this phenomenon and our proposed solution originates from these. Turkish is a pro-drop language, and the omission of object or subject pronouns is possible in the case of nominalized verbs. In the AMR representations, we add the omitted pronouns, which can be either a person, a thing, or an event, according to the context. Figure 5 provides such an example where the concept “person” is added since the readers pointed by the pronoun those have to be human. We use the concept “thing” to depict the omitted pronouns referring to objects, events, or ideas.

Figure 5. Annotation of omitted pronouns in nominalized verbs.

3.3.1 Person marking and the null subject

In Turkish, a predicate must contain a person marker. The doer of an event that the predicate represents is revealed by the personal suffixes concatenated to the end of the predicates. The explicit usage of the subject is optional. In the sentence “Kitap okuyorum.” (I am reading a book.), the suffix “-m” (the last letter of the verb which stands for I) indicates who is reading the book. This type of subject usage is highly common in Turkish and called “null-subject.” We should also note that personal markers may also appear on nominalized verbs (Figure 4b). In AMR representation, in the case of a null subject, we accept the subject indicated by the personal suffix (depicted with a nominative pronoun in the AMR notation parallel to English) as the related argument of the predicate. It is worth noting that, in case of a missing explicit subject within the sentence, the absence of any person marker on the predicate indicates the 3rd person singular subject. Figure 3 sample on the left provides such a case where “:ARG0 (o/o)” is the omitted pronoun s/he.

Spanish is also a null-subject language and Migueles-Abraira et al. (Reference Migueles-Abraira, Agerri and Diaz de Ilarraza2018) discuss this feature in terms of AMR. However, contrary to Turkish, Spanish has gender and they need to handle the 3rd person null subjects in a different manner. Although not as much as Turkish, Brazilian Portuguese is also seen as a partial null-subject language (Holmberg, Nayudu, and Sheehan Reference Holmberg, Nayudu and Sheehan2009), and Anchiêta and Pardo (Reference Anchiêta and Pardo2018a) discuss this situation in terms of AMR. In the case of null subject, they also fill the related argument implicitly inferred.

3.3.2 Modality

Modality is the phenomenon in which possible situations are discussed. In Turkish, modality suffixes are used to express modalities such as possibility, obligation, and permission. English AMR simply represents syntactic modals using predicate frames such as possible-01, likely-01, obligate-01, permit-01, and recommend-01. Linh and Nguyen (Reference Linh and Nguyen2019) also mention syntactic modalities for Vietnamese but do not follow the grouping of modalities proposed by the English AMR.

For Turkish, we map Turkish modality suffixes to some selected predicates without changing the sentence meaning in parallel with English AMR. This seems straightforward, but there are considerations to be made. Firstly, Turkish does not have a predicate for the sense of possibility like in English. While the English PropBank provides a frame for the sense of possibility, the Turkish PropBank does not. Therefore, we create a special frame “mümkün.01” (possible) which has one argument :ARG1 to represent the possible event (in Figure 6a). Secondly, modality markers may carry more than one sense, and as a result, one could map them to more than one predicate according to the context.

Table 2 shows some common modality suffixes with their corresponding verb frames. Sentences in the first two rows and the last three rows have the same modality markers (-Abil and -mAlI), although their senses are entirely different. Furthermore, a verb can have more than one modality suffix at the same time (Figure 6b). In this case, each suffix should be mapped to a proper predicate separately and represented in AMR. One should note that the expression of modalities is not provided only by modality markers; there are nominals which give a modality expression to the sentence. To make annotation consistent, we map these nominals to the same frames with the modality markers.

Table 2. Modality samples

Figure 6. Modality representation in Turkish AMR.

3.3.3 Voices

Turkish has four voice structures (viz., reciprocal, reflexive, causative, and passive) constructed through voice suffixes (VSs) attached to verbs. Voices describe the relationship between the predicate and the subject. As a result, when a verb takes a VS, its arguments’ number and type may change or stay the same (Göksel and Kerslake Reference Göksel and Kerslake2004). The change of argument structure of verbs affects their AMR representation as expected, which brings some issues. The Turkish PropBank does not have frames for such verbs and uses their stems with some additional features to represent them. From the AMR point of view, there are two possible solutions to address the issue. The first one is to create verb frames for all verbs inflected by VS; however, this approach causes a vast amount of verb frames, which is a situation that we avoid, as we discussed above. Furthermore, VSs are not DS and do not derive new verbs. Classes and the meanings of the stems stay the same; the only change is on their argument–predicate relations. Thus, we believe that this approach does not provide a proper solution. A second and more appropriate solution is to represent VS-inflected verbs by the use of their stem frames as suggested in Şahin and Adalı (Reference Şahin and Adalı2018). However, instead of adding additional arguments to verb frames as in Şahin and Adalı (Reference Şahin and Adalı2018), we propose a more AMR-oriented approach also compatible with the English AMR framework. In the following paragraphs, we detail the proposed approach. We should state that since passive voice does not cause any changes on the verb argument structure as in English, we handle it by leaving the argument ARG0 empty as has been done for Spanish (Migueles-Abraira et al. Reference Migueles-Abraira, Agerri and Diaz de Ilarraza2018).

Reciprocal verbs express actions that are performed together or against each other. They are formed with the reciprocal suffix -(I)ş which could be affixed to only a few transitive and intransitive verb stems, for example, “öpüşmek” (to kiss each other), “özleşmek” (to miss each other), and “gülüşmek” (to laugh together). Şahin and Adalı (Reference Şahin and Adalı2018) benefit from the number of agents who do the action; however, this approach is not suitable for AMR because it is insufficient to represent the meaning of verbs in cases of mutual involvement of the agents to the action. We propose that the agents that perform the action reciprocally have to be both ARG0 and ARG1 of the verb. In Figure 7a, the subjects first linked via the conjunction and and then are used as the arguments.

Figure 7. AMR representation of the verb voice structures.

Reflexive verbs are formed by combining the reflexive suffix -(I)n only with transitive verbs. Reflexive verbs are type of verbs that indicate actions that affect the person who performs the action either directly or indirectly, for example, “yıkanmak” (wash oneself—to take a bath), “taranmak” (to comb one’s hair), and “giyinmek” (to wear oneself—to get dress). Şahin and Adalı (Reference Şahin and Adalı2018) suggest to define a new semantic role such as A0A1 which accounts for multi-role for representing such verbs, as a future work. The reason for this suggestion is stated as the PropBank conventions not allowing to annotate one argument with two different roles. However, this is possible in AMR. Thus, for Turkish AMR, we solve this issue by making ARG0 and ARG1 of the verbal stem as the same. We believe, our solution is more convenient since it increases the compatibility of the representation with the other AMR frameworks and the solution presented above for reciprocal verbs. Figure 7b shows the AMR representation of the reflexive verb yıkan.

Our solution using reentrancy for reciprocal and reflexive voices is similar to the solution proposed for the pronoun “se” in Spanish (Migueles-Abraira et al. Reference Migueles-Abraira, Agerri and Diaz de Ilarraza2018), except that Migueles-Abraira et al. (Reference Migueles-Abraira, Agerri and Diaz de Ilarraza2018) add an extra concept in the case of reciprocal usage of this pronoun. In our solution, we intend not to distinguish reciprocal and reflexive representations since (1) in both cases the ones who do the action and the ones who are affected by the action are the same and (2) the original AMR conventions suggest that “AMR should abstract away from coreference gadgets like pronouns, zero-pronouns, reflexives, control structures, etc.” (Banarescu et al. Reference Banarescu, Bonial, Cai, Georgescu, Griffitt, Hermjakob, Knight, Koehn, Palmer and Schneider2013). However, we also agree with Migueles-Abraira et al. (Reference Migueles-Abraira, Agerri and Diaz de Ilarraza2018) that the use of some specific pronouns would help to differentiate the meaning. An alternative to our current solution might be to use some specific pronouns (e.g., “birbiri” (each other) for reciprocity and “kendi” (oneself) for reflexivity) in ARG1 of the predicates to differentiate the two phenomena. We believe the mentioned AMR convention may be reconsidered in the case of a universal schema covering MRLs.

The causative suffixes (-dIr, -t, -It, -Ir, -Ar, -Art) attach to transitive or intransitive verbs (Göksel and Kerslake Reference Göksel and Kerslake2004) to construct causative structures such as “boyatmak” (to make somebody paint something), “yaptırmak” (to make somebody do something), and “kestirmek” (to make somebody cut something). Şahin and Adalı (Reference Şahin and Adalı2018) introduce a new role ArgA to show the causer of an action. Although this approach seems a fairly neat solution to incorporate the verb framing of Turkish causative structures, we prefer not to use it with AMR compatibility concern in mind. In English, there is no need of an additional role to represent the causative structure since it is constructed by the predicate make whose arguments indicate the agents who do the action and who cause the action done. To make Turkish AMR parallel to English AMR, we prefer to create a new verb frame “yap.03” (an equivalent for make-02 in the English PropBank) and use it in the AMR representation of Turkish causative verbs. Figure 7c illustrates the AMR representation of the causative verb “boyat” (make somebody paint). It is worth mentioning that all these voices may be used as nested structures and the meaning should be considered according to the context during the AMR annotation. The addition of two consecutive causative suffixes may or may not mean differently than the single occurrence of the causative voice. For example, for the sentence “Bizim mimara evi boyattırdım” (I made our architect to make somebody to paint the house), two nested yap.03 predicates would be necessary in the AMR annotation.

4.1.1 Inter-step tree

The parser takes an input sample I=(V, A, morph, t, Prop), where

• • V = { $v{_i}$ $\mid$ i $\in$ [0,n], i $\in$ $\mathbb{N}$ } is a set of nodes representing word tokens in the sentence,Footnote m

• • A = { $a{_i{_j}}$ $\mid$ i,j $\in$ [0,n], i $\ne$ j, i,j $\in$ $\mathbb{N}$ } is a set of dependency relations between nodes $v{_j}$ (the head) and $v{_i}$ (the dependent),

• • morph represents morphological features of words,

• • t represents parts-of-speech tags of words,

• • Prop = { $prop{_i{_k}}$ $\mid$ i $\in$ (0,n], i $\in$ $\mathbb{N}^+$ , k $\in$ [0,m], k $\in$ $\mathbb{N}$ } represents a set of semantic layer tags, where $prop{_i{_k}}$ corresponds to $k^{\rm th}$ annotation of node $v{_i}$ and m the number of semantic layer tags that the node $v{_i}$ has.

We define the inter-step tree D = (C, R, NodeProperties), where C = { $c{_i}$ $\mid$ i $\in$ (0,n], i $\in$ $\mathbb{N}^+$ } represents a set of nodes, R = { $r{_j{_i}}$ $\mid$ i,j $\in$ (0,n], i $\ne$ j, i,j $\in$ $\mathbb{N}^+$ } represents a set of edges, and NodeProperties is a quadruple <morph, t, head node, dependency relation> consisting of the features of each node $c{_i}$ . $c{_i}$ and $r{_i{_j}}$ are defined as below, where orderof(j) represents the order of the predicate within the sentence. Since k=0 and k=1 are reserved for predicate declaration (see Table 3), the argument roles start from k=2.

$c{_i} = \left\{\begin{matrix}prop{_i{_1}} && \textrm{if } prop{_i{_0}}=\textrm{Y }\\[3pt] v_{i} && \textrm{otherwise}\end{matrix}\right.$ and $\quad r{_i{_j}} = \left\{\begin{matrix}prop{_i{_k}} && \textrm{if } k = \textrm{orderof}(j)+1 \\[3pt] a{_i{_j}} && \textrm{otherwise}\end{matrix}\right.$

Since the semantic layer tag $prop{_i{_k}}$ can be a verb frame or an argument relation or the letter “Y,” it can be expressed by a node or relation in the inter-step tree, depending on its type. When a node has more than one relation tag, the very first tag becomes $c{_i}$ , the rest is used to establish reentrancy connections (details in Section 4.1.2). The dependency components directly participate in the construction of D if they do not have any semantic layer tags. Figure 12a shows the inter-step tree of the sentence given in Table 3. The inter-step tree is constructed by the semantic layer tags which are verb frames (“bit.01” (end), “yürü.01” (walk), “de.01” (say)), relations (AMR-MNR, ARG1), and the TreeBank nodes (“bu” (this), “ilişki” (relationship)) and dependency relations (DETERMINER, COORDINATION). The word “ilişki” (relationship) has two argument relation tags A0 (ARG0) and A1 (ARG1) (Table 3), and A1 is used in the inter-step tree since it is the first tag.

Figure 12. Inter-step tree and AMR graph for “Bu ilişkiyi bitirelim, böyle yürütemeyeceğim, dedi.” (Let’s end this relationship, I can’t run it like this, she said).

4.1.2 Parser

We use a similar notation to Wang et al. (Reference Wang, Xue and Pradhan2015b) for the introduction of our parser. However, our parser does the alignment between text spans and AMR concepts simultaneously, and it differs from the mentioned study having different actions in a rule-based setting rather than a transition-based one.

We define our rule-based tree-to-graph parser as

Cr = (Cr, Actions, $Cr{_0}$ , Rules).

• • Cr is a set of parsing states,

• • Actions is a set of actions A: Cr $\rightarrow$ Cr,

• • $Cr{_0}$ is an initialization step where inter-step tree is built,

• • Rules is a set of conversion rules.

A parsing state is a couple (D, q), where q holds node indices according to the sentence word order, and it is used as a queue to process all nodes of the inter-step tree. The graph conversion starts with the construction of the inter-step tree and then continues with processing q. The parser starts with the first element of q and iterates by giving its related node in D and its properties to Rules where the next action is determined. At each iteration, the Rules set returns a set of actions according to the given node properties ([ $Rule(c{_i},NodeProperties{_i}) \rightarrow Actions{_a}$ ]), and the parser applies the action on D.

We have eight types of actions (Table 4) that will cover all possible situations in the conversion process. Pr(i) returns the parent index of a node at index i, Ch(i) returns all the children indexes of a node at index i, $\gamma : C \rightarrow R$ is a function that establishes an AMR relation between two input concepts, where the second argument of the function becomes the parent node after the action, $\zeta : C \rightarrow R$ deletes the relations between the current node and its parent. The function takes two arguments (i.e., the current node and its parent in focus). Since the initial inter-step tree is constructed from the dependency tree, the dependent could only have one head at the beginning but could have multiple heads as the AMR graph gets constructed. The focused parent may not be found directly and should be given as an argument to this function. $\delta : C \rightarrow C$ is a function that creates a new concept node from an existing node due to its morphological features and creates a relation between the new node and its parent (the current node). $\varphi : C \rightarrow C$ deletes a node given as an argument. $\iota : R \rightarrow L$ , where L is the AMR relation set, assigns a label to the given edge as an argument. The eight actions are as follows:

• • Add Edge: It simply adds an edge between the node in the queue with index i ( $c_{q_{i}}$ ) and the other node with index j ( $c{_{j}}$ ) in the inter-step tree. The newly created edge $r_{c_{q_{i}}c_{j}}$ is included into the edge set R. It also assigns a label l from AMR label set L to $r_{c_{q_{i}}c_{j}}$ .

• • Delete Edge: It deletes the edge between the node in the queue with index i ( $c_{q_{i}}$ ) and its parent. The removed edge $r_{c_{q_{i}}c_{Pr({q_{i}})}}$ is excluded from R.

• • Add Node: It creates a new node $c{_{k}}$ based on the node in the queue with index i ( $c_{q_{i}}$ ) and establishes an edge between $c{_{k}}$ and $c_{q_{i}}$ where the parent node is $c_{q_{i}}$ . The newly created node $c{_{k}}$ is included into the node set C.

• • Replace Head: It replaces the node in the queue $c_{q_{i}}$ with a new one $c{_{k}}$ . It first takes all children nodes of the node $c_{q_{i}}$ and then creates edges between the children and $c{_{k}}$ . The newly created node $c{_{k}}$ is included into the node set C and $c_{q_{i}}$ is excluded from C.

• • ReAttach: It deletes the edge between a node in the queue ( $c_{q_{i}}$ ) and its parent. A new edge is established between $c_{q_{i}}$ and a node $c{_{k}}$ .

• • Swap: It deletes the edge between a node in the queue ( $c_{q_{i}}$ ) and its parent. It creates a new edge between these two nodes in the opposite direction.

• • Merge: It creates a new node $c_{k}$ by combining the node in the queue with index i ( $c_{q_{i}}$ ) and its parent and connects $c_{k}$ to the grandparent of $c_{q_{i}}$ . The nodes $c_{q_{i}}$ and $c_{Pr({q_{i}})}$ are removed from the node set C.

Table 4. Actions

The parser processes q twice consecutively. The first process normalizes D either by a node addition or deletion and converts D to the graph form by adding reentrancies. The second process gives the graph its final shape by mapping nodes and relations with their AMR counterparts. We name these two steps graph conversion and post-process.

The graph conversion consists of three sub-steps: node removal, reentrancy, and suffix alignment. Nodes to get removed are for the words that do not contribute to the sentence meaning. These are determiners or intensifiers of the other nodes. The parser removes nodes connected to their heads with the relations DETERMINER and INTENSIFIER in the inter-step tree. However, this does not mean that all intensifiers and determiners do not contribute to the sentence meaning. Their meaning contributions depend on their usage and the whole sentence meaning. Our parser is not capable of distinguishing which ones should be removed or not. Reentrancies emerge when the same node participates in multiple relations. We call such nodes reentrancy nodes. The reentrancy nodes are the ones having more than one argument tags in their semantic layer (Table 3 node at the 2nd indice). As we mentioned in the previous subsection, the first tag is embedded in the inter-step tree. For the rest, in this step, the parser establishes new relations between reentrancy nodes and the most suitable nodes selected by the ruleset. As a result of this process, the inter-step tree turns into a graph. In Figure 12b, it is shown that the previously absent relation ARG0 (A0) is added into D between “ilişki” (relationship) and “yürü.01” (work).

Converting morphological suffixes to proper AMR components is the most important step of the Turkish AMR parsing. As discussed before, the majority of the meaning contributions come from these suffixes. The parser uses the given morphological properties of nodes ( $Node-Properties$ ). The following operations may be performed in accordance with the Turkish AMR framework (Section 3):

• • adding a null subject,

• • adding modalities,

• • adding polarity,

• • adding new nodes and relations coming from voice structures,

• • adding relations coming from case markers.

Figure 12b gives the automatically generated AMR graph for the studied example. As may be seen from the figure, the previously absent nodes “biz” (we) and “o” (s/he) are revealed by personal suffix markers extracted from NodeProperties and become the agents of the predicates “bitirelim” and “dedi.” The word “yürütemeyeceğim” (the verb run Footnote n in future tense with modality and negativity markers) has one causative suffix and multiple ISs (i.e., modality, negativity, and personal markers). The parser adds the concepts “yap.03” (make), “mümkün.01” (possible), “-” (minus), and “ben” (I) to represent causativity, modality, negativity, and the agent who does the action, respectively. It should be noted that the word “bitir” (the verb end) is constructed from the root word “bit” (to end) by the causative suffix -ir. However, since the morphological analyzer outputs its lemma as “bitir” instead of “bit” and misses to output the causative structure (Table 3 node at the 3rd indice), our AMR parser fails to extract this information from the node properties and to add the “yap.03” (make) concept in this example to represent causativity.

Post-processing maps non-AMR components that the previous stage has not mapped to AMR concepts and relations. The nodes that have abstract concepts in their representations are aligned with their AMR representations. On the other hand, the relations mapping could be either edge renaming or transformation of an edge to an equivalent AMR sub-graph. If the AMR specification has a relation that has the same meaning, edge renaming is straightforward as shown in Table 5. In Figure 12b, the parser maps AMR-MNR to the relation :manner and transforms COORDINATION to a sub-graph adding the node “and.” One should note that our parser is mostly developed on top of the syntactic features of words and sentences and is not good at capturing semantic relationships between sentence constituents. In Figure 12b, we see that the parser fails to construct the :cause relation since it could not get any clue about this semantic relation from the node properties.

Table 5. Direct mapping of PropBank relations to AMR relations

4.1.3 Evaluation

We evaluate the effectiveness of the parser (1) by comparing its outputs to gold standards and (2) using it for semi-automatic annotation. For the first set of evaluations, we use the Smatch score (Cai and Knight Reference Cai and Knight2013), an AMR evaluation metric that calculates the degree of overlapping between two formal semantic structures.Footnote o In the AMR case, two AMR graphs to be compared with each other are rewritten as logical propositions (i.e., triples), and the f-score between these triples in the graphs in terms of the propositional overlap against each other is calculated. For example, the triplet < ARG0(a, b)> shows that the two variables a and b are related in the AMR graph with the relationship ARG0. The produced variable names for the same concept in the two graphs may be different from each other, and Cai and Knight (Reference Cai and Knight2013) solve this problem by getting all possible triples and finding a subset that gives maximum f-score with the help of integer linear programming.

Our parser achieved a Smatch Score of 0.65 and 0.60 (on the Turkish AMR corpus Section 4.2) at the end of the first and second MAMA cycle iterations. One should note that similar to many parsing tasks in NLP, this is not an end-to-end parser and designed to be used with gold-standard dependency and PropBank annotations. In a real-world setting, our rule-based tree-to-graph AMR parser performance will be affected by the errors introduced by automatic morphological analysis, dependency parsing, and semantic role labeling. Still, we believe that this first Turkish AMR parser will act as a strong baseline for future studies on Turkish AMR parsing. As will be detailed in Section 4.2, our corpus contains 600 sentences with gold-standard dependency and PropBank annotations where the parser’s performance is measured as 0.61 and 100 sentences with automatically produced dependency and PropBank annotations where the parser’s performance is measured as 0.54 with an overall average of 0.60 Smatch score as given above. One should note that the automatic dependency parsing (Sulubacak and Eryiğit Reference Sulubacak and Eryiğit2018) and semantic role labeling (Şahin and Steedman Reference Şahin and Steedman2018) performances in Turkish are still not on par with English due to the low training data resources.

For the second set of evaluations, we create two experimental setups to measure the effects of the parser in the annotation process. First, we select two sets of 10 sentences from IMST with similar syntactic and semantic structures. The selected sentences are also similar in terms of sentence length and structural complexity. We then record the time spent by a single human annotator who annotates these two sets separately; for one of the sets, the annotation is realized from scratch, and for the other one, it is done via semi-automatic annotation, where the experienced human annotator corrects the outputs of the introduced parser. The elapsed times in both annotation processes are given in Table 6, which reveals a remarkable reduction in annotation times (of around two-thirds) when the parser is used as a pre-processor, and the human annotator corrects its outputs rather than annotating from scratch (manual annotation). One should note that the selected sentences were not very difficult and the time spent for the annotation of a single sentence may not be generalized.

Table 6. Annotation times

In the second experiment, we randomly select 25 additional sentences not annotated before from IMST. Two human annotators annotate these sentences, one working from scratch and one working on the outputs of the tree-to-graph parser. The inter-annotator agreement between the two human annotators is measured as 0.85 Smatch score. We also make an error analysis on the sentences where there is no agreement between our annotators and observe that the annotations produced by the human annotator working on the parser’s outputs better conform to the predicate frame names than the ones produced by the human annotator working from scratch. This is an expected outcome since our parser uses gold-standard predicate frame tags, which should be replaced with an automatic predicate disambiguator in a real scenario, while the human annotator working from scratch try to select them each time manually, which is error prone. On the other hand, we see that the parser directs the annotator to use more conjunctions (as exemplified in the previous section (Figure 12b) the use of “and” instead of the :cause relation) and possessiveness (instead of :topic, :part-of, etc.) in the complex sentences than needed. We observe that the human annotator corrected these most of the time (as may be observed from the Smatch scores between the parser and the human annotator above), but in some sentences with complex semantic structures, these could be missed. The parser also helps extensively to the human annotator in cases morphologically inferable (such as null subject, dropped pronouns, modality), which could be missed by the human annotator working from scratch.