2.1. Learning mechanisms

The experimental results surveyed above point toward the following developmental trajectory. Basic verb argument-structure knowledge appears to develop early, at fifteen to sixteen months for English learners, and emerges before infants identify moved arguments, such as those in wh-questions. What learning mechanisms might allow learners to identify these nonlocal argument dependencies in their input? This is not a trivial task. Movement dependencies are not always marked with consistent morphology: for instance, English wh-phrases take a variety of different forms. The class of wh-elements in any language will distribute in specific ways in the surface forms of sentences: for instance, English wh-words are clause-initial and frequently occur in questions. However, even if a learner can identify a word class with these particular surface distributional properties, it does not necessarily follow that these are wh-elements. Many languages have question particles that can appear at sentence boundaries in both wh- and polar questions. An example is the particle la in Tz’utujil Mayan, as in 2. A Tz’utujil learner needs a way to tell that la is a question particle and not a wh-word, and conversely an English learner needs a way to tell that what is a wh-word and not a question particle.

Moreover, in many languages, wh-phrases do not appear clause-initially. In wh-in-situ languages like Chinese, Japanese, and Korean, wh-phrases are pronounced in their thematic position local to the verb, but still take interrogative scope in a higher clausal position, as in 3. Learners of these languages must identify when an expression is in a nonlocal wh-dependency with a higher clausal node, even when it has not overtly moved to this position.Footnote ²

Thus, in order to identify wh-dependencies in their language, children must solve multiple problems. They need to learn whether their language fronts wh-phrases, and if so, which surface forms signal that this movement has occurred. In a language with wh-fronting, they also must identify the thematic position where the wh-phrase should be interpreted in relation to the verb. As noted above, wh-in-situ poses a different learning problem from wh-fronting: in wh-fronting, a wh-phrase is pronounced in the position where it takes interrogative scope, and learners must identify a nonlocal dependency with its thematic position, whereas in wh-in-situ, the wh-phrase is pronounced in its thematic position, and learners must identify a nonlocal dependency with its scope position. We focus here on the problem posed by wh-fronting, but return to consider wh-in-situ in the general discussion in Section 5.

In English, surface signals for wh-movement include not only wh-words, but also a variety of other reflexes of movement, such as prosodic marking and, in questions where the moved constituent is not a subject, subject-auxiliary inversion and do-support. Mature speakers of a language make efficient use of these signals in sentence processing to identify moved arguments and predict upcoming ‘gaps’ where they should be interpreted (Aoshima et al. Reference Aoshima, Phillips and Weinberg2004, Crain & Fodor Reference Crain, Fodor, Dowty, Karttunen and Zwicky1985, Frazier & Clifton Reference Frazier and Clifton1989, Frazier & Flores d’Arcais Reference Frazier and Flores d’Arcais1989, Sussman & Sedivy Reference Sussman and Sedivy2003, Traxler & Pickering Reference Traxler and Pickering1996). But children must first learn these signals in order to use them in parsing wh-dependencies. In languages like English, identifying the tails of these dependencies is particularly challenging, because the thematic positions of moved elements are phonologically null. How do learners identify a nonadjacent dependency where only one element appears overtly?

One possible piece of the puzzle comes from the literature on ‘expectation violation’ or ‘error-driven learning’ in other areas of cognitive development. A large body of work finds that infants use knowledge about the physical and social properties of objects and agents, alone or in combination with learned statistical contingencies, to make predictions about upcoming events (Denison & Xu Reference Denison, Xu, Kushnir, Benson and Xu2012, Kouider et al. Reference Kouider, Long, Le Stanc, Charron, Fievet, Barbosa and Gelskov2015, Stahl & Feigenson Reference Stahl and Feigenson2015, Reference Stahl and Feigenson2017, Téglás et al. Reference Téglás, Vul, Girotto, Gonzalez, Tenenbaum and Bonatti2011). Violations of these predictions may provide valuable opportunities for learning (Stahl & Feigenson Reference Stahl and Feigenson2015, Reference Stahl and Feigenson2017). For instance, an experiment in Stahl & Feigenson Reference Stahl and Feigenson2015 presented eleven-month-olds with events that either conformed with or violated object solidity. In one such event, a ball rolled down a ramp toward a solid wall, stopping behind an occluder. When the occluder was lifted, one group of infants saw that the ball had been stopped by the wall, while a second group of infants saw that the ball had apparently passed through the wall, violating their predictions about object solidity. After this event, both groups of infants were tested on their ability to map a novel property (e.g. squeaking) to the previously observed toy. Infants who had observed the prediction-violating event showed significantly greater learning than infants who had not. In a further experiment, infants who viewed these events were then given a choice to explore the ball or a novel object. Infants who had viewed the prediction-violating event chose to explore the ball more than infants who had not. Moreover, their exploration was consistent with testing the object’s solidity properties: they banged the ball against the table to a greater extent than infants who had seen a different event type. These results suggest that even very young learners are sensitive to inconsistency between their own predictions and observed events, and when they observe a situation where their predictions are violated, they exploit this opportunity to learn, explore, and test hypotheses about the potential cause of that violation.

We pursue the hypothesis that a similar form of expectation violation may underlie infants’ discovery of argument movement dependencies in languages like English. Here, it is not predictions about physical events that drive learning, but rather predictions about grammatical structure. On this hypothesis, verb argument-structure knowledge developmentally precedes argument movement acquisition because the former provides the basis for generating structural predictions—specifically, predictions about upcoming arguments of verbs. When infants encounter a case where an expected argument does not appear in its local position, they exploit this expectation violation to learn about the cause of the locally missing argument, scaffolding their identification of movement dependencies (Gagliardi et al. Reference Gagliardi, Mease and Lidz2016, Perkins Reference Perkins2019, Perkins & Lidz Reference Perkins and Lidz2020, Stromswold Reference Stromswold1995). For example, learners who know that a verb like fix requires a direct object might register that it is unexpectedly missing after the verb in a question like What did David fix?. This unsatisfied structural prediction may provide the basis of inferring the tail of a nonlocal argument dependency—a ‘gap’ of argument movement—even though it is silent. And it may compel learners to search the rest of the sentence for the cause of the missing argument, eventually identifying that another expression in the sentence (what) is satisfying the verb’s transitivity requirement nonlocally. This would allow them both to assign an appropriate parse to the sentence and to begin to learn how various types of nonlocal dependencies are realized: that is, that this question contains a wh-dependency, which is marked in English by various surface signals, such as what, do-support, and subject-auxiliary inversion.

In sum, we propose that the process of acquiring nonlocal dependencies follows three logically independent steps, which we together call Gap-Driven Learning (Perkins Reference Perkins2019, Perkins & Lidz Reference Perkins and Lidz2020):

1. (i) using knowledge of verb argument structure to detect argument gaps: predicted arguments that are unexpectedly missing in their local positions;

2. (ii) identifying what surface forms are correlated with these argument gaps; and

3. (iii) inferring what types of syntactic dependencies are responsible for those correlations.

Here, we investigate the gap-driven learning hypothesis specifically in the domain of direct object gaps. This decision is motivated by empirical evidence for early knowledge of verb transitivity (Jin & Fisher Reference Jin and Fisher2014, Lidz et al. Reference Lidz, White and Baier2017), making it plausible that direct object gaps are the type of argument gap that learners may be able to detect readily at the relevant stage of development. But how this knowledge is in place by this age raises its own learning problem, which must be addressed in order for gap-driven learning to be possible. Before children can identify when arguments have been moved, they cannot identify all instances of direct objects in sentences containing transitive verbs. How, then, do they arrive at the appropriate expectations that some verbs obligatorily require objects, such that they will be surprised when those objects are missing? Perkins et al. (Reference Perkins, Feldman and Lidz2022) investigate this question computationally and show that it is feasible for children to find their way around this learning problem. The learner in Perkins et al. Reference Perkins, Feldman and Lidz2022 assumes that it occasionally represents sentences erroneously and learns what portion of its input representations to treat as signal versus noise for the purpose of learning verb transitivity. When tested on the distributions of direct objects that a child at this age could identify in child-directed English, the model learned how to filter its data to correctly assign transitivity properties to the majority of the most frequent verbs in its input. This tells us that it is in principle possible for children to identify verb transitivity without accurately parsing argument movement, thereby providing a way for gap-driven learning to get started.

In this article, we present a computational model that instantiates the first two steps of learning under the gap-driven learning hypothesis. The learner builds off of the model in Perkins et al. Reference Perkins, Feldman and Lidz2022, using the approximate verb transitivity knowledge identified by that learner. Our model tracks statistical regularities in the surface morphosyntactic features of sentences in order to identify clusters of sentences that share distributional properties. At the same time, it tracks when its expectations of upcoming direct objects are violated, in order to infer which clusters of properties are correlated with potential direct object gaps. When tested on child-directed speech, we find that the model identifies the large majority of sentences with object movement. Furthermore, we show that prior knowledge of verb transitivity, even if rough and approximate, is important for this distributional learning process to be successful. The learner performs better if it uses transitivity knowledge to infer likely object gaps, rather than clustering sentences on the basis of their overt surface features alone. These findings demonstrate that a learner could in principle identify object movement dependencies in English by using unsatisfied structural predictions to guide distributional learning. As verb transitivity knowledge forms the basis for generating these structural predictions, this provides an account for the empirically attested order of argument structure and argument movement acquisition in early development.

3.1. Generative model

The data that our learner observes consists of the morphosyntactic features of sentences containing transitive, intransitive, or alternating verbs. The learner builds off of a first step of learning modeled in Perkins et al. Reference Perkins, Feldman and Lidz2022, which shows how some initial knowledge of verb transitivity properties might be acquired before a child can identify moved objects. That learner assumed that there are three transitivity categories to be identified—transitive verbs that require direct objects, intransitive verbs that disallow them, and alternating verbs that optionally allow them—and assigned verbs in its input to these three categories based on their distributions with direct objects in canonical postverbal positions, which English-learning infants can identify prior to eighteen months (Gagliardi et al. Reference Gagliardi, Mease and Lidz2016, Hirsh-Pasek & Golinkoff Reference Hirsh-Pasek, Golinkoff, McDaniel, McKee and Cairns1996, Jin & Fisher Reference Jin and Fisher2014, Lidz et al. Reference Lidz, White and Baier2017, Perkins & Lidz Reference Perkins and Lidz2020, Seidl et al. Reference Seidl, Hollich and Jusczyk2003). These initial transitivity assignments are imperfect, modeling the realistic assumption that a child’s knowledge of verb transitivity is likely to be approximate at this stage of development.

The current learner now assumes that there are two reasons why it might observe canonical direct objects or no direct objects after the verbs in the sentences that it observes. On the one hand, the transitivity of that verb determines whether it should always, never, or sometimes occur with a direct object. On the other hand, there may be a separate grammatical process, such as argument movement, that results in an apparent transitivity violation. The learner assumes that these transitivity violations are governed by latent ‘categories’ of sentences with shared grammatical properties. Each category has a particular parameter governing whether it produces object gaps: if it does, then observations of canonical direct objects in that category may no longer reflect the transitivity properties of these verbs, but may instead be due to other grammatical properties that produce ‘nonbasic’ word orders. These properties also give rise to the distributions of other morphosyntactic features of sentences in a particular category.

For instance, the learner might identify that a sentence like What did David fix? belongs to a category of other sentences that have object gaps and also tend to be questions with subject-auxiliary inversion, a form of do, and an unknown functional element sentence-initially (e.g. what). On the other hand, the learner might identify that a sentence like Your toy got broken belongs to another category of sentences that also have object gaps, but different morphosyntactic features: here, a form of get and the verbal suffix -en. The distributional features of the first sentence category are the footprints of object wh-questions in English; the features of the second category are the footprints of get-passives.

The learner does not know ahead of time how many sentence categories there will be or what the properties of those categories are. Using the distributions of direct objects and the other observed sentence features in its data, the learner infers what categories of sentences are present, what their distributional properties are, and which categories produce object gaps. This allows the learner to identify specific clusters of morphosyntactic features that are correlated with object gaps in different clause types, which may be candidates for entering into nonlocal movement dependencies.

More formally, we provide the graphical model for the learner in Figure 1. A graphical model provides a visual representation of the process by which the learner assumes its data are generated. Circular nodes represent random variables, and arrows represent conditioning relationships between variables. Shaded nodes represent variables with observed/known values; unshaded nodes represent variables whose values are unknown and must be inferred. Rectangular ‘plates’ indicate when a portion of the model is repeated over a particular range, denoted by the superscript in the right corner. See Griffiths et al. Reference Griffiths, Chater and Tenenbaum2024 for more information.

Figure 1.

Graphical model for Joint Inference Learner. Nodes correspond to random variables: the observed direct objects X and other features F in each sentence, the transitivity category T and rate of direct objects θ for each verb, the latent ‘category’ c of each sentence, the rate of direct objects δ ^(X) and other sentence features δ ^(F) produced by each category, and whether each category produces a transitivity violation e. Arrows denote conditioning relationships between variables.

Observations of direct objects are formalized as the Bernoulli random variable X. This variable encodes direct object data for each of the m sentences containing each of the V verbs in the model’s input, with a value of 1 if the sentence contains a direct object following the verb, and 0 if it does not. The model’s observations of the other n relevant morphosyntactic features of the sentence are represented by the vector of Bernoulli random variables $ \overrightarrow{F} $ . Specific details of this feature set are discussed in the next section.

The direct object observations X ^(v) for a given verb v can be generated by two processes: the transitivity of verb v, represented by the variables T and θ in the upper half of the model, or the other grammatical properties of the category that the sentence belongs to, represented by the variables c, e, and δ ^(X) in the lower half of the model. We describe each of these generative processes in turn.

In the upper part of the model, each observation X ^(v) of a direct object for a particular verb is conditioned on the parameter θ ^(v), a continuous random variable that controls the probability that verb v will be used with a direct object. θ ^(v) is conditioned on the variable T ^(v), a discrete random variable that can take on three values corresponding to transitive, intransitive, or alternating verbs. In order to model the hypothesis that learners are using prior knowledge of verb transitivity properties, we assume that the learner has approximate knowledge of these values of T for the set of verbs in the learner’s data, as acquired by the model in Perkins et al. Reference Perkins, Feldman and Lidz2022. This means that the learner knows some of the values of θ as well. If verb v is fully transitive, then the learner assumes that θ ^(v) = 1: the verb should always occur with a direct object. If the verb is fully intransitive, then θ ^(v) = 0: the verb should never occur with a direct object. If the verb belongs to the alternating category of T, then θ ^(v) takes an unknown value between 0 and 1 inclusive. The prior probability over θ in this case is a Beta(α,β) distribution, where the parameters α and β are counts of direct objects and no direct objects for verb v in sentence categories without transitivity violations, excluding the current category.

In the lower part of the model, each X ^(v) is conditioned on the discrete random variable c, defined for all positive integers, which represents the category that the sentence belongs to. These sentence categories also condition the other morphosyntactic features in the sentence, encoded in the vector $ \overrightarrow{F} $ . Each category c is assumed to reflect a particular set of underlying grammatical properties that give rise to the distributions of direct objects and other features of a sentence. The number and properties of these categories are a priori unknown, and the learner infers the properties that will allow it to explain the distributions of features and direct objects that it observes. Returning to our earlier examples, the learner might infer a value of c that encodes English wh-object questions, giving high probability to sentence-initial function words (i.e. wh-words), subject-auxiliary inversion, forms of do, and direct object gaps. Another inferred value of c might encode English get-passives, giving high probability to direct object gaps, forms of get, and the -en verbal suffix. The prior probability over c is a Dirichlet process (Ferguson Reference Ferguson1973), which gives a particular category prior probability proportional to the number of sentence observations already assigned to that category. This process also reserves a small nonzero probability for new categories, allowing the model to flexibly converge on the number of sentence categories that best explains the distributions in its data. By allowing the model to explore a potentially unbounded number of categories, this prior builds in the fewest possible assumptions about the number of categories required to explain the distributions in a given language or data set; however, this form of prior also biases the model to reuse categories whenever possible, and thus to keep the total number of categories small.Footnote ³ See Appendix A for details.

The random variables e, δ ^(X), and δ ^(F) represent the parameters of each of the sentence categories. The Bernoulli random variable e_c encodes whether a given category c produces transitivity violations. If e_c = 0, then the category does not produce transitivity violations, and all observations of a direct object in X ^(v) were generated by the transitivity properties of verb v. But if e_c = 1, then the category does produce transitivity violations, and the observations of direct objects X ^(v) were generated by a particular grammatical property of category c. The learner in Perkins et al. Reference Perkins, Feldman and Lidz2022 inferred that transitivity violations occurred approximately 19% of the time in sentences containing this same set of verbs in child-directed speech. In order to model the hypothesis that the current learner builds off of the knowledge gained in that previous stage of learning, our learner assumes that 19% is the prior probability that e_c = 1.Footnote ⁴

The random variable $ {\delta}_c^{(X)} $ represents the probability of observing a direct object in a category with transitivity violations—that is, whether the particular violation in that category produces object gaps, or whether it adds an apparent extra object that is not licensed by the verb. Intuitively, we can think of the probability that a sentence contains a direct object as depending on one of two biased coins. If e_c = 0 and the observation was generated by the verb’s transitivity properties, then one biased coin is flipped and the sentence contains a direct object with probability θ ^(v). But if e_c = 1 and the observation was generated by the grammatical properties of category c, then a different biased coin is flipped and the sentence contains a direct object with probability $ {\delta}_c^{(X)} $ . The parameter $ {\delta}_c^{(X)} $ is assumed to have a uniform Beta(1,1) prior distribution. This uniform prior means that it is equally likely a priori for a sentence category to create object gaps as it is to add extra objects. This form of prior builds in the fewest possible assumptions about the probability of observing a direct object versus an object gap within a sentence category. Analogous to $ {\delta}_c^{(X)} $ , the random variables in $ {\overrightarrow{\delta}}_c^{(F)} $ represent the probabilities of observing the other morphosyntactic features in a given sentence category. Each $ {\delta}_c^{(F)} $ is also assumed to have a uniform Beta(1,1) prior distribution, meaning that all features are equally likely a priori to be present as they are to be absent; this likewise builds in the fewest possible assumptions about the distributions of features within sentence categories.

3.2. Inference

The learner uses component-wise Gibbs sampling (Geman & Geman Reference Geman and Geman1984) to jointly infer the category of each observed sentence (c) and whether each category contains transitivity violations ( $ e $ ). We first initialize values of c and e for each sentence. Then, for each sentence, we calculate a posterior probability distribution over new category assignments given the observed data in X and F, the known verb transitivity properties T, and the other sentence category assignments and properties. We resample new values of c for each sentence sequentially from this posterior probability distribution. Finally, we use the new category values to resample values of e for each category from its posterior probability distribution, given the other model parameters. This cycle is repeated over many iterations until the model converges to a stable distribution over c and e. Details of the initialization and sampling procedure are provided in Appendix A.

4.1. Data

We prepared a data set from four parsed corpora in the CHILDES Treebank (Pearl & Sprouse Reference Pearl and Sprouse2013), which contains parse trees for child-directed English corpora on CHILDES (MacWhinney Reference MacWhinney2000). Details of these corpora are provided in Table 1. From these corpora, we selected sentences containing the verbs whose transitivity properties are known by our learner. Because a child’s knowledge of verb transitivity is likely to be imperfect before eighteen months of age, we base our learner’s knowledge on the transitivity classes inferred by the learner in Perkins et al. Reference Perkins, Feldman and Lidz2022, which provides a model of the previous stage of learning that our current model builds off of. We selected 18,503 sentences containing the verbs whose transitivity properties were inferred by the previous learner: these are the fifty most frequent transitive, intransitive, and alternating action verbs in these corpora. Because the previous learner assigned only 66% of these verbs to the correct transitivity category as specified in Perkins et al. Reference Perkins, Feldman and Lidz2022, this provides a noisy and imperfect source of knowledge for the current learner.Footnote ⁵ Table 2 provides the frequencies of these verbs, along with the transitivity categories assumed by our model.

Table 1.

Corpora of child-directed speech.

Table 2.

Known verbs and transitivity categories assumed by learner

We conducted an automated search over the Treebank trees for overt direct objects following each verb, as well as the morphosyntactic features of each sentence that our model observes. We assume that the learner’s inference is driven by information relevant to the predicate-argument structure of a sentence: morphosyntactic features pertaining to subjects, objects, and verbs. These features are listed in Table 3.

Table 3.

Direct objects and morphosyntactic features observed by learner (X and F). The presence of a direct object is the sole feature encoded by X. The remaining twenty-one features are encoded within the feature vector $ \overrightarrow{F} $ .

In selecting these features, we model a learner with the representational abilities of an infant between the ages of fifteen and eighteen months. Prior behavioral evidence finds that infants at these ages can use the word-order properties of their language to identify clause subjects and objects in their canonical positions (Gagliardi et al. Reference Gagliardi, Mease and Lidz2016, Hirsh-Pasek & Golinkoff Reference Hirsh-Pasek, Golinkoff, McDaniel, McKee and Cairns1996, Jin & Fisher Reference Jin and Fisher2014, Lidz et al. Reference Lidz, White and Baier2017, Perkins & Lidz Reference Perkins and Lidz2020, Seidl et al. Reference Seidl, Hollich and Jusczyk2003). They attend to auxiliaries and can detect when the order of a subject and auxiliary is inverted (Geffen & Mintz Reference Geffen and Mintz2015). They are able to segment a variety of verbal suffixes in English and other languages (Figueroa & Gerken Reference Figueroa and Gerken2019, Höhle et al. Reference Höhle, Schmitz, Santelmann and Weissenborn2006, Kim & Sundara Reference Kim and Sundara2021, Mintz Reference Mintz2013, Nazzi et al. Reference Nazzi, Barrière, Goyet, Kresh and Legendre2011, Santelmann & Jusczyk Reference Santelmann and Jusczyk1998, Soderstrom et al. Reference Soderstrom, Wexler and Jusczyk2002, Soderstrom et al. Reference Soderstrom, White, Conwell and Morgan2007, van Heugten & Shi Reference van Heugten and Shi2010). In addition to auxiliaries and verbal affixes, infants at these ages are sensitive to the syntactic properties of a handful of other functional categories: determiners (Cauvet et al. Reference Cauvet, Limissuri, Millotte, Skoruppa, Cabrol and Christophe2014, Hicks et al. Reference Hicks, Maye and Lidz2007, Höhle et al. Reference Höhle, Weissenborn, Kiefer, Schulz and Schmitz2004, Shi & Melançon Reference Shi and Melançon2010), pronouns (Cauvet et al. Reference Cauvet, Limissuri, Millotte, Skoruppa, Cabrol and Christophe2014), prepositions (Lidz et al. Reference Lidz, White and Baier2017), and negators (de Carvalho et al. Reference de Carvalho, Crimon, Barrault, Trueswell and Christophe2021). Although they may not know the categories of other functional elements, they are able to recognize them as functional as opposed to lexical on the basis of their phonetic and prosodic properties (Monaghan et al. Reference Monaghan, Chater and Christiansen2005, Shi et al. Reference Shi, Morgan and Allopenna1998, Shi et al. Reference Shi, Werker and Morgan1999).

In coding for the features in Table 3, we model an infant who can identify objects locally after verbs, but cannot yet identify nonlocal objects, such as fronted wh-phrases in wh-questions (Perkins & Lidz Reference Perkins and Lidz2021). This means that sentences like You’re eating and What are you eating? were both coded as not having a direct object from our learner’s perspective, even though the wh-word what acts as a nonlocal object in the second sentence of this pair. Instead, wh-words are coded as ‘unknown function words’, a hypercategory that includes all functional elements assumed to be unknown at this age: wh-words, complementizers, quantifiers, focus particles, and conjunctions other than and.

We also code for the pragmatic feature ‘question’, which represents whether an utterance has interrogative force. Empirical evidence suggests that infants in their second year of life understand when a speaker is seeking information (Casillas & Frank Reference Casillas and Frank2017, Goodhue et al. Reference Goodhue, Hacquard and Lidz2023, Luchkina et al. Reference Luchkina, Sobel and Morgan2018); see Carruthers Reference Carruthers2018 on ‘questioning attitudes’ as a basic component of human minds. They do so likely on the basis of distributional, prosodic, and sociopragmatic cues (such as pauses and eye gaze) that differentiate questions from assertions in child-directed speech (Yang Reference Yang2022). Young infants are sensitive to the prosodic and distributional differences between declaratives and polar questions (Frota et al. Reference Frota, Butler and Vigário2014, Geffen & Mintz Reference Geffen and Mintz2015, Soderstrom et al. Reference Soderstrom, Ko and Nevzorova2011). Although wh-questions differ from polar questions in their prosody (Geffen & Mintz Reference Geffen and Mintz2017), it is possible that infants may know that these sentences are interrogatives, even before they are aware that they contain wh-dependencies (Gagliardi et al. Reference Gagliardi, Mease and Lidz2016, Perkins & Lidz Reference Perkins and Lidz2020, Seidl et al. Reference Seidl, Hollich and Jusczyk2003). Questions were identified by the presence of a question mark in the transcription; this does not distinguish constituent questions from polar questions.

In coding for the feature ‘question’, we abstract away from the specific prosodic features that learners might rely on to distinguish interrogatives from declaratives, and wh-questions from polar interrogatives (Frota et al. Reference Frota, Butler and Vigário2014, Geffen & Mintz Reference Geffen and Mintz2015, Gryllia et al. Reference Gryllia, Doetjes, Yang and Cheng2020, Soderstrom et al. Reference Soderstrom, Ko and Nevzorova2011, Yang Reference Yang2022), which were not available in the corpora of child-directed speech used for our model’s data set. In abstracting away from the prosodic signal, we ask how far a learner might get on the basis of distributional morphosyntactic information. However, we do not intend this as a claim that children cannot or do not additionally attend to this richer prosodic information, and further work might extend the current model to operate over a prosodically enriched data set.

To verify the accuracy of our automated coding, a random sample of 500 sentences from the data set were separately hand-coded by two trained researchers. Percentage agreement between the hand-coding and automated coding ranged from 87–100% across the twenty-one features; interrater reliability was also 87–100%. See Appendix B for more detail.

The sentences in the data set were also coded for their underlying clause types,Footnote ⁶ listed in Table 4. These annotations were used as a gold standard to evaluate our model and were not part of the model’s data set. These clause types included three with movement: wh-questions, passives, and relative clauses. A given clause might be coded as multiple types: for example, as both a question and a passive. For sentences with multiple clauses, coding was conducted for the clause containing the verb of interest. Accuracy of clause-type coding was again evaluated by comparing against a 500-sentence sample hand-coded by two researchers. Percentage agreement between the hand-coding and automated coding ranged from 84–99% across the nine clause types (interrater reliability 87–99%); see Appendix B. Additional hand-coding was conducted for wh-questions and relative clauses in order to annotate the gap site in these sentences, which could not be reliably identified automatically for the entire data set.

Table 4.

Distribution of underlying clause types in data set.

4.2. Results

4.2.1. Sentence category distributions

Our joint inference model inferred thirty-nine total sentence categories, sixteen with transitivity violations and twenty-three without. To determine which of the model’s inferred transitivity-violating categories were ones that contained object gaps (versus other types of transitivity violations), we calculated the odds ratio of direct objects appearing in these categories. This measure divides the odds of observing a feature in a given category by the odds of observing that feature outside of that category; an odds ratio significantly greater than 1 indicates that a feature is more likely to be present within than outside of the category, and an odds ratio significantly less than 1 indicates that a feature is more likely to be absent. Significance was calculated using a Fisher’s exact test with a Bonferroni correction for multiple comparisons. See Appendix C for full details.

Of the sixteen transitivity-violating categories, fifteen had significantly lower odds (odds ratio less than 1) of producing direct objects; we call these ‘object gap’ categories. For each of the model’s categories, Figure 2 displays the proportion of the category made up of each underlying clause type. Note that these proportions do not necessarily sum to 1 because a single clause might be of multiple types. For example, the sentences in the model’s category 1 are entirely (1.00) wh-questions; this means that a given sentence in category 1 has a 100% probability of being tagged with the wh-question type in the gold-standard annotation. However, in category 2, a given sentence has a 95% probability of being a wh-question and also a 99% probability of being an embedded clause: this is a category that is predominantly long wh-questions, that is, those with wh-dependencies into embedded clauses.

Figure 2.

Proportions of clause types in inferred sentence categories, joint inference model.

In order to see whether the sentences in a given category predominantly belong to a particular clause type, versus being spread out among many different clause types, we calculated the purity of these categories when compared to the true underlying clause types in the corpora. Purity was calculated by counting the total number of sentences that belong to the predominant clause type in each category and dividing by the total number of sentences in the data set (Manning et al. Reference Manning, Raghavan and Schütze2008). Because a given sentence could belong to more than one clause type (i.e. both a wh-question and an embedded clause), we counted it as belonging to the predominant type in the category if that type was among those that the sentence belongs to. We note that this is a coarse approach and intend it only as a descriptive measure; our goal is not to evaluate the model on its clustering, but rather to evaluate it on whether it is able to find movement in its data, which we report in the following section. Given this approach, this measure has a minimum value of 0 if clusters are made up of a mixture of clause types, and a maximum value of 1 if clusters are made up of a single clause type. Our model’s overall cluster purity is 0.76, which tells us that the model’s categories were more likely to track one underlying clause type rather than a mixture.

The model inferred many more categories than necessary to identify the set of underlying clause types that it is being evaluated against. This is unsurprising: the learner was not given any information about how many clause-type categories were present, nor the grain size at which to perform its categorization. Instead, it was given leeway to posit as many categories as needed to explain the distributions of features and transitivity violations in its data. The model divided wh-questions among seven different categories: five transitivity-violating categories and two with no transitivity violations. These categories differentiate monoclausal from biclausal questions (e.g. What does he eat? vs. What would you like to read?), questions in the progressive aspect (e.g. What are you bringing?) from those in other aspects, and questions where the wh-word is sentence-initial from those where it is not (e.g. And what is he wearing?). The model also categorized subject questions separately from object and adjunct questions, and correctly identified subject questions as non-transitivity-violating. These distinctions may have implications for the learner’s ability to generalize about the surface forms that are distinctive of different types of movement dependencies, a point we return to in the following sections.

4.2.2. Accuracy on identifying object movement

Here, we ask how well our learner can identify instances of object movement in its data. Visually, we can see from Figure 2 that clause types with movement were more likely to be categorized in object-gap categories than in non-object-gap categories. To ask how well the model identified cases of object movement specifically, we compared its object-gap categories against the sentences that were coded as actually having object gaps in the corpus. The model’s accuracy is displayed in Figure 3 using three metrics. Precision measures the proportion of sentences in the model’s object-gap categories that contained object movement according to our gold standard—that is, the proportion of these categories made up of object wh-questions, object relative clauses, or passives. Recall measures the proportion of sentences with object movement in the corpus overall that were identified as belonging to one of the model’s object-gap categories. These metrics are not always aligned: it would be possible to achieve perfect recall by identifying all sentences as having object movement, but this would result in very poor precision. The F1 score, the harmonic mean of precision and recall, reflects the model’s overall accuracy by taking into account both of these metrics. For each of these metrics, we compare the model’s performance to a chance baseline, indicated by the dashed horizontal line. This represents the expected performance of a learner that randomly categorizes sentences as having transitivity violations that cause direct object gaps, by flipping a coin with weight 0.19, which is the probability of transitivity violations encoded in our learner’s prior.

Figure 3.

Accuracy on identifying sentences with object movement in three metrics: precision (proportion of model’s object-gap categories that contain object movement), recall (proportion of object movement in corpus identified by model), and F1 (harmonic mean of precision and recall).

The model achieved an F1 score of 0.50. Its recall was 0.80, indicating that it identified 80% of sentences with object movement in its data. This accuracy rate is substantially above chance performance. Its precision was 0.37, indicating that on average, 37% of the sentences within its object-gap categories had instances of object movement. This precision rate is also above chance, but shows us that the model did not always manage to isolate object movement from other clause types in its data. To examine this further, we plotted the distribution of movement and nonmovement types in the model’s object-gap categories in Figure 4. Object movement was the predominant clause type in 60% of these categories, but occurred alongside other movement types as well, particularly adjunct movement. The model appears to categorize adjunct movement together with object movement based on some surface distributional similarities: unlike subject movement, both object and adjunct movement contain subject-auxiliary inversion and can trigger do-support, even though adjunct movement does not tend to produce transitivity violations. The other 40% of the model’s object-gap categories predominantly comprised sentences without movement. Thus, while the learner achieved high accuracy in identifying sentences with object movement as such, in certain cases it categorized sentences with object movement together with other clause types.

Figure 4.

Distribution of movement types in model’s object-gap categories.

The model achieves this performance despite several factors that limit its accuracy. First, the model does not receive credit for identifying cases of movement other than wh-questions, passives, and relative clauses; other rarer cases of movement were more difficult to code automatically, and thus were not annotated in the gold-standard labels.Footnote ⁷ Second, the model infers object movement only from sentences that it believes violate verb transitivity: sentences with missing direct objects for verbs that it considers fully transitive. This means that the current evaluation measures how well the model was able to generalize from fully transitive verbs to verbs that also allow intransitive uses. Table 5 displays the proportions of sentences with object movement that the model correctly identified as having object gaps, broken down by the verb classes that comprised the model’s prior transitivity knowledge. The model achieved high recall even though the majority of sentences with object movement occurred with verbs that it believed to be alternating, rather than obligatorily transitive. Of the 1,369 sentences coded as having object movement in the corpus, only 299 contained known transitive verbs, compared to 1,055 containing known alternating verbs.Footnote ⁸ Nonetheless, the model achieved high accuracy across both the transitive and alternating verb classes. This tells us that it was able to generalize effectively: it used the presence of object gaps with known transitive verbs to identify the forms that object movement takes in its data, even with verbs that do not obligatorily require objects.Footnote ⁹

Table 5.

Proportion of object-movement sentences identified, by verb type.

In summary, our joint inference model performed significantly higher than chance in categorizing sentences with object movement in its data. It achieved a high recall rate, indicating that it was correctly able to identify the large majority of sentences with object movement that it encountered. Its accuracy was high for both transitive and alternating verbs, indicating that it was able to use the presence of transitivity violations with fully transitive verbs to identify direct object gaps with verbs that do not require objects. However, this object-gap inference produced a mixture of signal and noise: the sentences that the model categorized together with object movement also contained a variety of other movement and nonmovement clause types. This has potential implications for how informative the learner’s categories are for isolating object movement from other syntactic dependencies, a question we turn to next.

4.2.3. Identifying distinctive features of object movement

Under our hypothesis, the sentence categories inferred by the joint inference model are an intermediate step of learning. Jointly inferring how to categorize sentences according to their surface features, and which sentence categories contain object gaps, helps a learner identify the particular forms that characterize different types of object movement in the target language. Here, we ask how well the model identified which specific surface features are the footprints of object movement. To do this, we assessed which surface features are most distinctive in the categories that the model inferred to have object gaps. If these include the characteristic forms of English object movement dependencies, then the model’s sentence categories contain helpful information for identifying the ways that object movement can be realized in English.

To assess feature distinctiveness, we again calculated the odds ratio of each surface feature in the model’s argument-gap categories. Table 6 reports the features with odds ratios significantly greater than 1 for each of the model’s object-gap categories; full details are provided in Appendix C. Among these features are the characteristic forms of object movement dependencies in English. The categories that are predominantly wh-questions have greater odds of including subject-auxiliary inversion, do, and unknown function words sentence-initially or sentence-medially before the verb: these are wh-words. The categories predominantly made of passives have greater odds of including get or be, and -en, -ed, or irregular verbal morphology.

Table 6.

Features with significantly higher odds in object-gap categories.

However, the distinctive features of object-gap categories also include forms that are irrelevant to movement dependencies. These include many positional characteristics of subjects and verbs, but also some specific morphemes. For instance, be and -ing are distinctive of two of the model’s wh-question categories, and have is distinctive of several of the model’s passive categories. These features mark the realization of aspectual dependencies: be and -ing mark the progressive aspect, and have together with -ed or -en marks the perfect aspect. Thus, the model’s categories contain both signal and noise for learning which surface features are the footprints of movement rather than other syntactic dependencies.

In summary, the current learner successfully identified the forms that characterize the most frequent types of movement in English, but it also identified some irrelevant features that are accidentally correlated with these forms. This invites the question of how a learner could effectively use this information for further steps of learning—how a learner could separate signal from noise by explaining some correlations as movement and others as different dependencies. It is possible that the model’s ability to posit a potentially unbounded number of categories pushed it toward categories that are overly specific. Future work might test hypotheses about limits on the number of categories that a learner can posit, relaxing the ideal-learner assumption of this model in favor of one that more closely reflects the cognitive constraints that a child is operating within. It is unknown, however, whether this would lead the learner away from the accidental correlations that it identifies when its number of categories is unconstrained. Alternatively, it is also possible that a more sophisticated distributional learning mechanism might perform better. Further investigation is needed to determine whether the signal-to-noise ratio in the model’s categories improves if it infers argument gaps using not only missing direct objects, but also other required but missing arguments (subjects and prepositional objects). This would give the learner the opportunity to identify nonobject movement; it is an open question whether this could make its inference about categories with argument gaps more precise.

4.3. Model comparisons

Our model achieves above-chance performance on identifying sentences with object movement by jointly inferring two properties: how sentences should be categorized together according to their surface feature distributions, and which sentence categories violate expectations about verb transitivity. To evaluate how important this joint inference is, we compare our model to baseline learners that perform only one step of inference at a time.

4.3.1. No-Category baseline

If it did not matter that our learner categorized sentences according to their surface features, then a learner should do just as well at identifying object movement on a sentence-by-sentence basis, by noting when objects are unexpectedly missing for known transitive verbs. To test whether the model’s categorization process matters, we compared our model against a baseline learner that used only the presence or absence of direct objects in individual sentences, together with its knowledge of the transitivity properties of verbs in these sentences, to infer which sentences likely contain object gaps. Like our learner, this baseline model infers that an object gap is present when a transitive verb is unexpectedly missing its object. Unlike our learner, this model does not cluster sentences into categories according to their surface features, so it cannot draw inferences about which features are likely to be distinctive of sentences with object movement and cannot generalize the likely presence of an object gap from one sentence to another based on the similarity of their features.

This baseline learner has the architecture of the filtering model in Perkins et al. Reference Perkins, Feldman and Lidz2022, shown in Figure 5a. This is similar to the generative model in Figure 1, but omits the variables c, F, and δ ^(F). With the variable c omitted, the learner does not assume that its direct object observations are partially governed by latent categories of sentences; with F and δ ^(F) omitted, the learner does not observe or draw any inferences about the distributions of other surface features of sentences. Here, the variables e and δ ^(X) are not indexed by sentence category: e represents whether an individual sentence (rather than a sentence category) contains a transitivity violation, and δ ^(X) represents the rate of direct objects in individual sentences (rather than sentence categories) where transitivity violations are present. The learner’s inference procedure consists of learning which sentences contain transitivity violations given its assumptions about verb transitivity and the rate of violations, but no joint learning about sentence categories on the basis of their distributional features. We fixed the transitivity properties T of each verb and the parameter δ ^(X) to the values inferred by the learner in Perkins et al. Reference Perkins, Feldman and Lidz2022. We then sampled transitivity violations for each sentence in the corpus from the posterior probability distribution over e given X, T, and δ ^(X), integrating over θ. See Appendix A for details.

Figure 5.

Graphical models for (a) no-category baseline and (b) no-transitivity baseline.

To determine how well this ‘no-category baseline’ identified movement, we compared the sentences without direct objects that it inferred to have transitivity violations against the actual cases of object movement in the corpus. Its precision, recall, and F1 score are reported in Figure 3 above. The model achieved above-chance accuracy overall, but scored substantially lower than the joint inference model on all three metrics. This is because the baseline model’s only source of reliable information for object gaps comes from the small percentage of verbs that it believes to be obligatorily transitive; it uses no other features in the sentences to inform this inference. If we examine its identification of object movement across verb classes, we find that it achieved high accuracy (74%) on identifying object movement with fully transitive verbs. But for the much larger percentage of verbs that are alternating, it can only guess which sentences contain gaps, identifying only 34% of object movement with these verbs. Thus, our joint inference model’s ability to categorize sentences using a wide range of surface morphosyntactic features, and to generalize across sentences in a category, results in substantially better performance than inferring movement on a sentence-by-sentence basis from transitivity violations alone.

4.3.2. No-Transitivity baseline

Our second baseline comparison investigates how much prior verb transitivity knowledge constrains the learner’s identification of movement—specifically, how important it is that our learner uses transitivity violations in the process of categorizing sentences by their surface morphosyntactic features. We compare our model against a learner that performs this categorization without knowing which verbs require direct objects. Like our learner, this baseline model uses the surface features of sentences to cluster sentences into categories. Unlike our learner, this baseline model does not have any knowledge of which verbs are transitive, so it cannot track transitivity violations in order to infer that object gaps are present in some of its sentence categories. Instead, it treats direct object observations identically to other surface features: for this learner, all direct objects are governed by the grammatical properties of a sentence category, not by the transitivity classes of verbs in the sentences. This learner therefore runs the risk of inferring categories that mix together sentences with movement and sentences without.

The architecture of this ‘no-transitivity baseline’ is shown in Figure 5b. This assumes the lowest portion of the generative model in Figure 1, omitting the variables T, θ, and e. When the variables T and θ are omitted, the learner now assumes that all direct object observations X are generated by $ {\delta}_c^{(X)} $ , the grammatical properties of each sentence category, rather than by any properties of the verbs in these sentences. When the variable e is omitted, the learner no longer assumes that certain sentence categories contain transitivity violations. This means that its inference procedure consists of learning which sentence categories are present and which sentences belong to those categories, but no joint learning about transitivity violations in these categories. We sample category values for each sentence in the corpus from the posterior probability distribution over c given X and F, integrating over δ ^(X) and δ ^(F). See Appendix A for details.

Like our learner, the no-transitivity baseline inferred thirty-nine total categories. Of these, twenty-two had significantly lower odds of producing direct objects; we call these ‘object-gap’ categories, under the assumption that these are the learner’s candidate categories for object movement. Full details are provided in Appendix C. The proportions of underlying clause types in the learner’s categories are reported in Figure 6. These categories have similarly high purity to those inferred by the joint inference model: the baseline model’s overall cluster purity is 0.77, compared to 0.76 for the joint inference model. This shows that the morphosyntactic features being tracked by both learners are informative for differentiating the various underlying clause types in the corpus, even without knowledge of which verbs require objects.

Figure 6.

Proportions of clause types in sentence categories, no-transitivity baseline.

However, the baseline model’s categories did not successfully differentiate sentences with movement from sentences without. The learner inferred many more sentence categories that were candidates for object movement, leading to slightly higher recall than our joint inference learner (Figure 3 above). But its precision was quite poor, leading to a substantially worse F1 score. To examine the source of this worse precision, we plotted the distribution of movement and nonmovement types in the model’s object-gap categories in Figure 7. We find that object movement is the predominant clause type in only 27% of the learner’s object-gap categories, compared to 60% in our joint inference learner. This tells us that our learner’s ability to track transitivity violations is important for identifying categories of sentences with and without movement. While the distributions of morphosyntactic surface features of sentences convey a certain amount of information about the distinctions among different clause types, learning which of these distinctions signal movement, and which do not, requires the use of verb transitivity knowledge during distributional analysis.

Figure 7.

Distribution of movement types in object-gap categories, no-transitivity baseline.

4.4. Summary

In summary, our model identified 80% of sentences with object movement in child-directed speech, by tracking the surface morphosyntactic features of sentences that violate its expectations of verb transitivity. The model jointly infers how to categorize sentences according to their surface feature distributions and which of these sentence categories contain object gaps: unexpectedly missing objects of known verbs. This allowed the learner to generalize across sentences that share the same form and posit object gaps even for verbs that it does not know to be transitive. The learner performed substantially better than a baseline that relies only on known verb transitivity knowledge and does not categorize sentences on the basis of their surface feature distributions. This shows that the model’s categorization process is important. It also outperformed a baseline that categorizes sentences using their surface features alone, without knowing which verbs require objects. The baseline learner performed substantially worse at differentiating sentences with and without object movement, showing that verb knowledge is an important guide for identifying movement.