1. Introduction
The current study aims at deepening the understanding of the well-known subject/object asymmetry found in the comprehension and production of relative clauses (RCs) cross-linguistically by investigating the processing of internally headed relative clauses (IHRCs) through an eye-tracking paradigm. We consider Italian Sign Language (LIS) as empirical ground of investigation, and we examine the comprehension of internally headed subject relative clauses (SRCs) and object relative clauses (ORCs) across three populations of adults: deaf native signers, deaf non-native signers and LIS/Italian CODAs (children of deaf adults), i.e., hearing individuals who acquired LIS (from their deaf parents) and Italian (from the surrounding environment) from birth. Given the current lack of a clear definition of bilingualism (see Wagner et al., Reference Wagner, Bialystok and Grundy2022, for recent discussion), and for the sake of clarity, in this paper we adopt a narrow definition of (bimodal) bilingualism, according to which bimodal bilinguals are individuals who have been naturally and simultaneously exposed to a signed and a spoken language from birth. It follows that only CODAs adhere to this restrictive definition of bilingualism.Footnote 1 To our knowledge, this is the first study on the processing of RCs in CODAs. LIS/Italian CODAs are particularly interesting to study, and represent a privileged point of observation, as the two languages they are competent in not only differ in modality, but also with respect to the word order and the relativization strategy they display (see section 3 for further details). To allow for a comparison between the supposed subject/object asymmetry in the processing of IHRCs and externally headed relative clauses (EHRCs), displayed in Italian, CODAs have also been tested on the comprehension of Italian SRCs and ORCs, and their performance has been compared to that of a control group of Italian monolingual speakers.
The paper is organized as follows: section 2 presents the theoretical background; section 3 briefly describes the typological distance between LIS and Italian; section 4 outlines the goals and predictions of our study, along with methodological details, whereas in section 5 we present the results. Section 6 offers a discussion of the data, and section 7 draws the conclusions.
2. The theoretical background
2.1. Cross-linguistic variation in RC typologies
Cross-linguistically, RCs show a considerable range of variation in many features, among them, the position of the head with respect to the RC and its syntactic function within it.Footnote 2
The position of the head determines the typological distinction between EHRCs, as in (1a-b), and IHRCs, as in (2). In these examples (and throughout the paper), the head is in bold and the RC is within square brackets.
As examples in (1) show, in EHRCs the head is external to the RCs, which features a gap (<_>) corresponding to the interpretation of the head within the clause. In languages such as English, the external head precedes the RC, thus realizing so-called postnominal EHRCs (1a). Conversely, in languages such as Japanese or Mandarin, the external head follows the RC, resulting in prenominal EHRCs, exemplified in (1b).
(2) illustrates an IHRC, as the head thep is phonologically realized in a position internal to the RC, no gap is thus present within it. The example also shows that in some languages a definite strong determiner surfaces at the right edge of the RC (the in (2)).
2.2. The accessibility hierarchy and the theoretical proposals
With respect to the syntactic role carried out by the head noun within the RC, Keenan and Comrie (Reference Keenan and Comrie1977), building on a dataset of about fifty spoken languages, noticed that while all languages allow to relativize subject noun phrases (NPs), other syntactic positions are less accessible (i.e., less likely to function as the head of RCs). As a consequence of their finding, they propose that the syntactic functions the relative NP head may take on are not equally accessible, but vary according to the so-called Accessibility Hierarchy, illustrated in (3).
(3) Accessibility Hierarchy:
Subject > Direct Object > Indirect Object > Oblique > Genitive > Object of Comparison
The hierarchy shows that, although all languages display SRCs, fewer feature ORCs and even fewer display adjunct RCs.
The universality of the hierarchy, claimed by Keenan and Comrie (Reference Keenan and Comrie1977), has been confirmed in numerous subsequent cross-linguistic studies. Specifically, evidence has shown that ORCs (4b) are acquired later and are harder to process than SRCs (4a).
The asymmetry between SRCs and (direct) ORCs is also known as the subject advantage (see Lau & Tanaka, Reference Lau and Tanaka2021, for a recent and detailed review of the literature). As for the source of the asymmetry, there is no consensus. Several proposals have been advanced, which can be grouped into two main categories depending on the factors considered relevant.
On the one hand, we find theories which explain the increased complexity of ORCs in terms of linear order, or by relying on the issue of variation to a norm. This body of research is also known under the name of cue-based (or resource-based) accounts. To illustrate, the Dependency Locality Theory by Gibson (Reference Gibson, Marantz, Miyashita and O'Neil2000) proposes that English ORCs, as in (4b), impose a greater load on working memory than SRCs, (4a), since the filler (i.e., the head noun) is linearly further away from its associated gap within the RC and, thus, has to be retained in memory over a longer period of time. This theory predicts variations in the direction of the asymmetry (subject advantage vs. object advantage) based on the typology of RCs: in languages featuring postnominal EHRCs, the distance between the filler and its gap is linearly shorter in SRCs than in ORCs (as can be noticed in (4) above), thus leading to a subject advantage. Instead, in languages displaying prenominal EHRCs, such as Korean, where the linear distance between the filler and the gap is shorter in ORCs (5b) than in SRCs (5a), the opposite pattern is expected, hence leading to an object advantage.
Within cue-based theories, we also find accounts such as the Word Order Hypothesis (Bever, Reference Bever and Hayes1970), or the Perspective Maintenance Hypothesis (MacWhinney et al., Reference MacWhinney, Bates and Kliegl1984), arguing that in head-initial languages, SRCs are easier to process than ORCs because they respect the canonical word order, or the arguments' thematic roles, of prototypical transitive clauses (i.e., subject-verb-object, SVO), whereas ORCs do not, as they display the non-canonical surface word order object-subject-verb (OSV).
On the other hand, so-called structure-based accounts associate the source of the bias with the hierarchical syntactic structure by considering the structural distance between the head noun and the gap as the main predictor of complexity. Crucially, the distance is calculated in terms of syntactic nodes crossed by the head to reach its gap (e.g., Hawkins, Reference Hawkins1999; O'Grady, Reference O'Grady1997, Reference O'Grady and Kidd2011; O'Grady et al., Reference O'Grady, Lee and Choo2003), rather than linearly, hence leading to the universal prediction that SRCs (by virtue of having a head noun less embedded than ORCs) are easier to process across all types of EHRCs, whether post- or prenominal. A different type of structural account is Friedmann et al. (Reference Friedmann, Belletti and Rizzi2009), who argue for a hierarchical intervention effect and advance an account anchored within the Relativized Minimality framework (Rizzi, Reference Rizzi1990): in SRCs, no other NP can be found between the head noun and its gap, whereas in ORCs the subject of the RC hierarchically intervenes between them, leading to difficulties in reconstructing the dependency. The more features are shared between the head noun and the intervening element, the harder it will be to process the sentence (e.g., Adani et al., Reference Adani, van der Lely, Forgiarini and Guasti2010; Friedmann & Novogrodsky, Reference Friedmann and Novogrodsky2004, among many others).
2.3. Looking for a subject advantage in IHRCs
The aforementioned theories on the asymmetry between SRCs and ORCs are mainly based on post- and prenominal EHRCs. However, when it comes to the few studies exploring the asymmetry in languages featuring other types of RCs, such as IHRCs, controversial results emerge, which challenge the universality of the Accessibility Hierarchy and of the subject advantage. For instance, Hu et al. (Reference Hu, Cecchetto and Guasti2018) conducted an elicited production experiment and found that in Wenzhounese, a language allowing the head noun to be either external or internal to the RC, speakers opt for IHRCs when producing ORCs (thus showing an object advantage for this RC typology), while they display a subject advantage when producing EHRCs. This is in line with Jeon and Kim's (Reference Jeon and Kim2007) findings on Korean, a language which also displays both types of RCs, and in which second-language (L2) learners tend to produce slightly more ORCs than SRCs when using IHRCs, while the direction of the asymmetry is the opposite for EHRCs. Since both studies concern production, an explanation might be that, when asked to produce an ORC, speakers found it easier to choose an IHRC rather than an EHRC, hence increasing the frequency of that structure. This is confirmed by the acquisition pattern of these constructions. In his longitudinal study on the productions of three Korean children, Kim (Reference Kim1987) found that the first stage in the acquisition of RCs in Korean is represented by headless RCs, followed by IHRCs and then by EHRCs. Similarly, Lee (Reference Lee1991) investigated a corpus of spontaneous production of thirty-six Korean children and found that IHRCs surface earlier than EHRCs. Similar patterns have been reported for Cantonese (Yip & Matthews, Reference Yip and Matthews2007).
What we can draw from these studies is that the internally headed strategy seems to be selected to avoid the complexity of object extraction. By resorting to a different relativization strategy, asymmetries between SRCs and ORCs are less expected or can be harder to detect in these languages. To our knowledge, no study has been carried out on the comprehension of SRCs/ORCs in spoken languages that only allow the internally headed typology, thus, in the light of the few findings available, it is difficult to draw conclusions about the presence or absence of a subject advantage in the processing of this typology of relativization.
2.4. The subject/object asymmetry in sign languages
On a par with spoken languages, the growing literature on sign languages attests the same variation in the typologies of relativization available cross-linguistically.Footnote 3 To illustrate, (6) exemplifies an EHRC in German Sign Language (DGS), whereas (7) shows an IHRC in American Sign Language (ASL).Footnote 4
However, to date, sign languages have been scarcely included in the theoretical and empirical debate introduced above, with only two studies that have recently addressed the subject/object asymmetry across EHRCs and IHRCs. Hauser and Pozniak (Reference Hauser and Pozniak2019) adapted the eye-tracking Visual World Paradigm used in Pozniak (Reference Pozniak2018) to make it suitable to sign language investigation, and tested the comprehension of SRCs and ORCs in French and French Sign Language (LSF), both displaying (postnominal) EHRCs. They tested hearing French speakers and deaf LSF signers. Interestingly, they found a subject advantage for both French and LSF. The subject advantage found in LSF has been confirmed through the (offline) sentence-to-picture matching task developed by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021), who compared the comprehension of SRCs and ORCs across three populations of deaf signers, in three typologically different sign languages: LSF displaying EHRCs, LIS and Catalan Sign Language (LSC) featuring IHRCs. The study detected a clear subject advantage in LSF and LSC, whereas in LIS the asymmetry only surfaced in deaf late signers (exposed to LIS between 6 and 15 years old).
3. LIS and Italian RCs
Although used on the same territory, LIS and Italian differ greatly in their grammars.
LIS is a coherently head-final language, whose unmarked word order is subject-object-verb (SOV), as illustrated in (8).Footnote 5
According to Branchini and Donati (Reference Branchini, Donati and Lipták2009) and Branchini (Reference Branchini2014), LIS features restrictive RCs that display characteristics of (fronted) IHRCs.Footnote 6 Specifically, the head is realized inside the RC, in the position corresponding to its syntactic role within it. The head is (optionally) marked, through spatial agreement, by a sign (glossed pe) which is analysed as a determiner-like element co-referent with the head noun and surfacing at the right edge of the RC, as in (9a), or, less frequently, next to the head noun, as in (9b).
Moreover, as (9) shows, LIS RCs are marked by specific non-manuals (____rel), that is raised eyebrows, tensed eyes and cheeks and a forward head tilt. These non-manual markers can either spread over the entire RC, or occur only over the sign pe. Branchini and Donati's (Reference Branchini, Donati and Lipták2009) proposal is that the RC-final position of pe (9a) is a derived position, resulting from the (overt) movement of the relative determiner pe from an adnominal position, next to the head (a clause-internal D-position), to a clause-external C-position at the right periphery of the RC. In so doing, the movement of the relative determiner pe endows the RC with the necessary nominal features (Branchini, Reference Branchini2014; Branchini & Donati, Reference Branchini, Donati and Lipták2009).
Italian, on the contrary, is a head-initial language, whose unmarked word order is SVO. Headed restrictive RCs in Italian are externally headed, thus the RC contains a gap corresponding to the position where the head is interpreted inside the RC, and are introduced either by the relative complementizer che ‘that’, or by relative pronouns such as la quale ‘who’. (10) provides an example of an Italian SRC.Footnote 7
4. The current study
4.1. Goals and predictions
In this study, we aim to assess the alleged asymmetry between SRCs and ORCs in a sign language featuring the internally headed strategy, i.e., LIS, by using an eye-tracking paradigm. Moreover, we are interested in investigating whether the asymmetry holds across three populations of adult signers: deaf native signers (exposed to LIS from birth through their parents), deaf non-native signers (exposed to LIS between 1 and 18 years old) and hearing LIS/Italian bimodal bilingual CODAs (exposed to both LIS and Italian from birth). By including the CODA population, we also aim at finding possible correlations between performance and simultaneous bilingualism, as well as modality- or typology-related differences in the processing of the same structure in two different languages (i.e., LIS and Italian).
On the basis of the theoretical and empirical background outlined in section 2, we formulate some predictions.
First, if the Accessibility Hierarchy is truly universal, we predict that it should hold across modalities, thus we expect to find a subject/object asymmetry in LIS IHRCs as well.
With respect to the source of the asymmetry, we predict to find a subject advantage if complexity correlates with structural aspects, such as the level of embedding of the relativized NP. On the contrary, we expect an object advantage if the difficulty arises from linear cues, such as the distance between the head and the moved relative determiner pe, which, in LIS IHRCs, is greater in SRCs than in ORCs.
As far as the influence of factors such as age of first exposure (AoE) to LIS is concerned, if it has an impact on the processing of complex structures (as shown in the literature, see Zorzi et al., Reference Zorzi, Giustolisi, Aristodemo, Cecchetto, Hauser, Quer, Sánchez Amat and Donati2022, among others), we predict deaf native signers and CODAs to perform better than signers exposed to LIS later in life (i.e., non-native signers). Similarly, if early simultaneous bimodal bilingualism matters in terms of enhanced cognitive executive functions (the so-called bilingualism-related cognitive advantages, see Bialystok, Reference Bialystok2017, for a detailed review), we expect better performances in CODAs as compared to deaf native (and non-native) signers, whose exposure to Italian is often delayed or reduced.
Finally, we expect to replicate the subject advantage described for Italian in the literature (e.g., Guasti et al., Reference Guasti, Vernice and Franck2018), both across CODAs and monolingual speakers.
4.2. Participants
Seventy-two participants were recruited via social media, associations of interpreters and local associations for the deaf. All participants were adults and gave their written informed consent before taking part in the experiment.Footnote 8
We divided participants into three groups. The first group consisted of thirty deaf signers (21 females; mean age = 36.73 years; SD = 8.81), with normal or corrected-to-normal vision. They were included considering two criteria: deafness attested before the age of 3 and first exposure to LIS no later than age 18. By considering the AoE to LIS, we divided deaf participants into two groups: native signers and non-native signers. We considered native signers those with at least one deaf signing parent and exposed to LIS from birth. This corresponded to seventeen participantsFootnote 9 (11 females; mean age = 35.57 years, SD = 9.34). The remaining thirteen participants, exposed to LIS between 2 and 18 years of age (10 females; mean age = 37.92 years, SD = 7.75; mean AoE = 9.38 years, SD: 5.77) constituted the non-native group.Footnote 10
The second group included twenty-one hearing CODAs (17 females; mean age = 38.23 years, SD = 11.51), with normal or corrected-to-normal vision. The inclusion criteria required that they had at least one deaf signing parent, and exposure to LIS from birth.
The third group consisted of twenty-one hearing monolingual Italian native speakers (16 females; mean age = 38.28 years, SD = 12.48) with normal or corrected-to-normal vision, matched to the CODAs group by age and recruited as a control group. The inclusion criteria for this group were: exposure to Italian (from Italian parents) from birth, fixed residence in Italy from birth and L2 learnt after age 6.
All participants answered a questionnaire that allowed us to collect some biographical information useful to investigating possible correlations between sociolinguistic variables and performance in LIS. The questionnaire administered to deaf and CODA participants included specific questions addressing their signing background (for instance, AoE, rates of language use, LIS-related job, etc.).
As for deaf native signers, eleven participants out of seventeen declared using LIS and Italian daily, and the remaining six used LIS on a daily basis and Italian several times a week. Thirteen participants had a high school diploma, three had a bachelor's or master's degree and one participant had obtained a doctoral degree. Six participants worked as LIS teachers.
As for deaf non-native signers, eleven participants out of thirteen reported using LIS and Italian daily, whereas two used Italian on a daily basis and LIS several times a week. Seven participants held a high school diploma, and six a bachelor's or master's degree. One worked as an LIS teacher.
With respect to CODAs, sixteen participants out of twenty-one declared using LIS and Italian daily, four used Italian on a daily basis and LIS several times a week, and one participant used Italian daily and LIS rarely. Fourteen participants had a high school diploma, whereas seven held a bachelor's or master's degree. Eleven participants worked as professional LIS–Italian interpreters.
4.3. Methodology
The study used the adaptation to visual-only modality of the eye-tracking Visual World Paradigm conceived by Hauser and Pozniak (Reference Hauser and Pozniak2019). We created two versions of the same protocol: one to assess the comprehension of Italian RCs with CODAs and Italian monolinguals, and one for LIS RCs, to be administered to CODAs and deaf signers. In this visual-only adaptation, both the pictures and the linguistic stimulus were visually (rather than aurally) presented and appeared simultaneously on the same screen, as illustrated in Figure 1.
Figure 1. Display of stimuli and pictures in the two versions of the experiment created to assess the comprehension of SRCs and ORCs in LIS (above) and Italian (below).
As Figure 1 shows, the stimulus corresponded to a video of a signed (in the LIS experiment) or written (in the Italian experiment) RC. Participants were asked to look at both the pictures and the video and to select the correct picture corresponding to the video stimulus by fixing the gaze on it. Then, they had to confirm their answer by pressing the left or right button on a response device once the video ended. In the following sections, we describe the materials used, the design of the paradigm and the procedure of data collection.
4.3.1. Materials
The pictures used in the experiment, for both items and fillers, were taken from Knoeferle et al. (Reference Knoeferle, Crocker, Scheepers and Pickering2003).Footnote 11 Items corresponded to pairs of pictures representing the same event involving the same sets of three referents, albeit with inverted theta roles. Each picture was compatible with either a SRC or ORC interpretation. Thanks to this arrangement, four sentences could be derived from a pair of pictures: two SRCs (one corresponding to the right picture and one corresponding to the left picture) and two ORCs (one corresponding to the picture on the right and one corresponding to the picture on the left). To illustrate, the pair of pictures in Figure 1 corresponds to the RCs in Table 1.
Table 1. Example of experimental items paired with Figure 1
We used twenty-four pairs of pictures for items, so, for each version of the experiment, we created four lists of items, each list targeting twelve SRCs (six corresponding to the pictures on the left and six corresponding to the pictures on the right) and twelve ORCs (six for the pictures on the left and six for the pictures on the right). In so doing, the items were counterbalanced across conditions and position of the correct picture. Each participant was assigned a different list, such that each participant saw only one variant of the four possible RCs (see Table 1) for each item. Hauser and Pozniak (Reference Hauser and Pozniak2019), who created the protocol that we adopted, chose this combination to avoid having the correct answer always on the right or on the left, as well as to avoid possible influences of character saliency.
Following their adaptation to visual-only modality, for the Italian stimuli (both items and fillers) we created a video showing the written sentence appearing in chunks one after the other. Each chunk lasted 2 seconds on the screen. This strategy was meant to reproduce the sequentiality of signs in LIS discourse, hence making the two paradigms comparable. Chunks' duration was assessed through a pilot study involving two CODAs. The six chunks (between slashes) are exemplified in (11).
In the LIS version of the experiment, stimuli were provided through signed videos. The RCs used as stimuli were previously discussed with two LIS deaf native signers. When recording the stimuli, we controlled the mapping between characters in the pictures and spatial associations in the LIS sentences (i.e., verb agreement), in order to avoid a visual correspondence between the two. This was necessary to avoid the possibility that participants relied on the direct mapping between the sentence and the pictures to find the correct answer. A pilot study with three deaf LIS signers was conducted in order to check whether the speed of the stimuli recorded allowed them to perform the task. At normal speed, participants were not able to look at the sentence stimulus while simultaneously investigating the pictures, so we slightly reduced the speed of the videos and checked again with three other deaf LIS signers that the change in speed allowed them to perform the task without altering the comprehension and spontaneity of the LIS sentence. The videos of the signed RCs were then annotated through ELAN (Crasborn & Sloetjes, Reference Crasborn and Sloetjes2008) in order to identify the boundaries of each critical sign to be used for the analysis of eye-tracking data.
The twenty-four items (SRCs and ORCs) in each list were randomized with twenty-six fillers, which were the same across all four lists in both versions of the experiment. In the LIS experiment, fillers corresponded to question/answer pairs (12a), which were produced during discussion with the two deaf informants who recorded the items, and thus adopted. Fillers in the Italian version of the experiment corresponded to requests to find a referent not carrying out any action (12b).
Both items and fillers were introduced by a written (Italian experiment) or signed (LIS experiment) context (see the LIS example in (13)) presenting the referents that appeared in the subsequent stimulus (i.e., three characters or two objects), paired with the corresponding picture appearing on the upper portion of the screen. The characters were presented as still figures (without performing any action) in order to allow participants to familiarize with the characters and their associated sign without influencing their subsequent processing.
This was meant to ensure the same saliency to all referents (Hauser & Pozniak, Reference Hauser and Pozniak2019), and for the LIS experiment, to make sure participants were familiar with the sign adopted to identify a character, given the availability of sign variants in the Italian territory. The signed contexts were first discussed and then elicited with the two deaf informants.
4.3.2. Design
We used software Experiment Builder (SR Research Ltd, Reference SR Research2020) to build the paradigm. Both versions were composed of three parts: instructions, calibration and experiment. In the Italian version, instructions were provided through written texts on the screen, whereas in the LIS version they were provided through videos in LIS recorded by the same informants who produced the stimuli.
The experiment consisted of fifty-eight trials divided into three blocks: one block of eight practice trials (four items and four fillers), followed by fifty experimental trials (twenty-six filler trials fully randomized with twenty-four item trials), divided into two blocks of twenty-five trials each. Both practice and experimental trials had the same structure, that is: context > drift-check > stimulus > answer time > feedback. The feedback signal appeared only if the answer was not provided in time (see the experiment materials for a representative illustration of the trials' structure).
Each trial started with the context, after which a one-point calibration (drift-check) appeared in the upper-left corner of the screen to check that the participant's eye was properly tracked throughout the experiment. If this was the case, the trial continued by showing the stimulus (either an item or a filler). Following the design by Hauser and Pozniak (Reference Hauser and Pozniak2019), we divided the screen into two portions: on the upper one, we positioned the two competing pictures, whereas on the lower one we put the video for the stimulus, simultaneously appearing with the pictures. Each element, namely the two pictures and the video, was coded as area of interest.
The appearance of stimuli corresponded to the recording session of the experiment, that is, the portion of time during which participants' eye movements were recorded by the eye-tracker. Participants were instructed to investigate both the pictures and the stimulus simultaneously, and to fix the gaze on the picture corresponding to the stimulus as soon as they found it. The recording session was followed by an answer time, i.e., a 2-second span signalled by a green light appearing below the pictures. Here participants were asked to select the correct picture by pressing the left or right button on a button box, thus validating the answer they provided with their eye gaze. In case they did not answer in the allotted time, a red warning sign appeared in the centre of the screen. This procedure allowed us to record both online (eye data) and offline (accuracy) responses.
4.3.3. Procedure
Participants have been individually tested. CODAs took the LIS experiment first, and at least two weeks later they took the Italian experiment. Each session lasted about an hour and a half for LIS, and about forty minutes for the Italian experiment, including reading and signing the consent forms, answering the metadata questionnaire and performing the experiment. Before starting the test, participants took a short cognitive task, called The Odd One Out Cognitive Task (Giustolisi & Friedmann, Reference Giustolisi and Friedmann2019).Footnote 12 In addition, participants performed the Miles Test (Miles, Reference Miles1930), which allowed the detection of the participant's dominant eye to be recorded during the experiment.
Eye fixations were recorded using EyeLink Portable Duo (SR Research Ltd), laptop Host PC with Laptop-Mount and head-support, monocular mode, 500 Hz sampling rate, 9-point calibration and validation. The distance between the front of the camera and the participant's eyes was about 45 cm. We used the EyeLink button box as a response device to record accuracy data. Participants were allowed to take a break (during which instructions were repeated if necessary) after the eight practice trials and after the first twenty-five trials. Every time participants moved, calibration was performed again.
4.4. Analyses
4.4.1. Eye-tracking data (online measure): methodology of data analysis
We analysed and plotted our data through the Eyetracking-R package (Dink & Ferguson, Reference Dink and Ferguson2015) for all four populations (deaf native and non-native signers, CODAs and Italian monolinguals). Unless otherwise specified, all analyses were performed using the same linear mixed-effect model (Barr et al., Reference Barr, Levy, Scheepers and Tily2013). Independent variables were time (as a continuous variable) and RC type (subject/object). The latter was coded as ‘1’ for SRC, ‘0’ for ORC, and we applied mean centred coding. Dependent variable was the empirical logit of looks to the correct vs. incorrect picture from the RC onset until the end of the sentence (Barr, Reference Barr2008). The random variables were participants and items. We enforced zero correlations between random effects to avoid overparameterization or false convergence when necessary (Bates et al., Reference Bates, Mächler, Bolker and Walker2015).
4.4.2. Accuracy data (offline measure): methodology of data analysis
Accuracy measures were collected at the end of the sentence, by asking participants to press the button corresponding to their answer. Results were also analysed with R software via a linear mixed model through the glmer function (package lme4; Bates et al., Reference Bates, Mächler, Bolker and Walker2015). Responses were coded ‘1’ for correct answers and ‘0’ for incorrect answers. The independent variable was the type of RC (SRC vs. ORC). Participants and items were the random variables of the model. In LIS, we also included linguistic groups (CODAs, native, non-native) as an independent variable. When necessary, we enforced zero correlations between random effects to avoid overparameterization or false convergence.
5. Results
5.1. Italian
5.1.1. Eye-tracking data
Figure 2 shows the proportion of fixations on the correct picture (vs. the incorrect picture) in hearing Italian monolingual speakers (on the left) and CODAs (on the right) for Italian SRCs and ORCs. The time window goes from the (external) head noun to the end of the RC, that is, the object of the SRC, and the verb of the ORC.
Figure 2. Proportions of fixations (y-axis) on the correct picture in SRCs (red) and ORCs (blue) in Italian, by monolinguals (n = 21, on the left), and CODAs (n = 21, on the right). Time is represented along the x-axis. Each vertical black bar represents the average time of display for a given element of the RC.
In Figure 2, we see that Italian monolinguals fixated the correct answer earlier and in greater proportion when they were seeing a SRC rather than an ORC. The subject advantage is confirmed by the linear mixed model we ran, with an effect of clause type (estimate = .57, t = 5.87, p < 1.88 × 10−12), of time (Est. = .03, t = 127.930, p < 2 × 10−16), and an interaction between the two (Est. = .01, t = 28.31, p < 2 × 10−16).
Similar results are obtained for Italian RCs by the group of CODAs. Here again, there is a strong subject advantage captured both through an effect of RC type (Est. = 6.320 × 10−1, t = 5.218, p < 1.06 × 10−5), time (Est. = 2.572 × 10−2, t = 125.632, p < 2 × 10−16) and an interaction of the two (Est. = 1.234 × 10−2, t = 30.132, p < 2 × 10−16).
From these results, we can confirm the presence of a subject advantage in the processing of Italian RCs, as attested in previous research (see Guasti et al., Reference Guasti, Vernice and Franck2018, and reference therein).
5.1.2. Accuracy data
Accuracy scores in Italian SRCs and ORCs are plotted in Figure 3 for both CODAs and hearing monolingual speakers.
Figure 3. Accuracy in ORC (blue) and SRC (orange) comprehension in Italian, for CODAs (left, n = 21) and Italian monolinguals (right, n = 21). Black crosses represent the mean, and black horizontal lines represent the median of each group. The whiskers extend to the nearest data point, that is, no more than 1.5 times the interquartile range from the hinges.
CODAs and monolinguals performed at ceiling both in SRCs and ORCs. Both groups have very similar results, such that there is no effect of language group (Est. = .03, t = .03, p = .97), and no interaction between language group and clause type (Est. = −.008695, t = −.011, p = .99). There is, however, an effect of clause type, a subject advantage (CODAs: Est. = 2.810138, t = 4.991, p < 6.00 × 10−07; monolinguals: Est. = 2.801387, t = 4.705, p < 2.54 × 10−06).
5.2. Italian Sign Language
5.2.1. Eye-tracking data
Figure 4 reports the proportions of correct picture fixation when CODAs (on the left), deaf native signers (in the middle) and deaf non-native signers (on the right) were presented with an LIS SRC or an ORC. The time window spans from the subject of the RC to the relative determiner pe. Note that, contrary to the Italian version, the time at which a given critical sign appeared varied across sentences, thus the figure provides the average time of their display.
Figure 4. Proportions of fixations (y-axis) on the correct picture in SRCs (red) and ORCs (blue) for LIS by CODAs (n = 21, on the left), deaf native signers (n = 16, middle) and deaf non-native signers (n = 13, on the right). Time is represented along the x-axis. Each vertical black bar represents the average time of display for a given element of the RC.
As can be seen in Figure 4, for CODAs there is a clear difference across RC types, with the correct answer being fixated earlier and more accurately in SRCs than in ORCs. This subject advantage is confirmed statistically through the linear mixed model, with effects of clause type (Est. = .35, t = 2.60, p < .01), time (Est. = .009, t = 35.43, p < 1.132 × 10−272) and an interaction between these two variables (Est. = .01, t = 25.05, p < 1.25 × 10−137).
Interestingly, results are not so clear-cut for deaf native signers. The difference across clause type is not statistically significant (Est. = .24, t = 1.57, p = .12), although an interaction between conditions and time can be detected (Est. = .02, t = 19.42, p < 9.54 × 10−84), as well as an effect of time (Est. = .008, t = 27.72, p < 5.02 × 10−168). From these data, we see that deaf native signers started gazing more at the correct answer when they reached the relative determiner (pe). Their performance is slightly biased towards SRCs although it remains at chance levels.
The results of the last population of interest, deaf non-native signers, are presented in the graph on the right. Similarly to the results displayed by deaf native signers, non-native signers do not show a subject advantage. There is no effect of clause type (Est. = .12, t = .43, p = .67), although increasing over time (Est. = .008, t = 27, p < 3.53 × 10−159), and an interaction between clause type and time is detected (Est. = .01, t = 20.68, p < 2.89 × 10−94). The accuracy scores in both conditions remain very low throughout the sentence, even after the relative determiner is produced. It is puzzling to observe such low scores on the deaf population, both native and non-native. We address this point in the discussion.
5.2.2. Accuracy data
The mean accuracy of each language group (CODAs, deaf native signers and deaf non-native signers) in the comprehension of LIS SRCs and ORCs is presented in Figure 5.
Figure 5. Accuracy in ORC (blue) and SRC (orange) comprehension in LIS, for CODAs (left, n = 21), deaf native signers (middle, n = 16) and deaf non-native signers (right, n = 13). Black crosses represent the mean, and black horizontal lines represent the median of each group. The whiskers extend to the nearest data point, that is, no more than 1.5 times the interquartile range from the hinges.
As Figure 5 shows, CODA signers present much higher scores than deaf signers in both SRCs and ORCs. The subject advantage identified in eye-tracking data is confirmed in the offline measure with a clear effect of condition for CODAs (Est. = .77, t = 3.42, p < .0006), and it is detected for deaf native signers (Est. = .79, t = 3.59, p < .0003). No subject advantage is detected in the deaf non-native population again (Est. = .28, t = 1.19, p = .23). The language group is also an important factor since CODAs perform significantly better than deaf native (Est. = −1.09, t = −3.77, p < .0002) and non-native signers (Est. = −.82, t = −2.66, p < .008). The comparison between the accuracy data in native and non-native signers is not significant (Est. = .28, t = .87, p = .39).
6. Discussion
In this study, we aimed at contributing to the assessment of the alleged subject/object asymmetry in the understudied typology of IHRCs by using an eye-tracking paradigm and by considering a language in the visual–manual modality: LIS. Moreover, we were interested in investigating whether such asymmetry, if present, holds across different populations of adult signers: deaf natives and non-natives, and hearing LIS/Italian bimodal bilinguals (so-called CODAs). We also aimed at examining the impact of age of first-language exposure and simultaneous bilingualism on the processing of complex structures. To complete the picture, we wanted to verify the presence of modality- or typology-related differences in the comprehension of SRCs and ORCs in the two native languages of bimodal bilingual CODAs, i.e., Italian and LIS.
The results of our experiment corroborate the presence of a subject advantage in Italian RCs, through both online and offline measures, and across language groups (CODAs and Italian monolingual speakers). This was an expected outcome, which, nevertheless, adds novel results from a new paradigm (eye-tracking) and population (CODAs) to the literature, while also validating the visual-only adaptation by Hauser and Pozniak (Reference Hauser and Pozniak2019).
On the contrary, the picture that emerges for LIS is more complex. The data reveal that CODAs, on average, present higher accuracy scores than deaf signers. Moreover, the subject advantage is more consistent and salient in CODAs, being detected in both online and offline measures, whereas for deaf native signers it only surfaces in offline measures. As for deaf non-native signers, it is not detected at all. In what follows, we first discuss the general significance of our results in LIS from a theoretical perspective (section 6.1), and then we try to unveil the reasons behind the better performance of CODAs when compared to deaf signers (section 6.2), and the contrast with the results from a previous study (section 6.3).
6.1. Theoretical implications
The detection of a subject advantage in LIS RCs has two main theoretical implications. First, it shows that the predictions of the Accessibility Hierarchy hold across RC typologies (EHRCs and IHRCs), modalities (spoken and signed) and populations (CODAs and deaf native signers), thus supporting the claimed universality of the subject advantage.
Additionally, our findings provide evidence for a structural- rather than cue-based explanation for the source of the asymmetry. Let us check our results against the predictions of the different accounts outlined in section 2.2. Proposals relying on canonical word order are ruled out, since they fail to predict any asymmetry for LIS given that both subject and object IHRCs display the canonical SOV order. Similarly, accounts trying to explain the asymmetry in terms of linear interference of new referents or linear distance are excluded because SRCs are easier to understand than ORCs, despite featuring a greater number of referents occurring between the head noun and the relative determiner pe identifying the head, and despite displaying a greater linear distance between the moved relative determiner pe and its merging position (where it leaves a gap), as illustrated in the contrast between (14a) and (14b).
Conversely, accounts ascribing the source of the asymmetry to the structural distance between the head noun and the gap, and to the dependency created through the (overt or covert) movement of the head (e.g., Cole, Reference Cole1987; O'Grady et al., Reference O'Grady, Lee and Choo2003) seem to correctly capture the LIS data. Indeed, although LIS RCs do not involve the overt movement of the noun head, they do display the overt movement of the relative determiner pe from an adnominal position, next to the head noun (a clause-internal D-position), to a clause-external C-position in the right periphery of the RC (Branchini, Reference Branchini2014; Branchini & Donati, Reference Branchini, Donati and Lipták2009). The dependency created by the movement of pe is longer, and the structural position of the gap left by pe is more embedded when the head is the object, thus making ORCs more complex even within the internally headed type.
As for the impact of sociolinguistic factors such as AoE, our results confirm that a delayed exposure to LIS has long-lasting effects (e.g., Zorzi et al., Reference Zorzi, Giustolisi, Aristodemo, Cecchetto, Hauser, Quer, Sánchez Amat and Donati2022), leading non-native signers to face greater difficulties than native (deaf and hearing) signers.
Crucially, our data also present two unexpected outcomes, one being that CODAs outperformed deaf native (and non-native) signers; the other being the contrast with the results obtained by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021), who found a subject advantage only in late LIS signers. We address the two issues in turn in the following sections.
6.2. Early simultaneous bilingualism as an asset?
Since both CODAs and deaf native signers are exposed to LIS from birth, a different performance between these two populations was hard to foresee. To try to disentangle the reasons behind the obtained asymmetry, we consider the differences holding between the two groups, which are mostly correlated with their access to the spoken language (i.e., Italian), both as early input and education.
Deaf people's access to Italian varies greatly in quantity and quality depending on several variables, such as degree of hearing loss, age of onset, use and types of hearing devices, etc. This, along with the lack of accessibility (e.g., subtitles) in both private and public contexts, often leads to a reduced and delayed input, which affects their acquisition of the spoken language. Conversely, CODAs, being hearing, have an early and full exposure to Italian from birth. Thus, one possibility is that CODAs' early and simultaneous access to both Italian and LIS has allowed them to develop so-called bilingualism-related cognitive advantages, which, according to the literature on unimodal bilingualism, result from the constant need to manage two languages (e.g., Bialystok, Reference Bialystok2017). Specifically, we are referring to enhanced attentional control, which includes cognitive abilities such as task switching, decision making and conflict resolution (e.g., Bialystok & Craik, Reference Bialystok and Craik2022). Recent studies show that bimodal bilinguals do rely on language control when suppressing interference and/or inhibiting a language, especially the more dominant one, i.e., the spoken language (e.g., Blanco-Elorrieta et al., Reference Blanco-Elorrieta, Emmorey and Pylkkänen2018; Declerck et al., Reference Declerck, Meade, Midgley, Holcomb, Roelofs and Emmorey2021; Emmorey et al., Reference Emmorey, Li, Petrich and Gollan2020). Therefore, we might speculate that by engaging in language control by virtue of their early bilingualism, CODAs have developed enhanced cognitive abilities that helped them to be more effective in such a complex experiment as the one we have run, that required to perform two tasks simultaneously, namely, understanding syntactic structures while integrating non-linguistic information (i.e., pictures), and providing online (eye data) and offline (accuracy) responses (see Ostadghafour & Bialystok, Reference Ostadghafour and Bialystok2021, for similar hypotheses on unimodal bilinguals). However, the scarce literature on the cognitive advantages deriving from bimodal bilingualism does not point in this direction. When testing their executive functions, neither Emmorey et al. (Reference Emmorey, Luk, Pyers and Bialystok2008) nor Giezen et al. (Reference Giezen, Blumenfeld, Shook, Marian and Emmorey2015) found a significant difference in CODAs' performance when compared to monolinguals, hence failing to replicate the bilingual advantage that is described in the unimodal bilingualism literature (see also Olulade et al., Reference Olulade, Jamal, Koo, Perfetti, LaSasso and Eden2016, for neuroanatomical evidence). In the light of these findings, we need to find other explanations.
One alternative might be to consider another difference holding between the deaf and CODA populations, which concerns the possibility of developing a stronger metalinguistic awareness (i.e., the individuals' ability of paying attention, reflecting upon and making judgements on the structural features and functions of language; see Bialystok, Reference Bialystok2001). Due to the lack of accessible educational materials and of bimodal bilingual (LIS/Italian) policies in the Italian educational system,Footnote 13 deaf students face greater difficulties with formal instruction on Italian grammar at school, which compromise their development of an explicit knowledge of the spoken language. Conversely, it seems plausible that the accessible explicit metalinguistic education that CODAs receive helps them to build the architecture of complex constructions, which they transfer to analyse the structures at play in their sign language in a more systematic way as compared to deaf people. If this line of reasoning is correct, the explicit knowledge of the LIS grammar, that some CODAs and some deaf people develop in specific training to become interpreters or LIS teachers, might contribute to their competence and understanding of complex sentences to a lesser extent.
To assess whether the formal knowledge of LIS plays a role in processing, we ran two within-group post-hoc analyses comparing, on the one hand, the performance of CODAs working as LIS interpreters to that of CODAs whose profession did not involve receiving an explicit training on the LIS grammar, and on the other, the performance of deaf LIS teachers to that of deaf signers with LIS unrelated jobs. Crucially, no significant difference in performance was observed, neither for the CODA group (eleven LIS interpreters and ten non-interpreters), nor for the deaf signers' group (seven LIS teachers and twenty-two non-teachers). If LIS metalinguistic knowledge does not tip the balance either way, we might turn to consider the stronger metalinguistic awareness in Italian that CODAs develop during the school years, that can be transferred when processing the sign language. Similar conclusions about the possibility that metalinguistic awareness is shared across languages are also reached in studies on unimodal bilingualism (e.g., Torregrossa et al., Reference Torregrossa, Eisenbeiß and Bongartz2022, and references therein).
Besides the interesting questions and considerations that these issues raise, we interpret these results as evidence for the necessity that Italian policymakers improve the promotion of sign language knowledge and of bimodal bilingual education to guarantee a solid development of linguistic and non-linguistic abilities for deaf people of both LIS and Italian.
6.3. Same syntactic construction, two different outcomes
In this section, we address the puzzling contrast in participants' performance that we detected between our study and that of Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021), who assessed the comprehension of SRCs and ORCs in LIS through a sentence-to-picture matching task across deaf native and non-native signers (section 2.4). Specifically, the difference concerns the accuracy of deaf native signers in ORCs, which in Hauser et al.'s (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) study was around 75%, while in our study it decreases below 50%, yielding a subject advantage to surface. To account for such a decrease, we need to compare the tasks of the two studies. Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) used pictures displaying three characters as items, and participants had to choose one of the characters based on the RC they saw, by touching the laptop screen. In our experiment, instead, participants were presented with two pictures representing three characters each (i.e., six characters in total), and were asked to compare them while looking at the RC stimulus. While doing this, they had to select the correct picture by fixating the gaze on it, and then to validate the answer by using a button-box device within a 2-second span. The complexity of the task in our study (combining eye-tracking and accuracy) might have increased the distance between the comprehension of SRCs and ORCs (the latter usually more demanding), thus allowing us to identify a subject advantage with respect to an easier task where no asymmetry emerged.
Alternative explanations regard the stimuli themselves. Table 2 compares the structures used as stimuli by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) for SRCs (15a) and ORCs (15b), with the structures that we selected for SRCs (16a) and ORCs (16b).
Table 2. Illustrative sample of the structures employed for LIS SRCs and ORCs in the study by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021), (15a-b), as compared to ours (16a-b)
As we can see from the table, Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) generalized the sentence-initial position of the head (preferred by their informants), thus obtaining different linear orders for SRCs (SOV) and ORCs (OSV). Moreover, they employed a classifier handshape (CL) to mark the object within the RC in both SRCs and ORCs.Footnote 14 Conversely, in our study (being the first of its kind) we decided to use the unmarked structure described in the literature for LIS SRCs and ORCs (e.g., Branchini, Reference Branchini2014; Branchini & Donati, Reference Branchini, Donati and Lipták2009; Cecchetto et al., Reference Cecchetto, Geraci and Zucchi2006). In so doing, the head sits in the position corresponding to its syntactic function within the RC, thus both SRCs and ORCs exhibit the order SOV, and the head is only identified by means of the relative determiner pe. Assuming that deaf native signers indeed have at their disposal a second strategy to produce ORCs, i.e., by topicalizing the object head, the low accuracy we register in their responses might be due to a hesitation in processing the RC (according to the SOV or OSV order) leading to a reinterpretation of the arguments' syntactic roles when reaching the verb and the sign pe. Similarly, CODAs' good performance might suggest that they have only one strategy available, namely the head-in-situ one. Additionally, the higher accuracy in ORCs in the study by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) might be related to the fact that the object head is clearly identified thanks to cues absent in our study: its sentence-initial (i.e., topicalized) position and the presence of a classifier marking it. Among these cues, NP topicalization has been observed in previous studies as a simplification strategy (employed by children and adults) to overcome the complexity of object wh-questions (e.g., Guasti et al., Reference Guasti, Branchini and Arosio2012, and the literature cited therein). Topicalization might have contributed to a better comprehension of ORCs in the study by Hauser et al. (Reference Hauser, Zorzi, Aristodemo, Giustolisi, Gras, Sala, Sánchez Amat, Cecchetto and Donati2021) as well. We leave the testing of these hypotheses for future research.
7. Conclusions
In this paper, we present the first eye-tracking study assessing the subject/object asymmetry in the comprehension of internally headed SRCs and ORCs in LIS, across deaf native and non-native signers, and hearing CODAs.
From a theoretical perspective, our results confirm the predictions of the Accessibility Hierarchy (Keenan & Comrie, Reference Keenan and Comrie1977) advanced for spoken languages, and provide evidence for the validity of structural accounts to explain the source of the asymmetry.
At the empirical level, the research offers new data on complex sentence processing in LIS, which raise interesting questions regarding potential differences holding across deaf and hearing populations of LIS signers. Whether these differences correlate with cognitive advantages associated with early and simultaneous bimodal bilingualism, with enhanced metalinguistic awareness fostered by education or with the existence of different grammatical strategies, is left for future studies.
Besides the novel contribution that these results add to the literature, our research highlights the importance of an early, accessible and simultaneous exposure to both the sign and spoken language as a crucial step to guarantee a solid linguistic competence both in hearing and deaf signers.
Data availability statement
Experimental and supplementary materials (including the questionnaires), as well as the data and statistical analyses of this study are openly available in the OSF repository at: https://osf.io/97fhn/.
Acknowledgements
This research has received funding from the European Union's Horizon 2020 Research and Innovation Programme under grant Agreement No. 693349, and from the Department of Excellence Development Project ‘Linguistic and Cultural Diversity, Multilingualism and Multiculturalism for the Wellbeing of Persons and Communities’ funded by the Italian Ministry of University and Research (2018–2022). Our deepest gratitude goes to our LIS consultants, Gabriele Caia and Mirko Pasquotto, for their availability to discussion and for contributing to the recording of the LIS stimuli. We are very grateful to all the participants who took part in the study and in the various pilots we ran. We thank Céline Pozniak for her advice regarding the design of the experiment. We also thank the audience of the following conferences for their comments and suggestions: Architectures and Mechanisms for Language Processing (AMLAP 2022); Bilingualism Matters Research Symposium (BMRS 2022); Incontro di Grammatica Generativa (IGG 48) and Formal and Experimental Advances in Sign Language Theory (FEAST 2023).
Author contributions
All authors conceived and designed the study. E. F. prepared the experimental materials and implemented the tasks. C. H. analysed the data. All authors discussed the results. E. F. and C. H. drafted the manuscript; E. F., C. B. and C. H. revised the first draft. All authors have approved the manuscript before submission.
Competing interests
None.
Ethical standards
The study received approval by the Ethics Committee of Ca' Foscari University of Venice. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Open access
