1. Introduction
Working toward accessible communication means aiming for information to be available to everyone in a way that it is ‘accurate, clear, direct, precise and easy to understand’ (Perego, Reference Perego2020, 21). Everyone can benefit from making standard communication more accessible. In that regard, one could opt to use Plain Language (less-simplified variant of easy-to-understand languages) or Easy Language (more-simplified) (Matamala, Reference Matamala2022) to convey information in a clearer way.
Easy Language (EL) is a ‘language variety in which a set of recommendations regarding wording, structure, design and evaluation are applied to make information accessible to people with reading comprehension difficulties for any reason’ as stated in the ISO standard dedicated to it (ISO, 2023, 2). If the standardized recommendations are followed and applied, the resulting EL text should deviate from standard use with linguistic elements related to content, vocabulary and structure (Bernabé, Reference Bernabé2020; Lindholm & Vanhatalo, Reference Lindholm and Vanhatalo2021; Nomura et al., Reference Nomura, Nielsen and Tronbacke2010) and with graphic elements, such as visual support, design or layout (Nomura et al., Reference Nomura, Nielsen and Tronbacke2010).
The present study, as outlined in Section 2, investigates the effects of incorporating visual support in texts, in line with what is recommended in Easy Language guidelines. First, the theoretical foundations and previous research on which this study builds will be reviewed (Sections 1.1–1.3). We then provide a detailed explanation of the study itself, including its objectives (Section 2) and methodology (Section 3). The findings are presented in Section 4, followed by a discussion in Section 5. Finally, the paper concludes with a summary of the main results and a reflection on its limitations.
1.1. The use of images with text
1.1.1. The multimedia principle
Cognitive psychology has shown that mental imagery, concreteness and verbal association play an important role in supporting learning across different domains (Clark & Paivio, Reference Clark and Paivio1991). Building on that foundation, Cognitive Load Theory (CLT) (Sweller et al., Reference Sweller, van Merrienboer and Paas1998) argues that working memory has a limited capacity and operates through partially independent systems for processing verbal/auditory information and visual/spatial material, whereas long-term memory is essentially unlimited and stores structured knowledge that can become increasingly automated.
Mayer’s Cognitive Theory of Multimedia Learning (CTML) (Reference Mayer2014) integrates these perspectives, proposing that learners gain more from words and pictures presented together than from words alone. CTML is based on three assumptions: (1) visual and auditory information are processed through separate channels (drawing on dual-coding theory), (2) each channel has limited capacity (consistent with CLT) and (3) learning is an active process of selecting, organizing and integrating new information with prior knowledge. The theory also outlines twelve design principles, including the multimedia principle itself, which emphasizes the use of both verbal (text or narration) and visual (images, animations or video) representations to promote deeper understanding by engaging multiple processing channels.
Based on this theory, empirical research has demonstrated that combining text with images generally leads to more effective learning than relying on text alone or pictures alone, a phenomenon often referred to as the multimedia principle (Anglin et al., Reference Anglin, Vaez, Cunningham and Jonassen2004; Butcher, Reference Butcher and Mayer2014; Mayer, Reference Mayer2014). However, not all images are equally beneficial. Research shows that while relevant visuals can significantly enhance comprehension, decorative or tangential images may have little effect or even hinder learning. For instance, Sung and Mayer (Reference Sung and Mayer2012) found that graphics often increased learners’ enjoyment of online lessons but did not consistently improve understanding, highlighting the potentially seductive effect of irrelevant visuals. Similar findings are reported in Levie and Lentz (Reference Levie and Lentz1982), Guo et al. (Reference Guo, McTigue, Matthews and Zimmer2020) and Beymer et al. (Reference Beymer, Orton, Russell, Baranauskas, Palanque, Abascal and Barbosa2007). A more detailed discussion of empirical studies in this area will be presented in Section 1.2.
1.1.2. Visual support in Easy Language texts
Visual support is mentioned in some way and has a prominent position in all 23 reviewed Easy Language guidelines in González-Sordé and Matamala (Reference González-Sordé and Matamala2023). Photographs, illustrations and symbols may accompany text ‘so that if a reader encounters an unknown word, the visual representation can be used to bootstrap understanding of the text, allowing the reader to obtain the intended meaning’ (Rivero-Contreras et al., Reference Rivero Contreras, Engelhardt and Saldaña Sage2023, 2). Nevertheless, while EL guidelines widely promote the use of images, there is not a unanimous guidance on how to put this into practice. In fact, recommendations from different guidelines can be contradictory. For example, guidelines by The Department of Health in the UK (2010) state that images should be placed on the left of the text, while the Social Care Institute for Excellence’s Accessibility Guidelines (2005) defend that they should appear on the right. Since both were published in the UK, differences cannot be linked to cultural preferences.
It is also unclear which type of visual support is best to use. For example, in García-Muñoz (Reference García Muñoz2012, 74), it is advised to ‘explore if the public prefers drawings [or] photography […] and keep the same style in all the text’ Nevertheless, these guidelines do not explain how to determine what the public prefers. On this, Sutherland and Isherwood (Reference Sutherland and Isherwood2016, 308) determined that ‘aside from the simplified written text, it is not clear from the current paucity of experimental research whether other aspects of [EL] such as symbols, pictures (line drawings), or photographs necessarily enhance understanding […]. Overall, there is little supporting evidence in the literature for the recommendations contained in many [EL] design guidelines’.
Although literature reviews show that there has been a rise of empirical studies on EL (Chinn & Homeyard, Reference Chinn and Homeyard2017, 1190; González-Sordé & Matamala, Reference González-Sordé and Matamala2023; Lindholm & Vanhatalo, Reference Lindholm and Vanhatalo2021, 15; Rivero-Contreras & Saldaña, Reference Rivero-Contreras, Saldaña, Díez Mediavilla and Gutiérrez Fresneda2020; Sutherland & Isherwood, Reference Sutherland and Isherwood2016, 297–298), it is observably necessary to foster research that evaluates the effectiveness of certain aspects of this language variety. In the following section, we will explore some findings that the present study premises on.
1.2. Research background
Studies collected through the review of recent literature in the area will be presented according to their results: (1) the inclusion of visual support showed positive effects, (2) negative effects or no effects.
Rivero-Contreras et al. (Reference Rivero-Contreras, Engelhardt and Saldaña2021, Reference Rivero Contreras, Engelhardt and Saldaña Sage2023) showed that visual support facilitated sentence processing for adults with and without dyslexia (Reference Rivero-Contreras, Engelhardt and Saldaña2021) and with different levels of education (Reference Rivero Contreras, Engelhardt and Saldaña Sage2023). In Schatz et al. (Reference Schatz, Haberstroh, Bindel, Oswald, Pantel, Paulitsch, Konopik and Knopf2017), neurotypical participants performed better when the EL version was accompanied by simple illustrations than when it was not. Jones et al. (Reference Jones, Long and Finlay2007) found that comprehension scores of participants with mild or borderline learning disabilities were significantly higher for the symbolized texts than the non-symbolized ones. Hibbing and Rankin-Erikson (Reference Hibbing and Rankin-Erickson2003) found that the addition of illustrations to text could enhance struggling middle-school readers’ comprehension. Levie and Lentz (Reference Levie and Lentz1982) reviewed 55 experimental studies and found that meaningful, well-integrated visuals consistently improve comprehension and recall compared to text found alone, and that they are especially beneficial to poorer readers. When it comes to studies on people with intellectual disabilities (ID), Wadihah and Fauzi (Reference Wadihah and Fauzi2021) found that the use of images accompanying text improved reading comprehension for intellectually disabled students. Buell (Reference Buell2017) tested adults with an ID and found evidence to demonstrate that images decreased cognitive effort.
Hurtado et al. (Reference Hurtado, Jones and Burniston2014) and Saletta et al. (Reference Saletta, Kaldenberg, Rivera and Wood2019), on the other hand, found negative and uncertain effects of visual support when it comes to reading comprehension. Saletta et al. (Reference Saletta, Kaldenberg, Rivera and Wood2019) found that adding images in EL texts did not improve reading comprehension for young adults with intellectual or developmental disabilities. They tested three types of images (photographs, line drawings and control images), all with the same outcome. And Hurtado et al. (Reference Hurtado, Jones and Burniston2014) demonstrated that under certain conditions, photographs can even increase cognitive effort for readers with an ID. Additionally, this last study found no significant differences in comprehension (which was assessed through a questionnaire) between participants when showed a ‘picture only’ and when showed an EL text with a picture.
Likewise, Dye et al. (Reference Dye, Hare and Hendy2004) observed that the inclusion of photographs in the questionnaire did not affect the ability of adults with learning disabilities to complete it. Yaneva et al. (Reference Yaneva, Temnikova and Mitkov2016) also found no effects in comprehension after showing neurotypical adults and participants with autism EL texts both with and without pictures (photographs and icons). Finally, Poncelas and Murphy (Reference Poncelas and Murphy2007) declare that the participants with an ID did not understand any of the versions (text-based and symbol-based) of the simplified manifesto they showed them.
All said, extensive research confirms that adding relevant visuals to text improves learning (Sung & Mayer, Reference Sung and Mayer2012) and comprehension (Beymer et al., Reference Beymer, Orton, Russell, Baranauskas, Palanque, Abascal and Barbosa2007; Guo et al., Reference Guo, McTigue, Matthews and Zimmer2020; Levie & Lentz, Reference Levie and Lentz1982), whereas decorative images provide little benefit and can even distract readers (Guo et al., Reference Guo, McTigue, Matthews and Zimmer2020; Levie & Lentz, Reference Levie and Lentz1982).
As shown, there are not enough studies performed in the area to be able to have conclusive findings regarding what type of images to use, or whether visual support should be considered an essential feature of Easy Language. As stated in Hurtado et al., ‘the generalised use of text and picture formats for all people with an ID in spite of the scant evidence supporting its effectiveness is concerning’ (Hurtado et al., Reference Hurtado, Jones and Burniston2014, 822). The use of eye tracking in EL research is growing (Borghardt et al., Reference Borghardt, Deilen, Fuchs, Gros, Hansen-Schirra, Nagels, Schiffl and Sommer2021; Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020), but it has still not been used to its full potential.
1.3. Monitoring eye movements
Eye tracking measures eye position, eye movement and pupil size to detect zones in which the user has a particular interest at a specific time (González-Sánchez et al., Reference Gonzalez-Sanchez, Baydogan, Chavez-Echeagaray, Atkinson, Burleson and Jeon2017). One of the basic output measures of interest is fixations, which are eye movements that stabilize the retina over a stationary object of interest and indicate that the participant focused their attention on it (Duchowski, Reference Duchowski2007, 46). For instance, when looking at text, the fixation position indicates which part of the sentence is currently being processed to a certain extent (Borghardt et al., Reference Borghardt, Deilen, Fuchs, Gros, Hansen-Schirra, Nagels, Schiffl and Sommer2021). In that regard, eye movements have been shown to be directly influenced by textual variables (e.g., ‘increased linguistic complexity leads to increased fixation duration and decreased saccade length’ [Borghardt et al., Reference Borghardt, Deilen, Fuchs, Gros, Hansen-Schirra, Nagels, Schiffl and Sommer2021, 4]) and provide information on different cognitive processes (e.g., Rayner, Reference Rayner1998; White et al., Reference White, Heck, Jubran, Chroust and Bhatt2022) (see Section 3.5). Nevertheless, eye movements alone do not provide the complete picture of text processing and reading comprehension that we aim to build (Boland, Reference Boland, Carreiras and Clifton2004). For that reason, it is advised to combine eye-tracking data with other data (Van Gog et al., Reference Van Gog, Ericsson and Rikers2005), as has been done in the present study with comprehension, perceived difficulty and preference questions.
For years, researchers have been using eye tracking to explore the processing of information in text and images or the integration of information in both elements, with a recurrent focus on education and learning. For example, Hannus and Hyönä (Reference Hannus and Hyönä1999) determined that children (10-year-olds of high and low intellectual ability) inspected the illustrations in science textbooks minimally and that learning was mainly driven by the text. Zhao et al. (Reference Zhao, Schnotz, Wagner and Gaschler2014) later found that, although text (rather than a picture) is more likely used to construct mental models in initial coherence-formation processing of text and picture, secondary students seemed to also rely on the picture to answer the question after having had a first contact with the material. Additionally, Krejtz et al. (Reference Krejtz, Duchowski, Krejtz, Kopacz and Chrząstowski-Wachtel2016) showed that interactive elements are optimal to spur reading, leading to a more complete visual inspection of the material, rather than static illustrations.
Eye tracking has also already been used to study EL (Rivero-Contreras et al., Reference Rivero Contreras, Engelhardt and Saldaña Sage2023; Rivero-Contreras et al., Reference Rivero-Contreras, Engelhardt and Saldaña2021; Denzen et al., Reference Denzen, Santibáñez, Moore, Foley, Gersten, Gurgol, Majhail, Spellecy, Horowitz and Murphy2012; Yaneva et al., Reference Yaneva, Temnikova and Mitkov2016), as the tracking of eye movements is a non-invasive method to assess online cognitive skills and processes involved in reading (Mézière et al., Reference Mézière, Yu, Reichle, der Malsburg and McArthur2023; Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006; Rayner et al., Reference Rayner, Pollatsek, Ashby and Clifton2011). The German research group ‘Simply complex – Easy Language’ stands out in the use of this technology in EL research (Deilen, Reference Deilen, Hansen-Schirra and Maaß2020; Schiffl, Reference Schiffl, Hansen-Schirra and Maaß2020). The findings of Rivero-Contreras et al. (Reference Rivero-Contreras, Engelhardt and Saldaña2021) and Yaneva et al. (Reference Yaneva, Temnikova and Mitkov2016), which examine visual support through eye-tracking methods, have already been discussed in Section 1.2.
It is important to bear in mind that opting for this technology for a study on EL can also present some challenges. Participants with an ID are among the primary users of EL and require researchers to carefully consider necessary adaptations in the experiment’s design and implementation, as well as the associated ethical constraints (Borghardt et al., Reference Borghardt, Deilen, Fuchs, Gros, Hansen-Schirra, Nagels, Schiffl and Sommer2021; Csákvári & Gyori, Reference Csákvári and Gyori2015; Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020). Listed below are some challenges researchers may face when performing eye-tracking tests on users with an ID, compiled by Deilen and Schiffl (Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020); with comments from the authors on how these were acknowledged or resolved in the present study:
-
(1) Participants with an ID may not be able to communicate their desires or comprehend given information as easily as neurotypical participants (Csakvari and Gyori, Reference Csákvári and Gyori2015; Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020). For this reason, we provided all written information in EL (including all data collection information and consents) and frequently asked the participant if they wanted to carry on with the test or if they had any doubts.
-
(2) Participants with an ID may find it hard to limit their movement throughout the experiment due to memory or executive control deficits (Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020). In our study, we identified that the participants with an ID often turned their heads when the researcher started talking. For this reason, we had to frequently remind them to stay still and later review the gaze plotsFootnote 1 in search of any issues that might invalidate the recording of eye movements. This led to the rejection of 2 recordings.
-
(3) Participants with an ID are more prone to present problems of vision or wear thick glasses that interfere with an optimal calibrationFootnote 2 process (Warburg, Reference Warburg2001; Csakvari and Gyori, Reference Csákvári and Gyori2015), an issue that still has no general solution (Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020). In our study, we identified that 2 participants with an ID wore thick glasses that deformed their eyes, and another user with an ID squinted their eyes while reading. For this reason, we asked participants if they had any problems of vision we should acknowledge at the beginning of the test (3 users with an ID said they did), and we searched for any issues in the gaze plots once the test was finished. This issue led to the rejection of 2 additional recordings.
2. Aims of the study
As argued in the previous section, the limited research on the impact of visual support on EL comprehension does not help determine whether visual support is beneficial for a user facing reading or comprehension barriers. In fact, many of the reviewed studies showed no beneficial effects of visual support for the reader (e.g., Dye et al., Reference Dye, Hare and Hendy2004; Poncelas & Murphy, Reference Poncelas and Murphy2007; Yaneva et al., Reference Yaneva, Temnikova and Mitkov2016), as did two studies that resembled the present one in terms of aims and method (Hurtado et al., Reference Hurtado, Jones and Burniston2014; Saletta et al., Reference Saletta, Kaldenberg, Rivera and Wood2019).
The contribution of the present paper is threefold. First, we examine how the type of visual support used (illustrations, photographs, or no image) affects content comprehension, perceived difficulty and preference. Second, we monitor the reading behavior with the means of eye tracking. Finally, we study the effects of visual support in EL comprehension both on a group of people with ID and on a group of neurotypical participants. Our hypotheses areas follows :
H1. Visual support of any type has significant effects on text comprehension for readers with intellectual disabilities. Easy Language guidelines (García-Muñoz, Reference García Muñoz2012; ISO, 2023; Nomura et al., Reference Nomura, Nielsen and Tronbacke2010; UNE, 2018) suggest that appropriate visual support facilitates text comprehension. As per previous research (Section 1.2), a significant difference is expected for individuals with ID (Buell, Reference Buell2017; Wadihah & Fauzi, Reference Wadihah and Fauzi2021).
H2. People with and without intellectual disabilities explore the page differently. Authors expect the group with an ID to make more and longer fixations; thus, also a longer fixation time. This group is also expected to perform shorter saccades (see Section 3.5 for a discussion on eye-tracking metrics and how they can reveal how readers engage with and process textual information).
Finally, a part of our study is exploratory due to the lack of previous consistent findings. For this reason, we also formulate two open research questions:
Q1. How does the presence and type of visual support influence perceived level of text difficulty?
Q2. How does the presence and type of visual support influence the way readers explore the page?
3. Method
We designed an eye-tracking study combined with three questionnaires on comprehension, perceived difficulty and preference of visual support. This is a mixed design study which presents an independent variable with 3 levels: (1) no visual support, (2) illustration and (3) photograph; and the two groups of participants (ID/neurotypical) as the between-subjects factor.
Previous to the study, we performed a pilot test with 6 participants (medium age of 28.5; 5 women and 1 man). These participants had no cognitive disabilities or reading difficulties and followed the full procedure of the test. This experience allowed the researchers to perform slight changes to the design based on their observations and the analysis of the results. We will now expose in detail the final test design and method.
3.1. Participants
Initially, we aimed for a total of 60 participants, 30 in each group, but some of the users with an ID had to be discarded due to eye-tracking calibration or data collection problems (see Section 1.3). The final sample includes 52 participants (23 men and 29 women; mean age of 39.9 [SD = 14.5]). 26 of them had ID, and 26 did not. Participants with an ID (12 men and 14 women; mean age of 44.4 [SD = 13.2]) were recruited by contacting two local non-profit organizations that work on watching over the quality of life and giving jobs to people with intellectual or developmental disabilities (Som – Fundació, Taller Jeroni de Moragas). Neurotypical participants (11 men and 15 women; mean age of 34.8 [SD = 11.5]) were recruited through word-of-mouth promotion. In all cases, participation was strictly voluntary, and no participant was reimbursed.
All participants were native speakers of Catalan with normal or corrected vision (the recordings of 2 out of the 3 participants who declared having eye problems had to be discarded). Participants were not asked to provide a formal diagnosis of ID. Since having an ID was already a requirement to receive support from the collaborating foundations, this was considered sufficient to meet the diagnostic criterion. To ensure consistency within the group, two screening tests were administered, confirming that none of the participants showed intelligence levels or reading comprehension skills that differed significantly from the rest. Refer to Section 4.1 for the participants’ results on the screening tests on (1) reading comprehension and (2) intelligence and abstract reasoning.
3.2. Materials
3.2.1. Evaluation instruments
We used two screening tests, one that assessed reading comprehension (Test de lectura comprensiva [TLC]) (Comes, Reference Comes1990) and one that evaluated intelligence and abstract reasoning (Raven’s Progressive Matrices [RPM]) (Raven, Reference Raven1998), to collect data on these aspects for each participant and group. TLC has been proven to be a fully valid initial assessment of text comprehension skills in readers with varying proficiency levels (Baez-Naghelli et al., Reference Baez-Naghelli, Águila, Nieves, Grande and Mena2015; Del Cueto et al., Reference Del Cueto, Conte, Parellada and Roldan2019; Ferreres et al., Reference Ferreres, Abusamra, Casajús, Cartoceti, Squillace and Sampedro2009). Similarly, RPM has demonstrated strong validity and reliability and is widely regarded as an appropriate measure of nonverbal and general intelligence (Kazem et al., Reference Kazem, Alzubiadi, Yousif, Aljamali, Al-Mashhdany, Alkharusi, Al-Busaidi, Alsarmi, Al-Bulushi, Al-Bahrani and Al-Fori2007; Queiroz-Garcia et al., Reference Queiroz-Garcia, Espirito Santo and Pires2021).
Screening tests were conducted prior to the eye-tracking assessment and took approximately one hour per participant. The TLC and RPM tests were administered in group sessions, where participants completed the assessments individually and autonomously, all within the same room. Responses were recorded on paper, and participants could request assistance from the researcher if needed. In contrast, questions assessing comprehension, perceived difficulty and preference were administered on a separate day during individual one-on-one sessions with the researcher, held in the eye-tracker room. During these sessions, participants provided their responses orally.
All comprehension questions were of a similar low difficulty and asked on explicit information in the text (Fajardo et al., Reference Fajardo, Ávila, Ferrer, Tavares, Gómez and Hernández2014) and on elements that appeared or were related to the image. See an example:
Perceived difficulty was assessed through a Likert scale (Yaneva et al., Reference Yaneva, Temnikova and Mitkov2016) after each page, which presented options from 1 to 4 to avoid a neutral answer. This measure reflects the subjective impressions of the participants on text difficulty.
Finally, at the end of the experiment, participants were asked an open question on their preference in terms of visual support (‘would you rather read plain text, a text with a photograph or a text with an illustration?’).
3.2.2. Stimuli
Stimuli shown in the experiment were based on an EL brochure-style publication on tourist attractions of the Catalan towns of Rupit and Pruit (Vidal, Reference Vidal2021). The texts were in EL, included no low-frequency words and had low text complexity. Researchers chose this brochure due to its adequacy for the purpose of the test: images in it share the same representative function (Carney & Levin, Reference Carney and Levin2002), which facilitated the task of formulating adequate and homogeneous comprehension questions. The content in the original brochure was complemented with other information about the described locations from different tourist guides, and it was slightly adapted so all the pages shown (6 in total) (1) had the same layout, (2) had about the same length (between 65 and 70 words) and (3) presented a similar number of ideas. The Fernández Huerta readability indexFootnote 3 for all 6 texts was between 90.12 and 102.70, in which results over 90 are labelled as ‘very easy’ for adult readers. Sentences were presented in black 14-point Arial font on a white background, and paragraphs were double-spaced.
Each of the 6 texts had 3 different presentations, corresponding to the 3 levels of the independent variable: no visual support, photograph and illustration. All participants read the same six texts in the same order, but with different conditions regarding visual support. This was achieved through a Latin square array. For the sake of minimizing the possible influence of extraneous factors, a set of stimuli was arbitrarily assigned to each participant with only one criterion: making sure that in the end, the same number of participants would have read through each set of stimuli.
When it comes to images, stimuli presented (1) the original photographs used in the brochure and (2) illustration-style copies of them, created with digital illustration tools. All images were representational (Carney & Levin, Reference Carney and Levin2002) and relevant to the content in the text. Pages had a size of 1230 × 819. On the pages with visual support, a 520 × 390 (4:3) image appeared in the top left corner, as this is their usual placement in Spanish and Catalan EL texts (Plena Inclusión, 2021). Photographs and illustrations were in color and of high resolution. The right half of the page showed the text, which, as stated before, was always between 65 and 70 words in length and was presented in 10 lines. On pages where no visual support was provided, the text was centered on the screen. See Table 1 for an example.
Different versions of a stimulus
It is important to consider that a group of people belonging to the EL target group validated both the original text and the supporting images and considered them adequate for an EL adaptation before publication. Validation with the target group is strongly recommended to publish under the ‘Easy Language’ category. At Plena Inclusión, an advocacy movement that fights for the rights of people with intellectual and developmental disabilities in Spain, they validate images by asking the participants ‘what do you see in the image?’ and ‘how does the image relate to the text?’. If an issue arises, they ask the follow-up questions ‘how should the image be to explain this topic well?’ and ‘what do we do to ensure the image is properly related to the text?’ (Plena Inclusión, 2021).
3.3. Apparatus
Eye movements were recorded with a Tobii T60 (sampling rate of 60 Hz) in a desk monitor setup. Head movements were not restrained, although participants were given instructions to stay still while performing the test (as discussed in Section 1.3, this was hard for some participants with an ID). Viewing was binocular, and the movements of both eyes were recorded. Participants were seated approximately 60 cm away from the 43 cm display size monitor with a screen resolution of 1280 × 1024 pixels. The Tobii T60 eye tracker and the stimuli were controlled through a laptop. A different laptop was used to record the participant’s answers and comments in a digital document.
3.4. Procedure
Participants first signed the written informed consent and demographics form. They then performed the TLC and RPM tests in a group session. Later, they proceeded with the eye-tracking experiment in a separate room, with the only company of the observing researcher. For participants with an ID, the two sessions were conducted on different days to reduce the risk of fatigue (Deilen & Schiffl, Reference Deilen, Schiffl, Hansen-Schirra and Maaß2020) and to allow extra time for explanation and assistance. In contrast, the sessions for neurotypical participants were conducted on the same day. The eye-tracking test lasted around 40 minutes and for each participant we retrieved a maximum of 25 minutes of recorded eye movements (around 4 minutes per page of stimuli). The recording was paused while the participants answered the comprehension, perceived difficulty and preference questions, which explains the difference in these times.
The researcher always explained the procedure and calibrated the eye tracker before starting this last test. Then the participant read the first page of stimuli describing a tourist attraction on Rupit and Pruit and answered two multiple-answer comprehension questions orally (see the stimuli explained in Section 3.2.2). The researcher switched to the next screen once the participant told them they had finished reading. They could read the text again if they needed to, once the questions were asked, since the aim was not to measure short-term memory or recall. As mentioned, the eye tracker only recorded eye movements performed while the participants were reading the text prior to the posing of the question. Participants were also asked about their perceived difficulty of the text, which they also answered orally. This process was repeated until readers had read through all 6 pages of the stimuli.
The study protocol (CEEAH CA40) was approved by the Research Ethics Committee of the Universitat Autònoma de Barcelona in February 2022.
3.5. Eye-tracking measures
Eye-tracking measures can be categorized as local or global. Global measures are aggregated over broader regions, such as sentences or multiple sentences, and provide information on overall reading behavior, including individual differences between groups with varying reading skills (Mézière et al., Reference Mézière, Yu, Reichle, der Malsburg and McArthur2023, 428). For example, Rayner et al. (Reference Rayner, Chace, Slattery and Ashby2006) showed that reading more difficult texts results in longer average fixation durations. Similarly, Reichle et al. (Reference Reichle, Liversedge, Drieghe, Blythe, Joseph, White and Rayner2013) and Blythe and Joseph (Reference Blythe, Joseph, Liversedge, Gilchrist and Everling2012) found that less skilled readers tend to make more and longer fixations than skilled readers, with longer fixations reflecting prolonged processing times or processing difficulties.
Recent studies have also demonstrated that global eye-tracking measures can predict reading comprehension. Southwell et al. (Reference Southwell, Gregg, Bixler and D’Mello2020) had participants read long passages silently and then answer multiple-choice questions. They found that eye movements predicted comprehension scores with correlations ranging from 0.35 to 0.40, with more fixations and shorter fixation durations being associated with better performance. D’Mello et al. (Reference D’Mello, Southwell and Gregg2020) reported similar findings, further highlighting the predictive value of global measures.
In contrast, local measures provide more detailed insights into the specific cognitive processes affected by reading ability (Mézière et al., Reference Mézière, Yu, Reichle, der Malsburg and McArthur2023). However, because most research has emphasized global measures, it remains unclear whether local eye movements can reliably predict reading comprehension accuracy.
Given this background, the present study will focus on the global measures listed below, as they have been more consistently linked to reading comprehension outcomes and offer a robust starting point for examining individual differences in reading behavior.
-
1. Fixation count: the sum of the number of fixations (defined in Section 1.3) in an area (Clifton et al., Reference Clifton, Staub, Rayner, van Gompel, Fischer, Murray and Hill2007; Duchowski, Reference Duchowski2007).
Fixation count is an indicator of visual processing of that area or stimuli (e.g., the image, the text) (Krejtz et al., Reference Krejtz, Duchowski, Krejtz, Kopacz and Chrząstowski-Wachtel2016). More (and shorter) fixations may reflect attentive reading (Faber et al., Reference Faber, Bixler and D’Mello2018; Southwell et al., Reference Southwell, Gregg, Bixler and D’Mello2020) as the number of fixations tends to increase when the text is difficult (Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006) and more fixations are shown to be linked to help memory performance (Fehlmann et al., Reference Fehlmann, Coynel, Schicktanz, Milnik, Gschwind, Hofmann, Papassotiropoulos and de Quervain2020). Orduna-Hospital et al. (Reference Orduna-Hospital, Hernández-Aranda and Sanchez-Cano2023) showed that people with an ID tended to exhibit a higher number of fixations.
-
2. Average fixation duration: the average duration of a single fixation (White et al., Reference White, Heck, Jubran, Chroust and Bhatt2022).
Longer fixations are associated with reading difficulty, and poor readers tend to exhibit them (Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006) while shorter fixations are associated with better comprehension (Copeland & Gedeon, Reference Copeland and Gedeon2013; Southwell et al., Reference Southwell, Gregg, Bixler and D’Mello2020). The slower the cognitive processing, the longer the fixation (He et al., Reference He, Garrido, Sowman, Brock and Johnson2015). Studies suggest that unskilled readers (such as readers with an ID or learning disabilities) will have longer fixation duration (Everatt & Underwood, Reference Everatt and Underwood1994; Rivero-Contreras, Reference Rivero Contreras, Engelhardt and Saldaña Sage2023).
-
3. Total fixation time: the sum of all single fixations on a certain area (Clifton et al., Reference Clifton, Staub, Rayner, van Gompel, Fischer, Murray and Hill2007; Duchowski, Reference Duchowski2007).
It reflects encoding processes (Child et al., Reference Child, Oakhill and Garnham2020) and is an indicator of lexical access and integration (Inhoff & Rayner, Reference Inhoff and Rayner1986; Morton, Reference Morton1969; Whaley, Reference Whaley1978). A longer total fixation time is considered an indicator of a deeper and more effortful processing of visual information (Just & Carpenter, Reference Just and Carpenter1980; Krejtz et al., Reference Krejtz, Duchowski, Krejtz, Kopacz and Chrząstowski-Wachtel2016).
-
4. Average saccade amplitude: the average distance travelled by the eye between two fixation points (Paeye & Madelain, Reference Paeye and Madelain2011, 149).
A pattern of long saccades indicates greater exploration (Wang & Sparks, Reference Wang and Sparks2016). Poor readers tend to exhibit shorter saccades (Rayner et al., Reference Rayner, Chace, Slattery and Ashby2006). Orduna-Hospital et al. (Reference Orduna-Hospital, Hernández-Aranda and Sanchez-Cano2023) showed that people with an ID exhibited a higher number of saccades. Subjects without ID showed faster saccades and with a higher amplitude than those with ID.
Finally, areas of interest (AOIs) are an analytical tool that allows for calculating these measures and receiving quantitative, relevant metrics. Put simply, AOIs are the boundaries drawn around an area of each page of stimuli that fall into different categories. In the present study, researchers determined three types of AOIs: image, text and whitespace (see Image 1).
Groups of AOIs in a page of stimuli.
4. Results
To examine differences in the screening tests, comprehension, perceived difficulty and style preference, we have run an analogous 3-way analysis of variance and a series of independent t-tests. These results, as well as the ones from the participant’s eye movements, will be presented in this section. We used R for the statistical analysis. Results will be discussed in Section 5.
4.1. Screening tests: TLC and RPM
There were no intra-group outliers in the results from the screening tests. Participants with an ID got on average 33.7% of the answers correct in the TLC test on reading comprehension (SE = 0.829), whereas the neurotypical group scored 85.8% (SE = 0.443). The difference was significant, t(52) = 17.198, p < .001.
In regard to the RPM test on abstract reasoning and intelligence, the ID group got 62.9% of the questions right (SE = 1.604), whereas the neurotypical group obtained 95.1% of accurate answers (SE = 0.374). The difference was also significant (t(52) = 6.54, p < .001) (Tables 2 and 3).
T-Test results from the screening tests
Accuracy results from the screening tests
4.2. Comprehension, perceived difficulty and preference questions
We performed basic percentage and statistical calculus from the participants’ answers to the comprehension, perceived difficulty and preference questions and a Mann–Whitney U test to compare the tendencies for both groups. Both the comprehension and perceived difficulty questions showed relevant between-group differences. The question on the page layout preference, on the other hand, showed no between-group relevance (Table 4).
T-Test results from the comprehension, perceived difficulty and preference questions
Regarding the perceived difficulty of the texts, interestingly, the group with an ID gave a mean of 1.69 out of 4 on difficulty for all 3 types of visual support, showing their rating was independent of the variable. The neurotypical group rated 1.23 out of 4 in difficulty for those texts accompanied by photographs (p = .002) or illustrations (p < .001), and a slightly lower rate of 1.07 on average for those with no visual support (p = .002) and showing a relevant difference between groups for all three types of visual support (p < .001) (Table 5).
Perceived difficulty out of 4
Finally, when analyzing the answers to the comprehension questions, we can see a relevant between-group difference and a non-relevant within-group difference among participants. The inclusion of visual support is non-relevant for any of the groups (ID: p = .34; neurotypical: p = 1). Neither does the type of visual support show effects in the accuracy of the participants (ID: p = 1; neurotypical: p = .48), while the between-group difference is shown to be relevant (p < .001). The participants with an ID answered on average 55.4% of the questions correctly, against the 95.1% of the neurotypical group. These results are portrayed in Table 6.
Correct answers to comprehension questions out of 104
Finally, most participants (53.8%) showed no clear preference when it came to the type of visual support in the text. Among those who did show a preference, the majority of participants preferred the use of pictures (34.6% of ID, 26.9% of neurotypical). While the preference of no visual support is anecdotal and very low for both groups (only one participant and 3.8% in both groups), the between-group difference is only bigger when it comes to the preference of illustrations: only one participant (3.8%) chose this typology in the group with an ID, versus the 19.2% of participants in the neurotypical group. Nevertheless, between-group differences show no relevance (p = 1). These results are displayed in Table 7.
Visual support preference
4.3. Differences in eye movement characteristics
To verify whether there were differences in reading depending on the characteristics of the stimuli and the participant groups, as per our initial hypotheses, we ran a 3-way mixed design ANOVA with group (neurotypical versus ID) as the between-subjects variable, areas of the page (whitespace versus image versus text) and experimental condition (photograph versus illustration) as within-subjects independent variables.
Before conducting statistical tests, outlying fixations in terms of duration were identified using the standard ±1.58 IQR/sqrt(n) criterion (Chambers et al., Reference Chambers, Cleveland, Kleiner and Tukey1983). Fixations exceeding 1032 ms were classified as outliers, accounting for 11.79% of cases. Prior to analysis, these outliers were replaced with the largest non-outlying fixation duration (1031 ms), following a procedure known as winterization (Kuwak & Kim, Reference Kwak and Kim2017). A similar approach was applied to outliers in saccadic amplitude (11.17%).
4.3.1. Fixation count
The analysis on the fixation count yielded a significant difference between groups; individuals with an ID exhibited a higher number of fixations (M = 10.82, SE = .956) than the cognitively neurotypical (M = 5.63, SE = .996), (F (1, 46) < .001, p < .001, eta2 = .105).
The difference in the number of fixations for different types of images was not significant (F (1, 46) = .2, p = .660, eta2 < .001), nor was the interaction between group and type of image (F (1, 46) = .07, p = .793, eta2 < .001), showing that both groups had a similar number of fixations on the two types of images (illustrations and photographs).
The analysis also revealed significant differences in fixation count between areas of the page, (F (1.23, 56.54) = 27.32, p < .001, eta2 = .213), see Figure 1. Post hoc comparisons indicated that both groups had the highest number of fixations on whitespace (M = 12.89, SE = 1.65), followed by text (M = 8.51, SE = .59) and image (M = 3.25, SE = .231).
The interaction between participant group and AOI group on fixation count.
Interestingly, the interaction effect between participant group and areas of the page was significant, (F (1.23, 56.54) = .007, p = .007, eta2 = .065), see Figure 1. Post hoc comparisons indicated that participants with an ID had a significantly higher number of fixations on whitespace (M = 18.17, SE = 2.286) than the neurotypical participants (M = 7.62, SE = 2.383), (t(46) = 3.195, p < .001). They also looked significantly more at text (M = 10.55, SE = .827) than the neurotypical group (M = 6.48, SE = .862), (t(46) = 3.413, p = .001). The difference between groups was smallest for the image, yet still significant (t(46) = 2.050, p = .046). Participants with an ID exhibited more fixations (M = 3.73, SE = .319) than the neurotypical participants (M = 2.78, SE = .333).
4.3.2. Average fixation duration
Participants with an ID also showed significantly longer average fixations (M = 308, SE = 11.3) than the users in the neurotypical group (M = 266, 11.8), (F (1, 46) = .006, p = .012, eta2 = .066).
The analysis revealed a significant interaction effect between the group and the area of the page (F (1.44, 66.24) < .001, p < .001), see Figure 2. Post hoc comparisons suggested that the longest fixations of the group with an ID were on the text (M = 358, SE = 14.4), while their fixations on whitespace (M = 293, SE = 15.1) and the image (M = 274, SE = 12.7) were much shorter. Interestingly, neurotypical users show the longest fixations on images (M = 291, SE = 13.2), while their fixations on text (M = 261, SE = 15.1) and white space (M = 245, SE = 15.8) are shorter (F (1.44, 66.24) < .001, p < .001). Interestingly, the relevance of differences on average fixation duration between areas of the page for one group is opposed to the other: for the group with an ID, we only see a relevant difference between image and text (p < .001) and image and whitespace (p < .001), but not between image and whitespace (p = .5). On the contrary, the neurotypical group only show relevant differences in their average fixation duration between image and whitespace (p = .02), but not for the other two pairs of AOIs (image-text (p = .2), text-whitespace (p = .22)).
The interaction between participant group and AOI group on average fixation duration.
With this, there is a highly significant difference between the duration of the fixations on the text (t(46) = 4.635, p < .001) and on whitespace (t(46) = 2.181, p = .034) between the two groups, being much higher in both cases for the participants with an ID. This difference is not observed for images (t(46) = −.905, p = .37).
There was also a significant interaction between type of image and page area, (F(1.66, 76.42) = .039, p = .031), see Figure 3. Both groups performed longer fixations on the image when it was a photograph (M = 303, SE = 12.1) than when it was an illustration (M = 261, SE = 10.8), (t(46) = 3.035, p = .039), whereas type of image did not affect the average fixation durations on text (t(46) = 1.215, p = .231) and white space (t(46) = .262, p = .795).
The interaction between experimental condition and AOI group on average fixation duration.
4.3.3. Total fixation time
The difference in the total fixation time (tft) depending on the image type was not significant (F(1, 46) = .15, p = .696, eta2 = < .001). However, we observed a significant main effect of the page areas. Both groups showed a significantly higher tft on whitespace (M = 3626, SE = 496) and text (M = 2745, SE = 253) than on images (M = 914, SE = 70), (F (1.44, 66.31) < .001, p < .001, eta2 = .189) (see Figure 4).
The interaction between participant group and AOI group on total fixation time.
The participants with an ID fixated on whitespace for longer (M = 5406, SE = 686) than the neurotypical group (M = 1847, SE = 715), (t(46) = 3.592, p < .001). The same happens with text, where the group with an ID fixated for more time (M = 3818, SE = 350) than the neurotypical group (M = 1673, SE = 365), (t(46) = 4.243, p < .001). However, there was not a significant effect on total fixation time of the interaction between image type and participant group (F (1, 46) = .04, p = .846, eta2 < .001), and the two groups did not differ significantly in the time they looked at the images (t(46) = 1.292, p = .203).
4.3.4. Saccadic amplitude
Regarding saccadic amplitude, we observed a significant interaction effect between group and page area, (F(1.29, 59.35) = .04, p = .022, eta2 = .044). We see a significant between-group difference in the saccadic amplitude on image, with smaller saccades for the group with an ID (M = 5.77, SE = .692) than for the neurotypical group (M = 8.45, SE = .722), (t(46) = 2.680, p = .01, see Figure 5).
The interaction between participant group and AOI group on average saccade amplitude.
Analogous comparisons for text (t(46) = .239, p = .8121) and white space (t(46) = −.243, p = .8089) were not significant. In addition, participants performed smaller saccades on the photographs (M = 4.72, SE = .244) than on the illustrations (M = 5.48, SE = .244), (F(1, 46) = 6.47, p = .014, eta2 = .016). This effect of image type was quantified by group (F(1, 46) = .372, p = .06, eta2 = .016). The neurotypical group showed significantly bigger saccades for illustrations (M = 6.22, SE = .353) than photographs (M = 4.88, SE = .353), (t(46) = 3.100, p = .003), whereas the group with an ID showed small saccades for both types of images, with non-relevant differences (t(46) = .444, p = .6591). These results also show a significant between-group difference in saccadic amplitude for illustrations, with the neurotypical participants performing significantly bigger saccades (M = 6.22, SE = .353) than the participants with an ID (M = 4.74, SE = .338), (t(46) = 3.030, p = .004). This difference was not significant for saccades on photographs, which were small for both groups (t(46) = .5) (see Figure 6).
The interaction between participant group and experimental condition on average saccade amplitude.
5. Discussion
We will first review the hypotheses and contrast them with the results of our study (findings reject H1 and confirm H2). We will also answer the exploratory research questions Q1 and Q2. Later on, we will discuss other interesting findings outside the scope of our initial hypotheses.
H1. Visual support of any type has significant effects on text comprehension for readers with intellectual disabilities. Visual support inclusion was not related to accuracy in comprehension questions in any of the groups of participants. The type of visual support also showed no effects on reading comprehension. Therefore, we found no evidence of differential influence of photographs or illustrations helping in the comprehension of the text that they accompany.
H2. People with and without intellectual disabilities explore the page differently. The group with an ID was expected to make more and longer fixations, show a longer fixation time and perform shorter saccades. The total fixation time of the participants with an ID on the text was also higher than those in the neurotypical group, probably indicating a higher difficulty to decode the text or interpret its content (Rayner, Reference Rayner1998). The same happens with the average fixation duration: the group with an ID shows much longer fixations on text and whitespace, indicating a higher cognitive effort when reading (Southwell et al., Reference Southwell, Gregg, Bixler and D’Mello2020). Differently, the neurotypical group shows shorter fixations overall, with their longest ones being on the image (opposite to the results of the group with an ID), arguably because they may find more interest in exploring that area.
Regarding saccades, the neurotypical participants performed longer ones on the image area, possibly due to them searching for items or details in the scene. Saccades were also longer when the image was an illustration, showing extensive eye movements around the abstract picture. The overall smaller saccades of the group with an ID show that they were focusing on areas of interest on the page with more and longer fixations, rather than scanning and exploring around it.
Q1. How does the presence and type of visual support influence perceived level of text difficulty? There were no significant differences on the perceived difficulty of the text that could be linked to the inclusion or type of visual support.
Q2. How does the presence and type of visual support influence the readers’ eye movements? We found that saccades were longer on illustrations than on photographs, a difference that was especially significant in the neurotypical group. This group showed extensive eye movements around the abstract picture.
Having assessed the initially posed hypotheses and questions, it is also very interesting to see that the participants, especially those on the group with an ID, had the highest number of fixations and average total fixation time on whitespace. We suggest that this may have occurred because the texts shown lengthened between 65 and 70 words rather than being of sentence-length, and the participants might have needed to rest their gaze on the whitespace briefly; although it may also reflect a difference in how the two groups scan the scene and integrate the different areas of the page (Skversky-Blocq et al., Reference Skversky-Blocq, Shmuel, Cohen and Shechner2022).
Other aspects worth discussing would be the between-group difference on the answers on perceived difficulty, style preference and comprehension questions. There were no relevant differences on the perceived difficulty of the text that could be linked to the participant groups. Regarding preference on the inclusion and type of visual support in the page, most participants showed no preference or would rather not communicate it. Photographs were the preferred style for both groups, but it is very interesting to discuss the number of participants that chose illustrations, which is much higher in the neurotypical group (19.2%) than in the group with an ID (3.8%). This may possibly be due to participants with an ID being more aware and averse to infantile styles of visual support, as some participants in the group communicated to the researcher. Lastly, results make it clear that the comprehension questions were easy for the neurotypical group (95.1% of correct answers) and quite difficult to answer for the participants with an ID (55.4% of accuracy). It is hard to determine whether these results are fully due to the participants with an ID being unable to comprehend the needed information in the text, but it probably indicates a difference in the reading comprehension capacities of each group.
6. Conclusions
We were able to assess our initial hypotheses and shed light on the more exploratory aspects analyzed. On the one hand, contrary to H1, visual support was not related to accuracy in comprehension questions. This questions whether this should be such an established aspect of EL texts. On the other hand, H2 was confirmed, indicating that global eye-tracking measures are a good tool to predict the cognitive effort of processing different types of texts and that, although participants with an ID might need some adaptations in the test design and setting, it is possible and advisable to use this methodology with EL primary groups.
Additionally, there was no effect of the group or visual support on how difficult the participants perceived the text. In this regard, the way participants with an ID perceive the text in terms of difficulty does not match their lower results on comprehension, showing a misperception of their reading abilities or a reluctance to verbalize their struggles. Notably, neurotypical participants showed a higher preference for illustrations than that of participants with an ID, possibly reflecting a desire of the latter for less infantilizing support.
On this basis, the present study makes a meaningful contribution to the limited empirical evidence directly examining EL recommendations and offers valuable insights into the relationship among visual support, eye-tracking measures and reading comprehension. Nevertheless, it is important to acknowledge its limitations and suggest directions for future research.
6.1. Limitations and future research directions
First, regarding the eye-tracking measures used in the present study, global measures are not very informative about the specific cognitive processes affected by reading ability (Mézière et al., Reference Mézière, Yu, Reichle, der Malsburg and McArthur2023, 428). For example, we do not know whether the differences between groups in processing times are due to linguistic (e.g., lexical access) or non-linguistic (e.g., working memory) processes. Therefore, it would be valuable for future studies to assess local measures (although since most studies have focused on global measures, it remains uncertain whether local eye movements can also predict reading comprehension accuracy). Second, it would be valuable to examine within-group differences in comprehension by presenting participants with the same information in both standard language and EL, incorporating this as a within-group variable. Third, while we aimed for a larger sample, our final participant pool included 52 individuals (26 per group). Given the limited research involving adults with an ID in this area, we encourage future studies to replicate our findings or conduct similar investigations with larger samples.
We also observed unexpected results outside our original hypotheses that merit further study. For instance, the group with an ID reacted negatively to illustrations, while these were much more appreciated by the neurotypical group. In addition, many linguistic and graphical aspects of EL remain unexplored and should be addressed in future work. Validating EL adaptations empirically is key to ensuring their effectiveness and improving accessible communication.
Our study made a significant contribution to the limited empirical evidence directly assessing EL recommendations, although it is essential that future research continues to evaluate their validity and applicability.
Data availability statement
The data and analysis scripts related to the reported results are available on OSF: https://osf.io/ecwkx/?view_only=ba4285b1ccb44fb19ceb2c0f7bcc9ede
Acknowledgements
The authors thank the organizations SOM Fundació (Barcelona) and Taller Jeroni de Moragas (Valldoreix) for their collaboration in the study by facilitating contact with participants and providing a space to perform the experiment.
Funding statement
This work is part of Mariona González-Sordé’s PhD in Translation and Intercultural Studies at the Department of Translation and Interpreting and East Asia Studies of the Universitat Autònoma de Barcelona. The publication is partially based upon work performed within the COST Action LEAD-ME (CA19142), supported by COST (European Cooperation in Science and Technology) and within the EU-funded Innovation Action (IA) MediaVerse project (H2020-EU.2.1.1 programme). MediaVerse has received funding from the Horizon 2020 Research and Innovation Programme of the European Union, under grant agreement number 957252.
Competing interests
González-Sordé and Soler-Vilageliu are members of TransMedia Catalonia, a research group funded by the Catalan Ministry of Research and Universities (2021 SGR 00077).
Open access
