To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
This chapter focuses on learner translanguaging, analysing its characteristics, the factors that influence these characteristics, key functions of learner translanguaging and suggestions for facilitating learner translanguaging in class. Six characteristics of learner translanguaging are identified; it is agentive, purposeful, spontaneous and emergent, situated and contextual, identity-reflective and multimodal. Its manifestation is dependent on key influencing factors, including the classroom environment, aspects of learner identity, peer interaction and group dynamics, the type of activity involved, learner language proficiency and repertoire, and also the emotional and cognitive load experienced by learners at different points in the learning process. The chapter then introduces a project conducted by a Bolivian teacher to facilitate learner translanguaging, which then helps us, alongside other examples from research, to understand key functions of learner translanguaging, including when interacting with peers, interacting with the teacher and interacting with oneself. The chapter concludes by presenting a framework for facilitating learner translanguaging involving twelve recommendations.
This chapter explores multilingual communication, both in wider society and in educational contexts. It discusses the term ‘translanguaging’ in detail, providing and explaining useful definitions of the term. The chapter also identifies and unpacks key terminology that is often used within translanguaging theory and pedagogy. It then goes on to examine real examples from research on translanguaging, including in wider society, in the family and in the classroom. The chapter reveals that translanguaging is a natural part of who we are and what we do in our lives as bi/multilingual citizens. There is also time to explore the difference between translanguaging and terms such as ‘codeswitching’ and ‘using the L1’, explaining why we prefer to write and think about translanguaging. Finally, the chapter concludes with a historical discussion of how and why countries and education systems have tended to promote monolingual practices, particularly in classrooms, and the challenges that this presents for us as teachers interested in adopting more multilingual practices in our teaching.
Spoken discourse is a less commonly explored domain of corpus-based research. This chapter explores these challenges, as well as providing an overview of current corpus research on spoken discourse and emerging research areas. It concludes with a case study illustrating methods for these emerging research areas. The chapter highlights newer areas of corpus-based research such as fluency, prosody, and multimodal studies, language variation and change, and discourse-level units and includes an empirical study that illustrates two of these research areas through the use of fluency, prosody, and multimodal features across nurse–patient interactions. Rather than reporting on the interactions as a whole, the analysis also shows frequency of occurrence across discourse-level units: opening the visit, establishing the patient’s chief complaint, examination of the patient (both physical and patient history), counseling the patient (providing care options), and closing the interaction. The data comes from the US Nurse–Standardized Patient Corpus (Staples, 2015), and includes fifty interactions between nurses and standardized patients, who are actors trained to present the same case to multiple providers. The study demonstrates some of the more challenging elements of spoken discourse analysis and offers insights for the analysis of spoken discourse, particularly in these emerging research areas.
This chapter provides an overview of multimodal corpora, highlighting their significance in analyzing communication that integrates various modes such as speech, gesture, and gaze. The chapter defines both a mode and a multimodal corpus before introducing key types of multimodal corpora, including video-recorded interactions, corpora of signed languages, and datasets combining text and images. Prominent examples of multimodal corpora such as the ViMELF, FreMIC, and AMI corpora showcase the variety of multimodal corpora that exist and the various techniques employed to construct multimodal corpora. Challenges in constructing and annotating multimodal corpora are discussed, including temporal alignment, manual annotation efforts, and data-sharing limitations due to privacy concerns. The chapter emphasizes the importance of tailoring multimodal corpora to specific research questions while acknowledging the trade-offs of specialization. Advancements in technology, such as automated tagging and collaborative tools, are proposed as potential solutions to enhance accessibility and interoperability. A case study on turn management in virtual meetings illustrates the application of multimodal analysis, offering insights into how participants request and allocate turns in mediated contexts. The chapter serves as a guide for researchers navigating the methodological complexities of multimodal corpus-based studies.
This chapter contains an overview of William Burroughs’ collaborative work over the course of a half-decade career, including literary collaborations (with Kells Elvins, Jack Kerouac, Allen Ginsberg, Brion Gysin, and James Grauerholz) and interactions with filmmakers, musicians, and artists. The discussion is situated within the context of other literary collaborations, considerations of geography (who meets whom and where), mutual affinities that inspire and sustain joint projects, and contemporary theoretical work on authorship (e.g., Barthes, Foucault, Derrida, Deleuze and Guattari). In so doing, the chapter invites a reconsideration of notions of single authorship, and a fresh, expanded perspective on Burroughs’ oeuvre. Ultimately, the chapter reflects on the essential ingredients for successful collaborations, as well as various perils and challenges that accompany any collaborative endeavor.
Recent advances in visual-language models (VLMs) have led to significant improvements in a plethora of complex multimodal tasks like image captioning, report generation, and visual perception. However, generating text from meteorological data is highly challenging because the atmosphere is a chaotic system that is rapidly changing at various spatial and temporal scales. Given the complexity of atmospheric phenomena, it is critical to verifiably quantify the effectiveness of existing VLMs on weather forecasting data. In this work, we present SynopticBench, a high-quality dataset consisting of 1,367,041 text samples of Area Forecast Discussions created by the National Weather Service over the continental United States paired to images of 500 mb geopotential height, 2 m temperature, and 850 mb wind velocity in weather forecasts. We also present Synoptic Phenomena Alignment and Coverage Evaluation (Space), a novel evaluation framework that can be used to effectively estimate the quality of text descriptions of synoptic weather phenomena. Extensive experiments on generating forecast discussions using state-of-the-art VLMs show the sensitivity of existing evaluation metrics in this domain and enable further exploration into synoptic weather and climate text generation.
This special issue pursues a social semiotic study of improvisation. The approach considers the phenomenon both as a constitutive dimension of action and as a socially recognizable achievement. Contributions share a common focus on interaction, which is analyzed across multiple modalities including virtual reality, heavy machinery, paint and canvas, rock, theater, war, and the ethical relation between self and other.
This roundtable discussion convenes contributors to this special issue for reflections on the diversity of their research questions, approaches, and findings, as well as avenues for further inquiry. The discussion touches on several shared concerns, including the role of lexicalization in improvised forms becoming intelligible, enregistered, and reusable; how improvisational action might implicate action below the threshold of awareness; and more generally the relation between sign-processes and embodied action. In doing so, this roundtable discussion considers the importance of social semiotic analysis for understanding improvisation.
What should a nurse do when non-speakers of the local language come to the ward seeking information about a loved one? What should a receptionist do when they need to book an appointment and a language barrier takes them by surprise? How can an emergency call handler let a caller know that a human interpreter is being contacted? Chapter 3 examines circumstances in which the risks of multilingual AI are ostensibly low. It proposes a distinction between ancillary and core communication but argues that communicative settings are fluid. What starts as ancillary communication can easily turn into core care, so risk is not associated with specific roles or with levels of professional seniority. The chapter argues that, in the sectors under analysis, communication is rarely risk-free. Even where machine translation may not directly lead to harm or loss of life, it may be a feature of complex communicative environments which pose significant systemic risks.
The objective of this special issue is to present innovative research demonstrating that prosody needs to be reconceptualized as an inherently multimodal phenomenon, manifested across the spoken and/or visual domains. The studies included are organized into three core themes. Theme 1 addresses the temporal alignment of spoken and visual aspects of prosody, and how this is shaped by linguistic factors, speaker-specific traits (such as neurodiversity) and language learning patterns. Theme 2 deals with the coordination of spoken and visual aspects of prosody in conveying pragmatic intent, focusing on aspects such as negation, emotion and epistemic stance. Theme 3 explores how visual signals, including head movements and manual signals, fulfil essential prosodic roles across diverse sign language typologies. Taken together, the empirical evidence presented here shows that prosody is also embodied and that our bodily movements can manifest prosodic characteristics. On the one hand, they show the need to comprehensively re-evaluate our understanding of how speakers, listeners and learners engage with the prosodic dimension of language. On the other hand, they reveal that non-referential gestures are deeply meaningful and prosodically structured. Ultimately, visual cues are presented as indispensable for building accurate models of the human language capacity.
Addressing and predicting degenerative phenomena in domains such as health care and engineering, two fundamental fields of vital importance for society, offers valuable insights into early warning steps and critical event forecasting, leading to far-reaching implications for safety and resource allocation. By harnessing the power of data-driven insights, prognostics becomes the principal component of predicting such phenomena. Developing clustering techniques as feature extractors acts as an intermediate step between the raw incoming data and prognostics and provides the opportunity to unveil hidden relationships within complex datasets. However, when limited, noisy, and multimodal data are available in a label-free format, extensive preprocessing, and unreliable, complicated models are required for extracting meaningful features. This prohibits the development of adaptable methods in diverse domains that are in favor of robustness and interpretability. In this regard, this study introduces a novel unsupervised deep clustering model for feature extraction in degenerative phenomena. The model innovatively extracts prognostic-related features from raw data via clustering analysis, characterized by an increasing monotonic behavior representing system deterioration. This monotonicity is partial rather than complete, to incorporate the potential occurrence of oscillations in the degradation trajectory of the system or noise-related data, reflecting real-world scenarios. Its performance, robustness, generalizability, and interpretability are evaluated across diverse domains utilizing three datasets from health care and engineering featuring limited, noisy, high-dimensional, and multimodal raw signals. Results show that the model extracts meaningful prognostic-related features in both domains and all datasets, without a significant alteration in its architecture and independently of the chosen prognostic algorithm.
From the study of sign languages we know that the visual modality robustly supports the encoding of conventionalized linguistic elements, yet while the same possibility exists for the visual bodily behavior of speakers of spoken languages, such practices are often referred to as ‘gestural’ and are not usually described in linguistic terms. This article describes a practice of speakers of the Brazilian indigenous language Nheengatú of pointing to positions along the east-west axis of the sun’s arc for time-of-day reference, and illustrates how it satisfies any of the common criteria for linguistic elements, as a system of standardized and productive form-meaning pairings whose contributions to propositional meaning remain stable across contexts. First, examples from a video corpus of natural speech demonstrate these conventionalized properties of Nheengatú time reference across multiple speakers. Second, a series of video-based elicitation stimuli test several dimensions of its conventionalization for nine participants. The results illustrate why modality is not an a priori reason that linguistic properties cannot develop in the visual practices that accompany spoken language. The conclusion discusses different possible morphosyntactic and pragmatic analyses for such conventionalized visual elements and asks whether they might be more crosslinguistically common than we presently know.
Our focus on digital interaction in the history of English foregrounds the mutually transformative relationship between language and society, with technological affordances enabling (new) forms of social interaction, whilst impeding or remediating (older) communication practices. Early internet forum users maximised meaning-making with available linguistic resources, including pre-digital typographical and respelling practices. Today, within the diversity of digital Englishes, strategies typical of early digital interaction remain, reconfigured for users’ local language ideologies and community norms and expanded to incorporate multilingual practices and new semiotic modes. This chapter explores the sociopragmatic practices of identity and belonging across the digital age, from Usenet in the 1980s and SMS in the 2000s to Twitter in the 2020s, detailing a complex interplay between new communicative opportunities and long-established sociopragmatic practices originating offline. Our analysis points to a diversification of English-using internet users and an expansion of multilingual, multimodal repertoires which prompt a revisiting of traditional sociolinguistic conceptions of English.
Internet memes have been studied widely for their role in establishing and maintaining social relationships, and shaping public opinion, online. However, they are also a prominent and fast evolving multimodal genre, one which calls for an in-depth linguistic analysis. This book, the first of its kind, develops the analytical tools necessary to describe and understand contemporary 'image-plus-text' communication. It demonstrates how memes achieve meaning as multimodal artifacts, how they are governed by specific rules of composition and interpretation, and how such processes are driven by stance networks. It also defines a family of multimodal constructions in which images become structural components, while making language forms adjust to the emerging multimodal rules. Through analysis of several meme types, this approach defines the specificity of the memetic genre, describing established types, but also accounting for creative forms. In describing the 'grammar of memes', it provides a new model to approach multimodal genres.
This chapter first considers some correlations between memetic constructions and select figurative meanings, showing how our approach differs from existing multimodal metaphor approaches. As a case in point, the chapter presents an analysis of when-memes as relying on similative patterns of meaning, and also extends this discussion to include the family of If 2020 Was X memes.
This chapter outlines the reasons why a linguistically oriented book-length analysis of memes is a necessary step. It also previews the main theoretical tools to be used and highlights the ways in which this book differs from other books on memes. It includes a preview of the remaining chapters of the book.
This Element in Construction Grammar addresses one of its hottest topics and asks: is the unimodal conception of Construction Grammar as a model of linguistic knowledge at odds with the usage-based thesis and the multimodality of language use? Are constructions verbal, i.e. unimodal form-meaning pairings, or are they, or at least are some of them, multimodal in nature? And, more fundamentally, how do we know? These questions have been debated quite controversially over the past few years. This Element presents the current state of research within the field, paying special attention to the arguments that are put forward in favour and against the uni-/multimodal nature of constructions and the various case studies that have been conducted. Although significant progress has been made over the years, the debate points towards a need for a diversification of the questions asked, the data studied, and the methods used to analyse these data.
Blended, or hybrid, approaches to language learning continue to gain prominence. Resonant with established definitions of CALL, such approaches seek to promote ecological perspectives and embrace the ubiquity of technology. Questions of effectiveness and the justification of resources may result in a greater need for argument-based evaluation. Future research in blended language learning must take into account concepts inherent in multimodality, social semiotics, and computer mediated communication. Rather than revisit blended learning, however, porosity of environments may forge new metaphors of understanding and research.
Opening with an analysis of Instagram, Chapter 2 is concerned with how to think about postdigitality. Touching on multimodality, time-space-place, and responsive loops, this chapter highlights the contrast between digital life and postdigital life, unravelling the many dimensions of postdigitality. It concludes that postdigitality represents a world of symbiosis, whether that be of body and mind, physical life and screen life, representation and non-representation, immersion and connectivity, or interaction and convergence. These combinations are what lends digital media its unique power to move across time, space, and place. To explore these ideas, Chapter 2 analyses data which has been processed through ATLAS.ti to produce a list of postdigital keywords used by crescent voices.
After exploring the multimodal effects of BeReal, and the way in which it foregrounds place and event, this chapter explores the work of Hayles, Barad, and Braidotti, before utilizing New Literacy Studies to explore contrastive socio-cultural and social practices. The chief focus is on teasing out a theory of digital-materiality: not only what materialities and modes are present on screens, but also what inferences, values, and agendas these materialities carry. Postdigital lives entail entirely new relationships with materialities, though this does not mean a break with the physical and embodied, since postdigital life also contains many embodied ways of engaging with screens, ones which work across both the physical and the digital. This chapter attempts to conceptualize the distinct logic that people use to understand screens, while striking at more lived understandings of literacy. Consulting crescent voices on where they find comfort in their screen lives, this chapter reconciles people’s conflicting desires to pursue a flesh-and-blood life away from screens, as well as to use their screens to manifest and actualize the real aspects of their lives.