This chapter gives an overview of the relation between indexicality, deixis, and space in gesture from a semiotic and a linguistic point of view. Directive pointing gestures are not the only type of cospeech gestures that contributes to deixis. Iconic gestures that form part of the multimodal utterance may instantiate the targets to be pointed at and function as the deictic object of the deictic relation. In turn they may be interpreted as signs that stand for something else. A Peircean approach combined with a Bühlerian one, as suggested in this chapter, not only allows for a tertium comparationis with respect to the modality of the deictic and indexical signs under investigation. It also provides us with tools for representing semiotic processes like complex sign concatenation (e.g. deixis at signs vs. deixis at non-signs; deixis at metonymies or metaphors) as well as the collaborative creation of deictic space (sphere-like, map-like, screen-like; separated or shared) in multimodal interaction. The proposed schema of four semiotic subfields of space substantiates the view that space has to be thought of as a dynamic process of semiosis, not as a static entity.

This chapter presents the role and use of gesture in first language development and its integration in the child’s multimodal communicative system. It includes an overview of theories and methods that have triggered and facilitated the study of gestures in language development. The main issues are illustrated with detailed analyses of examples extracted from longitudinal data in English and French. The human communication system develops in a space of shared meanings in which adults socialize children into language in situated activities; consequently, this overview highlights the crucial role of caregivers in child–adult interactions. We first focus on the role of gestures in adults’ communicative input and then follow children’s development into the use of the adults’ multimodal communicative system. At the end of the developmental process, speech becomes clearly predominant but is both complemented and supplemented by other semiotic resources according to variables such as linguistic context, situation, interlocutor, activity, or discourse genre. Children learn to master the dynamic multimodal communicative system used around them and with them in their daily interactions.

As there are many different methods of linguistic analysis, there are many different ways of approaching gesture analysis. This chapter gives a selective overview of the current state of art on gesture coding and annotation systems. It opens with a discussion on the difference between coding and annotation, before it considers aims and challenges in gesture coding and annotation. Afterward, the chapter reviews existing systems and reflects on the interrelation between subject, research question, coding and annotation system. The chapter emphasizes that coding and annotations systems are always influenced by the particular theoretical framework in which they are situated. Accordingly, similar to the analysis of language, a theory-neutral analysis of gestures is not possible. Rather, theoretical assumptions influence subjects, aspects and levels of analysis and as such also make themselves visible in annotation systems. This will be illustrated by exemplary research topics in gestures studies: language, language development, cognition, interaction, and human–machine interaction. The account of the individual systems thereby does not aim at an extensive discussion, but rather focuses on their general logic for answering their particular research question. Here, differences between systems addressing the same research topic (e.g. language) as well as differences across research topics (e.g. language vs. interaction) will be explored. The chapter closes with some considerations on possible future developments.

Social media is resplendent with a creative blend of non-standardised graphical resources such as images, memes, digital stickers, avatars and GIFs that extend beyond the rigid parameters of Unicode emoji character encoding. This chapter explores how emoji interact with other kinds of visual resources beyond language in social media posts such as graphicons. The chapter aims to give the reader a sense of how a social semiotic intermodal approach furnishes a flexible toolkit for an analyst to explore emoji’s relations with other modes. It primarily analyses the meanings made through combinations of emoji, language and GIFs in tweets. The analysis reveals how graphicons such as GIFs and digital stickers often realise a salient ‘New’ of the posts wherein they occur, and thus foreground interpersonal meaning.

Conversation Analysis (CA) is a major contributing discipline to the study of language use and social action in context. Originating in the discipline of sociology, it forms the basis for the burgeoning field of interactional linguistics. This chapter offers an overview of major themes in the field. Beginning with a brief discussion of the intellectual background of the field, the chapter sketches three distinctive levels of analysis: sequential organization, practices of turn construction, and the organization of these practices as sets of resources for dealing with recurrent problems in the social organization of interaction. Sections of the chapter deal with sequence organization, preference, turn design, the fitting of talk to specific contexts and recipients (recipient design), progressivity, multimodality, and interaction in the context of specific social institutions such as medicine, legal discourse, and news conferences.

This chapter presents a view on context as understood within functional models of language, specifically the theoretical framework of Systemic Functional Linguistics (SFL). Amongst the functional approaches to language, SFL is recognized as a framework which has maintained an account of context that has prioritized its relationship with lexicogrammar, allowing it to make a causal connection between culture and language. The aim of this chapter is to highlight and explain the principal ways in which context works within the SFL framework and explore the main themes and parameters which situate context within an integrated theory of language as a semiotic resource. As no theory emerges in a vacuum, the first part of the chapter will consider the historical development of context as a concept within SFL theory with reference to how context is situated in other related functional grammars. Following this, we examine two areas of challenge related to the approach to context outlined in the chapter. Finally, the chapter concludes with closing remarks and key directions for future research in this area.

This chapter reflects on the changes in written communication associated with digital technology using some of the perspectives developed in earlier chapters. In particular, it focuses on differences in materiality and changes in display and design. When considering digital communication, there is plenty of evidence that writing is in ascendency, and that there is increased diversification in what we write and how we write it. New questions about who finds a voice and how we evaluate what is written have arisen. In times of post-truth, populism, political and religious extremism, writing can still be dangerous but it can also work to unite and consolidate, as it has done in the recent pandemic. If writing plays such a significant role in our social and cultural life, using it wisely is crucially important.

This chapter focuses on the shapes and forms that constitute writing, how it occupies graphic space and how it defines itself in relation to blank space and to other visual elements such as pictures, drawing and symbols. I consider how emphasis and authority are communicated and the ways in which distinctive layouts have become associated with different types of texts. Visual appearance is important when writing is used for decoration and embellishment, and in some cases, it becomes an art form in its own right. The chapter distinguishes between the display principle and the design principle.

The chapter ’Multimeowdality’ looks at the interplay of textual and visual elements in social media and approaches multimodality in computer-mediated communication (CMC) and computer-mediated discourse (CMD). Digital discourse has evolved from text-only discourse to multimodal interactions involving text, audio, video, and graphics. The chapter shows the CMC modes and semiotic modes used in multimodal interaction. Based on examples from cat-related digital spaces, it applies the faceted classification tool and the CMDA tool to describe communication on the interactive multimodal platforms. The chapter describes the various visual elements, such as photos, videos, memes, meme-like photos, GIFs, emoticons, emoji, and stickers, and discusses their functions in discourse.

Focusing on supermarkets in a Swiss Italian city, this chapter explores the interactional work that couples shopping together do before finally arriving at the counter to make a request. While some requests arise from a need that was stated at the very beginning of the shopping expedition (e.g., ‘I need cheese’) and which then guide the pair to a specific counter at the store, other requests arise from something said or seen in the course of traversing the store. It is shown that even needs or wishes that were expressed at the beginning of the interaction can undergo transformation. Potential purchasable items proffered by one member of the pair may be modified or rejected by the other, and in this process the pair enacts a particular kind of relationship, being a committed couple, in which both parties have a say in food that is to be purchased and prepared for specific meals. The study shows that requests emerge incrementally over the course of interactions, and are situated in particular interactional, material, and economy-oriented environments