1.1 The Impulse to Gesture

The idea of an impulse to gesture while speaking is an ideal starting point for this book. First of all, it captures an everyday view of gesturing as something uncontrollable and subconscious. I am not alone among gesture researchers to have made an addressee, upon mentioning this research area, suddenly claim heightened self-awareness of her gestures. Some people report immunity to this impulse – they say they ‘never’ gesture – whilst others are apparently overwhelmed by it – they ‘always’ gesture. In both cases, I observe a steady stream of recognisable gesture forms coherently organised and deployed as they speak. At the same time, the idea of an impulse to gesture captures a view of gestures that has arguably shaped the field of contemporary gesture studies; namely, the view that gestures are primarily a cognitive impulse, a spontaneous, unwitting, and idiosyncratic manifestation of thought (McNeill Reference McNeill1992, Reference McNeill2005, Reference McNeill2012, Reference McNeill2016).

The goal of this book is to explore a much lesser known side of the impulse to gesture. By ‘recognisable gesture forms’, I am not referring to emblematic gestures such as ‘thumbs up’ or to sustained body postures such as ‘arms crossed’ – for a dictionary combining emblems and postures see Morris (Reference Morris1994). I am referring to gestures that routinely connect with grammatical concepts in speech, and in the nature of that connection, exhibit regularity in their form, organisation, and function. This book is about those regularities. It is about the form and organisation of our impulse to gesture in connection with a particular grammatical concept in speech. My case will be based on negation.

Negation is a linguistic universal with clear grammatical and gestural manifestations. In grammar, a range of verbal particles and affixes explicitly express negation; they operate on the polarity of an utterance, and they impose positional constraints on syntax through negative node, scope, and focus – this understanding has been established for decades through various strands of linguistic, psycholinguistic, and logical–philosophical research (Horn Reference Horn1989). Meanwhile in gesture, the head shake is a famous expression of negation (partly because of its notorious cultural variations; Harrison Reference Harrison2013). People are also aware of an array of manual gestures associated with negation that appear to block, wipe away, or push against imaginary objects – and some of these have received scholarly attention since antiquity (Kendon Reference Kendon2004). By exploring the connection between grammatical and gestural manifestations of negation at the micro-level of utterances in face-to-face conversation, a novel understanding of the impulse to gesture emerges.

A decade of research into these connections in natural spoken language interaction shows that our impulse to gesture is constrained by kinesic, linguistic, conceptual, discursive, and interactive structures (Figure 1.1).

Figure 1.1 The impulse to gesture

These different layers of structure are inseparable within any one instance of gestural impulse. But they offer unique analytical angles to address gestures qualitatively and therefore constitute the sequentiality of chapters in this book. Furthermore, the identification of constraints in gesture through increasingly broad levels of structure stems from a methodological procedure for qualitative bottom-up gesture analysis developed within the Towards a Grammar of Gesture framework (henceforth ‘ToGoG’; Müller, Bressem, and Ladewig Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013). The research underpinning this book was crucially shaped through a series of ToGoG workshops and conference panels, and my main methodology for gesture description and analysis draws from ToGoG’s Linguistic Annotation System for Gestures (Bressem et al. Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013). For ToGoG, gesture analysis is a ‘procedure of discovery’ where each level of form description provides an empirical basis for the analysis of meaning. This book takes stock of my own discovery of the impulse to gesture through its connection to negation in speech.

In this introduction, I will begin by describing the origin of the project and its centrality to the cognitive linguistics framework. Then I will situate the work within the context of contemporary gesture studies and contextualise the approach I have adopted theoretically and methodologically, focusing on a particular type of gesture form and function. Next I will introduce the spoken language corpora that serve as the primary data source for this exploration and explain the software I have used to analyse them (ELAN annotation software). Finally, I will offer a synopsis of each chapter to help readers navigate the book.

1.2 Grammar in Motion and the Grammar–Gesture Nexus

The research underpinning this book began in the countryside surrounding Bordeaux, France with a project in applied cognitive linguistics called Grammar in Motion (Lapaire Reference Lapaire2005; see also Lapaire Reference Lapaire2013, Reference Lapaire2016). Contracted by Hachette Education and collaborating with a choreographer, Jean-Rémi Lapaire designed and filmed a series of gestural sequences designed to help English language teachers in France explain how English grammar works. Lapaire’s gesture creation and choreography were based on embodied approaches to grammar within the field of cognitive linguistics.

In cognitive linguistics, researchers analysing evidence from linguistic usage patterns have found that seemingly ‘abstract’ or ‘semantically empty’ grammatical markers, processes, and structures actually encode rich and dynamic patterns of conceptualisation (Langacker Reference Langacker1987, Reference Langacker1991a, Reference Langacker1991b, Reference Langacker2008; Lapaire Reference Lapaire, Oleksy and Stalmaszczyk2007). For Langacker (Reference Langacker1991a), grammatical structures are not empty but meaningful and ‘inherently symbolic, providing for the structuring and conventional symbolization of conceptual content’ (p. 1). The patterns of conceptualisation reflected in grammar shed light on embodied reasoning and schematic structures derived from experience and body-based interaction in the world (Heine Reference Heine1997; Lakoff and Johnson Reference Lakoff and Johnson1999). Bodily interaction with space, time, matter, and other bodies gives rise to conceptual metaphors (Lakoff and Johnson Reference Lakoff and Johnson1980), image schemas (Beate Reference Beate2005; Johnson Reference Johnson1987; Lakoff and Johnson Reference Lakoff and Johnson1999), mappings (Fauconnier Reference Fauconnier1997), blends (Fauconnier and Turner Reference Fauconnier and Turner2000), and other body-based conceptualisation structures (Heine Reference Heine1997), all of which are consequently structured and symbolised by grammar.

In Grammar in Motion, Lapaire (Reference Lapaire2005) examined the body-based conceptualisation patterns underpinning well-researched areas of English grammar to elaborate a theory of Kinegrams. According to Lapaire (Reference Lapaire, Oleksy and Stalmaszczyk2007), Kinegrams (from kinesis and gram) are ‘postural and gestural analogues of core grammatical phenomena’, created based on image-schematic, metaphoric, and conceptual blending analyses of grammatical meaning and processes (p. 7). The Kinegrams created for the Grammar in Motion project therefore visualise or ‘“act out” the semantic configurations and pragmatic mechanisms typically associated with selected grams or constructions’ (ibid.). For example, addressing the English modal verbs Lapaire (Reference Lapaire, Oleksy and Stalmaszczyk2007) wrote:

Kinegrams such as those for the modal verbs establish an important theoretical link between grammar and gesture. For me, they opened a space for empirical investigation into real-time dynamic connections between grammar and gesture in spoken language discourse and interaction.

The existence of a ‘grammar–gesture nexus’ – a systematic binding of grammatical and gestural form – challenges a mainstream view that gestures are primarily spontaneous, unwitting, and idiosyncratic manifestations of thought (McNeill Reference McNeill1992, Reference McNeill2005, Reference McNeill2012, Reference McNeill2016). In McNeill’s (Reference McNeill1992) original ‘speech-gesture nexus’ (p. 9), the temporal and semantic coordination between speech and gesture in spoken language discourse has nothing to do with grammar but instead reflects a psychological ‘growth point’. This growth point is ‘the initial unit of thinking for speaking out of which a dynamic process of organisation emerges’ (McNeill Reference McNeill2005: 17). Within the growth point, a ‘real-time dialectic’ occurs between static and dynamic modes of thought. The static mode of thought consequently manifests itself through conventional, linear, syntactic structures in speech – grammar – while the dynamic mode of thought manifests itself through imagery – gestures. Gestures are thus spontaneous creations based on unfiltered conceptual content, diametrically opposed to the ‘static structures’ or ‘chunks’ of linguistic structure (McNeill Reference McNeill2016) provided conventionally by grammar.

In the grammar–gesture nexus, ‘grammar’ is not a set of disembodied static structures. Grammar is a symbolic resource for speakers to shape and share embodied conceptualisation. Grammatical patterns construe thought patterns (Langacker Reference Langacker1987, Reference Langacker1991a, Reference Langacker1991b), and those thought patterns also motivate the gesture forms and functions that accompany speech. Grammar in the grammar–gesture nexus thus has salient symbolic and functional dimensions, both of which connect explicitly with gesture symbolism and function. By shaping and sharing conceptual content in particular ways, speakers ‘use’ grammar to achieve a range of functions. Speakers use grammar to hypothesise (conditionality), to request (question marking), to affirm (assertion), and to reject, oppose, and deny (negation). Following Givón (Reference Givón1993), from this perspective grammar is both a symbolic resource and ‘a set of strategies that one employs in order to produce coherent communication’ (p. 1). In focusing on the embodied, symbolic, and functional dimensions of grammar, similarities between grammar and gesture begin to emerge.

First, both grammar and gesture are embodied. Gesture is embodied not only in the literal sense that gesture involves the body, but also because the hands are an evolutionary source of embodied conceptualisation (Streeck Reference Streeck2009). As Streeck (Reference Streeck2009) notes, ‘no part of our body (except the eyes) is as important as the hand in providing us knowledge of the world’ (p. 4). The hands are thus a central tool for distributing and extending human cognition (Hutchins Reference Hutchins1995). As we interact manually with our world, for example, ‘fingers capable of grasping objects sort and categorise stimuli’ through peripheral neuron systems (Wilson and Folia Reference Wilson, Foglia and Zalta2017; cf. Pruszynski and Johansson Reference Pruszynski and Johansson2014). Second, both grammar and gesture are symbolic structures that speakers deploy for coherent communication. McNeill (Reference McNeill1992) notes that ‘gesture is a symbol in that it represents something other than itself’ (p. 20). Like grammatical symbolism, gestural symbolism also involves the pairing of physical form with conceptualisation. Langacker (Reference Langacker2008) explicitly includes ‘[u]nder the rubric phonological structure … not only sounds but also gestures’ (p. 29), consequently speculating elsewhere ‘whether such gesture should itself be considered linguistic in nature, that is, an inherent aspect of language structure’ (p. 249). Third, both grammar and gesture have a salient functional dimension. Speakers also ‘use’ gestures to achieve an array of communicative functions. As Kendon (Reference Kendon2004) observes, people may show ‘through visible bodily actions, that they are asking a question, making a plea, proposing an hypothesis, doubting the word of another, denying something or indicating agreement about it’ (p. 1).

By investigating these connections between grammar and gesture, this book describes the kinesic, linguistic, conceptual, discursive, and interactive structures that shape our impulse to gesture in interaction. The focus shifts from the raw cognitive–psychological side of gestural impulses to the systematic forms, organisation patterns, and functions that gestures exhibit in co-occurrence with core grammatical concepts structuring speech.

1.3 Gesture Form, Organisation, and Function

Gesture form refers to a salient kinesic feature of gestures: what they look like and how we can describe their appearance. Looking at a gesturing hand, gesture form can be described as the simultaneous combination of at least four form features. Every gesture has a handshape, a palm orientation, and a location in space; and most have a movement pattern (Bressem Reference Bressem, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013). The result of this combination is perceived holistically as ‘a gesture form’ and is only broken down for initial analytical purposes. This rudimentary level of gesture form can also be described in terms of form dimensions (Hassemer Reference Hassemer2015) or egocentrically, that is, from the perspective of physiological mechanism (Boutet Reference Boutet2010).

Gesture organisation refers to the temporal unfolding of gestures. Every gesture exhibits a basic temporal sequence akin to a beginning, middle, and end. More technically, phases of gestural action include a preparation phase, a stroke phase, various hold phases, and a retraction phase (Kendon Reference Kendon and Key1980, Reference Kendon2004). The preparation phase occurs at the beginning of gesture performance when the speaker’s hands are initially mobilised to perform a gesture. This initial movement often begins from a position of rest or ‘home position’ into a visible space immediately in front of the speaker’s body (Sacks and Schegloff Reference Sacks and Schegloff2002). Then, the stroke phase is characterised by clear and visible form features; movements are part of the gesture’s form as opposed to being a means to situate the gesture in a particular location; and the stroke is generally accepted as the moment when a gesture’s meaning is expressed and its functions are achieved (Kendon Reference Kendon2004). Hold phases are moments where a phase of gestural action is momentarily interrupted or paused – the hands are still but tense. The retraction phase occurs as the hands return to their position of rest. Together, these phases coordinate the unfolding of any particular gesture and orchestrate its momentum in relation to speech. Though the stroke is often seen as carrying the gesture’s meaning, all phases are potentially meaningful and provide speakers with interactive resources in face-to-face communication (see, for example, Cibulka Reference Cibulka2015).

Gesture forms and organisation patterns allow speakers to achieve an array of different functions through gesturing while speaking. To report a sample of well-understood functions, gestures have been shown to add informational content to utterances (Beattie and Shovelton Reference Beattie and Shovelton1999), assist speakers in lexical retrieval and other production processes (Hadar Reference Hadar1989), guide intrapersonal thinking processes (Goldin-Meadow Reference Goldin-Meadow2003), convey source domains of conceptual metaphors (Cienki and Müller Reference Cienki and Müller2008; Sweetser Reference Sweetser1998), replace words (Kendon Reference Kendon and Poyatos1988; McNeill Reference McNeill1992), mark up discourse structures, such as topic–comment (Kendon Reference Kendon1995, Reference Kendon2004), connect speech to material structures in the local environment (Goodwin Reference Goodwin, Duncan, Cassell and Levy2007), create cohesion over stretches of discourse (Chui Reference Chui2009; McNeill Reference McNeill1992), manage the interaction and distribution of turns (Bavelas et al. Reference Bavelas, Chovil, Lawrie and Wade1992), and contribute to speech act performance (Kendon Reference Kendon2004; Müller Reference Müller, Posner and Müller2004; Streeck Reference Streeck2009). While recent handbooks serve testimony to this diversity of gesture functions (Müller et al. Reference Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013b, Reference Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014), Kendon’s (Reference Kendon2004) distinction between the referential and pragmatic functions of gesture is most relevant here.

Kendon (Reference Kendon2004) observed that gesture functions could broadly be categorised into referential and pragmatic functions. When a gesture nuances, enhances, elaborates on, illustrates, depicts, or otherwise represents aspects of the co-occurring speech, gestures function ‘referentially’. In such cases, these are ‘gestures that are part of the referential content of their respective utterances’ (p. 158). When a gesture frames, presents, interprets, and structures aspects of the co-occurring speech, often in relation to the broader interaction, then gestures function ‘pragmatically’. Gestures that function pragmatically are ‘gestures that indicate something about the speaker’s attitude to the referential meaning or that contribute to the interpretive framework in terms of which this meaning should be treated’ (ibid.). Kendon (Reference Kendon2004) observed that gestures with pragmatic functions ‘serve to indicate the type of “act” or “move” the speaker is engaged in, how the speaker regards the utterance, or how the discourse is to be structured’ (p. 359).

Speech act performance, modality, and discourse structure emerge as functional dimensions shared by both grammar and gesture. Symbolic structures in grammar can be identified that speakers use to perform speech acts, adopt stance, and structure discourse; and likewise, symbolic structures in gesture have been identified that achieve those functions by shaping and sharing conceptual content in particular ways in the gestural modality. It is those structures within the gestural modality that I referred to earlier as ‘recognisable gesture forms’. Gestures that achieve pragmatic functions are recognisable because they have undergone conventionalisation. As Kendon (Reference Kendon2004) explains:

Gesture forms and functions thus range from spontaneous and idiosyncratic (i.e. ‘improvised’; Streeck Reference Streeck2009) to routine and conventionalised. Based on grammatical analysis, Langacker (Reference Langacker1987) has argued that ‘[l]anguage is a mixture of regularity and idiosyncrasy’ (p. 411). The co-existence of referential and pragmatic functions in gesture suggests that the mix observed for grammar is also characteristic of gesture. Within a grammar–gesture nexus, it is not that grammar is the regular dimension and gestures are the idiosyncratic dimension of language. Symbolic structures in both grammar and gesture exhibit a continuum between regularity and idiosyncrasy depending on their function in spoken discourse. While idiosyncrasy has primarily been studied through the imagistic function of gestures, the regularity of gesture has been explored, identified and documented most saliently in studies of ‘recurrent gestures’ (Ladewig Reference Ladewig2011, Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b) and ‘gesture families’ (Kendon Reference Kendon2004).

1.4 Recurrent Gestures and Gesture Families

Recurrency is a major feature of the impulse to gesture and thus a central theme of this book. Generally speaking, recurrency characterises the impulse to gesture because gestures continuously occur whenever people speak. Gestures have been observed to occur across speakers, contexts, languages, and cultures. More specifically, recurrency is a dimension of gesture form and function that leads to the repeated performance of similar gestures to achieve similar functions by different speakers in different contexts.

Any stream of gestures that accompanies speech will be populated with ‘recurrent gestures’ (Ladewig Reference Ladewig2011, Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b). Ladewig (Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b) defined the ‘recurrent gesture’ as a ‘stable form-meaning unit [that] recurs in different contexts of use over different speakers in a particular speech community’ (pp. 1559–60). The stability of recurrent gestures arises in the conventional pairing of a ‘formational core’ with a ‘semantic core’ that also often corresponds to a discursive function. Repertoires of recurrent gestures may be identified for a given linguistic community, such as German (Bressem and Müller Reference Bressem, Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b) and French (Calbris Reference Calbris1990). The formational core of a recurrent gesture can be described as a relatively fixed combination of gesture form features, and the semantic core can be described as the associated meaning or function that the combination conventionally encodes and achieves.

Several recurrent gestures are now well documented in the gesture studies literature (Figure 1.2). Examples include the ‘Palm Up Open Hand’ or ‘Palm Presenting’ gesture ([1]); the ‘Brushing Aside’ gesture; the ‘Cyclic’ gesture ([2]); and the ‘Horizontal Palm’ gesture ([3]). To describe them briefly, the Palm Up Open Hand or Palm Presenting gesture connects with speech acts of offering, presenting, and suggesting, and at the level of discourse the gesture can mark the introduction of a new topic (Kendon Reference Kendon2004; Müller Reference Müller, Posner and Müller2004). The Brushing Aside gesture has been observed among German and Spanish speakers to connect with rejections (Bressem and Müller Reference Bressem, Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Teßendorf Reference Teßendorf, Cornelia, Cienki, Fricke, Ladewig, McNeill and Bressem2014). The Cyclic gesture can represent ongoing activity referentially but also extends metaphorically to connect with elicitations, requests, and turn-holding (Ladewig Reference Ladewig2011, Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014a). Finally, the Horizontal Palm gesture is a recurrent gesture associated with negation and may indicate the speaker’s refusal (Calbris Reference Calbris2003; Harrison Reference Harrison2010; Kendon Reference Kendon2004).

Figure 1.2 Examples of recurrent gestures: ‘Palm Presenting’ [1], ‘Cyclic’ [2], and ‘Horizontal Palm’ [3]

Studies of recurrent gestures present an array of conventionalised symbolic structures in the gestural modality that occur repeatedly, that is, they recur in spoken language discourse data. Recurrent gestures were absent from the widely adopted ‘Kendon’s continuum’ – a continuum used to distinguish between various gesture types developed by McNeill (Reference McNeill1992) and credited to research conducted by Kendon (Reference Kendon and Poyatos1988). However, researchers have since inserted recurrent gestures between gesticulation and emblems (Cienki Reference Cienki, Kosecki and Badio2012; Ladewig Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014a). Any one stretch of spoken language discourse will be populated by such recognisable gesture forms, and they are contrasted with more idiosyncratic forms of gestural expression that may be termed ‘singular gestures’ (Müller Reference Müller2010). Following Müller (Reference Müller2010), Ladewig (Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b: 1559) describes the distinction as follows:

Recurrent gestures thus comprise a formational and semantic core. This core is typically based on or derived from schematised re-enactments of everyday manual actions (Calbris Reference Calbris1990, Reference Calbris2011; Kendon Reference Kendon2004; Ladewig Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b; Morris Reference Morris2002; Müller Reference Müller, Posner and Müller2004; Müller et al. Reference Müller, Bressem, Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013a; Streeck Reference Streeck2009). As Bressem et al. (Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013) write: ‘Gestures often constitute re-enactments of basic mundane actions, grounding the gestures’ communicative actions in real world actions’ (p. 1106; McNeill’s Reference McNeill2016 critique of this view is addressed in Chapter 7). In addition to the formational core, recurrent gestures tend to exhibit a number of form-variants. The precise performance of a recurrent gesture may vary depending on context and lead to utterance-specific or ‘local’ meanings (Kendon Reference Kendon2004). Form variants arise usually from variations in form parameters other than those that constitute the core, especially movement pattern and location. Form-variants are determined by context and lead to subtle semantic variations for each gesture in interaction with specific utterances (Kendon Reference Kendon2004; Calbris Reference Calbris2003, Reference Calbris2011; Ladewig Reference Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014b).

A recurrent gesture and its form-variants together constitute what Kendon (Reference Kendon2004) has called a ‘gesture family’. According to Kendon (Reference Kendon2004), gesture families are:

Gesture families are thus collections of form variants centred around a formational core. A gesture family structures and symbolises meaning conventionally in connection with a salient domain of speech act performance, modality, and discourse structure. The focus in this book is a particular gesture family associated with negation.

1.5 Negation and the Open Hand Prone Gesture Family

Recurrent gestures associated with negation were previously described in Kendon’s (Reference Kendon2004) context-of-use studies of gestures with ‘pragmatic’ functions. Kendon (Reference Kendon2004) studied gestures of the Open Hand and first identified two gesture families: Open Hand Supine and Open Hand Prone gestures. In the Open Hand Supine gesture family, the hand is open and the wrist is supine so that the palm is facing upwards. Gestures in the Open Hand Supine family were found to occur in contexts where the speaker was presenting, offering, or suggesting ideas (Müller Reference Müller, Posner and Müller2004). Gestures associated with negation, however, were described as part of the Open Hand Prone family.

In the Open Hand Prone family, according to Kendon (Reference Kendon2004) ‘the forearm is always in a prone position so that the palm of the hand faces either toward the ground or away from the speaker, depending upon how the elbow is bent’ (p. 248). Gestures where the palm of the hand faces towards the ground were called ‘Horizontal Palm’ or ‘ZP’ gestures, and these also involved an abrupt horizontal movement along the lateral axis. ZP gestures are apparently derived from the act of knocking something aside with the hand or using the open hand to cut through (Calbris Reference Calbris2003). Open Hand Prone gestures where the palm of the hand faces away from the speaker were called ‘Vertical Palm’ or ‘VP’ gestures. With VP gestures, the hand re-enacts a stopping action. When analysing the contribution of Open Hand Prone gestures to the utterances they were part of, Kendon (Reference Kendon2004) found that the gestures applied a ‘semantic theme of stopping or interrupting a line of action that is in progress’ (pp. 248–9). He found all gestures in this family to be performed ‘in contexts where something is being denied, negated, interrupted or stopped, whether explicitly or by implication’ (p. 248). Kendon (Reference Kendon2004) thus established a connection between Open Hand Prone gestures and negation in terms of context-of-use.

Gestures in the Open Hand Prone family have also been described by other gesture researchers. Connections between Open Hand Prone gestures and the expression of negation have now been observed from semiotic (Calbris Reference Calbris1990, Reference Calbris2005, Reference Calbris2011), praxeological (Streeck Reference Streeck2009), and conceptual perspectives (Bressem and Müller Reference Bressem, Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014a; Calbris Reference Calbris2003; Harrison Reference Harrison, Zlatev, Andrén, Johansson Falck and Lundmark2009a).

1.6 Corpus of Spoken Language Interactions

The primary corpus for research in this book contains conversations between Anglophones collected with their permission between 2007 and 2008 in both laboratory-like and natural settings. For the laboratory-like data, I invited pairs of speakers living in Bordeaux whose first language was English into a comfortable apartment setting to take part in a study. The participants were aged between twenty-three and thirty from North America, Canada, Ireland, and England. They included men and women, and in all but one case, the pairs knew each other prior to arriving at the apartment.

To stimulate conversation, each pair was asked to play a board game called Half-Minute Topics. The game required participants to take turns throwing a dice, then accordingly move a counter along a winding path of squares on the board. Each square contained a topic of conversation and the pair was instructed that upon landing in a given square, whoever threw the dice had to strike up a conversation with their partner based on that square’s topic. I took this game from an English Language Teaching resource pack for a lesson aimed at teaching aspects of English grammar, in particular negation and conditionals. Some of the topics had therefore been designed specifically to elicit negative speech acts. For example, a number of topics required participants to engage in discussions about unpleasant topics, such as boring household chores, annoying habits, and general dislikes.

Although the pairs sometimes began this task with an artificial conversation opener (e.g. ‘OK, your turn, tell me about … ’), the stimulus quickly led to spontaneous conversations about a multitude of topics that extended beyond those prescribed by the board game. When often the pairs digressed, I made no attempt to restrain conversations or to re-direct attention back to the game, and I allowed participants to converse for as long as they wanted. When they finished playing, I also asked the pairs to discuss whether they thought the game they had played would be suitable for English Language Teaching (a number of participants where also part-time English teachers). The goal of this add-on session was to collect a further 20 to 30 minutes of spoken interaction data from each pair. The participants agreed to take part in the research, but the design of the game and the focus of the study were not discussed. The overall amount of data collected in this laboratory-like setting comprised approximately four and a half hours of conversation between five pairs of English speakers (the interactions typically lasted between 45 and 60 minutes).

Added to this experimental data are recordings from various speakers collected in a diverse array of settings:

• natural interactions in English during summer 2007 on and around a campsite in southwest France, particular of one surfer in his late twenties interacting with friends at the campsite bar (approximately two hours);

• stories I recorded of an English man in his mid-sixties, whilst he cooked in his kitchen at home or on camping holidays (approximately four hours);

• two group discussions in English among students at the coffee shop of a British university campus in China (approximately two hours).

This 12-hour plus corpus of conversations in spoken English provides the primary basis for Chapters 2, 3, 4, and 5 of this book, each of which investigates a particular dimension of the impulse to gesture among English speakers. Chapters 6 and 7 continue to investigate the impulse to gesture but extend the study to include episodes of interaction in French, Chinese, and French Sign Language. Accordingly, those chapters are based on an additional three hours of recording:

• an informal business meeting in French between myself and a colleague in France as we prepared for a project in industry (approximately 90 minutes);

• an episode of interaction between lifeguards on a beach in southwest France (five minutes, extracted from author’s larger corpus of workplace interactions);

• an episode of interaction between a new home-owner and her builder in a newly constructed tower of apartments in China (approximately 30 minutes);

• a French sign language class (one hour).

In all cases, I avoided rearranging or disturbing the subjects requested permission, and followed Kendon (Reference Kendon2004) in aiming for recordings of ‘ordinary settings of people talking together, in most cases while they were in pursuit of their own purposes’ (p. 365). Recordings were all made with a camera on a tripod or hand held. The total data set of spoken interaction is therefore in the region of 15 hours and includes interactions between over thirty-five speakers. Added to this, I conducted two retrospective interviews with participants from the corpus and two focus groups about negation with undergraduate students at a university in China – these are reported in Chapter 8. Finally, Chapter 2 and 3 both include one example taken from video-recorded lectures publically available on YouTube.

1.7 Identifying the Grammar–Gesture Nexus

The goal of this book is to characterise the impulse to gesture through specific bindings of grammatical and gestural form that occur when English speakers conceptualise and express negation. In this grammar–gesture nexus, the speech involves utterances structured explicitly by linguistic negation including a grammatical particle like no, not, and nothing, as well as syntactic processes such as negative node, scope, and focus (Horn Reference Horn1989). The gestures involve a subset of recurrent forms belonging to the Open Hand Prone gesture family described by Kendon (Reference Kendon2004) that have been observed in connection with negation in a number of empirical studies (Bressem and Müller Reference Bressem, Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014a; Calbris Reference Calbris1990, Reference Calbris2003, Reference Calbris2005, Reference Calbris2011; Harrison Reference Harrison2009b, Reference Harrison2010, Reference Harrison2015; Kendon Reference Kendon2004; Streeck Reference Streeck2009). This specific grammar–gesture nexus is key to understanding the kinesic, linguistic, cognitive, and discursive structures that shape the impulse to gesture in spoken language interaction.

To identify utterances structured by linguistic negation, I followed what Horn (Reference Horn1989) describes as ‘the traditional criteria for negativity – the presence of a negative particle, its appearance in a specified syntactic location, and so forth’ (p. 34). As Huddleston and Pullum (Reference Huddleston and Pullum2005) write, negation in English is ‘marked by individual words (such as no, not, never) or by affixes within a word (such as -n’t, un-, non-)’ (p. 149; original emphasis). I initially ignored cognitively more complex types of negation such as implicit or inherent negation (Leech and Svartvik Reference Leech and Svartvik1994), however, they will be discussed where relevant; as will linguistic phenomena such as multiple negation and Negative Polarity Items (Lawler Reference Lawler2005; Horn and Wansing Reference Horn, Wansing and Zalta2017).

To create a corpus of negative utterances, I viewed the video recordings in ELAN annotation software (www.mpi.com). The graphical user interface of ELAN presents the video feed and options for creating analytical tiers. In a tier I called ‘speech’, I made annotations for each utterance that contained linguistic negation, effectively compiling a corpus of ‘negative utterances’. To then identify gestures associated with negation, I used the video feed to examine all the negative utterances and selected those utterances with which the speaker gestured for further study. Within those utterances, I followed Kendon’s (Reference Kendon2004) distinction between gestures with referential functions and gestures with pragmatic functions, then set out to identify gesture forms with form properties of gestures in the Open Hand Prone family: both the Vertical Palm and the Horizontal Palm manifestations first identified by Kendon (Reference Kendon2004). To illustrate this process, examples (i) through (v) below were initially identified in the corpus as negative utterances co-occurring with gestures:

1. (i) I don’t like getting my nose punched.

2. (ii) They are not real rocks.

3. (iii) They don’t broadcast it.

4. (iv) I don’t have to pay for that night.

5. (v) I don’t know.

Utterances (i) to (v) all contain grammatical forms of negation and therefore qualified to be in the initial corpus. More specifically here, they each contain the negative particle not or its clitic n’t, and these negative nodes are introduced either by an auxiliary are or so-called do-support. Syntactically the nodes are located immediately after the grammatical subject, and they project a negative scope over the predicate that follows. Thus in (i) the speaker negates any desire to be punched on the nose when boxing, while in (ii) it is the authenticity of rocks used on an indoor climbing wall that is being negated. The speaker in (iii) negates broadcasting information as a metaphor for talking publicly about private information, while the speaker in (iv) negates the obligation to pay a particular night’s fees on the campsite. The speaker in (v) negates having knowledge on a specific topic, in this case gender equality in France. Regardless of this variation, the grammatical forms and processes in these negative utterances conventionally encode negation and allow the speaker to negate a particular proposition.

Turning to the video feed, the speakers all perform gestures with their negative utterance so all instances were also included in the gesture corpus. However, the kinds of gestures they performed determined whether they would be included for further study into the relationship between gesture and negation. Only examples (iv) and (v) exhibited forms resembling members of the Open Hand Prone family, while examples (i), (ii), and (iii) exhibited forms that varied depending on the referential content of the utterance (Figure 1.3). Thus in example (i), as the speaker negates any desire to be punched on the nose when boxing, she brings her right hand up to her face and with an open hand makes contact with her nose [1]. The speaker’s gesture can be understood as connecting with the concept of nose expressed in speech, perhaps referring to her own nose. Importantly though, the gesture form does not refer to the linguistic negation in any way. Likewise in example (ii), as the speaker says ‘They are not real rocks’ she moves her open hands into the gesture space cupped with the palms facing upward, as if to enact holding a rock [2]. In example (iii), as the speaker says ‘They don’t broadcast it’ (using ‘broadcast’ metaphorically to mean ‘talk about’), he raises an open hand with the fingers extended into his upper right gesture space, and with the palm lateral, moves this hand away from his body twice [3].

Figure 1.3 Gestures with [1] ‘nose’, [2] ‘rocks’, [3] ‘broadcast’

Examples (i) to (iii) illustrate ways the gesture forms connect to the variable content of the negative utterances and the difference in gesture forms reflect that variability. However, in example (iv), as the speaker says ‘don’t’, he moves an open hand turned palm down abruptly along the horizontal axis (Figure 1.4[4]). This gesture is recognisable as the ‘Horizontal Palm’ observed in the Open Hand Prone family to occur in contexts where negation is being expressed (Kendon Reference Kendon2004); it co-occurs specifically with the node of negation (Harrison Reference Harrison2010) and researchers agree that the Horizontal Palm encodes negation via an underlying action of cutting or sweeping aside (Calbris Reference Calbris2003; Kendon Reference Kendon2004). When the speaker in example (v) says ‘I don’t know’, he raises both hands with open palms vertical into the gesture space and holds them there for the entirety of his utterance (Figure 1.4[5]). This gesture is recognisable as the ‘Vertical Palm’ gesture, also observed in the Open Hand Prone family to occur in contexts where negation is being expressed (Kendon Reference Kendon2004). The Vertical Palm also co-occurs specifically with the negative node (Harrison Reference Harrison, Müller, Cienki, Fricke, McNeill and Bressem2014b) and likewise can be related semantically to negation via an underlying action of blocking, stopping, or holding away (Bressem and Müller Reference Bressem, Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Bressem2014a; Calbris Reference Calbris2011; Kendon Reference Kendon2004).

Figure 1.4 Gestures that connect to [4] ‘don’t’ and [5] ‘don’t’

These five examples capture the broad distinction between the referential and pragmatic functions of gestures and they exemplify gestures in the Open Hand Prone family. Examples (i) to (iii) illustrate how the form of gestures that connect to referential content of the negative utterances varies depending on the nature of the object being negated. Examples (iv) and (v) illustrate the horizontal and vertical variants of Open Hand Prone forms that gestures exhibit when they connect to the negation in the utterance. Gestures connecting to the referential content of the utterances can be said to reflect the spontaneous, unwitting, and idiosyncratic side of the impulse to gesture, with each form potentially reflecting the speaker’s individual experience, viewpoint, and understanding. Gestures explicitly connecting to negation, on the other hand, exhibit recognisable forms that are coordinated temporally with the negative particle. They capture the systematic and constrained side of the impulse to gesture, and they constitute a basis for analysing the grammar–gesture nexus.

1.8 Methods of Gesture Analysis

The over-arching research methodology and paradigmatic stance towards gestures throughout this book stems from a ‘form-based view’ or ‘linguistic approach’ to gestures (Bressem Reference Bressem, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013; Bressem et al. Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013; Müller et al. Reference Müller, Bressem, Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013a). A number of research content and practical considerations will be offered to introduce this approach.

The underlying approach to gestures adopted here can be considered ‘form-based’ because it ‘gives [gesture] form a prominent role in the process of description and analysis’ (Bressem et al. Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013: 1100). The description and analysis of each gesture associated with negation began with an in-depth description of the gesture form along with an interpretation of the kinds of meanings the form could produce (i.e. bottom-up). Although the presence of grammatical negation in the co-occurring speech was a key criterion to identify utterances for the grammar–gesture corpus, the details of this verbal context were then not taken into account during the first critical stages of gesture analysis. The initial focus of gesture analysis was the form features, organisational structures, and possible semantic derivations of the gesture as a meaningful action.

To describe gesture form, organisation, and derivation, I adopted Bressem et al.’s (Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013) Linguistic Annotation System for Gestures which proposes a sequence of steps to take when analysing gestures:

The first stage of description of gesture form focuses on the four salient parameters: Handshape, Orientation, Position, Movement type (Bressem Reference Bressem, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013). Recall that gestures in the grammar–gesture nexus corpus were initially selected because of their potential family resemblance to gestures in the Open Hand Prone family (Kendon Reference Kendon2004). Carrying out a form feature analysis served to verify the formational core of each gesture in the corpus and identify and describe variations in form along the different parameters. Based on that analysis, a more detailed understanding of the formational core was possible and a number of gestural variations were identified for both vertical and horizontal manifestations of the gesture.

Each variation was then examined with the goal of discerning the motivation of the gesture forms. The Linguistic Annotation System for Gestures takes into account the mode of representation, underlying action, and salient image schema. Examining the mode of representation involves asking what kind of action are the hands engaged in when they gesture and offers insight to potential meanings of the gesture. Müller et al. (Reference Müller, Bressem, Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013a) proposed four modes of representation – ‘the hand acts, the hand molds, the hand draws (or traces), and the hand represents’ (p. 712). Identifying the mode of representation leads naturally to proposing an underlying action. While moulding, drawing, and representing can constitute underlying actions, when the hand ‘acts’, further speculation is required about what kind of action the gesture seems to be performing.

The fine-grained description and analysis of the gesture form provided by this approach results in a vocabulary for talking about the gestures in non-semantic terms regardless of the verbal utterance they are part of. An empirical basis is thus provided to consider the gesture as a communicative action itself rather than a mode of expression dependent on speech (Müller Reference Müller, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013). Within the form-based approach, this empirical basis is a pre-requisite to then analyse how the form interacts in specific instances or ‘locally’ as an integral part of speech, which depending on the research questions can focus on the level of prosody, syntax, semantics, and pragmatics (Bressem et al. Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013). As Bressem et al. (Reference Bressem, Ladewig, Müller, Müller, Fricke, Ladewig, McNeill and Teßendorf2013) state, with this method ‘a gesture’s meaning is determined in a (widely) context-free analysis of its form, which grounds the later context-sensitive analysis of gestures’ (p. 1100). This grounding process is highly qualitative with the initial categories developed during context-free analysis then reiteratively pursued, evaluated, and revised in order to find a ‘goodness of fit’ with the contextualised data (Dörnyei Reference Dörnyei2007).

1.9 Road Map

The sequentiality of the chapters in this book is guided by the logic of the ToGoG form-based or linguistic approach to gesture (Müller et al. Reference Müller, Bressem, Ladewig, Müller, Cienki, Fricke, Ladewig, McNeill and Teßendorf2013a). Generally speaking, the chapters move from the micro-level of identifying and describing recurrent patterns at the utterance level (Chapter 2), to issues of linguistic organisation (Chapter 3), conceptualisation (Chapter 4), discourse cohesion (Chapter 5), discourse coherence (Chapter 6) and interaction (Chapter 7). Each level sheds light on the various principles and constraints that lead to the centrality of recurrent co-occurrences between grammatical form and gesture form to spoken language interaction, allowing the book to end with an Impulse Theory of how, when, and why we gesture (Chapter 8). The different contributions of each chapter may be summarised as follows.

Chapter 1. The Impulse to Gesture: Spontaneous but Constrained. This chapter has introduced the idea of an impulse to gesture and described the genesis of the grammar–gesture nexus in association with the conceptualisation and expression of negation. The chapter narrowed down the focus of gesture studies to a particular subset of gestures called recurrent gestures and gave an overview of previous work on the Open Hand Prone gesture family related to negation. Finally, the corpora used in this research were presented and the methodology was described.

Chapter 2. The Grammar–Gesture Nexus: A Mechanism for Regularity in Gesture. Chapter 2 develops the notion of a grammar–gesture nexus – systematic recurrent bindings between grammatical and gestural form that constitute a mechanism for the regularity we observe in spontaneous co-speech gesture. Three grammar–gesture nexus are identified and presented. They each illustrate the conventional pairing of a negative grammatical form or construction with a variant of the Horizontal Palm gesture. Variations in the gesture form stem from variations in the different manual action they reproduce, which connects to the co-occurring linguistic segment in terms of joint realisation of meaning and/or function. The variants reproduce actions I describe as ‘striking aside’, ‘clearing a space’, and ‘cutting through’ that bind respectively with clausal negations (not), exclusions (with the adverbial just), and rejections (with negative adjectives and adverbs in the construction It’s X).

Chapter 3. Sync Points in Speech: Evidence of Grammatical Affiliation for Gesture. Chapter 3 posits the grammar of linguistic concepts as an organisational principle for the impulse to gesture. We illustrate how gestures associated with negation may occur with a range of elements in the utterance, including negative particles and elements in the scope they project, such as Negative Polarity Items and focused elements. Examining how speakers prepare, perform, hold, and retract their gestures in relation to these elements reveals a grammatical affiliation between speech and gesture, with the grammar of negation yielding sync points that constrain the organisation of gesture in relation to speech.

Chapter 4. Gesture as Construal: Blockage, Force, and Distance in Space and Mind. Moving from the mechanics of the grammar–gesture nexus to its cognitive basis, Chapter 4 adopts a cognitive-linguistic perspective on a class of gestures associated with negation and views them as tools for dynamic construal operations with speech. The Vertical Palm gesture may reproduce various embodied actions including blocking, stopping, pushing, throwing, and wiping away. When speakers perform this gesture in conjunction with negative structures and speech acts, they construe their negation in terms of the physical action they reproduce. A conceptual semantic analysis of such multimodal utterances shows how gestures operate on a similar conceptual basis to the negative construals identified by linguists working with speech. Speakers use gesture in physical space to construe negation and their negative speech act as expressions of distance, force, and absence in conceptual space. Furthermore, their construals of space are sensitive to the position of their addressee(s) in the real-time interactive space.

Chapter 5. Gesture Sequences: Wrist as Hinge for Shifts in Discourse. Chapter 5 extends the concept of a grammar–gesture nexus from individual bindings at the utterance level to a mechanism for cohesion across linguistic segments at the level of discourse. We focus on a particular sequence of recurrent gestures that speakers use to frame their verbal content and indicate linguistic, logical, and rhetorical links between otherwise separate elements in the co-occurring speech. The basic sequence invariably involves a Palm Up gesture and a Palm Down gesture, specifically the Palm Presenting gesture and the Horizontal or Vertical Palm gesture. In sequencing, the rotation of the wrist(s) between the two gestures is key as it relates to the shift in discourse underway. These discourse shifts may include, for example, a logical consequence or the resolution of a conflict. Palm Up and Palm Down gestures have previously been isolated and opposed, but this study of their sequencing sheds light on a gesture-based construction that operates on co-occurring speech, regardless of its content.

Chapter 6. Patterns of Gesturing: The Business of ‘Horizontal Palming’. Taking an entire interaction as the analytical unit now, Chapter 6 examines the impact of genre, style, and identity on the impulse to gesture. Maintaining a focus on gestures associated with negation but shifting language to French, we scrutinise one speaker’s repeated use of the Horizontal Palm gesture over the course of a 90-minute informal business meeting. Our analysis starts from the micro-kinesic context of individual occurrences of the gestures at the utterance level upwards (and outwards) by considering the increasingly broad semantic, pragmatic, and discursive structures that operate reflexively on the speaker’s impulse to gesture. The broader communicative purpose of the interaction, the role the speaker was adopting, and the position of this interaction within a chain of related interactions (inter-textuality) are shown to be essential elements shaping a ‘gestural genre’ that accounts for the recurrency of gestures associated with negation at the micro-level. The chapter invites us to view recurrency not only as a product that leads to ‘recurrent gestures’, but also as a dynamic process that results in ‘recurrent gesturing’.

Chapter 7. Wiping Away: Embodied Interaction in Speech and Sign. Chapter 7 puts the impulse to gesture within more complex ecological contexts of situated activity in order to examine the embodied actions motivating gestures associated with negation. To do this, we study interactive contexts in which the real-world action of ‘wiping away’ – using the open palm to remove or eliminate something from a surface – plays a central role in the ongoing interaction. These contexts showcase how such actions acquire communicative properties in interaction, as well as how they connect to the gestures associated with negation that continue to occur with the ongoing speech. These observations provide the basis to then explore the wiping away gesture as a tool for embodied interaction, both in spoken language and in sign language. The relation between action, gesture, and language emerges as inseparable over the course of this chapter.

Chapter 8. Impulse Theory: How, When, and Why We Gesture. The recurrent form-function pairings studied throughout this book raise important questions about the nature of gesture and its relation to linguistic structure in interaction. Chapter 8 presents Impulse Theory to account for how, when, and why, we gesture in this conventional and systematic way. Based on conclusions from the previous chapters, it begins by answering the question of what is the impulse to gesture. This answer involves several key constructs addressed throughout the book – the grammar–gesture nexus, sync points in speech, conventionality, gestural competence, and the action–gesture–grammar link – each of which provides connection points to evaluate how current approaches diverge and converge. Having situated Impulse Theory, we evaluate it with reports of interview and focus groups held with the conversationalists from our corpora. We then extend the theory by illustrating its application to examine other linguistic concepts multimodally – namely, progressive aspect – before concluding the book with some ongoing challenges for research into the impulse to gesture.