Age of first exposure (AoFE) is an important factor that influences the quality of L2 acquisition. This study aims to investigate the AoFE effect on the contextual learning of L2 novel words at the neural level, as measured by the N400 component from event-related potentials (ERPs). Eighty-eight participants were recruited for the experiment of L2 pseudoword learning, which includes a learning session and a testing session. The participants’ EEG data were recorded from the testing session, and the N400 effect was derived from target words that were either congruous or incongruous with the context. The linear mixed model and multiple regression model revealed a positive AoFE effect on the N400 effect in discourses that were designed for testing retrieval of episodic and semantic memory even after accounting for the variance contributed by several confounding factors. In addition to AoFE, the effects of total L2 exposure, L2 proficiency and personality on the L2 novel word learning performance indicated by the N400 effect were also confirmed in the statistical results.

With a focus on politicians’ and medical experts’ gratitude expressions in UK government COVID-19 briefings, this research describes how perspective and intensity were modulated in expressing gratitude to realise different pragmatic intentions. This corpus-assisted analysis finds that retrospective or prospective gratitude expression was adopted by the two British elite groups to build solidarity (encouraging) and/or make requests (directing) for protecting public health. Gratitude of varying intensities was expressed (e.g. by highlighting metaphorical dimensions such as WIDTH and DEPTH) to correspond to the importance of a benefit (judged by how much the given benefit matches the receiver’s needs and preferences) and/or to implicitly display the evaluation of the benefactor’s responsibility and efforts. We tentatively formulate a dynamic model of gratitude expression in public discourse and shed light on the metaphorical conceptualisation of English gratitude expression and the power of gratitude expression in boosting social cohesion and directing social actions in a discourse of crisis.

This essay is the introduction to the Special Issue ‘Events in language and mind: Theoretical and empirical advances in the event integration theory’. We first review Leonard Talmy’s event integration theory in addition to some critiques of this framework. Following this, we point to some empirical research inspired by this framework, which explores the interaction between language and cognition. We then briefly introduce the papers in this volume and discuss their contributions to the event integration framework. We conclude with some limitations, questions and future directions.

This study investigated whether speakers use multimodal information (speech and gesture) to differentiate the physical and emotional meanings of the polysemous verb touch. We analyzed 302 hand gestures that co-occurred with this perception verb. For each case, we annotated (1) the meaning of touch (physical vs. emotional), (2) the gesture referent speakers physically touched (other-touch vs. self-touch), (3) the personal pronoun following the verb and (4) if they used intensifiers and negation. There were three main findings. First, we have seen that when speakers express the physical meaning, they are likely to reach an external referent (other-touch), but when they imply the emotional meaning, they tend to touch their own body (self-touch). Second, the most frequent co-speech gesture (chest-touching gesture) was associated with the emotional meaning, uncovering the metaphor the heart is container for emotions. Third, this study showed that the physical meaning of touch usually coexists with a wide variety of personal pronouns and negation words; in contrast, the emotional meaning of touch occurs primarily with the pronoun me and it is usually modified by intensifiers. Thus, speakers use both speech and gesture to differentiate the meanings of the polysemous verb touch.

A well-known method of studying iconic words is through the collection of subjective ratings. We collected such ratings regarding familiarity, iconicity, imagery/imageability, concreteness, sensory experience rating (SER), valence and arousal for Mandarin ABB words. This is a type of phrasal compound consisting of a prosaic syllable A and a reduplicated BB part, resulting in a vivid phrasal compound, for example, wù-mángmáng 雾茫茫 ‘completely foggy’. The correlations between the newly collected ABB ratings are contrasted with two other sets of prosaic word ratings, demonstrating that variables that characterize ABB words in an absolute sense may not play a distinctive role when contrasted with other types of words. Next, we provide another angle for looking at ABB words, by investigating to what degree rating data converges with corpus data. By far, the variable that characterizes ABB items consistently throughout these case studies is their high score for imageability, showing that they are indeed rightfully characterized as vivid. Methodologically, we show that it pays off to not take rating data at face value but to contrast it with other comparable datasets of a different phenomenon or data about the same phenomenon compiled in an ontologically different manner.

Research using masked priming and parafoveal preview techniques has shown that visual letter similarity has an impact on word processing during the initial stages in Latin-derived scripts. However, these effects appear to be absent in Arabic. One reason for this discrepancy could be attributed to the distinctive features of the Arabic script, which includes numerous letters sharing a basic form while varying in the location or number of diacritics. To shed light on this issue, the present study employed Arabic letters rather than words in two masked priming experiments: an alphabetic decision task and a letter-matching task. Both experiments showed that visually similar letters were more effective as primes than visually dissimilar letters. These findings suggest that the processes of letter identification in Arabic and Latin scripts may be roughly alike, implying that differences in visual letter similarity across scripts may arise at later stages of processing.

Distributional semantic representations were used to investigate crossmodal correspondences within language, offering a comprehensive analysis of how sensory experiences interconnect in linguistic constructs. By computing semantic proximity between words from different sensory modalities, a crossmodal semantic network was constructed, providing a general view of crossmodal correspondences in the English language. Community detection techniques were applied to unveil domains of experience where crossmodal correspondences were likely to manifest, while also considering the role of affective dimensions in shaping these domains. The study revealed the existence of an architecture of structured domains of experience in language, whereby crossmodal correspondences are deeply embedded. The present research highlights the roles of emotion and statistical associations in the organization of sensory concepts across modalities in language. The domains identified, including food, the body, the physical world and emotions/values, underscored the intricate interplay between the senses, emotion and semantic patterns. These findings align with the embodied lexicon hypothesis and the semantic coding hypothesis, emphasizing the capacity of language to capture and reflect crossmodal correspondences’ emotional and perceptual subtleties in the form of networks, while also revealing opportunities for further perceptual research on crossmodal correspondences and multisensory integration.

Reading official letters and being able to react to them appropriately is part of the daily life of every adult in many countries. Although the history of the plain language movement dates to the past century, it is only now that efforts are being made in Poland to adapt official documents to their audience. In this paper, we describe the results of a study (N = 685) in which we examined how particular linguistic forms and text structure affect the reception of an official-looking text – comprehension as well as emotions experienced, trust in the office, feelings of helplessness in the situation of reading the writing, and many other variables. Based on Imbir’s (2016, New Ideas in Psychology, 43, 39–49), two-system theory of emotions, we aim to examine how emotions – triggered by these complex stimuli – affect their processing. We hope that the results of our interdisciplinary research will answer the questions and hypotheses posed by researchers and find application in the reforms currently underway. We believe that by linking findings from cognitive psychology, the psychology of emotions and linguistics, we will also expand knowledge in the broad discussion on overcoming barriers between administrative offices and target audiences.

In this work, we use a mathematical model of the property listing task dynamics and test its ability to predict processing time in semantic and lexical decision tasks. The study aims at exploring the temporal dynamics of semantic access in these tasks and showing that the mathematical model captures essential aspects of semantic access, beyond the original task for which it was developed. In two studies using the semantic and lexical decision tasks, we used the mathematical model’s coefficients to predict reaction times. Results showed that the model was able to predict processing time in both tasks, accounting for an independent portion of the total variance, relative to variance predicted by traditional psycholinguistic variables (i.e., frequency, familiarity, concreteness imageability). Overall, this study provides evidence of the mathematical model’s validity and generality, and offers insights regarding the characterization of concrete and abstract words.

In this study, our objective was to explore the impact of hearing loss on the conceptual system underlying word meaning. We collected perceptual strength norms for 200 Italian words from early deaf individuals with limited or no access to auditory information and compared them to existing norms from hearing individuals. For each word, participants provided perceptual strength ratings for each perceptual modality. Our results revealed a significant reduction of the auditory modality in the norms provided by deaf individuals compared to the hearing population. However, we did not observe an overall decrease in reported perceptual strength. Interestingly, we found a heightened involvement of other sensory modalities accompanied by reduced modality exclusivity in the conceptualization of words, indicating that deaf individuals heavily rely on information coming from the other perceptual modalities to form concepts. These findings suggest that hearing loss leads to a reorganisation of word conceptualization, characterised by increased multisensoriality. Importantly, although diminished, the auditory modality remains present, suggesting that deaf individuals can still infer auditory-associated knowledge about words to some extent.

This paper presents an audiovisual perceptual analysis of the wh-question and wh-exclamation intonation in Brazilian Portuguese using auditory–visual congruent and incongruent stimuli, to investigate the relative importance of each modality in signaling pragmatic meanings. Ten Brazilian Portuguese speakers (five female) were filmed while producing both speech acts 10 times. Next, artificial stimuli were created: audio and visual cues were either matched (audio and video from the same speech act) or mismatched (audio and video from the different speech acts), resulting in 10 congruent and 10 incongruent stimuli of the wh-questions and the wh-exclamations. The perceptual experiment was taken by 36 Brazilians who identified the stimulus as a question or an exclamation. Results from the logistic regression showed that the factor ‘congruence’ was significant and had a significant interaction with ‘speakers’, which means that the congruent stimuli increased the comprehension of the Brazilian Portuguese wh-questions and wh-exclamations. In contrast, the incongruent stimuli tended to lower listeners’ identification, but to a degree depending on individual speakers’ strategies. Although variation in the accuracy of expressing both speech acts was also found across speakers, this study corroborates that the visual channel impacts the perceptual identification of the pragmatic intonation function of distinguishing sentence mode.

Embodied imagery hypothesis proposes the activation of perceptual-motor systems during language processing. Previous studies primarily used concrete visual stimuli to investigate mental imagery in language processing by native speakers (NSs) and second language (L2) learners, but few studies employed schematic diagrams. The study aims to investigate mental imagery in processing prepositional phrases by English NSs and L2 learners. Using image-schematic diagrams as primes, we examine whether any mental imagery effect is modulated by target preposition (over, in), the abstractness of meaning (spatial, extended), and stimulus onset asynchrony (SOA; 1,040 ms, 2,040 ms). A total of 79 adult L2 learners and 100 NSs of English completed diagram–picture matching and semantic priming phrasal decision tasks. Results revealed interference effects on L2 processing of over phrases and under 2,040 ms SOA, but no such effects were observed in the NS group. The selective interference effects in L2 suggest different mental imagery patterns between L1 and L2 processing, and processing schematic diagram primes requires high cognitive demands, potentially leading to difficulties in integrating visual and linguistic information and making grammaticality judgments. The findings partially validate schematic diagrams as visual representations of concepts and suggest the need for further examination of schematic diagrams with varying degrees of complexity.

A commonly held assumption is that demonstration and pantomime differ from ordinary action in that the movements are slowed down and exaggerated to be better understood by intended receivers. This claim has, however, been based on meagre empirical support. This article provides direct evidence that the different functional demands of demonstration and pantomime result in motion characteristics that differ from those for praxic action. In the experiment, participants were dressed in motion capture suits and asked to (1) perform an action, (2) demonstrate this action so that somebody else could learn how to perform it, (3) pantomime this action without using the object so that somebody else could learn how to perform it, and (4) pantomime this action without using the object so that somebody else could distinguish it from another action. The results confirm that actors slow down and exaggerate their movements in demonstrations and pantomimes when compared to ordinary actions.

The literature on face emojis raises the central question whether they should be treated as pictures or conventionalized signals. Our experiment addresses this question by investigating semantic differences in visually similar face emojis. We test a prediction following from a pictorial approach: small visual features of emojis that do not correspond to human facial features should be semantically less relevant than features that represent aspects of facial expressions. We compare emoji pairs with a visual difference that either does or does not correspond to a difference in a human facial expression according to an adaptation of the Facial Action Coding System. We created two contexts per pair, each fitted to correspond to a prominent meaning of one or the other emoji. Participants had to choose a suitable emoji for each context. The rate at which the context-matching emoji was chosen was significantly above chance for both types of emoji pairs and it did not differ significantly between them. Our results show that the small differences are meaningful in all pairs whether or not they correspond to human facial differences. This supports a lexicalist approach to emoji semantics, which treats face emojis as conventionalized signals rather than mere pictures of faces.

In everyday life, visual information often precedes the auditory one, hence influencing its evaluation (e.g., seeing somebody’s angry face makes us expect them to speak to us angrily). By using the cross-modal affective paradigm, we investigated the influence of facial gestures when the subsequent acoustic signal is emotionally unclear (neutral or produced with a limited repertoire of cues to anger). Auditory stimuli spoken with angry or neutral prosody were presented in isolation or preceded by pictures showing emotionally related or unrelated facial gestures (angry or neutral faces). In two experiments, participants rated the valence and emotional intensity of the auditory stimuli only. These stimuli were created from acted speech from movies and delexicalized via speech synthesis, then manipulated by partially preserving or degrading their global spectral characteristics. All participants relied on facial cues when the auditory stimuli were acoustically impoverished; however, only a subgroup of participants used angry faces to interpret subsequent neutral prosody. Thus, listeners are sensitive to facial cues for evaluating what they are about to hear, especially when the auditory input is less reliable. These results extend findings on face perception to the auditory domain and confirm inter-individual variability in considering different sources of emotional information.

Word-object and action-object learning in children aged 30 to 48 months appears to develop at a similar time scale and adheres to similar attentional constraints. However, children below 36 months show different patterns of learning word-object and action-object associations when this information is presented in a bimodal context (Eiteljoerge et al., 2019b). Here, we investigated 12- and 24-month-olds’ word-object and action-object learning when this information is presented in a unimodal context. Forty 12- and 24-month-olds were presented with two novel objects that were either first associated with a novel label (word learning task) and then later with a novel action (action learning task) or vice versa. In subsequent yoked test phases, children either heard one of the novel labels or saw a hand performing one of the actions presented with the two objects on screen while we measured their target looking. Generalized linear mixed models indicate that 12-month-olds learned action-object associations but not word-object associations and 24-month-olds learned neither word- nor action-object associations. These results extend previous findings (Eiteljoerge et al., 2019b) and, together, suggest that children appear to learn action-object associations early in development while struggling with learning word-object associations in certain contexts until 2 years of age.

Social hierarchical information impacts language comprehension. Nevertheless, the specific process underlying the integration of linguistic and extralinguistic sources of social hierarchical information has not been identified. For example, the Chinese social hierarchical verb 赡养, /shan4yang3/, ‘support: provide for the needs and comfort of one’s elders’, only allows its Agent to have a lower social status than the Patient. Using eye-tracking, we examined the precise time course of the integration of these semantic selectional restrictions of Chinese social hierarchical verbs and extralinguistic social hierarchical information during natural reading. A 2 (Verb Type: hierarchical vs. non-hierarchical) × 2 (Social Hierarchy Sequence: match vs. mismatch) design was constructed to investigate the effect of the interaction on early and late eye-tracking measures. Thirty-two participants (15 males; age range: 18–24 years) read sentences and judged the plausibility of each sentence. The results showed that violations of semantic selectional restrictions of Chinese social hierarchical verbs induced shorter first fixation duration but longer regression path duration and longer total reading time on sentence-final nouns (NP2). These differences were absent under non-hierarchical conditions. The results suggest that a mismatch between linguistic and extralinguistic social hierarchical information is immediately detected and processed.

Executive functions (EFs) in both regulatory and meta-cognitive contexts are important for a wide variety of children’s daily activities, including play and learning. Despite the growing literature supporting the relationship between EF and language, few studies have focused on these links during everyday behaviours. Data were collected on 208 children from 6 to 12 years old of whom 89 were deaf children (55% female; M = 8;8; SD = 1;9) and 119 were typically hearing children (56% female, M = 8;9; SD = 1;5). Parents completed two inventories: to assess EFs and language proficiency. Parents of deaf children reported greater difficulties with EFs in daily activities than those of hearing children. Correlation analysis between EFs and language showed significant levels only in the deaf group, especially in relation to meta-cognitive EFs. The results are discussed in terms of the role of early parent–child interaction and the relevance of EFs for everyday conversational situations.

The embodied view of semantic processing holds that readers achieve reading comprehension through mental simulation of the objects and events described in the narrative. However, it remains unclear whether and how the encoding of linguistic factors in narrative descriptions impacts narrative semantic processing. This study aims to explore this issue under the narrative context with and without perspective shift, which is an important and common linguistic factor in narratives. A sentence-picture verification paradigm combined with eye-tracking measures was used to explore the issue. The results showed that (1) the inter-role perspective shift made the participants’ to evenly allocate their first fixation to different elements in the scene following the new perspective; (2) the internal–external perspective shift increased the participants’ total fixation count when they read the sentence with the perspective shift; (3) the scene detail depicted in the picture did not influence the process of narrative semantic processing. These results suggest that perspective shift can disrupt the coherence of situation model and increase the cognitive load of readers during reading. Moreover, scene detail could not be constructed by readers in natural narrative reading.

Conceptual metaphor theory (CMT) holds that most conceptual metaphors are processed unconsciously. However, whether multiple words can be integrated into a holistic metaphoric sentence without consciousness remains controversial in cognitive science and psychology. This study aims to investigate the role of consciousness in processing Chinese nominal metaphoric sentences ‘A是B’ (A is[like] B) with a psychophysical experimental paradigm referred to as breaking continuous flash suppression (b-CFS). We manipulated sentence types (metaphoric, literal and anomalous) and word forms (upright, inverted) in a two-staged experiment (CFS and non-CFS). No difference was found in the breakthrough times among all three types of sentences in the CFS stage, while literal sentences were detected more slowly than either metaphoric or anomalous sentences in the non-CFS stage. The results suggest that the integration of multiple words may not succeed without the participation of consciousness, let alone metaphoric processing. These findings may redefine ‘unconscious’ in CMT as ‘preconscious’ and support the indirect access view regarding how the metaphoric meaning is processed in the brain.