This article sheds light on the significant yet nuanced roles of shame and guilt in influencing moral behaviour, a phenomenon that became particularly prominent during the COVID-19 pandemic with the community’s heightened desire to be seen as moral. These emotions are central to human interactions, and the question of how they are conveyed linguistically is a vast and important one. Our study contributes to this area by analysing the discourses around shame and guilt in English and Japanese online forums, focusing on the terms shame, guilt, haji (‘shame’) and zaiakukan (‘guilt’). We utilise a mix of corpus-based methods and natural language processing tools, including word embeddings, to examine the contexts of these emotion terms and identify semantically similar expressions. Our findings indicate both overlaps and distinct differences in the semantic landscapes of shame and guilt within and across the two languages, highlighting nuanced ways in which these emotions are expressed and distinguished. This investigation provides insights into the complex dynamics between emotion words and the internal states they denote, suggesting avenues for further research in this linguistically rich area.

Word-object and action-object learning in children aged 30 to 48 months appears to develop at a similar time scale and adheres to similar attentional constraints. However, children below 36 months show different patterns of learning word-object and action-object associations when this information is presented in a bimodal context (Eiteljoerge et al., 2019b). Here, we investigated 12- and 24-month-olds’ word-object and action-object learning when this information is presented in a unimodal context. Forty 12- and 24-month-olds were presented with two novel objects that were either first associated with a novel label (word learning task) and then later with a novel action (action learning task) or vice versa. In subsequent yoked test phases, children either heard one of the novel labels or saw a hand performing one of the actions presented with the two objects on screen while we measured their target looking. Generalized linear mixed models indicate that 12-month-olds learned action-object associations but not word-object associations and 24-month-olds learned neither word- nor action-object associations. These results extend previous findings (Eiteljoerge et al., 2019b) and, together, suggest that children appear to learn action-object associations early in development while struggling with learning word-object associations in certain contexts until 2 years of age.

A commonly held assumption is that demonstration and pantomime differ from ordinary action in that the movements are slowed down and exaggerated to be better understood by intended receivers. This claim has, however, been based on meagre empirical support. This article provides direct evidence that the different functional demands of demonstration and pantomime result in motion characteristics that differ from those for praxic action. In the experiment, participants were dressed in motion capture suits and asked to (1) perform an action, (2) demonstrate this action so that somebody else could learn how to perform it, (3) pantomime this action without using the object so that somebody else could learn how to perform it, and (4) pantomime this action without using the object so that somebody else could distinguish it from another action. The results confirm that actors slow down and exaggerate their movements in demonstrations and pantomimes when compared to ordinary actions.

The embodied view of semantic processing holds that readers achieve reading comprehension through mental simulation of the objects and events described in the narrative. However, it remains unclear whether and how the encoding of linguistic factors in narrative descriptions impacts narrative semantic processing. This study aims to explore this issue under the narrative context with and without perspective shift, which is an important and common linguistic factor in narratives. A sentence-picture verification paradigm combined with eye-tracking measures was used to explore the issue. The results showed that (1) the inter-role perspective shift made the participants’ to evenly allocate their first fixation to different elements in the scene following the new perspective; (2) the internal–external perspective shift increased the participants’ total fixation count when they read the sentence with the perspective shift; (3) the scene detail depicted in the picture did not influence the process of narrative semantic processing. These results suggest that perspective shift can disrupt the coherence of situation model and increase the cognitive load of readers during reading. Moreover, scene detail could not be constructed by readers in natural narrative reading.

Executive functions (EFs) in both regulatory and meta-cognitive contexts are important for a wide variety of children’s daily activities, including play and learning. Despite the growing literature supporting the relationship between EF and language, few studies have focused on these links during everyday behaviours. Data were collected on 208 children from 6 to 12 years old of whom 89 were deaf children (55% female; M = 8;8; SD = 1;9) and 119 were typically hearing children (56% female, M = 8;9; SD = 1;5). Parents completed two inventories: to assess EFs and language proficiency. Parents of deaf children reported greater difficulties with EFs in daily activities than those of hearing children. Correlation analysis between EFs and language showed significant levels only in the deaf group, especially in relation to meta-cognitive EFs. The results are discussed in terms of the role of early parent–child interaction and the relevance of EFs for everyday conversational situations.

Social hierarchical information impacts language comprehension. Nevertheless, the specific process underlying the integration of linguistic and extralinguistic sources of social hierarchical information has not been identified. For example, the Chinese social hierarchical verb 赡养, /shan4yang3/, ‘support: provide for the needs and comfort of one’s elders’, only allows its Agent to have a lower social status than the Patient. Using eye-tracking, we examined the precise time course of the integration of these semantic selectional restrictions of Chinese social hierarchical verbs and extralinguistic social hierarchical information during natural reading. A 2 (Verb Type: hierarchical vs. non-hierarchical) × 2 (Social Hierarchy Sequence: match vs. mismatch) design was constructed to investigate the effect of the interaction on early and late eye-tracking measures. Thirty-two participants (15 males; age range: 18–24 years) read sentences and judged the plausibility of each sentence. The results showed that violations of semantic selectional restrictions of Chinese social hierarchical verbs induced shorter first fixation duration but longer regression path duration and longer total reading time on sentence-final nouns (NP2). These differences were absent under non-hierarchical conditions. The results suggest that a mismatch between linguistic and extralinguistic social hierarchical information is immediately detected and processed.

The literature on face emojis raises the central question whether they should be treated as pictures or conventionalized signals. Our experiment addresses this question by investigating semantic differences in visually similar face emojis. We test a prediction following from a pictorial approach: small visual features of emojis that do not correspond to human facial features should be semantically less relevant than features that represent aspects of facial expressions. We compare emoji pairs with a visual difference that either does or does not correspond to a difference in a human facial expression according to an adaptation of the Facial Action Coding System. We created two contexts per pair, each fitted to correspond to a prominent meaning of one or the other emoji. Participants had to choose a suitable emoji for each context. The rate at which the context-matching emoji was chosen was significantly above chance for both types of emoji pairs and it did not differ significantly between them. Our results show that the small differences are meaningful in all pairs whether or not they correspond to human facial differences. This supports a lexicalist approach to emoji semantics, which treats face emojis as conventionalized signals rather than mere pictures of faces.

In this study, our objective was to explore the impact of hearing loss on the conceptual system underlying word meaning. We collected perceptual strength norms for 200 Italian words from early deaf individuals with limited or no access to auditory information and compared them to existing norms from hearing individuals. For each word, participants provided perceptual strength ratings for each perceptual modality. Our results revealed a significant reduction of the auditory modality in the norms provided by deaf individuals compared to the hearing population. However, we did not observe an overall decrease in reported perceptual strength. Interestingly, we found a heightened involvement of other sensory modalities accompanied by reduced modality exclusivity in the conceptualization of words, indicating that deaf individuals heavily rely on information coming from the other perceptual modalities to form concepts. These findings suggest that hearing loss leads to a reorganisation of word conceptualization, characterised by increased multisensoriality. Importantly, although diminished, the auditory modality remains present, suggesting that deaf individuals can still infer auditory-associated knowledge about words to some extent.

Reading official letters and being able to react to them appropriately is part of the daily life of every adult in many countries. Although the history of the plain language movement dates to the past century, it is only now that efforts are being made in Poland to adapt official documents to their audience. In this paper, we describe the results of a study (N = 685) in which we examined how particular linguistic forms and text structure affect the reception of an official-looking text – comprehension as well as emotions experienced, trust in the office, feelings of helplessness in the situation of reading the writing, and many other variables. Based on Imbir’s (2016, New Ideas in Psychology, 43, 39–49), two-system theory of emotions, we aim to examine how emotions – triggered by these complex stimuli – affect their processing. We hope that the results of our interdisciplinary research will answer the questions and hypotheses posed by researchers and find application in the reforms currently underway. We believe that by linking findings from cognitive psychology, the psychology of emotions and linguistics, we will also expand knowledge in the broad discussion on overcoming barriers between administrative offices and target audiences.

This essay is the introduction to the Special Issue ‘Events in language and mind: Theoretical and empirical advances in the event integration theory’. We first review Leonard Talmy’s event integration theory in addition to some critiques of this framework. Following this, we point to some empirical research inspired by this framework, which explores the interaction between language and cognition. We then briefly introduce the papers in this volume and discuss their contributions to the event integration framework. We conclude with some limitations, questions and future directions.

This paper presents an audiovisual perceptual analysis of the wh-question and wh-exclamation intonation in Brazilian Portuguese using auditory–visual congruent and incongruent stimuli, to investigate the relative importance of each modality in signaling pragmatic meanings. Ten Brazilian Portuguese speakers (five female) were filmed while producing both speech acts 10 times. Next, artificial stimuli were created: audio and visual cues were either matched (audio and video from the same speech act) or mismatched (audio and video from the different speech acts), resulting in 10 congruent and 10 incongruent stimuli of the wh-questions and the wh-exclamations. The perceptual experiment was taken by 36 Brazilians who identified the stimulus as a question or an exclamation. Results from the logistic regression showed that the factor ‘congruence’ was significant and had a significant interaction with ‘speakers’, which means that the congruent stimuli increased the comprehension of the Brazilian Portuguese wh-questions and wh-exclamations. In contrast, the incongruent stimuli tended to lower listeners’ identification, but to a degree depending on individual speakers’ strategies. Although variation in the accuracy of expressing both speech acts was also found across speakers, this study corroborates that the visual channel impacts the perceptual identification of the pragmatic intonation function of distinguishing sentence mode.

In this work, we use a mathematical model of the property listing task dynamics and test its ability to predict processing time in semantic and lexical decision tasks. The study aims at exploring the temporal dynamics of semantic access in these tasks and showing that the mathematical model captures essential aspects of semantic access, beyond the original task for which it was developed. In two studies using the semantic and lexical decision tasks, we used the mathematical model’s coefficients to predict reaction times. Results showed that the model was able to predict processing time in both tasks, accounting for an independent portion of the total variance, relative to variance predicted by traditional psycholinguistic variables (i.e., frequency, familiarity, concreteness imageability). Overall, this study provides evidence of the mathematical model’s validity and generality, and offers insights regarding the characterization of concrete and abstract words.

Embodied imagery hypothesis proposes the activation of perceptual-motor systems during language processing. Previous studies primarily used concrete visual stimuli to investigate mental imagery in language processing by native speakers (NSs) and second language (L2) learners, but few studies employed schematic diagrams. The study aims to investigate mental imagery in processing prepositional phrases by English NSs and L2 learners. Using image-schematic diagrams as primes, we examine whether any mental imagery effect is modulated by target preposition (over, in), the abstractness of meaning (spatial, extended), and stimulus onset asynchrony (SOA; 1,040 ms, 2,040 ms). A total of 79 adult L2 learners and 100 NSs of English completed diagram–picture matching and semantic priming phrasal decision tasks. Results revealed interference effects on L2 processing of over phrases and under 2,040 ms SOA, but no such effects were observed in the NS group. The selective interference effects in L2 suggest different mental imagery patterns between L1 and L2 processing, and processing schematic diagram primes requires high cognitive demands, potentially leading to difficulties in integrating visual and linguistic information and making grammaticality judgments. The findings partially validate schematic diagrams as visual representations of concepts and suggest the need for further examination of schematic diagrams with varying degrees of complexity.

Distributional semantic representations were used to investigate crossmodal correspondences within language, offering a comprehensive analysis of how sensory experiences interconnect in linguistic constructs. By computing semantic proximity between words from different sensory modalities, a crossmodal semantic network was constructed, providing a general view of crossmodal correspondences in the English language. Community detection techniques were applied to unveil domains of experience where crossmodal correspondences were likely to manifest, while also considering the role of affective dimensions in shaping these domains. The study revealed the existence of an architecture of structured domains of experience in language, whereby crossmodal correspondences are deeply embedded. The present research highlights the roles of emotion and statistical associations in the organization of sensory concepts across modalities in language. The domains identified, including food, the body, the physical world and emotions/values, underscored the intricate interplay between the senses, emotion and semantic patterns. These findings align with the embodied lexicon hypothesis and the semantic coding hypothesis, emphasizing the capacity of language to capture and reflect crossmodal correspondences’ emotional and perceptual subtleties in the form of networks, while also revealing opportunities for further perceptual research on crossmodal correspondences and multisensory integration.

Research using masked priming and parafoveal preview techniques has shown that visual letter similarity has an impact on word processing during the initial stages in Latin-derived scripts. However, these effects appear to be absent in Arabic. One reason for this discrepancy could be attributed to the distinctive features of the Arabic script, which includes numerous letters sharing a basic form while varying in the location or number of diacritics. To shed light on this issue, the present study employed Arabic letters rather than words in two masked priming experiments: an alphabetic decision task and a letter-matching task. Both experiments showed that visually similar letters were more effective as primes than visually dissimilar letters. These findings suggest that the processes of letter identification in Arabic and Latin scripts may be roughly alike, implying that differences in visual letter similarity across scripts may arise at later stages of processing.

This study addresses the importance of orofacial gestures and acoustic cues to execute prosodic patterns under different communicative settings in Farsi. Given that Farsi lacks morpho-syntactic markers for polar questions, we aim to determine whether specific facial movements accompany the prosodic correlates of questionhood in Farsi under conditions of degraded information, that is, whispering and wearing face masks. We hypothesise speakers will employ the most pronounced facial expressions when whispering questions with a face mask to compensate for the absence of F0, reduced intensity and lower face invisibility. To this end, we conducted an experiment with 10 Persian speakers producing 10 pairs of statements and questions in normal and whispered speech modes with and without face masks. Our results provide support to our hypotheses that speakers will intensify their orofacial expressions when confronted with marked conditions. We interpreted our results in terms of the ‘hand in hand’ and ‘trade-off’ hypothesis. In whispered speech, the parallel realisation of longer word duration and orofacial expressions may be a compensatory mechanism for the limited options to convey intonation. Also, the lower face coverage is mutually compensated for by word duration and intensified upper facial expressions, all of which in turn support the trade-off hypothesis.

A well-known method of studying iconic words is through the collection of subjective ratings. We collected such ratings regarding familiarity, iconicity, imagery/imageability, concreteness, sensory experience rating (SER), valence and arousal for Mandarin ABB words. This is a type of phrasal compound consisting of a prosaic syllable A and a reduplicated BB part, resulting in a vivid phrasal compound, for example, wù-mángmáng 雾茫茫 ‘completely foggy’. The correlations between the newly collected ABB ratings are contrasted with two other sets of prosaic word ratings, demonstrating that variables that characterize ABB words in an absolute sense may not play a distinctive role when contrasted with other types of words. Next, we provide another angle for looking at ABB words, by investigating to what degree rating data converges with corpus data. By far, the variable that characterizes ABB items consistently throughout these case studies is their high score for imageability, showing that they are indeed rightfully characterized as vivid. Methodologically, we show that it pays off to not take rating data at face value but to contrast it with other comparable datasets of a different phenomenon or data about the same phenomenon compiled in an ontologically different manner.

This study investigated whether speakers use multimodal information (speech and gesture) to differentiate the physical and emotional meanings of the polysemous verb touch. We analyzed 302 hand gestures that co-occurred with this perception verb. For each case, we annotated (1) the meaning of touch (physical vs. emotional), (2) the gesture referent speakers physically touched (other-touch vs. self-touch), (3) the personal pronoun following the verb and (4) if they used intensifiers and negation. There were three main findings. First, we have seen that when speakers express the physical meaning, they are likely to reach an external referent (other-touch), but when they imply the emotional meaning, they tend to touch their own body (self-touch). Second, the most frequent co-speech gesture (chest-touching gesture) was associated with the emotional meaning, uncovering the metaphor the heart is container for emotions. Third, this study showed that the physical meaning of touch usually coexists with a wide variety of personal pronouns and negation words; in contrast, the emotional meaning of touch occurs primarily with the pronoun me and it is usually modified by intensifiers. Thus, speakers use both speech and gesture to differentiate the meanings of the polysemous verb touch.

Japanese pitch accent is phonemic, making it crucial for second-language learners to acquire. Building on theories of multimodal learning, the present study explored how auditory, visual and gestural training of Japanese pitch accent affected behavioral, neural and meta-cognitive aspects of pitch perception across two experiments. Experiment 1 used a between-subjects pre/posttest design to train native English speakers to perceive Japanese pitch accents in one of the following three conditions: (1) baseline (audio + flat notation), (2) pitch height notation (audio + notation mimicking pitch height) and (3) pitch height notation + a left-hand gesture (L-gesture) (to engage the contralateral right hemisphere specialized for suprasegmental pitch processing). Our results indicated that (2) pitch height notation training was most robust in its benefits, as participants in this condition improved on trained and novel words alike. Experiment 2 used a within-subjects design to extend Experiment 1 in three ways: adding a right-hand gesture (R-gesture) condition (to engage more segmental language areas in the left hemisphere), introducing a neural correlate of cognitive load (measured by EEG alpha and theta power) and performing a metacognitive subjective assessment of learning (e.g., ‘Which training did you find the most helpful?’). The results showed that although there were no differences among our four training conditions on learning outcomes or EEG power, participants made the most positive subjective evaluations about pitch height notation and R-gesture training. Together, the results suggest that there may be a ‘just right’ amount of multimodal instruction to boost learning and increase engagement during foreign language pitch instruction.