Distributional semantic representations were used to investigate crossmodal correspondences within language, offering a comprehensive analysis of how sensory experiences interconnect in linguistic constructs. By computing semantic proximity between words from different sensory modalities, a crossmodal semantic network was constructed, providing a general view of crossmodal correspondences in the English language. Community detection techniques were applied to unveil domains of experience where crossmodal correspondences were likely to manifest, while also considering the role of affective dimensions in shaping these domains. The study revealed the existence of an architecture of structured domains of experience in language, whereby crossmodal correspondences are deeply embedded. The present research highlights the roles of emotion and statistical associations in the organization of sensory concepts across modalities in language. The domains identified, including food, the body, the physical world and emotions/values, underscored the intricate interplay between the senses, emotion and semantic patterns. These findings align with the embodied lexicon hypothesis and the semantic coding hypothesis, emphasizing the capacity of language to capture and reflect crossmodal correspondences’ emotional and perceptual subtleties in the form of networks, while also revealing opportunities for further perceptual research on crossmodal correspondences and multisensory integration.

Self-recognition is of great significance to our sense of self. To date, disturbances in the processing of visual self-recognition are well studied in people with schizophrenia, whereas relatively few studies have focused on the processing of self in other domains, such as auditory. An investigation of auditory self-recognition contributes to delineate changes related to self and the potential roots of the described psychopathological aspects connoting schizophrenia. By applying unimodal task and multisensory test, this study investigated auditory self-recognition in people with schizophrenia under unimodal and bimodal circumstances. Forty-six adults diagnosed with schizophrenia and thirty-two healthy controls were involved in this study. Results suggested that people with schizophrenia seemed to have significantly lower perceptual sensitivity in detecting self-voice, and also showed stricter judgment criteria in self-voice decision. Furthermore, in the presentation of stimuli that combined the stimulation of others’ faces with one’s own voice, people with schizophrenia mistakenly attributed the voices of others as their own. In conclusion, altered auditory self-recognition in people with schizophrenia was found.

Studies suggest a greater reliance on visual information for maintaining balance in anxious subjects. Nevertheless, links between this supposed preferred visual processing and spatial orientation have not yet been evaluated. Two groups of subjects differing in their level of trait anxiety were formed. Equipped with a head-mounted visual display, they learned a virtual corridor using passive translation but active rotation, both with normal and with two different conflicting sensory conditions. After two visual navigation trials in the corridor, they were blindfolded and asked to reproduce the same trajectory from memory. In addition, subjects drew a map of the remembered corridor. Anxious subjects were comparable to non-anxious subjects when asked to reproduce the trajectory from memory, but exhibited a deficit when drawing a map of the corridor they were in. The results do not support the hypothesis that anxious subjects use preferentially one type of sensory cue over another for spatial orientation, but instead manifest difficulties in constructing more global representations of space.

The primary aim of this study was to evaluate the effect of auditory feedback in a VR system planned for clinical use and to address the different factors that should be taken into account in building a bimodal virtual environment (VE). We conducted an experiment in which we assessed spatial performances in agoraphobic patients and normal subjects comparing two kinds of VEs, visual alone (Vis) and auditory–visual (AVis), during separate sessions. Subjects were equipped with a head-mounted display coupled with an electromagnetic sensor system and immersed in a virtual town. Their task was to locate different landmarks and become familiar with the town. In the AVis condition subjects were equipped with the head-mounted display and headphones, which delivered a soundscape updated in real-time according to their movement in the virtual town. While general performances remained comparable across the conditions, the reported feeling of immersion was more compelling in the AVis environment. However, patients exhibited more cybersickness symptoms in this condition. The result of this study points to the multisensory integration deficit of agoraphobic patients and underline the need for further research on multimodal VR systems for clinical use.

Individuals are constantly bombarded by sensory stimuli across multiple modalities that must be integrated efficiently. Multisensory integration (MSI) is said to be governed by stimulus properties including space, time, and magnitude. While there is a paucity of research detailing MSI in aging, we have demonstrated that older adults reveal the greatest reaction time (RT) benefit when presented with simultaneous visual-somatosensory (VS) stimuli. To our knowledge, the differential RT benefit of visual and somatosensory stimuli presented within and across spatial hemifields has not been investigated in aging. Eighteen older adults (Mean = 74 years; 11 female), who were determined to be non-demented and without medical or psychiatric conditions that may affect their performance, participated in this study. Participants received eight randomly presented stimulus conditions (four unisensory and four multisensory) and were instructed to make speeded foot-pedal responses as soon as they detected any stimulation, regardless of stimulus type and location of unisensory inputs. Results from a linear mixed effect model, adjusted for speed of processing and other covariates, revealed that RTs to all multisensory pairings were significantly faster than those elicited to averaged constituent unisensory conditions (p < 0.01). Similarly, race model violation did not differ based on unisensory spatial location (p = 0.41). In summary, older adults demonstrate significant VS multisensory RT effects to stimuli both within and across spatial hemifields.

We investigated the effect of auditory–visual sensory integration on visual tasks that were predominantly dependent on parvocellular processing. These tasks were (i) detecting metacontrast-masked targets and (ii) discriminating orientation differences between high spatial frequency Gabor patch stimuli. Sounds that contained no information relevant to either task were presented before, synchronized with, or after the visual targets, and the results were compared to conditions with no sound. Both tasks used a two-alternative forced choice technique. For detecting metacontrast-masked targets, one interval contained the visual target and both (or neither) intervals contained a sound. Sound–target synchrony within 50 ms lowered luminance thresholds for detecting the presence of a target compared to when no sound occurred or when sound onset preceded target onset. Threshold angles for discriminating the orientation of a Gabor patch consistently increased in the presence of a sound. These results are compatible with sound-induced activity in the parvocellular visual pathway increasing the visibility of flashed targets and hindering orientation discrimination.