"Spatiotemporal Dynamics underlying Multi-sensory Processing in the context of Motor Activity and Social Perception

Human perception emerges from the integration of sensory information across space and time, enabling adaptive interaction with the external world. A fundamental challenge for the brain is to dynamically coordinate multisensory signals in relation to ongoing actions and the presence of other agents. The present doctoral thesis investigates the spatiotemporal mechanisms underlying (multi)sensory processing in motor and social contexts, with a particular focus on peripersonal space (PPS) and the visual perception of sociality. Building on theories that describe perception as emerging from the lawful spatiotemporal structure of the sensory world, this work examines how spatial proximity to one’s body and temporal contingency shape sensory integration in dynamic contexts and perception of inter-agents’ movement coupling. Across four experimental studies, behavioral, electrophysiological, immersive virtual reality, and computational approaches are combined to address the aims of this thesis. First, the thesis investigates how motor execution modulates spatially dependent multisensory (audio-tactile) interactions within PPS. Behavioural results from audio– tactile tasks (Experiments 1 & 2) reveal that arm movement execution reduces the well- described impact of looming auditory stimuli on tactile processing, independently of the stimulated body part. Accordingly, EEG evidence (Experiment 3) shows that motor activity selectively alters the temporal dynamics of multisensory integration, primarily affecting late integrative stages rather than early sensory processing. These findings suggest that action-dependent PPS modulations reflect changes in central multisensory processing rather peripheral sensory gating. Second, the thesis examines social modulation of PPS by testing how socially meaningful cues influence multisensory spatial processing (Experiment 4). Using a visuo–tactile task implemented in immersive virtual reality, results demonstrate that social appearance, operationalized through attire in a workplace scenario, systematically reshapes spatially dependent multisensory interaction effects. This may be further modulated by individual differences in leadership style, indicating that multisensory processing underlying PPS representation is sensitive to abstract social attributions and social-cognitive traits. Third, the thesis addresses the temporal dimension of social perception by investigating whether visual inter-agent temporal coupling constitutes a cue for attributing social interactions (Experiment 5). Computational analyses based on motion-energy extraction from third-person videos reveal that temporally correlated motion between agents, quantified via inter-agent motion coupling, reliably predicts observers’ judgments of sociality across multiple independent datasets. These findings support the view that social interaction perception relies on early, visually grounded spatiotemporal mechanisms, consistent with models proposing a dedicated visual pathway for social processing. Together, the results demonstrate that multisensory and social perception are governed by flexible spatiotemporal representations that are dynamically reshaped by action execution and social meaning. The thesis advances a unified framework in which peripersonal space and social perception emerge from shared principles of spatiotemporal contingency, highlighting perception as an adaptive interface for interaction with both objects and other agents.