arXiv cs.AI· Yuheng Zhang, Jie Zhang, Yuecong Min, Yunwei Zhao, Meichen Liu, Qiankun Li, Shiguang Shan·· 12 小时前AI 评分20
PI-MLLM：受心理物理学启发的多模态大语言模型，用于解压视频情感分析
PI-MLLM: Psychophysics-Inspired Multimodal Large Language Model for Oddly Satisfying Video Emotion Analysis
AI 导读
研究者构建了 OSVD 数据集，包含 12,000 段标注的 5 秒片段，每段在纯视频、纯音频和视听三种条件下提供观众情感标签及感官证据解释。基于该数据集提出 PI-MLLM，融合模态特定动态对齐（MSDA）与基于联盟的视听归因（CAVA），后者用 Shapley 归因判断主导模态及视听证据是协同还是冲突。实验显示该模型在观众情感预测和感官解释质量上均有提升。
正文 · 原文
Abstract:Oddly Satisfying Videos (OSVs) can evoke strong viewer responses through repetitive motion, visual tactility, and ASMR-like sounds despite containing few explicit semantic emotion cues. This makes OSVs a challenging setting for Video Emotion Analysis (VEA), since most existing methods focus on semantic video content and under-model the physical sensory dynamics that shape viewer responses. To support Oddly Satisfying Video Emotion Analysis (OSVEA), we construct the Oddly Satisfying Video Dataset (OSVD), comprising 12,000 annotated 5-second clips. Each clip provides viewer-emotion labels under video-only, audio-only, and audio-visual conditions, together with explanations grounded in visual and acoustic sensory evidence. Building on OSVD, we propose PI-MLLM, a psychophysics-inspired multimodal large language model that integrates Modality-Specific Dynamics Alignment (MSDA) and Coalition-Based Audio-Visual Attribution (CAVA). MSDA uses auxiliary prediction objectives to align modality representations with visual motion and acoustic energy-onset dynamics. CAVA evaluates emotion scores across modality coalitions and applies Shapley attribution to identify the dominant modality and determine whether visual and auditory evidence is synergistic or conflicting. Experiments show that PI-MLLM improves both viewer-side emotion prediction and sensory-grounded explanation quality, suggesting the value of physical sensory cues to video emotion analysis.
| Comments: | 12 pages, 5 figures, 8 tables; includes supplementary material |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28264 [cs.CV] |
| (or arXiv:2610.28264v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28264 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuheng Zhang [view email]
[v1]
Tue, 20 Oct 2026 14:20:39 UTC (794 KB)
来源：arXiv cs.AI · arxiv.org