arXiv cs.CL· Guimin Hu, Qiwei Peng·· 2 天前AI 评分22
FaShift：面向多模态学习的因子级选择性分布偏移处理框架
Beyond Invariance: Factor-wise Selective Treatment of Shifts for Multimodal Learning
AI 导读
FaShift 提出因子级分布偏移处理框架，通过分解多模态表征并评估各因子的分布变化、干预效应与跨环境稳定性，将潜在因子区分为非预测性、稳定预测性和预测有用但不稳定三类，分别进行抑制、保留或衰减处理，而非统一强制不变性。
正文 · 原文
Abstract:Distribution shifts pose a fundamental challenge to robust multimodal learning. Existing methods typically promote invariance by suppressing environment-dependent variations, implicitly applying a uniform treatment to shifted information. However, shifted information can have different predictive consequences: some contributes little to prediction, some remains reliably informative, and some exhibits environment-dependent predictive effects. This raises a fundamental question: which shifted information should be suppressed, preserved, or attenuated? To address this question, we propose FaShift, a factor-wise shift framework that diagnoses latent representation factors according to their distributional changes, intervention effects, and cross-environment stability. By factorizing multimodal representations and evaluating these complementary properties, FaShift distinguishes non-predictive factors, stably predictive factors, and predictively useful but unstable factors. It then selectively suppresses, preserves, or attenuates their contributions, rather than enforcing uniform invariance. Experiments on MOSI, MOSEI, and CH-SIMS show that FaShift achieves the best ID accuracy and F1 across all three datasets. Compared with the strongest competitors reporting OOD results, it improves OOD accuracy/F1 by 1.5/3.6 points on MOSI and 1.7/2.7 points on MOSEI, while achieving the best reported OOD accuracy and F1 on CH-SIMS.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.21403 [cs.CL] |
| (or arXiv:2610.21403v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21403 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Qiwei Peng [view email]
[v1]
Thu, 15 Oct 2026 07:21:45 UTC (6,028 KB)
来源：arXiv cs.CL · arxiv.org