arXiv cs.AI· Pengyang Shi, Jinlu Zhang, Chunze Lin, Yizhou Wang, Li Yi·· 1 天前AI 评分31
CFGComposer：通过 Copula 分数校正实现组合式音频-文本到动作生成
CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction
AI 导读
CFGComposer 是一个联合训练的音频-文本到动作（AT2M）扩散模型，通过 CFG 类比估计 copula 分数 s_I 并以 ε 空间校正注入，弥补朴素 ε 混合（即 Product-of-Experts）遗漏的跨模态交互。
正文 · 原文
Abstract:Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score $s_I$. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score $s_I$: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates $s_I$ via a CFG analogy and injects it as an $\epsilon$-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR $0.9916$, FID $27.18$) with GT-level beat alignment (BA $0.1826$ vs. GT $0.1743$), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15101 [cs.SD] |
| (or arXiv:2611.15101v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15101 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pengyang Shi [view email]
[v1]
Wed, 11 Nov 2026 06:28:18 UTC (8,640 KB)
来源：arXiv cs.AI · arxiv.org