arXiv cs.CL· Shashi Kumar, Esa\'u Villatoro-Tello, Sergio Burdisso, Kadri Hacioglu, Thibault Ba\~neras-Roux, Hasindri Watawana, Dairazalia Sanchez-Cortes, Srikanth Madikeri, Petr Motlicek, Andreas Stolcke·· 12 小时前AI 评分20
面向 LLM 语音识别的对话音频上下文抽象压缩
Abstract Compression of Conversational Audio Context for LLM-based ASR
AI 导读
研究提出 Abstract Compression，将前序轮次的音频序列压缩为固定数量的 latent token 并显式保留对应转写文本，以降低 LLM-based ASR 使用对话上下文的开销。经监督多轮训练后，对话上下文主要帮助识别上下文实体；在域内和域外测试集上，压缩模型以更小的前序音频占用恢复了部分原始上下文条件带来的增益。论文已被 IEEE SLT 2026 接收。
正文 · 原文
Abstract:Standard LLM-based speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether multimodal context from prior turns improves LLM-based ASR and how to represent that context efficiently. We find that, after supervised multi-turn training, conversational context mainly helps with the recognition of contextual entities. However, conditioning on raw context is expensive because the prior-turn audio embedding sequence grows rapidly with conversation length. To address this, we propose Abstract Compression, a learned latent abstraction of prior-turn audio that replaces each prior-turn audio sequence with a fixed number of latent tokens while retaining corresponding transcripts explicitly. On both in-domain and out-of-domain test sets, the compression-enabled model recovers part of the gains of raw-context conditioning with a smaller prior-turn audio footprint. We also provide targeted analyses of the compression setup and its tradeoffs.
| Comments: | 7 pages; accepted to IEEE SLT 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2603.26246 [cs.CL] |
| (or arXiv:2603.26246v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.26246 arXiv-issued DOI via DataCite |
Submission history
From: Shashi Kumar [view email]
[v1]
Fri, 27 Mar 2026 10:09:30 UTC (1,818 KB)
[v2]
Tue, 20 Oct 2026 16:12:39 UTC (1,860 KB)
来源：arXiv cs.CL · arxiv.org