arXiv cs.AI· Mingwei Fu, Zheshu Song, Zhikai Zhou, Shen Huang, Shidong Shang, Mengxiao Zhu·· 3 天前AI 评分23
ScreenASR-Bench：带视觉上下文的语音识别分层基准
ScreenASR-Bench: A Tiered Benchmark for Speech Recognition with Visual Context
AI 导读
研究者推出 ScreenASR-Bench，一个面向带视觉上下文语音识别的分层基准，包含显式文本证据（ETE）、显式视觉证据（EVE）和隐式语义证据（ISE）三个递进难度层级。
正文 · 原文
Abstract:Current ASR models perform well on general benchmarks but still struggle with acoustically ambiguous words. Visual context can aid recognition in screen-based scenarios, yet existing benchmarks provide limited insight into the benefits of different types of visual evidence. To address this gap, we introduce ScreenASR-Bench , a tiered benchmark with three progressively challenging levels: Explicit Textual Evidence (ETE), based on direct on-screen text; Explicit Visual Evidence (EVE), using visible content rather than on-screen text; and Implicit Semantic Evidence (ISE), requiring scene-level inference. Extensive experiments on ASR models, Large Audio-Language Models (LALMs) and Omni models show that incorporating visual context substantially improves recall over the audio-only setting across all three model families. The gains are largest for ETE and smallest for ISE, indicating that models leverage direct visual evidence more effectively than indirect semantic cues. To facilitate future research on visually grounded speech recognition, we publicly release ScreenASR-Bench.
| Comments: | 5 pages |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16260 [cs.AI] |
| (or arXiv:2610.16260v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16260 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheshu Song [view email]
[v1]
Mon, 12 Oct 2026 06:36:20 UTC (5,228 KB)
来源：arXiv cs.AI · arxiv.org