arXiv cs.CL· Zhiyun Shi·· 12 小时前AI 评分52
VLMs Read Aloud：VLM 识别与使用视觉文本之间的差距
VLMs Read Aloud: The Gap Between Recognizing and Using Visual Text
AI 导读
论文提出 VLM 存在“读用差距”，即模型能逐字转录图像中的文字，却仍无法据此作答；把转录文本回喂给模型后，其表现接近直接读原文。识别与使用对分辨率要求不同，Qwen3-VL-8B 的识别在 15 像素字符时饱和，使用要到 24 像素才跟上，从 2B 扩到 32B 只提升识别、不缩小差距。
正文 · 原文
Abstract:Visual-text compression renders long contexts as images and is engineered around how much of the text a vision-language model (VLM) can recover. Yet recovered text is not usable text. VLMs can transcribe the text in an image word for word and still fail to answer from it; given back their own transcript, they answer nearly as well as from the original text. We call this difference the read-use gap. Recognizing and using visual text have different resolution requirements: Qwen3-VL-8B's recognition saturates at 15-pixel characters while use catches up only at 24 pixels, and scaling from 2B to 32B raises recognition without narrowing the gap. Putting the evidence into words closes it: thinking VLMs recite the evidence in their thinking, and their own transcript, written without seeing the question, recovers most of what thinking adds. In a preregistered held-out test, the gap ranks how much eight VLMs gain from their own words over pixels (Spearman $\rho$ = 0.86), which recognition rates do not. We propose ALOUD, which spends visual resolution on finding and recognizing evidence and language tokens on using it. Without keeping the source text, it doubles the accuracy of a frozen Qwen3-VL-8B on long documents compressed 5x and 10x: at 5x, from 27.7% to 54.8%, near the original text's 58.5%, with a context smaller than the compressed pages themselves. For less than the cost of reading the original text, it leads at equal budget every method that does without it, at every level, and beats AGAR and VERA, which paste the original text back; at 5x it also beats LensVLM-9B, which is trained and keeps the original text. Visual-text compression is limited not by how much text a model can recover, but by how much it can use.
| Comments: | 23 pages, 7 figures, 21 tables |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.01517 [cs.CV] |
| (or arXiv:2611.01517v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01517 arXiv-issued DOI via DataCite |
Submission history
From: Zhiyun Shi [view email]
[v1]
Sun, 1 Nov 2026 03:51:38 UTC (3,930 KB)
[v2]
Tue, 3 Nov 2026 12:49:32 UTC (3,930 KB)
来源：arXiv cs.CL · arxiv.org