arXiv cs.AI· Slimane Larabi·· 3 天前AI 评分18
超越黑盒：面向文本嵌入的视觉接地
Beyond Black Boxes: Visual Grounding for Text Embeddings
AI 导读
研究者提出一种基于草图的视觉接地框架，将文本表示转换为视觉地形草图，以可视化文本嵌入并探索其在信息检索与推理中的潜力。在 MS-COCO 数据集上的评测显示，SBERT 在严格 top-1 检索准确率上仍占优，但在 top-5 检索上与草图方法表现相当。将 SBERT 与草图方法级联后优于单独使用 SBERT，表明草图表示若直接在文本上训练，或对信息检索有显著潜力。
正文 · 原文
Abstract:In this paper, our goal is to represent text embeddings as sketches for three main reasons. First, to visualize text representation and to explore their potential their potential for AI explainability. Second, to investigate the potential benefits of the sketch-based representation for information retrieval. Third, to explore their potential for reasoning, drawing inspiration from the concept of mental imagery. We propose a sketch-based visual grounding framework that transforms text representations into visual topographic sketches. Our approach aims to emphasis the semantics content encoded in the text embeddings through the corresponding sketch.
Evaluations are conducted on the MS-COCO dataset to examine the utility of these sketches for text retrieval. The comparative study of SBERT and Sketch-based method reveals that SBERT retains an advantage in strict top-1 retrieval accuracy while achieving comparable performance to the sketch-based method in top-5 retrieval. A cascaded approach combining SBERT and the sketch-based method outperforms SBERT alone, suggesting that the Sketch-based representation could offer significant potential for information retrieval if trained directly on text.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18047 [cs.AI] |
| (or arXiv:2610.18047v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18047 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Slimane Larabi [view email]
[v1]
Tue, 13 Oct 2026 09:37:27 UTC (1,648 KB)
来源：arXiv cs.AI · arxiv.org