arXiv cs.CL· Qianqi Yan, Yichen Guo, Ching-Chen Kuo, Shan Jiang, Hang Yin, Yang Zhao, Xin Eric Wang·· 12 小时前AI 评分38
OmniTrace：面向全模态 LLM 生成时来源归因的统一框架
OmniTrace: A Unified Framework for Generation-Time Source Attribution in Omni-Modal LLMs
AI 导读
OmniTrace 通过读取生成过程中后层注意力，将每句生成内容归因到文本块、图像及音视频秒级来源单元，无需额外前向传播和逐模态调参。在 Qwen3-Omni-30B、MiniCPM-o 4.5 和 Qwen2.5-Omni-7B 上，单次生成即可恢复 leave-one-out 归因 77-96% 的效果，而后者最多需 41 次额外前向。
正文 · 原文
Abstract:An omni-modal language model can answer a question about a multimodal document, a recording, or a video, but it cannot show which part of the input the answer came from. Asking the model to cite its evidence yields what it says it used; measuring what it used has so far required rerunning it many times with parts of the input removed, a cost that grows with every second of audio and video. We show that the generation pass already holds most of this information. OmniTrace attributes each generated sentence to source units (text chunks, images, and seconds of audio or video) by reading late-layer attention as mass over units with no extra forward pass and no per-modality tuning; head weights can be trained where ablation data are available. With fixed causal test setting, the units OmniTrace selects are causally important for responses and for individual sentences in video, audio, and image-text question answering on Qwen3-Omni-30B, MiniCPM-o 4.5 and Qwen2.5-Omni-7B. With one generation pass, OmniTrace recovers 77-96% of the effect of leave-one-out attribution, which needs up to 41 extra passes. OmniTrace also agrees best with human-marked evidence: its response-level rankings reach an AUROC of 0.83 on documents and 0.64 on video, against 0.76 and 0.60 for the strongest baseline.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Multimedia (cs.MM) |
| Cite as: | arXiv:2604.13073 [cs.CL] |
| (or arXiv:2604.13073v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.13073 arXiv-issued DOI via DataCite |
Submission history
From: Qianqi Yan [view email]
[v1]
Fri, 20 Mar 2026 17:25:00 UTC (1,585 KB)
[v2]
Thu, 15 Oct 2026 20:15:24 UTC (4,404 KB)
来源：arXiv cs.CL · arxiv.org