arXiv cs.CL· Yikai Zhao, Saurabh Pandey, Pradeep Kumar Misra·· 13 小时前AI 评分44
作者写的与 Agent 读到的：企业文档摄取中的静默线索丢失
What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion
AI 导读
主流文档转换器在把 Word/PDF 转成 token 时会丢失作者用格式编码的含义，导致 Agent 自信地答错。在 76 份真实政策文档任务上，忽略格式的摄取使 micro F1 从 0.690 降至 docling 的 0.150 和纯文本的 0.075，该差距在 Claude Opus/Sonnet/Haiku 五款模型与 GPT-5.6 上均复现。
正文 · 原文
Abstract:Enterprise agents act on documents. Before an agent can follow a policy, answer a compliance question or file a ticket, some ingestion layer must turn a Word file or PDF into tokens - and that layer is where an agent deployed in the wild silently loses information. We show that one layer is lost by every mainstream converter - the meaning an author encodes in how text looks - and that the loss makes agents confidently wrong rather than visibly incomplete. We build a task from 76 real policy documents: recover the requirements the author singled out, scored against the file's own OOXML properties by a deterministic matcher with no LLM judge. Formatting-blind ingestion collapses from micro F1 0.690 to 0.150 (docling) and 0.075 (plain text), a gap that replicates across six answering models from two vendors - five Claude models spanning the Opus, Sonnet and Haiku tiers, and OpenAI's GPT-5.6. An agent that cannot see the cue guesses, and docling's false-positive rate on requirements the author did not flag is four times ours. A span-level leave-one-out localises the loss to two cues - highlight (+0.302) and colour (+0.262), both Holm-adjusted p < 1e-4 - while bold, italic, monospace and strikethrough each sit within +/-0.003 of zero. The same asymmetry appears in PDF. Colour is also the cue no mainstream converter preserves and no parsing benchmark scores, and it is common: 32.8% of 6,460 documents carry a deliberate colour or highlight, 10.3% on an obligation. A pre-registered test shows that typing a cue for the agent adds +0.085 beyond merely preserving it (Holm-adjusted p = 0.049). We validate the labels against rendered page images through an independent modality (kappa = 0.72) and bound what wording alone can achieve (AUC 0.664). For agent builders the practical finding is that ingestion, not reasoning, is where this reliability is won or lost, and that the fix is cheap and measurable.
| Comments: | 28 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.30026 [cs.CL] |
| (or arXiv:2610.30026v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30026 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yikai Zhao [view email]
[v1]
Wed, 21 Oct 2026 16:51:25 UTC (802 KB)
来源：arXiv cs.CL · arxiv.org