arXiv cs.CL· Zehao Lin, Kai Chen, Xixuan Hao, Erenkai Zhu, Chunyu Li, Helan Hu·· 11 小时前AI 评分26
更好的置信度何时能改善知识图谱审计？
When Does Better Confidence Improve a Knowledge-Graph Audit?
AI 导读
研究通过可修复性、选择余量与标签有效性，分析为何LLM构建知识图谱中更好的错误预测对审计改进有限。在Re-DocRED的跨文档图上，错误率×扇出队列比同等动作的联合模型多留3.98-4.74个百分点错误，而开放共享动作可减少0.89-1.43个百分点残差错误。通用LLM验证在四组模型预算配置中的三组上带来提升。
正文 · 原文
Abstract:Confidence estimates guide human review in LLM-built knowledge graphs, but better error prediction can yield little improvement in an audit. We study this gap through repairability, selection headroom and label validity. Modelling non-adaptive auditing as budgeted covering, we characterise the minimal decision-relevant error statistics and bound the regret of scalar-preserving first-order planning by a structural mass $\Delta^{\mathrm{cl}}$. We simulate ideal repair against benchmark-aligned labels for five extraction configurations and two cross-document graphs on Re-DocRED, followed by frozen confirmation on test and replication on 500 hash-selected training documents. On the held-out cross-document graphs, an error-rate $\times$ fan-out queue leaves 3.98-4.74 percentage points more errors than a joint model with the same actions. By comparison, opening shared actions for that joint planner reduces residual errors by 0.89-1.43 points relative to its own fact-only plans, in point estimates. Equivalence between joint and independent-block models remains inconclusive. Generic LLM verification improves three of four model-budget configurations on each held-out set. LLM adjudication identifies possible benchmark omissions.
| Comments: | 27 pages, 4 figures, 19 tables. Ancillary files include a public rendering of the confirmation protocol and summary results |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.05877 [cs.CL] |
| (or arXiv:2611.05877v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05877 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zehao Lin [view email]
[v1]
Wed, 4 Nov 2026 03:12:37 UTC (1,045 KB)
来源：arXiv cs.CL · arxiv.org