arXiv cs.CL· Guanjie Cheng, Jianfeng Gao, Yufei Tong, Xinkui Zhao, Shuiguang Deng·· 11 小时前AI 评分31
DuSE-Bench：评估对话与文档双源演化下的联合推理基准
DuSE-Bench: Benchmarking Joint Reasoning over Evolving Dialogue and Documents
AI 导读
DuSE-Bench 是一个包含 1,000 个任务、覆盖五种场景的基准，其中 700 个任务所需前提同时来自对话与文档，用于评估双源演化下的联合推理。在 11 种基线配置上，联合接地任务的内容准确率最高仅 56.7%，且内容准确率提升可能伴随响应类型选择变差，内容正确的回答也常缺乏对原始证据的完整归因。
正文 · 原文
Abstract:Persistent Web information assistants must combine evolving document facts with user requirements established and revised across sessions. Yet systematic evaluation of how changes in both sources affect response requirements and evidence attribution remains limited. To address this gap, we introduce Dual-Source Evolution Benchmark (DuSE-Bench), a benchmark comprising 1,000 tasks across five scenarios, including 700 whose required premises originate in both dialogue and documents. An evolving Support Graph guides task construction and preserves links from updated facts and reused conclusions to original turns and passages. Task annotations distinguish grounding source from state dependence and enable separate evaluation of response content, response type, and source attribution. Experiments across 11 baseline configurations reveal persistent challenges in joint reasoning over evolving dialogue and documents, with content accuracy on jointly grounded tasks reaching at most 56.7%. Moreover, improvements in content accuracy can coincide with poorer response choice, while content-correct responses frequently lack complete attribution to original evidence. Matched evidence substitutions further show that the effect of replacing one source's evidence depends on support from the other. These findings motivate coordinating dialogue memory and document retrieval to apply current user requirements and document facts together while preserving their original support. The data are available at this https URL.
| Comments: | 12 pages, 4 figures |
| Subjects: | Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2611.00755 [cs.CL] |
| (or arXiv:2611.00755v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00755 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yufei Tong [view email]
[v1]
Sat, 31 Oct 2026 09:04:59 UTC (1,028 KB)
来源：arXiv cs.CL · arxiv.org