arXiv cs.CL· Junhao Chen, Yu Huang, Siyuan Li, Rui Yao, Hanqian Li, Hanyu Zhang, Jungang Li, Jian Chen, Bowen Wang, Xuming Hu·· 14 小时前AI 评分34
KnowMT-Bench：面向多轮对话知识密集型长文本问答的基准测试
KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues
AI 导读
研究者提出 KnowMT-Bench，用于系统评估 LLM 在医学、金融、法律等知识密集型领域的多轮长文本问答能力，填补现有基准仅限单轮对话的空白。该基准采用动态评估设置，让模型根据逻辑递进的问题序列自行生成多轮对话历史。实验显示 LLM 在多轮对话中事实能力与信息传递效率均明显下降，RAG 可逆转这一性能退化，主动上下文去噪也有较稳定缓解效果。
正文 · 原文
Abstract:Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains. However, existing benchmarks are limited to single-turn dialogue, while multi-turn dialogue benchmarks typically assess other orthogonal capabilities rather than knowledge-intensive factuality. To bridge this critical gap, we introduce \textbf{KnowMT-Bench}, a new benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields, including medicine, finance, and law. To faithfully assess the model's real-world performance, KnowMT-Bench employs a dynamic evaluation setting where models generate their own multi-turn dialogue histories given logically progressive question sequences. Our experiments on a diverse suite of LLMs show a clear degradation in both factual capability and information delivery efficiency in multi-turn dialogue. We further probe the underlying causes and investigate mitigation strategies spanning domain-specific finetuning, active context denoising, and RAG. The results show domain-dependent gains from finetuning, while active context denoising and RAG more consistently mitigate these issues, with RAG notably reversing this performance degradation. These findings underscore the importance of our benchmark for evaluating LLMs' conversational factual capabilities in real-world applications. Code and data are publicly available.\footnote{\href{this https URL}{\texttt{this https URL}}}
| Comments: | Accepted by EMNLP 2026 Findings |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2509.21856 [cs.CL] |
| (or arXiv:2509.21856v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2509.21856 arXiv-issued DOI via DataCite |
Submission history
From: Yu Huang [view email]
[v1]
Fri, 26 Sep 2025 04:32:29 UTC (1,610 KB)
[v2]
Mon, 26 Oct 2026 07:41:31 UTC (1,646 KB)
来源：arXiv cs.CL · arxiv.org