arXiv cs.CL· Qian-Wen Zhang, Biao Liu, Fang Li, Jie Wang, Lingfeng Qiao, Yifei Yu, Di Yin, Xing Sun·· 2 天前AI 评分35
FactGuard：检测不可回答问题以提升长上下文推理可靠性
Towards Reliable Long-Context Reasoning: Detecting Unanswerable Questions via FactGuard
AI 导读
研究者提出 FACTGUARD 多任务工作流及 FACTGUARD-BENCH 基准，包含 25,220 个样本、上下文长度最高 128K，针对证据缺失与误导性证据场景。对 11 个开源与闭源 LLM 的评测显示，随上下文增长模型难以拒绝不可回答问题，最佳模型准确率仅 71.62%；用该基准训练后整体准确率提升至 81.17%。
正文 · 原文
Abstract:Despite the success of Large Language Models (LLMs) in long-context tasks, their reliability in recognizing unanswerable questions remains a critical challenge. In information-seeking scenarios, models often suffer from overconfidence and hallucinations when evidence is missing or misleading. In this paper, we propose FACTGUARD, a systematic multi-task workflow for generating evidence-grounded answerable and unanswerable questions. We introduce FACTGUARD-BENCH, a benchmark comprising 25,220 examples with context lengths up to 128K, specifically targeting Lack of Evidence and Misleading Evidence scenarios. Evaluations of eleven representative open-source and proprietary LLMs reveal a significant reliability gap: models struggle to reject unanswerable questions as context scales, with the leading model achieving only 71.62% accuracy. By training with FACTGUARD-BENCH, we improve overall accuracy to 81.17% and significantly enhance models' reasoned rejection capabilities. Our work provides a scalable path toward building more faithful and trustworthy long-context LLMs. Our source is publicly available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2504.05607 [cs.CL] |
| (or arXiv:2504.05607v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2504.05607 arXiv-issued DOI via DataCite |
|
| Journal reference: | EMNLP 2026 |
Submission history
From: Qian-Wen Zhang [view email]
[v1]
Tue, 8 Apr 2025 01:45:16 UTC (1,225 KB)
[v2]
Thu, 15 Oct 2026 08:24:58 UTC (4,070 KB)
来源：arXiv cs.CL · arxiv.org