arXiv cs.AI· Daeyong Kwon, Soyoung Yoon, Seung-won Hwang·· 14 小时前AI 评分34
SAFE：用 LLM 作为验证器实现基于证据的多跳推理框架
SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning
AI 导读
SAFE 是一个 LLM-as-verifier 框架，在生成过程中逐步校验中间推理步骤而非仅判断最终答案，将推理拆解为知识图谱三元组形式的原子化证据单元。在三个多跳 QA 基准上，SAFE 平均提升准确率 8.8 个百分点。该工作已被 EMNLP 2026 主会接收，代码与数据集已公开。
正文 · 原文
Abstract:Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning. We propose SAFE, an LLM-as-verifier framework for evidence-grounded multi-hop QA. Rather than judging only the final answer after generation, SAFE verifies reasoning during generation by checking intermediate steps against the provided passages and previous reasoning trajectory. To make this process checkable, SAFE decomposes reasoning into atomic, evidence-grounded units represented with Knowledge Graph (KG) triples. At train-time, SAFE verifies benchmark supervision under KG-grounded constraints and constructs reliable verifier training data. At inference-time, an external verifier checks each generated step, identifies invalid reasoning, and provides correction feedback before errors propagate. Across three multi-hop QA benchmarks, SAFE improves accuracy by 8.8 pp on average. These results show that evidence-grounded multi-hop QA benefits from shifting LLM-based evaluation from post-hoc answer judgment to stepwise reasoning verification.
| Comments: | Accepted to EMNLP 2026 (Main Conference). Github: this https URL | Dataset: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.01993 [cs.CL] |
| (or arXiv:2604.01993v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.01993 arXiv-issued DOI via DataCite |
Submission history
From: Daeyong Kwon [view email]
[v1]
Thu, 2 Apr 2026 12:59:30 UTC (2,777 KB)
[v2]
Tue, 9 Jun 2026 17:29:55 UTC (2,397 KB)
[v3]
Sun, 23 Aug 2026 03:17:47 UTC (2,397 KB)
[v4]
Thu, 29 Oct 2026 06:25:32 UTC (2,398 KB)
来源：arXiv cs.AI · arxiv.org