arXiv cs.CL· Ning Li, Zixuan Guo, Yan Xu, Wenbo Fei, Yifan Niu, Chang Luo, Yasheng Wang, Weiwen Liu, Yong Yu, Weinan Zhang·· 10 小时前AI 评分31
BALTO：面向幻觉缓解的平衡 Token 级策略优化框架
BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
AI 导读
BALTO 提出一种平衡 Token 级策略优化框架，通过提取可核查的事实性声明、对照参考上下文验证并映射为 Token 级标签，将概率质量从无支撑内容重新分配给忠实内容。在 ConFiQA、RAGTruth 和 FinLLM-Eval 上，BALTO 在全部六种模型-基准组合中取得最高忠实度，Q-Score 持续优于现有后训练基线。代码已开源。
正文 · 原文
Abstract:Hallucinations remain a major obstacle to deploying large language models (LLMs) in knowledge-intensive settings, where generated responses must be faithfully grounded in provided evidence. Reinforcement learning (RL) is a promising direction for hallucination mitigation, but response-level faithfulness rewards suffer from a granularity mismatch: localized hallucinations can cause supported content to receive spurious penalties. Although recent work introduces fine-grained feedback such as claim-level verification and token-level rewards, unbalanced credit assignment can still induce length, verbosity, or optimization-noise biases. We propose BALTO, a Balanced Token-level Policy Optimization framework for hallucination mitigation. BALTO extracts checkable factual claims, verifies them against the reference context, and projects claim-level judgments to token-level labels. A balanced token-level credit assignment mechanism is introduced into the framework. This design redistributes probability mass from unsupported content toward faithful content, rather than suppressing the entire response. We systematically analyze the limitations of response-level rewards from a theoretical standpoint, and prove BALTO's advantages in training stability and optimization efficiency for hallucination mitigation. Experiments on ConFiQA, RAGTruth, and FinLLM-Eval show that BALTO achieves the highest faithfulness across all six model--benchmark settings and consistently outperforms existing post-training baselines in Q-Score, demonstrating a stronger faithfulness--informativeness trade-off. The code is released at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.15893 [cs.CL] |
| (or arXiv:2606.15893v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.15893 arXiv-issued DOI via DataCite |
Submission history
From: Ning Li [view email]
[v1]
Sun, 14 Jun 2026 16:25:59 UTC (284 KB)
[v2]
Mon, 22 Jun 2026 03:05:35 UTC (284 KB)
[v3]
Tue, 10 Nov 2026 17:43:32 UTC (291 KB)
来源：arXiv cs.CL · arxiv.org