arXiv cs.CL· Jingfeng Chen, Raghuveer Thirukovalluru, Junlin Wang, Kaiwei Luo, Bhuwan Dhingra·· 12 小时前AI 评分31
ACPO：面向长文本问答的自监督原子一致性偏好优化
Atomic Consistency Preference Optimization for Long-Form Question Answering
AI 导读
研究者提出原子一致性偏好优化（ACPO），一种无需外部监督的自监督偏好微调方法，利用多个随机回答中单个事实的一致性信号筛选高质量与低质量数据对，用于模型对齐。在 LongFact 和 BioGen 数据集上，ACPO 在 Phi-3 与 Llama3 上的平均表现超过强监督对齐基线 1.95 分，且不依赖 GPT-4 等更强模型或外部知识库。
正文 · 原文
Abstract:Large Language Models (LLMs) often produce factoid hallucinations - plausible yet incorrect answers. A common mitigation strategy is model alignment, which improves factual accuracy by training on curated (factual, non-factual) pairs. However, this approach often relies on a stronger model (e.g., GPT-4) or an external knowledge base to assess factual correctness that may not always be accessible. Addressing this, we propose Atomic Consistency Preference Optimization (ACPO), a self-supervised preference-tuning method that enhances factual accuracy without external supervision. ACPO leverages atomic consistency signals (i.e., the agreement of individual facts across multiple stochastic responses) to identify high- and low-quality data pairs for model alignment. Despite being fully self-supervised, ACPO outperforms the strong supervised alignment baseline by 1.95 points averaged across Phi-3 and Llama3 on the LongFact and BioGen datasets, demonstrating its effectiveness in improving factual reliability without relying on external models or knowledge bases.
| Comments: | 13 pages, 1 figure |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2505.09039 [cs.CL] |
| (or arXiv:2505.09039v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2505.09039 arXiv-issued DOI via DataCite |
Submission history
From: Raghuveer Thirukovalluru [view email]
[v1]
Wed, 14 May 2025 00:39:47 UTC (2,142 KB)
[v2]
Mon, 10 Nov 2025 04:08:26 UTC (1,855 KB)
[v3]
Fri, 16 Oct 2026 13:28:17 UTC (1,850 KB)
来源：arXiv cs.CL · arxiv.org