arXiv cs.AI· Jian Hong, Chen Cheng, Quan Liu, Yuhao Chen, Enhong Chen·· 12 小时前AI 评分34
STAIF：面向复杂指令遵循的分阶段优化框架
STAIF: A Stage-wise Optimization for Complex Instruction Following
AI 导读
研究者提出 STAIF 分阶段优化框架，将主观软约束的对齐与客观可验证硬约束的优化解耦：第一阶段用多负样本偏好优化提升对软约束的敏感度，第二阶段用 RLVR 强化硬约束的严格满足。团队同时构建了约 31,000 条中英双语多约束指令数据集 STAINSTRUCT，在代表性基准上取得 SOTA 表现并展现出泛化能力。
正文 · 原文
Abstract:Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often underemphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.
| Comments: | 16 pages, 6 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.22649 [cs.AI] |
| (or arXiv:2607.22649v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.22649 arXiv-issued DOI via DataCite |
Submission history
From: Chen Cheng [view email]
[v1]
Fri, 26 Jun 2026 06:14:07 UTC (10,361 KB)
[v2]
Fri, 16 Oct 2026 10:00:07 UTC (10,395 KB)
来源：arXiv cs.AI · arxiv.org