arXiv cs.CL· Guangzeng Han, Xiaolei Huang·· 2 天前AI 评分27
什么样的指令微调数据才算好？一项基于上下文学习视角的研究
What Makes Good Instruction-Tuning Data? An In-Context Learning Perspective
AI 导读
研究者提出基于加权上下文影响（wICI）的指令数据选择框架，通过衡量候选样本对语义相近样本降低指令遵循难度的程度来筛选数据。在多个模型和基准上，该方法在受限数据预算下持续优于现有基线，并发现样本难度与上下文影响呈负相关。该工作已被 ACL 2026 主会接收。
正文 · 原文
Abstract:Instruction-tuning datasets often contain substantial redundancy and low-quality samples, necessitating effective data selection methods. We propose an instruction data selection framework based on weighted in-context influence (wICI), which measures how effectively each candidate example reduces instruction-following difficulty for semantically related peers. Through systematic experiments, we address three key questions: what constitutes effective instruction tuning data from an in-context perspective, whether sample difficulty correlates with in-context influence, and how in-context influence translates to instruction tuning effectiveness. Experiments across multiple models and benchmarks demonstrate that our method consistently outperforms existing baselines under constrained data budgets, while empirically showing that sample difficulty negatively correlates with in-context influence.
| Comments: | ACL 2026, main conference |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2604.25132 [cs.CL] |
| (or arXiv:2604.25132v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.25132 arXiv-issued DOI via DataCite |
Submission history
From: Guangzeng Han [view email]
[v1]
Tue, 28 Apr 2026 02:09:28 UTC (685 KB)
[v2]
Thu, 15 Oct 2026 04:18:21 UTC (685 KB)
来源：arXiv cs.CL · arxiv.org