arXiv cs.CL· Christophe Khalil, Aya Mourad·· 12 小时前AI 评分39
指令税：基础模型在事实召回上可胜过指令微调版本
The Instruction Tax on Factual Recall: Base Models Can Beat Their Instruction-Tuned Counterparts
AI 导读
在 AKBC LM-KBC 2026 共享任务中，基础模型以文本补全方式提示取得 0.6836 分，超过指令微调模型聊天界面的 0.5681 分，也超过 0.2964 的基线两倍以上。任务要求模型在 32B 参数以内、不检索不微调地生成主谓关系对的全部可能宾语。作者将指令微调导致的事实召回下降称为“指令税”，并指出其源于模型权重而非提示方法。
正文 · 原文
Abstract:We present our solution to the AKBC LM-KBC 2026 Shared Task, where we achieved a score of 0.6836 on the official leaderboard, more than double the baseline of 0.2964. This task requires generating all possible objects for subject-relation pairs using a language model with up to 32 billion parameters, with no retrieval or fine-tuning involved. We show that the prompting methodology is as important as the model architecture: while the instruction-tuned model version achieves 0.5681 with its chat interface, the base model achieves 0.6836 when prompted using text completion. Thus, instruction tuning decreases factual recall; we refer to this effect as the instruction tax, which is caused by the model's weights rather than the prompting method. Our final solution is fairly straightforward: we prompt the base model with some examples from the training data, generate 64--256 outputs, and retain only the repeated ones. Two specialized methods are then applied for each relation: one improves the recall of large answer sets, and the other verifies a person's vitality before predicting their place of death.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28407 [cs.CL] |
| (or arXiv:2610.28407v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28407 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aya Mourad [view email]
[v1]
Tue, 20 Oct 2026 15:48:56 UTC (32 KB)
来源：arXiv cs.CL · arxiv.org