arXiv cs.AI· Jianguo Huang, Yao Long Teng, Yanchen Deng, Bo An·· 3 天前AI 评分31
System One 模型的顺序不变决策：Runoff 方法
Consider, Then Choose: Order-Invariant Decisions in System One Models
AI 导读
针对 System One 模型在选择题中答案依赖选项顺序的问题，研究者提出 Runoff 方法：先对每个选项单独打分保留少量候选，再让每个候选遍历所有位置进行比较，无需训练、两次前向传播即可完成，构造上保证答案与选项顺序无关。在两个开源模型和五个数据集上，Runoff 在打乱选项后答案零变化，而循环平均法变化达 9.3%；20 个选项时准确率相当且快三到四倍。
正文 · 原文
Abstract:System One models answer typed questions in a single forward pass and promise the same answer for the same input. For choice questions, this promise fails: the answer depends on the order of the options, increasingly so as they grow in number, and averaging over their cyclic shifts reduces this dependence without removing it. Scoring each option alone, which the order cannot affect, shows that the order changes the answer mostly when the model is unsure among its top few options, even though the listwise answer looks confident. We therefore propose $\textbf{Runoff}$, which considers, then chooses: it scores each option alone to keep a few candidates, then compares them in listings where each candidate takes every position. It needs no training and can run in two forward passes, and its answer does not depend on the option order by construction. On two open models and five datasets, it changes no answer under shuffling, against 9.3% for cyclic averaging, and with 20 options it is about as accurate and three to four times faster. Wrapped around eight System One models, it removes the order dependence of the five benchmarks, while cyclic averaging changes up to 15.4%.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18233 [cs.AI] |
| (or arXiv:2610.18233v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18233 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianguo Huang [view email]
[v1]
Tue, 13 Oct 2026 12:02:46 UTC (192 KB)
来源：arXiv cs.AI · arxiv.org