arXiv cs.CL· Tardelli Ronan Coelho Stekel·· 13 小时前AI 评分34
联合还是逐选项？1700万到10亿参数紧凑编码器用于闭集决策
Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters
AI 导读
针对闭集决策（如工具选择），研究比较了编码器联合读取选项与逐选项打分两种方式。在3200万参数、三个随机种子的实验中，逐选项打分在37项基准决策套件的留出样本上，相同优化步数下机会校正技能高出5.8分，训练算力近似匹配时高出2.8分，但需5至13倍的编码器前向次数。
正文 · 原文
Abstract:Many language-model pipelines end in a closed-set decision, such as selecting a tool. Prompting a large language model for it is slow and order-sensitive. We ask how small an encoder for such decisions can be and whether it should read the options jointly or separately. In a three-seed study at 32 million parameters, per-option scoring beats joint scoring on a held-out sample of a 37-benchmark decision suite by 5.8 points of chance-corrected skill at matched optimizer steps and by 2.8 at approximately matched training compute; two external benchmarks agree, though not significantly on their unseen tasks. Crossing reader with initialisation shows that the advantage requires a reranker initialisation, whose query-candidate pair input matches per-option reading; from the raw encoder the two readers are indistinguishable. Per-option scoring needs 5 to 13 times more encoder passes: at most twice the median latency, but more than four times the mean. Of six released joint encoders (17 million to 1 billion parameters), the 68- and 150-million-parameter models exceed a 435-million-parameter DeBERTa-v3 zero-shot classifier by 4.9 and 5.9 accuracy points on 209 overlap-audited System1-Arena tasks and are at least on par with it on the 145 tasks unseen in training. The smallest answers a median request in 14 milliseconds on 16 processor threads, against 728 for the classifier. Reordering options changes 12 to 23% of the joint models' answers (per-option scoring: none). Up to 68 million parameters, they lead their size range on a public leaderboard's private, mainly in-distribution tests.
| Comments: | 18 pages, 8 figures, 8 tables. Code: this https URL ; models: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.30970 [cs.CL] |
| (or arXiv:2610.30970v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30970 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tardelli Ronan Coelho Stekel [view email]
[v1]
Thu, 22 Oct 2026 09:14:44 UTC (251 KB)
来源：arXiv cs.CL · arxiv.org