arXiv cs.CL· Agnibh Karmakar, Mayur Parvatikar, Shreyash Dhoot, Amit Dhanda, Aman Chadha, Kapil Wanaskar, Vinija Jain, Amitava Das·· 10 小时前AI 评分31
Swiss-Knife:可重构的外部化多目标解码时对齐框架
Swiss-Knife: A Framework for Reconfigurable Externalised Multi-Objective Alignment at Decode Time
AI 导读
Swiss-Knife 是一个在解码时进行外部化多目标对齐的框架,将对齐规范作为运行时对象,支持热插拔评分模块、批归一化、成对聚合算子与选择规则。
正文 · 原文
Abstract:Decode-time alignment methods steer a frozen language model by scoring candidate continuations with an external reward and selecting the maximiser. We argue that this shared design is a single degenerate point in a much larger space. We introduce Swiss-Knife, a framework for externalised multi-objective alignment in which the alignment specification is a first-class runtime object: hot-swappable scoring blades, a batch normaliser, a pairwise aggregation operator, and a selection rule. Six axioms characterise the admissible aggregation operators, and we prove a representation theorem: every operator satisfying them has the form $R_i = \sum_{j \neq i} g((\mu_i - \mu_j)/s(\sigma_i,\sigma_j))$, a two-parameter family containing probit and logistic comparison rules and pointwise argmax as named coordinates. Within it, pairwise aggregation is Lipschitz-stable under adversarial reward contamination while argmax is not, and Candidate-Batch Normalization (CBN) makes the weight simplex invariant to the rescalings under which reward models are only ever identified. Our reference instantiation pairs DPO-LoRA blades with an uncertainty-aware pairwise tournament. Sweeping the helpfulness/honesty/harmlessness simplex, it attains the best balanced frontier of six decode-time methods (harmonic $F_1$ 0.797 vs. 0.750 for the strongest baseline, $p < 10^{-14}$) with the lowest refusal rate and highest helpfulness of any arm, and reconfigures its objectives in 0.05 ms with no gradient computation.
| Comments: | 19 pages, 3 figures, includes technical appendix and proofs |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2609.22226 [cs.CL] |
| (or arXiv:2609.22226v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.22226 arXiv-issued DOI via DataCite |
Submission history
From: Agnibh Karmakar [view email]
[v1]
Thu, 3 Sep 2026 01:31:53 UTC (89 KB)
[v2]
Tue, 17 Nov 2026 21:43:37 UTC (89 KB)
[v3]
Thu, 19 Nov 2026 05:41:01 UTC (89 KB)
来源:arXiv cs.CL · arxiv.org