跳到正文
arXiv cs.CL· Agnibh Karmakar, Mayur Parvatikar, Shreyash Dhoot, Amit Dhanda, Aman Chadha, Kapil Wanaskar, Vinija Jain, Amitava Das·· 10 小时前AI 评分31

Swiss-Knife:可重构的外部化多目标解码时对齐框架

Swiss-Knife: A Framework for Reconfigurable Externalised Multi-Objective Alignment at Decode Time

AI 导读

Swiss-Knife 是一个在解码时进行外部化多目标对齐的框架,将对齐规范作为运行时对象,支持热插拔评分模块、批归一化、成对聚合算子与选择规则。

正文 · 原文

View PDF HTML (experimental)

Abstract:Decode-time alignment methods steer a frozen language model by scoring candidate continuations with an external reward and selecting the maximiser. We argue that this shared design is a single degenerate point in a much larger space. We introduce Swiss-Knife, a framework for externalised multi-objective alignment in which the alignment specification is a first-class runtime object: hot-swappable scoring blades, a batch normaliser, a pairwise aggregation operator, and a selection rule. Six axioms characterise the admissible aggregation operators, and we prove a representation theorem: every operator satisfying them has the form $R_i = \sum_{j \neq i} g((\mu_i - \mu_j)/s(\sigma_i,\sigma_j))$, a two-parameter family containing probit and logistic comparison rules and pointwise argmax as named coordinates. Within it, pairwise aggregation is Lipschitz-stable under adversarial reward contamination while argmax is not, and Candidate-Batch Normalization (CBN) makes the weight simplex invariant to the rescalings under which reward models are only ever identified. Our reference instantiation pairs DPO-LoRA blades with an uncertainty-aware pairwise tournament. Sweeping the helpfulness/honesty/harmlessness simplex, it attains the best balanced frontier of six decode-time methods (harmonic $F_1$ 0.797 vs. 0.750 for the strongest baseline, $p < 10^{-14}$) with the lowest refusal rate and highest helpfulness of any arm, and reconfigures its objectives in 0.05 ms with no gradient computation.
Comments: 19 pages, 3 figures, includes technical appendix and proofs
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Machine Learning (stat.ML)
Cite as: arXiv:2609.22226 [cs.CL]
  (or arXiv:2609.22226v3 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.22226

arXiv-issued DOI via DataCite

Submission history

From: Agnibh Karmakar [view email]
[v1] Thu, 3 Sep 2026 01:31:53 UTC (89 KB)
[v2] Tue, 17 Nov 2026 21:43:37 UTC (89 KB)
[v3] Thu, 19 Nov 2026 05:41:01 UTC (89 KB)

来源:arXiv cs.CL · arxiv.org