arXiv cs.CL· Ziyang Luo, Zhongyao Chu, Xinjie He, Youting Wang, Xukui Qin, Runxiong Wu, Yan-Syuan Chen·· 11 小时前AI 评分40
YOPO：在冻结语言模型的单次前向传播中同时完成回答与弃答
You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model
AI 导读
YOPO 让冻结的语言模型在一次前向传播中同时完成回答与弃答，在 αNLI 上将 Qwen2.5-1.5B 的三分类准确率从 0.38 提升到约 0.75，并在 1.5B、3B、7B 上以一半成本保持与两遍参考方案 0.02 以内的差距。
正文 · 原文
Abstract:A frozen language model under-uses the evidence in its hidden states and answers confidently when the context contains no answer. We fix both on the residual stream without changing any weight. For accuracy, a small trained network (the probe) adds an input-dependent vector to the residual at several middle layers; it raises Qwen2.5-1.5B on $\alpha$NLI from 51% to 73% accuracy. For abstention, a fixed vector (the direction), the mean hidden state of contexts that contain the answer minus that of contexts that do not, scores the hidden state; a low score abstains. On four QA datasets with human answerability labels (Qwen2.5-7B), moving the direction to a dataset it was not fit on costs under 3 points of decision accuracy (from 0.83 in domain) on average and still beats asking the model whether its context suffices; the same judgment trained as a classifier into the probe's activation edit or into LoRA weights loses 16 and 22 points and ends below asking. We set out to put both functions on a single forward pass, and found that the probe's edit changes the very hidden state the direction scores, degrading it. A small map -- trained without labels, only to reconstruct the unedited hidden state from the edited one -- removes 63% of that interference on 1.5B. Refit on the hidden-state pairs of LoFiT or LoReFT, the same recipe repairs those methods too: the steering module is replaceable. The resulting system, YOPO, answers and abstains in one forward pass. On $\alpha$NLI it raises three-way accuracy from 0.38 to about 0.75 on 1.5B, and stays within 0.02 of a two-pass reference on 1.5B, 3B and 7B, at half the cost. We also observe improvements on the four QA datasets, where answerability is human-labeled and answers are generated as free-form text. Near-parity with the two-pass reference holds across nine backbones spanning six model families without backbone-specific hyperparameter tuning.
| Comments: | 31 pages. Ziyang Luo and Zhongyao Chu contributed equally |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.14465 [cs.CL] |
| (or arXiv:2608.14465v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.14465 arXiv-issued DOI via DataCite |
Submission history
From: Ziyang Luo [view email]
[v1]
Fri, 14 Aug 2026 16:44:35 UTC (111 KB)
[v2]
Thu, 29 Oct 2026 19:18:14 UTC (115 KB)
来源：arXiv cs.CL · arxiv.org