arXiv cs.CL· Sheng Xu, Zhen Chen, Junhua Wang, Boyuan Huang, Ke Jia, Jiadun Zhu, Yiming Xu·· 2 天前AI 评分18
仅幅度 FFN 干预实现工具结构化 LLM 推理：方法、门控评估协议与跨模型实证结果
Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results
AI 导读
研究提出 Amplitude Gating（AG）推理时干预方法，在不改动预训练权重的前提下屏蔽 FFN 激活坐标，通过六处 SwiGLU 位点区分共享与仅门控输入，并用幅度排序熵掩码及 Newton-Schulz 窗口排序变体提供无需重训练的前向路径。
正文 · 原文
Abstract:We study Amplitude Gating (AG), an inference-time intervention that masks feed-forward network (FFN) activation coordinates while keeping pretrained weights fixed. Six SwiGLU sites separate shared and gate-only inputs, expanded gate/value channels, and the FFN output. Magnitude-ranked entropy masks and a Newton-Schulz windowed ranking variant provide alternative forward paths without retraining. The corrected September collection covers three Qwen checkpoints and eight text/tool endpoints. Under frozen local text scorers, the displayed Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61 and +2.67 percentage points (pp), and static ToolBench in/out-domain endpoints by +1.64 and +2.82 pp. Comparing the lower historical/September AG mean with the higher baseline mean leaves positive margins on these routes. This supports robustness to the two observed executions, rather than formal statistical significance. JSON gains are small and displayed L-Eval effects are negative. These endpoints measure lexical or answer/action-name agreement rather than executed-tool success. Protocol-specific offline selector and Per-Sample Fix-Harm Evaluation (PFHE) analyses complement the fixed-route results. Current experimental data do not establish a significant random-forest (RF) benefit; further experiments are needed. AG provides a task-dependent intervention surface, with causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost still open.
| Comments: | 48 pages, 11 figures, 29 tables. Revised abstract, introduction, and methods; expanded figures; clarified intervention-site definitions, baseline variation, and RF results |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.11183 [cs.CL] |
| (or arXiv:2607.11183v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.11183 arXiv-issued DOI via DataCite |
Submission history
From: Zhen Chen [view email]
[v1]
Mon, 13 Jul 2026 07:28:55 UTC (2,045 KB)
[v2]
Tue, 14 Jul 2026 08:45:42 UTC (2,045 KB)
[v3]
Wed, 5 Aug 2026 06:56:38 UTC (1,708 KB)
[v4]
Thu, 15 Oct 2026 01:06:08 UTC (2,084 KB)
来源：arXiv cs.CL · arxiv.org