arXiv cs.CL· Zhibo Hu, Chen Wang, Yanfeng Shu, Liming Dong, Hye-young Paik, Liming Zhu·· 12 小时前AI 评分20
语言模型中的方向敏感扰动影响：多项扰动共享失效动态
Many Perturbations, Shared Failure Dynamics: Tracing Direction-Sensitive Perturbation Influence in Language Models
AI 导读
研究提出一种方向敏感、相对读出的语义脆弱性解释：任务保持的改写可让正确答案变错，而更大的内部变化却可能不影响答案。纵向-横向分解与紧凑的储备-传输模型在 Qwen3-32B 和 Gemma 4 31B 的留出族与条目上预测失效，向 ReClor 的迁移有效但依赖参数化。匹配干预显示方向携带标量几何之外的因果信息，但失效很少跨选项顺序持续，说明共享预测规律与上下文相关的方向效应并存。
正文 · 原文
Abstract:Why can a task-preserving rewrite turn a correct answer into a failure while a larger internal change leaves the answer intact? We develop a direction-sensitive, readout-relative account of semantic fragility: heterogeneous language changes can share geometric and readout relationships without sharing an input-independent failure direction. Clean decision reserve, representation transformation, and response alignment provide complementary descriptions of this process. An exact longitudinal-transverse decomposition identifies directional information discarded by scalar transmission. A compact reserve-and-transmission model predicts failures across held-out families and items in Qwen3-32B and Gemma 4 31B, with useful but parameterization-dependent transfer to ReClor. Matched interventions show that orientation carries causal information beyond scalar geometry. Among naturally failed rewrites, attention-component tests on disjoint original items and subsequent answer-order controls favor the natural paired direction over tested random and same-answer-letter donor directions. Yet failures rarely persist across option orders, and current-order corrections use recomputed clean references. Shared predictive regularity thus coexists with context-dependent directional effects. Numerical path integration reduces, but does not eliminate, finite-patch approximation error within a fixed context; a larger discrepancy from the natural rewrite effect remains. A complementary fixed-CoT cue study extends this account to influence unacknowledged in recorded reasoning: readout associations transfer across domains, although the positive-mean causal criterion fails and signed-effect prediction remains post hoc. Together, these results support shared relations among reserve, transformation, and response while distinguishing local causal sensitivity from a complete explanation of natural failure.
| Comments: | 42 pages, 6 figures |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.32309 [cs.CL] |
| (or arXiv:2610.32309v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32309 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhibo Hu [view email]
[v1]
Fri, 23 Oct 2026 08:09:28 UTC (2,309 KB)
来源：arXiv cs.CL · arxiv.org