跳到正文
arXiv cs.AI· Avinash Goutham Aluguvelly·· 10 小时前AI 评分31

权重空间编程修复视觉语言模型:SmolVLM-256M 与 LFM2-VL-450M 实验揭示修复依赖故障计算痕迹

Weight Space Programming for Vision Language Model Repair

AI 导读

研究者在 SmolVLM-256M 和 LFM2-VL-450M 的权重中植入故障,测试编码智能体能否无梯度直接修复权重。智能体在未被告知故障位置的情况下找回了大部分被缩放的投影器行和视觉编码器噪声,但修复依赖 32 位运算在 16 位权重中留下的低位痕迹;抹去这些低位后,LFM2-VL 准确率仅恢复约 4 个点,而未舍入时为 17 个点。作者建议权重修复研究披露权重数值格式并加入舍入对照组。

正文 · 原文

View PDF HTML (experimental)

Abstract:In weight-space programming, a language model edits a network's weights directly, without gradients. We ask what a successful repair of this kind relies on. We plant faults in the weights of two vision-language models, SmolVLM-256M and LFM2-VL-450M, and check each edit made by a coding agent against the original weights. Without being told where to look, the agent finds rescaled projector rows and noise in the vision encoder, infers the unknown scale factors, and returns most or all of the rescaled weights to their exact original values and accuracy to near its original level. These repairs depend on a trace of how the faults were computed. The original weights have 16-bit precision, and faults computed in 32-bit arithmetic leave low bits that only the damaged weights carry, a cue that a real failure need not leave. When we round those bits away, each model answers our test questions as before, but the agent, unaided, no longer repairs either model: on LFM2-VL accuracy recovers by about 4 points, against 17 from the unrounded weights. Marking the damaged weights again in their low bits brings the edits back to them. Rounded weights still hold statistical evidence of the rescaling. On SmolVLM the agent uses it once a sentence in the prompt, which we wrote knowing the fault, names such a statistic; on LFM2-VL, whose scale factors are larger, most sessions find the rescaled rows unaided, but none finds the noise. Swapped rows or columns, which keep the set of stored values, are not found unaided on either model within our budgets. We recommend that studies of weight repair report the number format of the weights and include a rounded control.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23615 [cs.CV]
  (or arXiv:2611.23615v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23615

arXiv-issued DOI via DataCite

Submission history

From: Avinash Goutham Aluguvelly [view email]
[v1] Tue, 17 Nov 2026 23:07:36 UTC (298 KB)
[v2] Thu, 19 Nov 2026 08:12:10 UTC (324 KB)

来源:arXiv cs.AI · arxiv.org