arXiv cs.CL· Zhiyun Shi·· 11 小时前AI 评分42
Legible：让微调写入的新事实能被模型正确读取，破解两跳推理诅咒
Write Facts the Way Models Read Them: Lifting the Two-Hop Curse
AI 导读
研究者提出 Legible 方法，用一个损失项让微调写入的新事实在桥接短语处与模型自身对该实体的读取表示对齐，无需多跳训练数据。在 Qwen2.5、Llama-3 和 Qwen3 上，Legible 在 MQuAKE 两跳诅咒问题上的组合成功率超过模型自身知识，桥接准确率达 CaKE 的 1.8-2.6 倍，并在 RippleEdits 及 14B 规模上解除该诅咒。
正文 · 原文
Abstract:A language model that learns a new fact by fine-tuning can repeat it but cannot reason with it: taught that a person is a citizen of a new country, it still cannot say, without chain-of-thought, who the head of state of that person's country is. This two-hop curse is widely treated as a reasoning deficit, and the strongest knowledge editors answer it with synthesized reasoning data. We show that the reasoning is intact and the writing is wrong. Fine-tuning writes a new fact as an answer to be said, whereas the second hop reads the bridge ("that person's country") as an entity, from the middle layers on: patching the model's own representation of the entity into the bridge at those layers largely restores the second hop, yet after fine-tuning the bridge there is no closer to the new entity than before the edit. We therefore propose read-equivalent writing (Legible): one loss term makes the bridge phrase, at the layer where the second hop finishes reading it, match the model's own reading of the new entity, and keeps the same relation's other subjects unchanged. Phrases and targets come from the edited model itself, and no multi-hop training data is used. Across Qwen2.5, Llama-3 and Qwen3, Legible (I) composes learned facts more often than the model composes its own knowledge, on MQuAKE's two-hop-curse questions; (II) reaches 1.8-2.6x the bridge accuracy of CaKE, whose reasoning data composes no better than plain fine-tuning, and has the best average over composition, accuracy and locality against four families of writers; and (III) lifts the curse on RippleEdits, on the benchmark of Balesni et al., who named the curse, and at 14B.
| Comments: | 16 pages, 4 figures, 15 tables |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.01519 [cs.CL] |
| (or arXiv:2611.01519v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01519 arXiv-issued DOI via DataCite |
Submission history
From: Zhiyun Shi [view email]
[v1]
Sun, 1 Nov 2026 03:52:31 UTC (1,156 KB)
[v2]
Tue, 3 Nov 2026 13:09:26 UTC (1,161 KB)
来源：arXiv cs.CL · arxiv.org