arXiv cs.CL· Ziyu Mu, Zehua Wang·· 14 小时前AI 评分18
Ouro 循环语言模型研究：后期循环如何重定向两跳检索
Cross-Loop Query Handoff: How Later Loops Redirect Two-Hop Retrieval in the Ouro Looped Language Models
AI 导读
研究通过在同层、同注意力头、同最终提示位置间移植注意力 query，考察 Ouro-1.4B 和 Ouro-2.6B 的后期循环如何修复两跳检索。在合成上下文问题中，把 loop-3 的 query 集合注入 loop 1，可救回 90% 和 97% 在 loop 1 答错、loop 3 答对的问题，而打乱来源仅 24% 和 21%。
正文 · 原文
Abstract:How do later loops in a pretrained looped language model repair two-hop retrieval? We study Ouro-1.4B and Ouro-2.6B by transplanting attention queries between loops at the same layer, head and final prompt position. On synthetic in-context questions, injecting the layer-indexed collection of loop-3 queries throughout loop 1 rescues 90% and 97% of questions answered incorrectly at loop 1 and correctly at loop 3, versus 24% and 21% for shuffled donors. The rescue also holds under greedy generation in Ouro-2.6B; in Ouro-1.4B it is limited to four-candidate ranking. Crossed donor-recipient swaps favor retrieval through the donor's bridge entity over copying its answer, although a substantial positional component remains in Ouro-2.6B. Late keys and values rescue far fewer targets. Freezing queries at their loop-1 values also damages two-hop accuracy more than a matched question that supplies the bridge. On natural questions, rescue is smaller and replicates only in Ouro-2.6B, and crossed swaps do not establish bridge redirection. These results identify a transferable collection of late-loop queries that supports bridge-conditioned retrieval on synthetic scenes; they do not locate where the bridge is computed or establish a single-vector handoff.
| Comments: | Code and reproducibility: this https URL 16 pages, 1 figure, 7 tables |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.34639 [cs.LG] |
| (or arXiv:2610.34639v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34639 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zehua Wang [view email]
[v1]
Sun, 25 Oct 2026 23:46:03 UTC (23 KB)
来源：arXiv cs.CL · arxiv.org