arXiv cs.AI· Pawel Olszowiec, Paulina Tomaszewska, Maciej \.Zelaszczyk, Michal Byra, Neo Christopher Chung·· 3 天前AI 评分23
循环语言模型潜在推理机制监测研究
Monitoring latent reasoning in looped language models
AI 导读
研究构建了基于机制可解释性的五视角检查协议，对四个家族的八个循环语言模型进行分析，区分了提示仅在循环状态内传递的 carry loops（Ouro、LoopUS）与每次迭代重新提供提示的 re-read loops（Parcae、Raven 家族的 Rec-OLMo 和 Huginn）。结果揭示了循环语言模型在安全性和高效使用方面的关键洞察。
正文 · 原文
Abstract:Looped language models (LMs) reuse one block of layers several times per token generation, promising more latent computation at no additional storage cost. Despite their rising popularity, the analysis of looped LMs is still in its infancy: little is known about whether the iterative transformations actually improve their capabilities, or whether these models remain stable under prompt perturbation. We construct an inspection protocol rooted in Mechanistic Interpretability comprising five diverse lenses. We apply it to examine eight looped LMs from four families. Models differ in many aspects but the key one is whether the prompt is transmitted only within the looped state (carry loops: Ouro, LoopUS) or re-supplied at every iteration (re-read loops: Parcae, and Rec-OLMo & Huginn from the Raven family). The results of our study provide valuable insights concerning safety and efficient use of looped language models.
| Comments: | 39 pages, 25 figures, 11 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.0; I.2.7 |
| Cite as: | arXiv:2610.18127 [cs.AI] |
| (or arXiv:2610.18127v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18127 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pawel Olszowiec [view email]
[v1]
Tue, 13 Oct 2026 10:39:05 UTC (1,501 KB)
来源：arXiv cs.AI · arxiv.org