arXiv cs.CL· Ziang Ye, Wentao Shi, Lei Chen, Zhengzhou Cai, Yu Wang, Zhiyuan Yao, Zihan Wang, Xierui Song, Fuli Feng, Yixin Cao·· 6 天前AI 评分31
ReTRAIL：让长程 LLM 智能体回退环境、复用探索经验
Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents
AI 导读
研究者提出 ReTRAIL 框架，让 LLM 智能体在长程任务中把环境回退到早期状态，同时保留有效进展并复用被放弃分支中学到的知识。
正文 · 原文
Abstract:Large language model agents increasingly operate in long-horizon environments where early mistakes can trigger cascading failures and trap agents in dead-end trajectories. Agents must therefore be able to proactively recover from such states and turn their exploratory experience into guidance for subsequent decisions. We introduce ReTRAIL, a framework that enables agents to rewind the environment to earlier states while preserving valid progress and carrying forward knowledge learned from abandoned exploration. The environment state moves backward while the agent's knowledge continues to move forward. ReTRAIL trains a shared policy with two complementary signals: State-Anchored GRPO compares alternative continuations from revisited states to improve execution, while a branch-advancement reward guides reflection through the progress of subsequent exploration. Experiments across four interactive benchmark MineSweeper, Sokoban, ALFWorld and ScienceWorld show that simply adding a rewind primitive provides limited benefit and can even degrade performance. In contrast, ReTRAIL enables agents to rewind from unproductive states and use insights from abandoned branches to guide subsequent exploration. These results demonstrate the value of coupling environment rewinding with experience reuse within an ongoing task for long-horizon task solving. The code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13058 [cs.AI] |
| (or arXiv:2610.13058v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13058 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziang Ye [view email]
[v1]
Fri, 9 Oct 2026 06:11:39 UTC (1,156 KB)
来源：arXiv cs.CL · arxiv.org