arXiv cs.CL· Dongzhuo Li, Fangzhou Cheng, Weiwei Li, Kevin Han, Haixu Ma, Zhongchang Sun, Ruiyi Li, Serena Li, Xiangjun Fan, Lizhu Zhang, Yuandong Tian·· 3 天前AI 评分45
TreeFlex：让智能体在自己的搜索树上导航
TreeFlex: Agents Navigating Their Own Search Trees
AI 导读
TreeFlex 是一个让智能体自身策略决定搜索位置的框架：模型以 Reflection 和 Execution 两种角色，在 harness 维护的搜索树上选择加深、分支、回访或停止，并用树感知的 GRPO 联合训练导航、任务动作、检查点与方案评审。
正文 · 原文
Authors:Dongzhuo Li, Fangzhou Cheng, Weiwei Li, Kevin Han, Haixu Ma, Zhongchang Sun, Ruiyi Li, Serena Li, Xiangjun Fan, Lizhu Zhang, Yuandong Tian
Abstract:A ReAct-style agent accumulates observations, intermediate results, and failed attempts in a single context. As this context grows, stale and off-path information consumes its limited capacity and obscures relevant evidence, making focused reasoning increasingly difficult. We introduce TreeFlex, a framework in which the agent's own policy decides where to search. The agent records each intermediate state, with a short summary, as a node in a search tree maintained by the harness. The policy is a single language model that acts in two roles. In the Reflection role, it has a global view of the tree through these summaries and picks a node to resume, together with an instruction. In the Execution role, the harness restores the path that led to that node, and the model carries out the instruction with the usual task tools until it saves a new node or proposes a solution, which Reflection then approves or sends back to another node. Through these choices, the policy decides whether to deepen the current line, branch from or revisit an earlier state, verify a proposal, or stop, without a fixed traversal order or a score for each node. We train navigation, task actions, checkpointing, and proposal review jointly with a tree-aware adaptation of GRPO, using task-level outcome rewards augmented by process rewards. After small-scale RL of a 4B model, with the same active-context limit for all methods, TreeFlex reaches 48.8% accuracy on a challenging deep-research benchmark, versus 41.5% for flat ReAct and 42.8% for FoldAgent, and its Pass@8 is about ten points higher than either. On SWE-bench Verified, with all three systems trained with RL from the same 9B model, TreeFlex resolves 54.9% of tasks, versus 43.3% for FoldAgent and 45.2% for its step-wise variant.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20334 [cs.AI] |
| (or arXiv:2610.20334v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20334 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Dongzhuo Li [view email]
[v1]
Wed, 14 Oct 2026 15:16:50 UTC (1,453 KB)
来源：arXiv cs.CL · arxiv.org