arXiv cs.AI· Yufeng Tian, Jiahao Tang, Minghao Wu·· 3 天前AI 评分32
执行后学习：用动作特定信用分配提升策略能力
Learning after Execution with Action-Specific Credit
AI 导读
研究提出"执行后学习"方法，将程序执行结果作为中间状态，为同一策略触发额外的学习交互，并用动作特定信用分别给初始动作和纠正动作分配信用。在 Image-to-TikZ 的 945 个留出样本上，单次调用渲染成功率达 92.70%，高于继续单阶段 GRPO 训练相同迭代次数后的 88.25%。该纠正能力依赖中间状态，无法通过继续单阶段训练复现，代码已开源。
正文 · 原文
Abstract:Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70\%, compared with 88.25\% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14573 [cs.AI] |
| (or arXiv:2610.14573v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14573 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yufeng Tian [view email]
[v1]
Sat, 10 Oct 2026 04:45:16 UTC (1,638 KB)
来源：arXiv cs.AI · arxiv.org