arXiv cs.AI· Wenbo Huang, Jianxin Bi, Shunyu Yang, Ruiyang Si, Rui Ni, Qiang Wang, Shulong Jiang, Yintianrun Zhang, Feixiang Ruan, Kewei Zhang, Ye Huang, Shukai Gong, Daquan Zhou·· 12 小时前AI 评分39
学习任务还是学习验证器？重新审视具身智能自我改进中的评判标准暴露
Learning the Task or Learning the Verifier? Rethinking Criteria Exposure in Embodied Self-Improvement
AI 导读
针对具身智能体在机器人仿真基准上的自我改进，研究提出"评判标准暴露"（CE）概念：将基准的成功标准直接暴露给学习智能体后，智能体能快速生成在给定标准下成功率更高的代码，但标准稍作改动（不偏离原指令）后这一优势即消失，且所习得技能跨任务迁移效果差。作者进一步提出"指令忠实标准"，实验显示其更能反映智能体生成程序的真实性能。
正文 · 原文
Authors:Wenbo Huang, Jianxin Bi, Shunyu Yang, Ruiyang Si, Rui Ni, Qiang Wang, Shulong Jiang, Yintianrun Zhang, Feixiang Ruan, Kewei Zhang, Ye Huang, Shukai Gong, Daquan Zhou
Abstract:Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.
| Comments: | 22 pages, 8 figures, 10 tables. Project page: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2611.08318 [cs.AI] |
| (or arXiv:2611.08318v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08318 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenbo Huang [view email]
[v1]
Thu, 5 Nov 2026 17:14:37 UTC (2,921 KB)
来源：arXiv cs.AI · arxiv.org