arXiv cs.AI· Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge, Ashmita Kapoor, Tanya Dixit·· 2 天前AI 评分40
Agent Gym：通过人在回路反馈持续评估与进化 LLM 智能体的框架
Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
AI 导读
研究团队提出 Agent Gym，一个模块化、领域无关的框架，可将任意现有 LLM 智能体包裹进持续评估与进化循环，无需修改智能体源码即可在部署后修正行为。框架包含 Act、Evaluate、Investigate、Correct、Learn、Observe 六项可组合能力，核心技术为带 21 个条件算子和三级动作的确定性-LLM 混合修正引擎，以及三层调查架构。
正文 · 原文
Abstract:Large Language Model (LLM) agents deployed in production environments face a fundamental tension: the agent's behavior is frozen at deployment time, while the business rules and edge cases it must handle continue to evolve. Existing approaches address agent construction and one-time evaluation but provide no structured mechanism for continuous post-deployment behavioral correction without modifying the agent's source code. Most of the approaches offered in the market, require intense collection of logs and traces, and re-examining the agent design by the engineering team, a process which is heavy, long and negates the economical value of agentic transformation. We introduce Agent Gym, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop. The framework provides six composable capabilities --- Act, Evaluate, Investigate, Correct, Learn, and Observe --- organized across three architectural zones: a constitution layer that codifies domain knowledge in configuration artifacts, a runtime inference pipeline that chains acting, investigation, and adaptive correction, and a learning loop that enables subject matter experts to discover and validate new correction rules through natural language interaction. The key technical contributions include a hybrid deterministic-LLM correction engine with 21 condition operators and three-tier actions, a three-layer investigation architecture for ground-truth-free compliance validation, and a programmatic safety loop that guarantees rule correctness before human approval. We further introduce the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency. An open-source reference implementation for invoice processing demonstrates that the framework is fully operational and ready for adoption.
| Comments: | 15 pages, 4 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.15591 [cs.AI] |
| (or arXiv:2608.15591v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.15591 arXiv-issued DOI via DataCite |
Submission history
From: Tanya Dixit [view email]
[v1]
Sun, 16 Aug 2026 07:27:26 UTC (28 KB)
[v2]
Wed, 14 Oct 2026 23:32:49 UTC (93 KB)
来源：arXiv cs.AI · arxiv.org