arXiv cs.AI· Sachin Gupta·· 11 小时前AI 评分31
AgentTrustCI：为 Java LLM 智能体提供构建期策略、接地与轨迹测试的 JUnit 5 框架
AgentTrustCI: Build-Time Policy, Grounding, and Trajectory Tests for Java LLM Agents
AI 导读
AgentTrustCI 是一个基于 Java/JUnit 5 的框架，将 LLM 智能体是否选用允许工具、答案是否基于工具输出、是否遵循既定工作流等应用层预期，表达为对录制轨迹的策略、接地与轨迹检查，并支持确定性回放以实现可重复的构建期测试。
正文 · 原文
Abstract:Infrastructure security does not establish whether an LLM agent selected an allowed tool, grounded its answer in tool output, or followed a required workflow. AgentTrustCI is a Java/JUnit 5 framework that expresses these application-layer expectations as policy, grounding, and trajectory checks over recorded traces. A deterministic replay implementation supports repeatable build-time tests; a judge interface provides an extension point for optional semantic evaluation. On 55 synthetic cases spanning seven enterprise domains, the full pipeline correctly classifies all 30 injected faults and 25 passing controls, including ten non-trivial controls, and misses none of 12 designated critical cases. Each single-layer configuration correctly classifies 35 cases because the corpus deliberately isolates ten faults per layer. A grounded answer obtained through a disallowed sequence illustrates the additional coverage of trajectory checks. Repeated measurements quantify validator execution separately from process startup and reporting. These results establish coverage of the authored regression suite, not production accuracy. We describe a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve. The contribution is a practical pattern for expressing behavioral contracts within an existing Java release pipeline.
| Comments: | Accepted for publication in the 2026 International Conference on Computer and Applications (ICCA) |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.06674 [cs.SE] |
| (or arXiv:2611.06674v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06674 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sachin Gupta [view email]
[v1]
Wed, 4 Nov 2026 15:46:11 UTC (82 KB)
来源：arXiv cs.AI · arxiv.org