arXiv cs.CL· Raffi Khatchadourian·· 13 小时前AI 评分31
DFAH：面向工具调用型 LLM 金融智能体的确定性-忠实性可重放性保障框架
Replayable Financial Agents: A Determinism-Faithfulness Assurance Harness for Tool-Using LLM Agents
AI 导读
研究者提出确定性-忠实性保障框架 DFAH，用于规范金融智能体重放实验中的配置控制、执行记录保存与行为比对。基于 2957 条合成合规分诊与数据操作任务的执行记录，某数据操作配置在 84% 的案例中重复了相同决策，但有序工具名序列仅在 26% 的案例中重复。该框架区分最终决策、工具名集合与有序工具名三类比对层级，并将证据支持度作为独立评估目标。
正文 · 原文
Abstract:Replaying a financial agent requires specifying the conditions of a rerun, preserving its execution record, and deciding which parts of its behavior to compare. We introduce the Determinism-Faithfulness Assurance Harness (DFAH), a framework and practical protocol for organizing these choices. The framework distinguishes comparisons of final decisions, tool-name sets, and ordered tool names, while treating support from available evidence as a separate evaluation target. We illustrate the framework using 2,957 recorded executions in synthetic compliance-triage and data-operations tasks. One data-operations configuration repeated its recorded decision across all trials on 84% of cases, while its ordered tool-name sequence repeated on 26%. Conversely, some compliance cases retained the same tool sequence while recorded decisions changed. A case-level example shows how repeated labels accompanied different lookups and escalation calls. These observations demonstrate how the comparison level changes the description of repeated behavior. The resulting protocol connects configuration control, preservation of each attempt, comparison of records, and inspection of changed cases. DFAH provides a practical foundation for designing replay experiments and locating execution differences during the development and evaluation of financial agents.
| Comments: | 12 pages, 2 figures, 5 tables. Replay framework and descriptive experiments in synthetic financial tasks. Includes an offline reproduction package. Original version accepted at the 2nd ICLR Workshop on Advances in Financial AI (2026) |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.11; D.2.5 |
| Cite as: | arXiv:2601.15322 [cs.AI] |
| (or arXiv:2601.15322v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2601.15322 arXiv-issued DOI via DataCite |
Submission history
From: Raffi Khatchadourian [view email]
[v1]
Sat, 17 Jan 2026 19:47:55 UTC (78 KB)
[v2]
Sat, 7 Mar 2026 20:27:07 UTC (105 KB)
[v3]
Sun, 20 Sep 2026 17:27:42 UTC (95 KB)
[v4]
Wed, 21 Oct 2026 14:26:55 UTC (173 KB)
来源：arXiv cs.CL · arxiv.org