跳到正文
arXiv cs.MA· Meenakshi Rajpurohit, Anna Phan, Ravikumar Komandur Narayanan, Gunanidhi Ramakrishnan, Nathan Howland·· 9 小时前AI 评分42

FinVeriQA:面向金融问答的研究智能体,涵盖路由、检索、MCP 工具调用与来源感知的证据验证

FinVeriQA: Financial Research Agents for Question Answering: Routing, Retrieval, MCP Tool Use, and Provenance-Aware Evidence Verification

AI 导读

开源多智能体金融问答原型 FinVeriQA 用紧凑型类型化决策模型 Laya 负责路由,Python 负责全部数值计算,Mistral-7B 撰写带引用的答案,并通过 MCP server 暴露整条流水线。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) write fluent financial analyses but miscompute ratios, recall stale prices, and route questions to the wrong tool with high confidence. We ask which component should do each job in a financial question-answering assistant, and study it with FinVeriQA, an open-source multi-agent research prototype over SEC filings and live market data: a compact typed-decision model (Laya) routes each question and screens out advice requests, Python computes every number, retrieval and tools fetch evidence, Mistral-7B writes a cited answer, a code validator checks that cited IDs exist and that numbers match computed notes, and a Model Context Protocol (MCP) server exposes the pipeline. In live experiments, Laya routed better than Mistral-7B (0.85 vs. 0.52; overall 0.86 vs. 0.68, McNemar p = 0.0003); a 12B Mistral decided no better, and DeepSeek-R1-8B decided better (0.79 vs. 0.68, p = 0.0192) but about 9.9 times more slowly. Mistral-7B computed 29% of 100 financial calculations correctly against 100% for Python, a live quote cut price error from 210% to 0.00%, and hybrid retrieval reached 76% Hit@5 on 50 filing questions. Yet in a live case study, both agent answers passed validation while describing AMD sample data under NVIDIA's and Alphabet's names. A controlled upper-bound study on the FinRank benchmark explains why: when one figure in the shared evidence is corrupted, self-consistency stays confident (agreement 0.55 to 0.59 while accuracy falls from 0.50 to 0.00), and a shared-evidence check (the kind our validator performs) catches 35% of the errors, whereas a check against the original passage catches 77% ($p = 5.96 \times 10^{-8}$); for DeepSeek-R1-8B, 68% vs. 90%. Verification must be independent of the evidence the writer saw.
Comments: 15 pages, 4 figures, 11 tables. Code: this https URL
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Multiagent Systems (cs.MA)
Cite as: arXiv:2611.25071 [cs.CL]
  (or arXiv:2611.25071v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.25071

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Meenakshi Rajpurohit [view email]
[v1] Wed, 18 Nov 2026 19:13:45 UTC (406 KB)

来源:arXiv cs.MA · arxiv.org