AuditFlow：面向 CFO 办公室的自主审计证据审查系统

Autonomous audit-evidence review for the Office of the CFO.

AuditFlow turns controlled financial-document bundles into review-ready audit workpapers. Five specialized agents cross-reference evidence, identify bounded exceptions, route them to a reviewer, and compile the result into a single workpaper.

Built for Syndicate by Maximor — AO Hackathon, Track 2: Autonomous Office of the CFO.

🚀 Live Demo · Backend API · Demo Video

The deployed demo runs the real five-agent pipeline rather than a static frontend mock.

Why AuditFlow?

Audit review is often slowed less by the final judgment than by collecting evidence, checking whether records agree, and documenting exceptions.

• 5 specialized AI agents with bounded responsibilities
• 14 controlled benchmark cases spanning clean, missing, conflicting, and multi-issue evidence
• Evidence-backed human review instead of silently accepting exceptions
• Real document upload with Claude's native PDF/image understanding, alongside the benchmark suite
• Second-pass evidence resolution on ambiguous cases, with disagreement surfaced rather than silently resolved
• Persistent reviewer decisions, building a record of confirmed vs. overturned findings

Benchmark results

14 cases · repeated-run results: 13 pass every run, 1 is known to vary. In repeated runs (3 per case), 13 of 14 cases produced the expected findings and routing every time. The ambiguous mixed goods/service case (CASE_06) occasionally returns no finding. The tax-calculation case (CASE_14) used to be detected in only about half of runs; after a change to how the tax arithmetic is checked it was detected in 20 of 20 test runs and in 9 of 9 live runs. Earlier single-run verification showed 100% routing accuracy, but repeated runs show that is not stable. We are working on CASE_06.

Product Preview

CASE_09 · complex exception routed to review

CASE_01 · clean transaction auto-cleared

Controlled benchmark bundle selector

Neatlogs execution observability

Five-agent pipeline

Intake → Evidence → Anomaly → Decision → Workpaper

The bounded exception types are duplicate invoice, amount mismatch, missing PO, missing receipt, vendor mismatch, date inconsistency, currency mismatch, and tax mismatch.

Evidence Resolver

On transactions where the Evidence Agent's first pass lands below a confidence threshold, a second independent pass runs and the two are compared. If they agree, the higher-confidence result is kept. If they disagree, both sets of findings are unioned rather than one being silently picked, and the disagreement is surfaced in the UI. This targets low-confidence ambiguity; it does not catch confidently wrong Evidence results (see the Known limitations section below).

Architecture and model strategy

AO (Agent Orchestrator) coordinates the pipeline; a FastAPI backend exposes the full run via POST /run-case (benchmark cases) and POST /run-case-upload (real documents); Neatlogs provides LLM tracing for workflow execution, model usage, latency, token consumption, and cost. Anthropic Claude is used with a focused mixed-model strategy:

The Anomaly Agent uses the stronger model because exception classification is the most nuanced reasoning step.

Benchmark and review controls

The benchmark contains 14 controlled scenarios: clean cases, duplicate invoices, amount and currency mismatches, tax calculation errors, missing POs and receipts, vendor mismatches, date inconsistencies, multi-issue cases, and incomplete or conflicting evidence. These are evaluation fixtures—not customer audits.

Known limitations: one case varies between runs. CASE_06 is an intentionally more ambiguous mixed goods/service scenario whose Evidence Agent output sometimes reports nothing missing, after which a finding is often not raised. The Evidence Resolver does not fully handle this: it only runs a second pass when evidence confidence is below 0.7, and in our tests the wrong Evidence outputs had confidence of 0.90 or higher, so the second pass would not have been triggered. The tax check covers the arithmetic sum only: a tax amount that contradicts a stated rate while the sum still reconciles is not detected. Documented honestly rather than hidden.

Detected exceptions are routed to a reviewer, who can inspect the finding, evidence, confidence, rationale, and workpaper context. The current UI supports Clear exception, Request evidence, and Escalate actions, and every decision is persisted server-side (POST /feedback) and viewable in a decision history dashboard (GET /feedback/history).

Upload Documents

Real PDF and image documents can be uploaded and run through the full five-agent pipeline via POST /run-case-upload, using Claude's native document understanding for extraction — no separate OCR step. The 14-case benchmark remains available separately for reproducible evaluation.

Stack and deployment

Run locally

Prerequisites: Python 3.13.5, Node.js, and ANTHROPIC_API_KEY. Set NEATLOGS_API_KEY to enable tracing.

git clone https://github.com/abhijitmanna912001/auditflow.git
cd auditflow
python3.13 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

If pip install fails with externally-managed-environment, you're on a system Python (e.g. Homebrew) that blocks global installs by design — the venv step above avoids this.

# Backend
ANTHROPIC_API_KEY=your-key uvicorn orchestration.api:app --host 127.0.0.1 --port 8000

# Frontend, in a second terminal
cd frontend
npm install
npm run dev

The API defaults to http://127.0.0.1:8000; set NEXT_PUBLIC_API_BASE_URL to point the frontend at another backend.

API

POST /run-case
Content-Type: application/json

{ "case_id": "CASE_09" }

The response is the Workpaper Agent's review-ready JSON output.

POST /run-case-upload?case_id=<label>&use_resolver=true
Content-Type: multipart/form-data

files: one or more PDF/PNG/JPG/JPEG/WEBP files

Same response shape as /run-case, run against real uploaded documents. use_resolver=true routes Evidence through the second-pass resolver.

POST /feedback
Content-Type: application/json

{ "case_id": "CASE_09", "document": "INV-1009-A", "finding": "...", "agent_action": "human_review", "decision": "confirmed" }

Persists a reviewer decision (confirmed, overturned, or evidence_requested) tied to a specific finding.

GET /feedback/history

Returns every persisted reviewer decision, oldest first.

Repository map

agents/         Five agent implementations and unit tests
orchestration/  FastAPI pipeline endpoint
dataset/        14 ground-truth cases and text document fixtures
evaluation/     Benchmark scoring and report comparison
frontend/       Next.js reviewer workspace
docs/           Agent contract and product screenshots

Next steps

• Automated regression testing using the persisted reviewer feedback (overturned findings as a regression set)
• Workpaper export and broader audit-domain coverage
• Additional anomaly types beyond the current eight

Team

• Abhijit Manna — agent logic, orchestration, evaluation, backend reliability, deployment
• Garvit Mathur — dataset, frontend, testing, integration, observability, deployment preparation

Built for Syndicate by Maximor — AO Hackathon · Track 2: Autonomous Office of the CFO.