arXiv cs.CL· Shamim Hasan, Samiya Ahmed Chowdhury·· 12 小时前AI 评分18
孟加拉语幻觉检测系统：检索与程序化推理结合，Kaggle 私有 macro-F1 达 0.956
Bengali Hallucination Detection with Retrieval and Programs: A System Study
AI 导读
一项 arXiv 系统研究提出仅推理的孟加拉语幻觉检测方案，结合响应条件查询重写、混合检索与任务专用模型评判，六条检索路径由路由器分派至 27B 文本评判模型或可执行 Python 的 14B 智能体。
正文 · 原文
Abstract:Bengali hallucination detection must distinguish unsupported factual claims from incorrect reasoning, often without a supplied reference passage. We present and characterize an inference-only system that combines response-conditioned query rewriting, hybrid retrieval, and task-specific language-model judging. Six retrieval routes search question-answer and passage collections; a router directs each example to a 27B textual judge or a 14B agent that can compare retrieved answers, reason directly, or generate and execute Python. We characterize the system using version-specific code, saved predictions, execution traces, and corpus metadata from the Olikbochon challenge. Kaggle associates a displayed private macro-F1 of 0.956 with Version 2; its saved run processes 2,516 examples in 220.99 minutes on two T4 GPUs. Of 253 math-routed examples, 88 terminate through candidate matching, while execution traces expose 49 stderr-bearing run events across 35 examples. A later run changes six predictions despite identical core worker code. A bounded audit of 1,090 benchmark-related corpus records finds exact question or context overlap for 38 processed evaluation inputs, precluding an unqualified claim of corpus-independent generalization. The contribution is a concrete retrieval-and-computation judging architecture with an auditable operational characterization, not a controlled estimate of component gains. We distinguish the reported leaderboard result from independently recomputed diagnostics and identify the evidence needed for stronger effectiveness and reproducibility claims.
| Comments: | 8 pages |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22892 [cs.CL] |
| (or arXiv:2610.22892v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22892 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shamim Hasan `` [view email]
[v1]
Fri, 16 Oct 2026 03:46:27 UTC (19 KB)
来源：arXiv cs.CL · arxiv.org