arXiv cs.CL· Khaled Ziani·· 10 小时前AI 评分22
基于微调 Gemma-4-12B-it 的阿拉伯语伊斯兰问答幻觉检测与答案验证系统
Detecting Hallucinations and Recovering Verified Answers in Arabic Islamic Question Answering
AI 导读
针对 HalluScoring 2026 Task 2.1，该系统用微调后的 google/gemma-4-12B-it 模型配合确定性解码，在 600 条伊斯兰问答实例（341 条幻觉、259 条非幻觉）上实现幻觉检测 Macro-F1 0.928、标签准确率 0.935，答案选择准确率 0.895，综合得分 0.912。
正文 · 原文
Abstract:Large language models can generate fluent responses to Islamic questions while introducing factual errors that are difficult to identify. This paper presents our system for \textsc{HalluScoring 2026} Task 2.1, \textit{Islamic Hallucination Detection and Find the Truth}. The task requires a unified two-step prediction: determining whether an Arabic answer generated by an LLM is hallucinated and selecting the verified answer from six closely related candidate options. We use the Islamic knowledge dataset provided by the shared task, which contains 600 question--answer instances, including 341 hallucinated and 259 non-hallucinated answers. Our system is based on the fine-tuned \texttt{google/gemma-4-12B-it} model and uses deterministic decoding during inference. The generated outputs are normalized to extract the hallucination label and the selected option. The system achieves a Macro-F1 score of 0.928 and a label accuracy of 0.935 for hallucination detection, together with an option accuracy of 0.895 for answer selection. These results yield a combined score of 0.912, demonstrating strong performance across both stages of the task. The lower option-selection accuracy indicates that distinguishing the verified answer from plausible alternatives remains more challenging than detecting hallucinated responses.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.03720 [cs.CL] |
| (or arXiv:2608.03720v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.03720 arXiv-issued DOI via DataCite |
Submission history
From: Khaled Ziani [view email]
[v1]
Tue, 4 Aug 2026 14:19:53 UTC (13 KB)
[v2]
Tue, 10 Nov 2026 06:36:06 UTC (13 KB)
来源：arXiv cs.CL · arxiv.org