面向波斯语医疗模型的单次前向不确定性头：声明级幻觉检测

View PDF HTML (experimental)

Abstract:Hallucination detection is particularly important for medical language models, but repeated-sampling approaches are computationally expensive. A faster alternative is a single-pass uncertainty head that predicts hallucination risk from a frozen generator's internal signals. Existing uncertainty heads consume backbone-specific features and tokenization, motivating adaptation when the backbone or language changes. We study two Persian medical models, Gaokerena-V and Gaokerena-R, on a 168-question Iranian medical entrance examination. Across five generations per question, Gaokerena-V produces the same option on only 14 questions and Gaokerena-R on 37, compared with 168 for Med-Gemma, indicating substantial response variability in the Gaokerena models. We therefore adapt the LLM Uncertainty Head (LUH) framework to these models and construct two paired claim-level hallucination datasets directly in Persian, with 1,600 responses per backbone. Lightweight heads are trained on frozen-backbone attention maps and token probabilities. On held-out test splits, the heads achieve PR-AUCs of 0.4820 and 0.4652, corresponding to 2.30 and 2.66 times their respective random baselines, and ROC-AUCs of 0.7852 and 0.7810. The resulting detectors require neither retrieval nor repeated sampling at inference.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2610.03482 [cs.CL]
	(or arXiv:2610.03482v3 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.03482 arXiv-issued DOI via DataCite

Submission history

From: Mehrdad Ghassabi [view email]
[v1] Fri, 2 Oct 2026 15:50:46 UTC (11 KB)
[v2] Mon, 5 Oct 2026 08:35:10 UTC (11 KB)
[v3] Wed, 7 Oct 2026 18:13:33 UTC (12 KB)