arXiv cs.CL· Furong Jia, Yuan Pu, Finn Guo, Monica Agrawal·· 2 天前AI 评分36
轻量概率基线 FBPR 在 MedQA 诊断题上可媲美 LLM
Counting Clues: A Lightweight Probabilistic Baseline Can Match an LLM
AI 导读
研究者提出 Frequency-Based Probabilistic Ranker（FBPR），用基于概念—诊断共现统计的平滑朴素贝叶斯给选项打分，在 MedQA 最可能诊断题上达到与对应 LLM 相当的表现。
正文 · 原文
Abstract:Large language models (LLMs) excel on multiple-choice clinical diagnosis benchmarks, yet it is unclear how much of this performance reflects underlying probabilistic reasoning. We study this through questions from MedQA that ask for the most likely diagnosis. We introduce Frequency-Based Probabilistic Ranker (FBPR), a lightweight method that scores options with a smoothed Naive Bayes over concept-diagnosis co-occurrence statistics from a large corpus. When co-occurrence statistics are sourced from Dolma, the pretraining corpus for OLMo, and RedPajama, an open reproduction of the LLaMA pretraining-data recipe, FBPR achieves comparable performance to the corresponding LLMs. Furthermore, direct LLM inference and FBPR exhibit complementary strengths, correctly answering largely different subsets of questions with overlap barely above chance. Combining them with a simple hybrid method yields a notable gain over either method alone. These findings show that explicit probabilistic baselines can account for a substantial portion of the benchmark performance on diagnosis questions, while serving as historically grounded reference points and complementary signals for hybridization.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.12868 [cs.CL] |
| (or arXiv:2512.12868v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2512.12868 arXiv-issued DOI via DataCite |
Submission history
From: Furong Jia [view email]
[v1]
Sun, 14 Dec 2025 23:00:10 UTC (1,196 KB)
[v2]
Tue, 27 Oct 2026 01:37:22 UTC (1,779 KB)
来源：arXiv cs.CL · arxiv.org