arXiv cs.CL· Mukand Krishna, Anoushka Harit, Chenghao Xiao, Zhongtian Sun·· 12 小时前AI 评分31
研究揭示 LLM 医学验证缺陷：能识别同源试验却仍重复计入证据
Recognizing Study Lineage Is Not Enough: Over-Crediting Same-Trial Evidence in LLM-Based Medical Verification
AI 导读
基于 14 个经来源审计的临床家族和五种模型条件测试发现，GPT-5.5 在三个来源家族中均能以 10/10 识别独立更新，却在 5/5 独立聚合调用中过度计入同一试验的第二篇论文。Qwen3-4B 与 Ministral-3-8B 也呈现同样的跨任务反差。加入研究分组指令后，GPT-5.5 在后续纯来源家族中的过度计入从 4/5 降至 0/5。
正文 · 原文
Abstract:Publication count is not study count. A model can identify shared trial lineage correctly in a direct task yet over-credit a second publication from the same trial when aggregating evidence. We evaluate this behavior in medical LLM verification using 14 source-audited clinical families and five model conditions. The families span provenance, clinical applicability, result direction, and composition; the provenance families provide the direct lineage tests. Recognition and aggregation are separate stateless calls, and repeated calls assess within-input stability. Across all three original provenance families, GPT-5.5 identifies the independent update in 10/10 recognition calls per family but over-credits the same-trial report in 5/5 separate aggregation calls per family. Selected Qwen3-4B and Ministral-3-8B conditions show the same cross-task contrast. A study-grouping instruction reduces GPT-5.5 over-credit from 4/5 to 0/5 in each later pure-provenance family. Other model conditions show mixed behavior, and rejected outputs limit several comparisons. The results identify a behavioral gap between elicited lineage recognition and evidence aggregation. The code and data artifacts are available at this https URL.
| Comments: | 20 pages, 5 figures, 17 tables; 8 pages of main content, with references and appendices thereafter |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.23721 [cs.CL] |
| (or arXiv:2610.23721v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23721 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mukand Krishna [view email]
[v1]
Fri, 16 Oct 2026 16:42:10 UTC (348 KB)
来源：arXiv cs.CL · arxiv.org