arXiv cs.AI· Feng Han, Ayako Yagahara, Kenji Hirata, Atsushi Yoshida, Kohsuke Kudo·· 4 小时前AI 评分20
BERT 与开源大模型在日语临床细粒度命名实体识别上的受控对比
Fine-Grained Japanese Clinical Named Entity Recognition: A Controlled Comparison of BERT and Open-Weight Large Language Models
AI 导读
研究在含 113 类实体的 iCorpus 上对比两个微调 BERT-CRF 基线(relaxed F1 0.836-0.841)与九个开源大模型,零样本最高仅 0.337。
正文 · 原文
Abstract:Background: Generative large language models (LLMs) offer an alternative to encoder-based clinical named entity recognition (NER), but their performance on fine-grained Japanese extraction tasks and sensitivity to generation settings remain uncertain. Methods: We compared two fine-tuned BERT-CRF baselines with nine open-weight LLMs on iCorpus, which defines 113 entity categories. The evaluation used 433 sentences from 36 test documents; 1,672 sentences were used for training. LLMs were evaluated using zero-shot prompting with and without JSON Schema constraints and three-seed QLoRA fine-tuning. Fine-tuned models were assessed under different output-token limits, including 900 and larger model-specific limits of 2,600 or 4,096 tokens. Performance was measured using sentence-level label-surface matching with relaxed and exact-surface F1. Results: The highest zero-shot relaxed F1 was 0.337, compared with 0.836-0.841 for the BERT baselines. At the larger output limits, all eight fine-tuned LLMs with at least 13B total parameters exceeded both BERT baselines (relaxed F1, 0.882-0.914; exact-surface F1, 0.840-0.880). Across three-seed means, increasing the output limit improved relaxed F1 by 0.052-0.085, alongside higher recall and fewer length-limited outputs. JSON Schema constraints changed relaxed F1 by -0.014 to +0.008 among seven models outside the two reasoning-oriented configurations, while medical-general model pairs showed no consistent advantage for medical specialization. Conclusions: On this historical Japanese case-report benchmark, task-specific fine-tuning combined with adequate generation capacity was associated with stronger LLM extraction performance. Output-token limits materially affected comparisons with BERT, whereas enforcing output structure alone did not overcome low zero-shot accuracy.
| Comments: | 33 pages, 3 figures, 5 tables; includes appendices with supplementary tables and one supplementary figure |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7; J.3 |
| Cite as: | arXiv:2611.22219 [cs.AI] |
| (or arXiv:2611.22219v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22219 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Feng Han PhD. [view email]
[v1]
Tue, 17 Nov 2026 05:01:24 UTC (1,574 KB)
来源:arXiv cs.AI · arxiv.org