arXiv cs.CL· Tam\'as Janusko, Sven Kleinkop, Ricardo Bochnia, Gunnar Hempel, Steffen Tomschke, J\"urgen Anke, Maik Thiele·· 11 小时前AI 评分29
通过软提示调优让 LLM 评审对齐人类标注策略
Adapting LLM Judges to Observed Human Annotation Policy through Soft Prompt Tuning
AI 导读
研究用软提示调优让开源 LLM 评审更贴近人类标注策略，在 FairBotBench 上训练并在偏见、毒性、个性化三个维度评估。结合文本提示与可学习连续提示嵌入的混合软提示调优，一致优于未调优基线；Qwen 3.5 9B 调优后在三个维度上均超过两个托管参考评审。
正文 · 原文
Abstract:Large Language Models (LLMs) are increasingly employed as automated judges for evaluating chatbot responses. While proprietary models can achieve high agreement with human annotators, they remain costly and difficult to deploy in practical settings. Lightweight open-source LLM judges provide an attractive alternative but typically exhibit lower agreement with human ratings. In this paper, we investigate whether soft prompt tuning can align open-source LLM judges more closely with human annotation policy. We train soft prompts on the FairBotBench benchmark and evaluate the adapted judges across the dimensions of bias, toxicity, and personalization. Our findings demonstrate that hybrid soft-prompt tuning, which combines textual prompts with learned continuous prompt embeddings, consistently improves agreement with human annotations over the corresponding untuned baselines. For Qwen 3.5 9B, the adapted judges surpass both hosted reference judges across all three evaluation dimensions.
| Comments: | CONVERSATIONS 2026, International Symposium on Chatbots and Human-Centred AI (Preprint) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2611.07126 [cs.CL] |
| (or arXiv:2611.07126v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07126 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Maik Thiele [view email]
[v1]
Wed, 4 Nov 2026 20:48:34 UTC (383 KB)
来源：arXiv cs.CL · arxiv.org