arXiv cs.CL· Minsuh Joo, Youna Kim, Choonghyun Park, Soo Kyung Kim, Hyunsoo Cho·· 3 天前AI 评分22
MAPLE：面向 LLM 评估的多轴参数化知识提取透镜
MAPLE: A Multi-Axis Parametric Knowledge Extraction Lens for LLM Evaluation
AI 导读
研究者提出 MAPLE 框架，用于刻画知识提取增强（KEE）技术对 LLM 参数化知识提取的影响，其诊断指标在表层、概率和超越采样三个层面区分答案恢复与干扰、未恢复失败中的 gold-token 排名变化，以及固定预算采样遗漏答案的恢复情况。
正文 · 原文
Abstract:LLMs absorb vast knowledge from training data, yet their inherently probabilistic nature offers no guarantee that this knowledge will surface in their outputs. To bridge this gap, Knowledge Extraction Enhancement (KEE) techniques such as activation steering, decoding adjustments, and prompting modify inference, potentially reinforcing existing predictions or surfacing less accessible answers. Yet their evaluations centered on aggregate accuracy, which obscures diverse effects by collapsing recovered failures and disrupted successes into a single score. We introduce MAPLE, a unified framework for characterizing how KEE techniques affect parametric knowledge extraction. Its diagnostic metrics disentangle these effects at the surface, probability, and beyond-sampling levels by measuring answer recovery and disruption, gold-token rank shifts on unrecovered failures, and recovery of answers missed by fixed-budget sampling. We evaluate six KEE techniques spanning decoding, activation steering, and prompting across three model families: OLMo2, Qwen2.5, and Llama3. Our analysis reveals distinct effects and trade-offs across techniques, highlighting the value of MAPLE for understanding and comparing KEE beyond aggregate accuracy.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19827 [cs.LG] |
| (or arXiv:2610.19827v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19827 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Minsuh Joo [view email]
[v1]
Wed, 14 Oct 2026 09:02:05 UTC (1,362 KB)
来源：arXiv cs.CL · arxiv.org