BenchPreS：评估持久记忆 LLM 的上下文感知个性化偏好选择性基准

BenchPreS: A Benchmark for Context-Aware Personalized Preference Selectivity of Persistent-Memory LLMs

AI 导读

研究者提出 BenchPreS，用 Misapplication Rate（MR）和 Appropriate Application Rate（AAR）两项指标评估持久记忆 LLM 能否在第三方沟通场景中恰当应用或抑制用户偏好。结果显示，即使前沿 LLM 也难以做到上下文敏感，偏好依从性更强的模型过度应用率更高，推理能力与基于提示词的防御均无法完全解决该问题。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor prompt-based defenses fully resolve this issue. These results suggest current LLMs treat personalized preferences as globally enforceable rules rather than as context-dependent normative signals.

Comments:	Accepted at EMNLP 2026
Subjects:	Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:	arXiv:2603.16557 [cs.AI]
	(or arXiv:2603.16557v2 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2603.16557 arXiv-issued DOI via DataCite

Submission history

From: Sangyeon Yoon [view email]
[v1] Tue, 17 Mar 2026 14:19:05 UTC (2,556 KB)
[v2] Wed, 14 Oct 2026 04:23:03 UTC (2,557 KB)

来源：arXiv cs.CL · arxiv.org