arXiv cs.AI· Garry Yang, Zizhe Chen, Xinru Chen, Yongqiang Chen, Jianxiang Wang, Deyu Zou, Linyi Ding, Jialiang Wu, Yunzhong He, Yu Gong, James Cheng, Huaixiao Tou·· 12 小时前AI 评分31
APeB：大语言模型智能体个性化能力基准测试
APeB: Benchmarking Personalization Ability of Large Language Model Agents
AI 导读
研究者提出个性化商品搜索（PPS）测试平台并构建 Agent Personalized Benchmark（APeB），用于评估 LLM 智能体在原始模糊查询和多样交互历史下的个性化能力。
正文 · 原文
Authors:Garry Yang, Zizhe Chen, Xinru Chen, Yongqiang Chen, Jianxiang Wang, Deyu Zou, Linyi Ding, Jialiang Wu, Yunzhong He, Yu Gong, James Cheng, Huaixiao Tou
Abstract:LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.
| Comments: | Accepted to Findings of EMNLP 2026. Code and data:this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2607.03162 [cs.AI] |
| (or arXiv:2607.03162v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2607.03162 arXiv-issued DOI via DataCite |
Submission history
From: Haochen Yang Mr [view email]
[v1]
Fri, 3 Jul 2026 10:03:35 UTC (33,705 KB)
[v2]
Thu, 27 Aug 2026 14:22:53 UTC (33,706 KB)
[v3]
Fri, 16 Oct 2026 07:23:05 UTC (33,706 KB)
来源：arXiv cs.AI · arxiv.org