arXiv cs.CL· Caixin Kang, Ke Wang, Yuchuan Wu, Xiaoqiang Wang, Ling Li, Bingxue Qiu, Yongbin Li, Yoichi Sato·· 3 天前AI 评分25
HICO-Bench：什么让社交智能体的交互真正有效？从配对世界中的可验证结果学习
What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds
AI 导读
研究者提出 HICO-Bench，包含 180 个日常协调场景，两个持有私有信息的智能体对话后各自记录商定计划，由程序检查 Feasibility、LLM 评判 Integrity，两者同时通过才算 Verified Success。
正文 · 原文
Abstract:Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social agents mainly rely on LLM judges that rate the conversation, without checking the resulting plan. We introduce HICO-Bench, a benchmark of 180 everyday coordination scenarios, such as resource allocation, scheduling, and authorization, in which two agents with private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), and an LLM judge checks Integrity (the conversation supports the recorded plan); passing both is a Verified Success. Each scenario has two versions that share the same public context but differ in one private condition, so the correct plan differs between them. Across thirteen LLMs, 36% of conversations rated highly for goal completion fail Verified Success. To learn from outcomes that arrive only at the end of a conversation, we propose Paired-Credit Policy Optimization (PaCPO). After each turn, PaCPO asks the agent for a tentative plan, checks it in both versions of the scenario, and rewards the turns after which the tentative plans fit their respective versions. PaCPO raises Qwen3-8B's Verified Success from 20.4% to 28.3%.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI) |
| Cite as: | arXiv:2610.20195 [cs.CL] |
| (or arXiv:2610.20195v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20195 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Caixin Kang [view email]
[v1]
Wed, 14 Oct 2026 13:44:30 UTC (1,174 KB)
来源：arXiv cs.CL · arxiv.org