HEART-Bench：LLM 能否做出人格一致的决策？

View PDF HTML (experimental)

Abstract:LLM agents in interactive entertainment and social applications, including companions, role-play characters, and persona clones, need decisions grounded in personality and past experiences. We introduce HEART-Bench to evaluate memory-grounded persona-consistent response selection. The benchmark combines 11 Big Five-anchored synthetic characters, each with 1,000 autobiographical memories, and 64 DIAMONDS-guided decision scenarios into 673 multiple-choice questions. Domain experts independently compare candidate responses, with a senior expert resolving disagreements and revising responses when needed; other characters' adjudicated responses to the same scenario serve as distractors. In the main evaluation, models receive character IDs and occupations, scenarios, and options, with retrieved memories added in memory-augmented settings. Detailed personality-and-value profiles are withheld. We evaluate 16 LLMs under No-Retrieval, Naive-RAG, Mem0, and PersonaDB. Accuracy without memories stays near the 25% chance level, and the best memory-augmented configuration reaches 64.9%. On a matched 150-item subset, human annotators reach 80.8%, compared with 64.0% for the best model. Memory-content ablations show the importance of retrieving experiences that match both the target character and the decision context.

Comments:	GitHub: this https URL
Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2605.30058 [cs.CL]
	(or arXiv:2605.30058v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2605.30058 arXiv-issued DOI via DataCite

Submission history

From: Yuling Shi [view email]
[v1] Thu, 28 May 2026 15:08:03 UTC (2,839 KB)
[v2] Tue, 27 Oct 2026 15:32:45 UTC (1,179 KB)