PLLB：用语言瓶颈实现策略学习

View PDF HTML (experimental)

Abstract:Modern AI systems such as self-driving cars and game-playing agents can achieve superhuman performance, but often lack human-like generalization, interpretability, and inter-operability with human users. Inspired by the rich interactions between language and decision-making in humans, we introduce Policy Learning with a Language Bottleneck (PLLB), a framework enabling AI agents to generate linguistic rules that capture the high-level strategies underlying rewarding behaviors. PLLB alternates between a *rule generation* step guided by language models, and an *update* step where agents learn new policies guided by rules, even when a rule is insufficient to describe an entire complex policy. Across five diverse tasks, including a two-player signaling game, maze navigation, image reconstruction, and robot grasp planning, we show that PLLB agents are not only able to learn more interpretable and generalizable behaviors, but can also share the learned rules with human users, enabling more effective human-AI coordination. We provide source code for our experiments at this https URL .

Comments:	Accepted to TMLR (2026), minor edits
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as:	arXiv:2405.04118 [cs.LG]
	(or arXiv:2405.04118v4 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2405.04118 arXiv-issued DOI via DataCite

Submission history

From: Megha Srivastava [view email]
[v1] Tue, 7 May 2024 08:40:21 UTC (11,826 KB)
[v2] Wed, 26 Mar 2025 20:53:57 UTC (13,578 KB)
[v3] Wed, 4 Feb 2026 00:34:02 UTC (3,170 KB)
[v4] Thu, 8 Oct 2026 00:00:30 UTC (3,170 KB)