arXiv cs.AI· Christopher Solinas, Radovan Halu\v{s}ka, Martin Schmid, Michael Buro, Nathan R. Sturtevant, Michael Bowling·· 3 天前AI 评分26
Belief Embedding Tree Search：用生成分布嵌入做信念树搜索的蒙特卡洛规划器
Belief Embedding Tree Search
AI 导读
研究者提出 Belief Embedding Tree Search（BETS），一种蒙特卡洛规划器，用生成分布嵌入（GDE）表示每个节点的信念，并通过 POMDP 动力学更新，信念模型与策略、价值网络一同从经验中训练，以 POMDP 的真实隐藏状态为训练目标。
正文 · 原文
Abstract:Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of actions and observations. They summarize everything the history says about the state, so planning in a POMDP is planning in a Markov decision process (MDP) over beliefs. Belief MDPs have continuous state spaces even for finite POMDPs, and exact planning in them is intractable in general. Planners must therefore approximate beliefs and estimate values. Monte Carlo planners do both inside a search tree, with an approximate belief at every node and a value estimate at every leaf. Approximations usually consist of sets of sampled states, with leaf values coming from rollouts, hand-designed bounds, or learned networks. Policies and values computed from these approximations are limited by their quality, and prior methods with learned networks rely on hand-chosen summaries of the sampled states. A belief representation should be computable from the parent node's representation through the POMDP's dynamics, let the search sample from it, and be learned so that it provides the information that policies and value functions need. We present Belief Embedding Tree Search (BETS), a Monte Carlo planner that meets these requirements by representing each node's belief with a generative distribution embedding (GDE) while updating it through the POMDP dynamics. BETS trains its belief model alongside its policy and value networks through experience, with the realized hidden state of the POMDP as the belief model's training target. We evaluate BETS on six configurations of RockSample, LightDark, and PocMan against BetaZero and three planners without learned components. BETS is competitive or leading on all six, while every baseline falls far behind on at least one. Where beliefs are highest-dimensional, it outperforms them by a large margin, even without search.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.15046 [cs.AI] |
| (or arXiv:2610.15046v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15046 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Christopher Solinas [view email]
[v1]
Sat, 10 Oct 2026 16:36:45 UTC (62 KB)
来源：arXiv cs.AI · arxiv.org