arXiv cs.AI· Galen J. Wilkerson·· 13 小时前AI 评分34
自组织强化学习：无学习器的群体如何逼近探索-利用最优解
Self-organizing reinforcement learning: a learner-free swarm approaches the explore-exploit optimum
AI 导读
一项 arXiv 研究显示，一个无学习器、无奖励信号的极简觅食群体通过 Lévy 飞行探索、邻近网络阈值级联招募和拥挤分散，动态逼近多智能体资源共享博弈的 ideal-free/Nash 最优分配，约 150 步达到该最优形态（投入觅食者跟随资源丰富度的斜率为 0.94），移除招募与移动的耦合后匹配度从 0.94 跌至 0.08。
正文 · 原文
Abstract:The explore/exploit trade-off is the core problem of reinforcement learning (RL). We show that a minimal foraging swarm approaches its optimum - the ideal-free / Nash allocation of a multi-agent resource-sharing game (an equilibrium, not a return maximum) - with no learner and no reward signal. Agents explore by heavy-tailed (Levy) flights, recruit neighbours to a discovery through a threshold cascade on their proximity network, and disperse as a patch crowds; this cascade cycle balances exploration and exploitation and drives foragers toward the ideal free distribution. The swarm attains the shape of this optimum (committed foragers track richness with slope 0.94) but not its exact equilibrium - it stays measurably exploitable - reaching it dynamically (~150 steps) through the coupling of recruitment to movement: remove the coupling and matching collapses (0.94 -> 0.08). The RL sits at the level of the problem and the allocation reached, not the micro-rule: no step evaluates payoff (reward-free by construction, though the crowd signal is informationally tied to payoff), and the swarm reaches the allocation without learning it (quality plateaus and re-settles after resources move, no cumulative gain). Against a trained baseline the comparison is two-sided: in an easier, disembodied game a learner is more precise, while in the same embodied environment the reward-free swarm reaches ~65% of a validated oracle ceiling at zero training and leads until trained PPO and centralised-critic (CTDE) learners overtake it (~74-75%, 16 seeds, all above the swarm mean) after ~4,000 of 15,000 training steps - its advantage is in budget, not asymptote. In this one setting we read it not as beating RL but as a strong zero-shot, reward-free operating point: a demonstration that the self-organizing physics of a minimal swarm can carry a collective close to the optimum RL seeks, with none of RL's machinery.
| Comments: | 16 pages, 9 figures, code and data: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI); Mathematical Physics (math-ph); Adaptation and Self-Organizing Systems (nlin.AO); Biological Physics (physics.bio-ph) |
| MSC classes: | 91A22, 68T05, 92D50 |
| ACM classes: | I.2.11; I.2.6 |
| Cite as: | arXiv:2610.25113 [cs.LG] |
| (or arXiv:2610.25113v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.25113 arXiv-issued DOI via DataCite |
Submission history
From: Galen Wilkerson [view email]
[v1]
Sun, 18 Oct 2026 08:45:51 UTC (301 KB)
[v2]
Tue, 20 Oct 2026 19:20:05 UTC (310 KB)
来源：arXiv cs.AI · arxiv.org