arXiv cs.AI· Illia Horenko·· 12 小时前AI 评分31
熵视角下的 AI 韧性、数据极简学习与最优控制
Entropic perspective on AI resilience, data-minimal learning and optimal control
AI 导读
一项研究从熵优化视角提出对抗攻击算法，可基于低成本可验证的数学判据证明对抗攻击的全局最优性与唯一性。该视角还催生“自攻击”学习算法，用数学判据替代训练中的一个智能体来生成最优新数据点。在流体力学与生物医学的合成及真实问题上，这类算法能用少得多的训练数据训练出简单数个数量级的模型，复杂度和预测性能均优于随机森林、深度神经网络及基于 Transformer 的基础模型等当代 AI 工具。
正文 · 原文
Abstract:Behind the general terms like adversarial attacks and generative adversarial networks (GANs), such criteria as AI resilience and AI robustness gained a crucial importance, in trying to make the emergent AI tools reliable, energy-efficient and safe. From mathematical perspective, these problems boil down to finding a mathematically-sound quantification of the smallest-sufficient amounts of the perturbations of function arguments that lead to the largest possible perturbations of the approximated function values. Here, the entropy-optimising perspective on adversarial algorithms from Artificial Intelligence (AI) is proposed to attack this problem. It is shown that adopting this perspective helps proving computational conditions for the global optimality and uniqueness of adversarial attacks based on cheaply-verifiable mathematical criteria. Further, it is shown how this perspective can be used for developing self-attacking learning algorithms, that generate optimal new data points for training, replacing one of the trained agents with this mathematical criterium. On a broad selection of various synthetic and real-life problems from hydrodynamics and biomedicine, it is shown that such self-attacking learning algorithms allow training orders-of-magnitude simpler and cheaper models with superior performance - and can be used to directly train the optimal controls in complex systems, requiring only a small fraction of the training data and outperforming the as-comprehensive-as-the-author-was-able-to-find set of contemporary AI tools (like boosted Random Forests, deep neural networks, foundational models based on transformer architectures, etc.) in complexity (measured as the model descriptor length) and predictive performance.}
| Subjects: | Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| MSC classes: | 68T01 (Primary), 68T99, 68Q32, 86A22, 92C50 (Secondary) |
| Cite as: | arXiv:2310.14421 [stat.ML] |
| (or arXiv:2310.14421v5 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2310.14421 arXiv-issued DOI via DataCite |
Submission history
From: Illia Horenko Dr. [view email]
[v1]
Thu, 19 Oct 2023 10:36:02 UTC (4,342 KB)
[v2]
Mon, 6 Nov 2023 15:57:27 UTC (4,348 KB)
[v3]
Tue, 14 Nov 2023 18:13:13 UTC (4,449 KB)
[v4]
Wed, 15 Nov 2023 11:05:34 UTC (4,449 KB)
[v5]
Tue, 20 Oct 2026 11:25:18 UTC (5,919 KB)
来源：arXiv cs.AI · arxiv.org