arXiv cs.AI· Dmitry Redko (Applied AI Institute), Albert Fazlyev (AI Talent Hub, ITMO University), Konstantin Sozykin (Applied AI Institute), Maria Ivanova (YSDA, Applied AI Institute), Evgeny Burnaev (Applied AI Institute), Egor Shvetsov (Applied AI Institute)·· 10 小时前AI 评分48
预训练先验还是搜索反馈?LLM 智能体在硬件感知代码优化中的行为研究
Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization
AI 导读
三项受控实验发现,LLM 在代码优化任务中高度依赖预训练先验,而非环境反馈或智能体结构。纯黑盒优化中 LLM 表现为贪心优化器;零样本 kernel 生成时显式输入尺寸信息无可测影响,模型在不同尺寸和温度下收敛到相同 kernel 参数,面对罕见 kernel 尺寸时性能急剧下降。反馈循环优化中 CUDA 随迭代单调提升,TVM IR 却持续退化,表明低密度语言下 kernel 优化会变差。
正文 · 原文
Abstract:LLM discovery and optimization systems are increasingly applied across domains, implementing a common propose-evaluate-revise loop. Such optimization or discovery progresses via context conditioning on received feedback from an environment. However, as modern LLM agents are increasingly complex in their structure, it is difficult to evaluate which components contribute the most, and when and how this exploration may fail. We answer these questions through three controlled experiments. Our findings: (1) In pure black-box optimization, LLMs act as greedy optimizers. (2) In zero-shot kernel generation, providing explicit input-size information has no measurable effect, models converge to the same kernel parameters regardless of size or temperature, as though the size instruction were invisible. Moreover, when tasked to perform kernel optimization for uncommon kernel sizes, performance sharply degrades regardless of the language used. (3) In feedback-loop kernel optimization, CUDA improves monotonically under iterative feedback, while TVM IR actively degrades, which demonstrates that kernel optimization degrades when models operate with low-density language. Our results conclude that LLMs in code optimization tasks highly depend on pretrained priors rather than provided feedback or agentic structure.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE) |
| Cite as: | arXiv:2605.19782 [cs.AI] |
| (or arXiv:2605.19782v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19782 arXiv-issued DOI via DataCite |
Submission history
From: Dmitry Redko [view email]
[v1]
Tue, 19 May 2026 12:48:48 UTC (41,572 KB)
[v2]
Wed, 18 Nov 2026 15:24:14 UTC (17,208 KB)
来源:arXiv cs.AI · arxiv.org