arXiv cs.CL· Zhengxiang Huang, Yujie Sun, Shengheng Chen, Zhaode Wang, Chaoyue Niu, Zeyu Zhao, Chengfei Lv, Fan Wu, Guihai Chen·· 10 小时前AI 评分39
EngineAgent：让端侧 LLM 推理引擎具备推理栈层级感知与硬件能力
EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
AI 导读
EngineAgent 是一个自动化端侧 LLM 模型适配的智能体系统，在覆盖 3 个移动端 LLM、4 组引擎-后端组合（CPU/GPU/NPU）的 12 项任务上实现 75% 端到端成功率，最强编码智能体基线为 33.3%。
正文 · 原文
Abstract:Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.
| Subjects: | Operating Systems (cs.OS); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Software Engineering (cs.SE) |
| Cite as: | arXiv:2611.14356 [cs.OS] |
| (or arXiv:2611.14356v1 [cs.OS] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14356 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhengxiang Huang [view email]
[v1]
Tue, 10 Nov 2026 17:16:47 UTC (7,831 KB)
来源：arXiv cs.CL · arxiv.org