arXiv cs.CL· Jarrod Ragsdale, Rajendra Boppana·· 13 小时前AI 评分29
ShIOEnv：面向语法约束合成与执行行为建模的命令评测环境
ShIOEnv: A Command Evaluation Environment for Grammar-Constrained Synthesis and Execution Behavior Modeling
AI 导读
ShIOEnv 是一个兼容 Gymnasium 的 Bash 环境，可合成带执行标注的 shell 输入轨迹，记录 stdout、stderr、退出状态及结构化文件系统与环境补丁。
正文 · 原文
Abstract:We introduce ShIOEnv, a Gymnasium-compatible Bash environment for synthesizing execution-annotated shell input traces. Datasets containing such traces are useful for training models to perform execution-free shell simulation, which in turn is useful for deploying honeypots for cyber deception. Existing datasets, such as NL2CMD and other Bash command corpora, do not include complete execution artifacts (stdout, stderr, exit status, and changes to the system's state) associated with a command's execution. ShIOEnv synthesizes shell inputs, executes them in a controlled system context, and records stdout, stderr, exit status, and structured filesystem and environment patches. Unconstrained (random) generation of shell inputs (command and arguments) results in multiple instances of arguments that are mostly redundant or result in execution errors. We introduce grammar-constrained synthesis to generate syntactically valid arguments with high input irreducibility, which we estimate through self-ablative contrasts of execution artifacts. We prepare 1.6M samples using grammar-constrained and unconstrained synthesis and collect their execution artifacts for 86 Linux/Bash utilities. Using this dataset, we train small language models (CodeT5) and evaluate their accuracy in predicting the actual execution behavior of attacker-sourced Bash commands. These models outperform commonly used rule-based simulation (Cowrie) and prompted large language models by up to 14.2 pp in output edit similarity (O-ES). We enhance the NL2CMD dataset by constructing 500K samples executable in a grounded system context along with their complete execution artifacts. We empirically demonstrate that models trained on grammar-constrained samples outperform those trained on NL2CMD samples by 22.5 pp in O-ES. ShIOEnv, command grammars, and 2.1M samples used in our primary evaluations are released for use by other researchers.
| Comments: | 25 pages, 7 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2505.18374 [cs.CL] |
| (or arXiv:2505.18374v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2505.18374 arXiv-issued DOI via DataCite |
Submission history
From: Jarrod Ragsdale [view email]
[v1]
Fri, 23 May 2025 21:00:57 UTC (4,487 KB)
[v2]
Thu, 5 Mar 2026 00:21:25 UTC (444 KB)
[v3]
Thu, 22 Oct 2026 15:49:19 UTC (704 KB)
来源：arXiv cs.CL · arxiv.org