arXiv cs.CL· Ziming Dai, Dabiao Ma, Ziheng Guo, Jack Dong, Zimu Zhou·· 9 天前AI 评分22
LLM-BlockFE：LLM 引导的分块特征工程，通过可执行程序搜索将长文本转为预测特征
Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search
AI 导读
研究者提出 LLM-BlockFE，一个 LLM 引导的离线特征构建框架，将长文本转为可执行特征程序，从而避免在线推理时调用 LLM。该方法通过增量追加不可变代码块构建特征程序，并引入基于深度校准信用分配的块级回滚机制、交错推进多条独立搜索轨迹。
正文 · 原文
Abstract:Industrial risk-control systems typically rely on structured-data models for efficient prediction, yet substantial valuable information remains embedded in unstructured long text. Extracting this information through manual feature engineering is labor-intensive, while requiring a large language model (LLM) to process every real-time input may not meet practical deployment requirements. To address this challenge, we propose LLM-BlockFE, an LLM-guided offline feature construction framework that converts long text into executable feature programs, thereby avoiding LLM calls during online inference. LLM-BlockFE constructs feature programs by incrementally appending immutable code blocks and evaluates candidate features using a downstream model. To address the tendency of conventional greedy search to become trapped in suboptimal solutions, our method introduces a block-level rollback mechanism based on depth-calibrated credit allocation and advances multiple independent search trajectories in an interleaved manner, reducing redundant exploration by sharing fixed descriptions of each trajectory's exploration direction. After the search, the resulting programs are frozen and deployed to extract structured features for downstream prediction models. Across two public and two private datasets, LLM-BlockFE achieves absolute AUC improvements of 0.0069 to 0.0358 over the strongest baseline on each dataset in the full-dataset comparison. Post-launch monitoring across five deployed financial risk-control applications shows absolute KS improvements of 0.02 to 1.56 percentage points over the existing manually designed strategy.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.12390 [cs.LG] |
| (or arXiv:2610.12390v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12390 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziming Dai [view email]
[v1]
Thu, 8 Oct 2026 17:39:11 UTC (794 KB)
来源：arXiv cs.CL · arxiv.org