arXiv cs.AI· Eun Go, Rohan Deb, Arindam Banerjee·· 2 天前AI 评分26
FinPILOT：面向 RL 交易智能体的推理时策略优化框架
Plan Before You Trade: Inference-Time Optimization for RL Trading Agents
AI 导读
FinPILOT 是一个受 MPC 启发的插件式推理时策略优化框架，利用多步价格预测器在每步决策前优化预训练策略，无需改动原始 RL 训练流程。在 DJ30 股票与外汇数据上，用 PPO、SAC、A2C、TD3、DDPG 五种预训练策略测试 30 种设置，28 种提升总回报、27 种提升 Sharpe 比率。预测误差越小，FinPILOT 的性能提升幅度与一致性越高。
正文 · 原文
Abstract:Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for policy adjustments using price forecasts at inference time. We propose FinPILOT (Financial Plugin Inference-time Learning for Optimal Trading), a plugin inference-time policy optimization framework inspired by Model Predictive Control (MPC). Under the price-taker assumption, portfolio actions do not affect future market prices, allowing a multi-step price forecaster to provide imagined market trajectories without iterative action-conditioned rollouts. At each decision step, FinPILOT uses the forecaster's predicted price trajectory to construct an allocation-based imagined return objective and optimizes the pretrained policy at inference time before executing one step of the trade. Our framework is compatible with any pretrained policy and adapts the policy using the forecaster's predictions without modifying the original RL training pipeline. Evaluated with five pretrained policies (PPO, SAC, A2C, TD3, DDPG) across DJ30 equities and foreign exchange (FX) over three rolling held-out test years (30 settings in total), FinPILOT improves Total Return in 28 of 30 settings and Sharpe ratio in 27 of 30, with gains spanning both stochastic and deterministic policies. Direct single- and multi-period portfolio optimizers supplied with the same forecasts do not reproduce these aggregate gains under our evaluation protocol. Further, using controlled forecasts at calibrated quality levels, we show that both the magnitude and consistency of FinPILOT's performance increase as forecast error is reduced.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2605.12653 [cs.LG] |
| (or arXiv:2605.12653v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12653 arXiv-issued DOI via DataCite |
Submission history
From: Eun Go [view email]
[v1]
Tue, 12 May 2026 18:58:03 UTC (199 KB)
[v2]
Thu, 15 Oct 2026 05:36:39 UTC (600 KB)
来源：arXiv cs.AI · arxiv.org