arXiv cs.AI· Charin Polpanumas·· 14 小时前AI 评分31
Vendster：评估预测工具质量如何影响 LLM 智能体在买卖谈判中行为的评测竞技场
Vendster: An Evaluation Arena to Investigate How Forecasting Tool Quality Affects LLM Agent Behaviors in Buyer-Vendor Negotiation
AI 导读
Vendster 是一个评测竞技场，用于测量预测工具的偏差与预测区间宽度如何改变两个 LLM 智能体之间的买卖折扣谈判结果。在基于 M5 数据集的 Vendster-M5 实例中，模型生成的预测使销售目标达成率最高提升 17 个百分点（p<0.001），但 relWQL、MASE 等标准预测指标无法显著预测谈判结果。
正文 · 原文
Abstract:Time-series forecasters are ranked almost entirely by accuracy metrics, but a forecast's decision value, its effect on the downstream decision, need not track accuracy and is rarely measured. This problem compounds as large language model (LLM) agents increasingly consume forecasters as tools, mediating them through stochastic, opaque policies. We present Vendster, an evaluation arena measuring how a forecaster's bias and prediction-interval width change the outcome of a buyer--vendor discount negotiation between two LLM agents. In the reference instance Vendster-M5, sampling base demand patterns from the M5 dataset, we evaluate two frontier-closed LLM families (Claude Opus 4.6 and GPT-5.6 Sol) and two open ones (DeepSeek-V3.2 and Qwen3-235B-A22B). Model-generated forecasts raise sales target attainment by up to +17 percentage points (p<0.001), yet standard forecast evaluation metrics such as relWQL and MASE do not predict negotiation outcomes at a statistically significant level. To isolate the causal structure, we substitute in synthetic forecasts with controlled bias and interval width. Under-prediction bias significantly improves outcomes for agents that react more to the forecasts---dose-response slopes -0.387 and -0.416 (p<0.001). Interval width has no statistically significant effect at zero bias. These findings indicate that the direction of forecast bias and agent forecast-reactivity are more strongly associated with downstream outcomes than forecast accuracy. Vendster is fully customizable and released under CC BY-NC 4.0 at this https URL.
| Subjects: | Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.38292 [cs.GT] |
| (or arXiv:2610.38292v1 [cs.GT] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38292 arXiv-issued DOI via DataCite |
Submission history
From: Charin Polpanumas [view email]
[v1]
Wed, 28 Oct 2026 05:23:48 UTC (257 KB)
来源：arXiv cs.AI · arxiv.org