arXiv cs.CL· Hy Nguyen, Thomas James Elton, Ella Wilson, Andrew Reeson, Cecile Paris, Josiah Poon, Jonathan K. Kummerfeld·· 11 小时前AI 评分40
VisCodeBench：面向真实场景文本到可视化生成的新基准数据集
VisCodeBench: a new benchmark dataset for Text-to-Vis reflecting real-world practice
AI 导读
VisCodeBench 是一个端到端 text-to-vis 基准，含教育、科学和大众场景共 2,672 个样本，覆盖不同复杂度。评测显示强系统基础执行已接近解决，但最佳数据相似度仅 0.81，最难子集降至 0.69；ClaudeCode-Opus-4.7 的图表正确率比 Codex-GPT-5 高约 5 分且快约 20 秒。
正文 · 原文
Abstract:Generating visualisations from natural language requests (text-to-vis) enables users to explore data more efficiently. While LLMs can complete simple visualisation requests, their performance on challenging ones is not well covered by existing benchmarks. We introduce VisCodeBench, an end-to-end text-to-vis benchmark with 2,672 samples from three different domains: educational, scientific, and general audience scenarios with varying levels of complexity. We benchmark three visualisation systems---Codex, Claude Code, and LIDA---paired with a range of open and commercial LLMs across seven metrics covering execution, chart content, and visual quality. While basic execution is near-solved for strong systems, no system comes close to reproducing the reference data, with the best displayed data similarity reaching only $0.81$ and falling to $0.69$ on our hardest subset for this metric. We further find that frontier models separate on chart correctness and efficiency, with ClaudeCode-Opus-4.7 improving chart correctness by around 5 points over Codex-GPT-5 while running roughly 20 seconds faster. Separately, we find that system design matters as much as the underlying model: despite both running on GPT-5, Codex clearly outperforms LIDA, showing that the surrounding prompting, execution, and refinement pipeline can shape outcomes as much as the model itself. Finally, latency remains a practical obstacle, as even the strongest systems take 39-78 seconds to produce results. VisCodeBench is available at \href{this https URL}{this https URL} and accessed via a leaderboard; the full dataset is kept private to prevent LLM training use.
| Comments: | Accepted to COLM 2026. this https URL |
| Subjects: | Computation and Language (cs.CL); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2611.02353 [cs.CL] |
| (or arXiv:2611.02353v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02353 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ngoc Gia Hy Nguyen [view email]
[v1]
Mon, 2 Nov 2026 01:00:32 UTC (3,338 KB)
来源：arXiv cs.CL · arxiv.org