arXiv cs.CL· Evanfiya Logacheva, Arto Hellas, Tsvetomila Mihaylova, Juha Sorva, Ava Heinonen, Juho Leinonen·· 11 小时前AI 评分29
基于话语相似度的上下文学习如何影响图表代码生成的忠实度与质量
In-Context Learning for Diagram Code Generation: What is the Effect on Faithfulness and Quality?
AI 导读
研究发现，用"话语分析到图表"示例的非常规 ICL 方法可将图表生成的上下文忠实度幻觉几率降低近 70%，但 ICL 对图表美观呈现和表层连贯性无显著影响。G-Eval 评测显示该新任务格式下完美忠实度结果的几率提升近 4.5 倍。研究对比了零样本指令生成与基于话语相似度示例的 ICL 方法，采用人工与自动双重评估。
正文 · 原文
Abstract:We investigate how provision of in-context examples based on discourse similarity affects quality and hallucination in diagram code generation from educational materials. Using a sample of texts as sources for diagram generation, we test two methods using a conventional text-to-diagram task format, instruction-based zero-shot generation and ICL with text-to-diagram examples, and an unconventional ICL method with a discourse analysis-to-diagram example, which requires a model to generate a diagram from a discourse analysis. We evaluate AI-generated diagrams for faithfulness to provided context and their quality outcomes using both human and automated evaluation. Bayesian data analysis of the human diagram evaluation shows that there is significant evidence that the unconventional task format improves context faithfulness in diagram generation compared to the zero-shot instruction with reducing the odds of context faithfulness hallucination by nearly $70\%$; however, we do not find the ICL having a significant effect on the diagram aesthetic presentation and superficial coherence. Evaluation with the LLM-as-a-judge G-Eval metric shows nearly 4.5-fold increased odds of the perfect faithfulness outcome for the novel ICL task. Our study provides a novel insight into ICL format effect on superficial fluency and faithfulness in context-grounded structured diagram code generation.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.20476 [cs.CL] |
| (or arXiv:2601.20476v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.20476 arXiv-issued DOI via DataCite |
Submission history
From: Evanfiya Logacheva [view email]
[v1]
Wed, 28 Jan 2026 10:45:28 UTC (1,034 KB)
[v2]
Fri, 21 Aug 2026 14:06:11 UTC (888 KB)
[v3]
Wed, 4 Nov 2026 14:33:42 UTC (8,532 KB)
来源：arXiv cs.CL · arxiv.org