arXiv cs.CL· Hongxiang Zhang, Yuan Tian, Tianyi Zhang·· 12 小时前AI 评分31
ATAR：面向大语言模型的注意力对齐推理方法
Attention-Aligned Reasoning for Large Language Models
AI 导读
研究者提出 ATAR 推理方法，利用推理链固有结构引导 LLM 注意力，在六个基准上超越 SOTA 方法，最高取得 15.39% 的绝对提升。使用 ATAR 后，同规模的“非推理”模型在多数基准上达到甚至超过推理模型的表现。消融实验显示注意力对齐组件贡献显著，且改进在不同注意力引导后端下均保持有效。
正文 · 原文
Abstract:Large Language Models (LLMs) tend to generate a long reasoning chain when solving complex tasks. However, as the reasoning chain extends, critical intermediate steps and the original prompt will be buried in the context, receiving insufficient attention and leading to errors. In this work, we present ATAR, a novel reasoning method that leverages the inherent reasoning structure to steer LLM attention. Our experiments show that ATAR outperforms SOTA methods across six benchmarks, achieving up to 15.39% absolute improvement. Furthermore, with ATAR, "non-reasoning" models achieve comparable or even better performance compared to reasoning models of the same size in most benchmarks. Finally, our ablation studies show that the attention alignment component contributes significantly, and that these improvements are persist under different attentionsteering backends.
| Comments: | EMNLP 2026 (Findings) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2510.03223 [cs.CL] |
| (or arXiv:2510.03223v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.03223 arXiv-issued DOI via DataCite |
Submission history
From: Hongxiang Zhang [view email]
[v1]
Fri, 3 Oct 2025 17:56:33 UTC (791 KB)
[v2]
Fri, 27 Mar 2026 17:14:20 UTC (1,086 KB)
[v3]
Tue, 20 Oct 2026 17:39:02 UTC (1,101 KB)
来源：arXiv cs.CL · arxiv.org