arXiv cs.CL· Kaixiang Shu·· 11 小时前AI 评分24
GPT-2 Small、Qwen2.5-1.5B 与 Llama-3.2-1B 中的共享关系表征与竞争通路
Shared relations, specific choices: conditional sensitivity and competing pathways in language models
AI 导读
在 GPT-2 Small、Qwen2.5-1.5B 和 Llama-3.2-1B 的受控实体绑定实验中，跨查询角色共享的低维关系背景可迁移到新名称，移除后恢复独立训练的关系方向能让输出对比更接近原值。
正文 · 原文
Abstract:How does a relational representation shared across queries drive a query-specific choice? Controlled entity-binding experiments with GPT-2 Small, Qwen2.5-1.5B and Llama-3.2-1B reveal a low-dimensional relational background that is predominantly shared across query roles and transfers to new names. Same-domain interactions persist when another attribute domain is queried. After removing this component, restoring independently trained relation directions brings output contrasts closer to their original values than sign-reversed or equal-norm random controls in all three models. Yet a transferable relational representation need not have a transferable functional sign. In Qwen2.5, fixing model weights, the training-mean relation vector and edit dose yields opposite selection effects across binary and lexical-mapping prompts with different answer vocabularies. A factorial contrast of downstream sensitivities across attribute-query combinations determines the shared direction's local signed selection gain. In a separate eight-condition protocol, mean local gain is positive in Qwen2.5 and negative in Llama3.2. Late-component clamps resolve opposing pathway contributions: in Llama3.2, negative transmission through selected attention-only paths is partially offset by positive transmission through paths traversing both selected attention and MLP sets. Source-coordinate analyses establish distributed factor coverage, hidden-receiver readouts reveal layout-dependent combination, and instance-geometry interventions separate response direction from amplitude. These findings show how conditional sensitivity and competing pathways give shared relational structure its effect on specific choices.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.01809 [cs.CL] |
| (or arXiv:2611.01809v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01809 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kaixiang Shu [view email]
[v1]
Sun, 1 Nov 2026 11:25:07 UTC (798 KB)
来源：arXiv cs.CL · arxiv.org