跳到正文
arXiv cs.AI· Eduardo C. Garrido-Merch\'an·· 4 小时前AI 评分31

大语言模型能否零样本完成权重空间编程?结构化数据集目标变量预测与分类研究

Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets

AI 导读

研究检验大语言模型能否在无梯度下降、无优化器、无代码执行的条件下,仅凭完整小数据集和前馈网络架构,直接写出每个权重与偏置的数值。在涵盖玩具问题、隐藏旋转伪装的合成问题、经典基准与真实工程回归的评测中,冻结网络在线性最优处匹配闭式解,在三个非线性层级达到生成过程的不可约误差,并在其中两个层级上超过包括梯度下降训练的同类架构在内的所有拟合基线;两个失败层级均因待发现结构与其分析坐标斜交。

正文 · 原文

View PDF HTML (experimental)

Abstract:Weight-space programming fixes every parameter of a feed-forward network by hand, with no gradient descent anywhere in the construction. Every demonstration so far starts from a target function handed to the model as a rule system, making it a compilation problem. A supervised dataset withholds that specification, so the procedure must do the statistical inference itself and then express the result in the parameterisation it was given. We ask whether a large language model can do both. It is shown a small dataset in full, with the architecture of a feed-forward network, and writes out every weight and bias as a literal number, with no optimizer, no gradient, no code execution, and no chance to evaluate it first. Each construction is sealed and pre-registers its expected score. The graded suite runs from two toy problems, through nine synthetic problems disguised by a hidden rotation, to the classical benchmarks and one real engineering regression. The frozen network matches the closed-form optimum wherever that optimum is linear, reaches the irreducible error of the generating process on three rungs where it is not, and beats every fitted baseline on two of those, including the same architecture trained by gradient descent. It fails on two rungs, both times because the structure to be found is oblique to the coordinates it analysed. Three further constructors separate two failure modes a single score conflates: one returns no parameter vector at all on the hardest rungs, another delivers everywhere while writing networks worse than predicting the mean. One of those failures proves avoidable: another constructor solves the oblique-hinge rung by a margin that survives multiple-comparison correction. What predicts whether the hidden layer earns its keep is the training error of a linear separator, which the composite complexity index misses.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.19292 [cs.AI]
  (or arXiv:2611.19292v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.19292

arXiv-issued DOI via DataCite

Submission history

From: Eduardo C. Garrido-Merchán [view email]
[v1] Sat, 14 Nov 2026 09:37:17 UTC (79 KB)
[v2] Tue, 17 Nov 2026 09:50:04 UTC (80 KB)

来源:arXiv cs.AI · arxiv.org