跳到正文
arXiv cs.AI· Ilker Oguz, Colin Christian, Christophe Moser, Demetri Psaltis, Gordon Wetzstein·· 10 小时前AI 评分48

编码智能体实现光学神经网络原位训练

In Situ Training of Optical Neural Networks by Coding Agents

AI 导读

研究者提出“optics as code”方法,让编码智能体仅凭测量数据训练光学神经网络,无需逐设备校准或硬件梯度。智能体通过返回探测相机图像的 harness 形成对光学的物理理解,并用可解释光学元件重新参数化网络。该方法在分类和图像编码任务上优于无梯度方法,以更低成本接近经校准模型的反向传播效果。

正文 · 原文

View PDF HTML (experimental)

Abstract:Optical neural networks promise fast, energy-efficient computing, but every device departs from its design and must be trained with the hardware. In situ training relies on an expert-built calibrated digital model of each device or on gradient-free trial and error. Coding agents, large language models that write and run programs, have shown an emergent ability to understand and edit the weights of digital neural networks. Here we introduce optics as code, a method in which a coding agent trains an optical neural network from measurements alone, through a harness that returns camera images of its probes and requires a hypothesis per request. We discovered that the agent forms a physical understanding of the optics and reparametrizes the network with interpretable optical elements tuned on the hardware. It outperforms gradient-free methods in classification and image encoding, approaching backpropagation through a calibrated model at a fraction of the cost. Needing neither per-device calibration nor gradients through the hardware, and improving with more capable models, optics as code offers a practical route to large, scalable optical neural networks and a general tool for calibrating and optimizing optical experiments.
Comments: 14 pages, 4 figures. Our project website is at this https URL
Subjects: Optics (physics.optics); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.23875 [physics.optics]
  (or arXiv:2611.23875v1 [physics.optics] for this version)
  https://doi.org/10.48550/arXiv.2611.23875

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ilker Oguz [view email]
[v1] Wed, 18 Nov 2026 05:29:13 UTC (5,923 KB)

来源:arXiv cs.AI · arxiv.org