Neural Expansion：深度神经网络泛化机制的统一几何框架

View PDF HTML (experimental)

Abstract:Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space. Using Generalization Intervals (GIs), we quantify direction-specific regions around training samples where the model's predictions remain stable. Our extensive empirical studies across diverse architectures and datasets reveal that training leads to progressive expansion of ray coverage, particularly along directions with low Jacobian singular values. This mechanism helps explain a large fraction of correctly classified test inputs, adversarial examples, and even mislabeled or out-of-distribution samples through the geometry of ray coverage rather than instance-level memorization. By connecting input-space curvature to predictive behavior, our framework provides a mechanistic foundation that complements existing global theories of deep learning.

Comments:	Accepted at NeurIPS 2026. Includes supplementary material. Code: this https URL
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.23716 [cs.LG]
	(or arXiv:2610.23716v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.23716 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Chashi Mahiul Islam [view email]
[v1] Fri, 16 Oct 2026 16:38:02 UTC (2,138 KB)