Generalization

Standard initialization in neural networks negatively impacts generalization ability under Neural Tangent Kernel theory, contradicting real-world performance, urging the development of improved theore…

Boosting GNN expressivity and generalization: Novel node individualization schemes lower sample complexity, improving substructure identification.

Latent Diffusion Transformers (DiTs) achieve almost-linear time training and inference through low-rank gradient approximations and efficient criteria, overcoming high dimensionality challenges.

Unlocking the scaling secrets of structured state-space models, this research identifies novel scaling rules for improved stability, generalization, and hyperparameter transferability, revolutionizing…

Passive data collection in complex social systems invalidates standard AI model validation; new methods are needed.

SGD can train neural networks to learn low-dimensional polynomials near the information-theoretic limit, surpassing previous correlational statistical query lower bounds.

Training AI models on AI-generated data leads to performance degradation, known as model collapse. This paper offers analytical formulas that precisely quantify this effect in high-dimensional regress…

Under-parameterized linear regression models can surprisingly exhibit double descent, contradicting traditional bias-variance assumptions.

New theoretical analysis reveals optimal binning strategies for minimizing bias in expected calibration error (ECE), improving machine learning model calibration evaluation.

Over-parameterized gradient descent dynamically adapts to signal structure, improving sequence model generalization and outperforming fixed-kernel methods.

Weighted pretrained features implicitly regularize models, and this paper reveals equivalent paths between weighting schemes and ridge regularization, enabling efficient hyperparameter tuning.

Graph Neural Networks’ (GNNs) computational power precisely mirrors that of arithmetic circuits, as proven via a novel C-GNN model; this reveals fundamental limits to GNN scalability.

New, tighter generalization bounds are derived for a class of stochastic learning algorithms that generate absolutely continuous probability distributions; enhancing our understanding of their perform…

Two-stage recommender systems using tree structures achieve better generalization with more branches and harmonized training data distributions across stages.

New information-theoretic generalization bounds, based on conditional f-information, improve existing methods by addressing unboundedness and offering a generic approach applicable to various loss fun…

Unlocking deep learning’s generalization mystery, this research pioneers a theoretical understanding of memorization neural network generalizability, revealing critical network structural requirements…

Eigen-SAM significantly boosts generalization in deep learning by directly addressing SAM’s limitations through explicit top Hessian eigenvalue regularization.

Drift-Resilient TabPFN masters temporal data shifts!

Cross-environment Intra-class Alignment (CIA) and its label-free variant, CIA-LRA, significantly improve node-level OOD generalization on graphs by aligning representations and eliminating spurious fe…

This work delivers dimension-free deterministic equivalents for random feature regression, revealing sharp excess error rates and scaling laws.