Deep Learning

New ‘Feature Speed Formula’ predicts & controls deep learning’s hierarchical feature learning by linking hyperparameter tuning to the angle between feature updates and backward pass.

Researchers found that superior OOD detection performance comes at the cost of reduced generalization. Their novel Decoupled Uncertainty Learning (DUL) algorithm harmonizes OOD detection and generali…

Researchers unveil the Infeasibility Theorem, proving optimal class-incremental learning is impossible with discriminative models due to task confusion, and the Feasibility Theorem, showing generative…

Shortcut back-propagation and an evolutionary training framework conquer gradient vanishing in spiking neural networks, drastically improving training and achieving state-of-the-art accuracy.

Leveraging symmetry in automated equation discovery improves accuracy and simplicity of learned governing equations, enhancing robustness against noise and achieving higher success rates across divers…

Swift Sampler (SS) automates the learning of efficient data samplers for deep learning, achieving significant performance gains (e.g., 1.5% on ImageNet) with minimal computational cost using only 10 p…

Deep learning’s Adam-style optimizers exhibit a surprising surge phenomenon: optimal learning rates initially increase, then decrease, before converging to a non-zero value as batch size grows.

SLATE: Supra-Laplacian encoding for spatio-temporal Transformers achieves state-of-the-art dynamic link prediction by innovatively using a multi-layer graph representation and a unique cross-attention…

Neural network hyperparameter transferability across vastly different model sizes is achieved via a newly discovered property called ‘Super Consistency’ of loss landscapes.

Sumba: a novel forecasting model achieves up to 8.5% improvement by using a structured matrix basis to generate dynamic spatial structures with lower variance and better interpretability.

SICSM, a novel framework, integrates selective SSMs and GFNs to accurately infer complex dynamical system structures from irregularly sampled, partially observed trajectories.

Researchers extended stochastic optimal control theory to infinite-dimensional spaces, enabling the creation of diffusion bridges for generative modeling in function spaces, demonstrating applications…

Deep kernel machines now achieve 94.5% accuracy on CIFAR-10, matching neural networks, by using stochastic kernel regularization to improve generalization.

Adaptive learning rate tuners often underperform; Curvature Dynamics Aware Tuning (CDAT) prioritizes long-term curvature stabilization, outperforming tuned constant learning rates.

GRAPHSSM: a novel graph state space model efficiently captures temporal graph dynamics, overcoming limitations of existing sequence models.

STk: a novel, differentiable module solves Top-k problems in neural networks without extra time/GPU memory, boosting performance in long-tailed learning.

STMixer: a novel SNN architecture enabling high performance on both synchronous and asynchronous neuromorphic hardware, achieving comparable results to spiking transformers with drastically lower powe…

Spiking Graph Neural Networks (SGNNs) on Riemannian Manifolds achieve superior performance and energy efficiency via a novel Manifold Spiking GNN (MSG).

PGLDM, a novel algorithm, accurately identifies shared and private dynamics in two generalized-linear time series, improving model accuracy and enabling lower-dimensional latent state representations.

Spatio-Spectral GNNs synergistically combine spatial and spectral graph filters for efficient, global information propagation, overcoming limitations of existing methods.