🏢 EPFL

Metric backbone graph sparsification surprisingly preserves community structure, offering an efficient and robust method for analyzing large networks.

Weight decay’s role in modern deep learning is surprisingly multifaceted, impacting optimization dynamics rather than solely regularization, improving generalization and training stability.

SuperDeepFool: a fast, accurate algorithm generating minimal adversarial perturbations, significantly improving deep learning model robustness evaluation and adversarial training.

SGD surprisingly diminishes network rank, unlike GD, due to a repulsive force between eigenvalues, offering insights into deep learning generalization.

Revolutionizing LLM training: Constant learning rate with cooldown replaces cosine schedule, enabling cost-effective scaling experiments!

SAMPa: Parallelizing gradient computations in Sharpness-Aware Minimization (SAM) achieves a 2x speedup and superior generalization.

FENS: a novel federated ensembling scheme that boosts one-shot federated learning accuracy to near iterative FL levels, while maintaining low communication costs.

Transformers’ learning dynamics depend heavily on initialization and Markovian data properties, leading to either global or local minima; this paper proves this, offers initialization guidelines, and …

Mirror descent’s implicit bias on separable data is formally characterized, revealing convergence towards a maximum margin classifier determined by the potential’s ‘horizon function’.

GRAPHEDX, a novel neural network, accurately estimates graph edit distance with varying operation costs, outperforming existing methods.

ConStruct: Generating realistic graphs with guaranteed structural properties via constrained diffusion.

Fine-tune personalization in federated learning to beat adversarial clients; collaboration level depends on data heterogeneity and adversary fraction.

DenseFormer enhances transformers by adding a depth-weighted averaging step, improving data efficiency and outperforming baselines in memory and inference time without increasing model size.

CoBo: A novel bilevel optimization algorithm for collaborative learning surpasses existing methods by efficiently selecting helpful clients, resulting in superior performance and scalability.