Representation Learning

Boost time-series model accuracy with Segment, Shuffle, and Stitch (S3)! This simple layer shuffles data segments to enhance representation learning, improving classification, forecasting, and anomaly…

Schur Nets boost higher-order GNNs by efficiently exploiting local graph structure for automorphism equivariance, achieving improved performance without the computational burden of traditional methods…

First finite-sample analysis of interventional causal representation learning shows that surprisingly few samples suffice for accurate graph and latent variable recovery.

This paper proposes a lightweight and scalable k-mer based model for metagenomic binning, achieving comparable performance to computationally expensive genome foundation models while significantly imp…

MUSE, a novel graph anomaly detection method, leverages multifaceted summaries of reconstruction errors, achieving state-of-the-art performance by addressing limitations of existing Graph-AE-based met…

Random pixel projections outperform complex online continual learning methods for image classification, challenging assumptions about representation learning.

Pure message passing in graph neural networks can accurately estimate common neighbor heuristics for superior link prediction.

Researchers achieve provably optimal memory capacity in transformer-compatible Hopfield models by framing the problem as an optimal spherical code arrangement, resulting in a novel sublinear time algo…

ProtGO: A novel unified framework integrating protein sequence, structure & function for superior representation learning, significantly outperforming current methods.

Noise Regularization rescues Deep Canonical Correlation Analysis from model collapse!

POSEIDON: a novel foundation model for PDEs achieves significant gains in accuracy and sample efficiency, generalizing well to unseen physics.

PLIP: Novel language-image pre-training framework excels at person representation learning, surpassing existing methods on various downstream tasks thanks to its three pretext tasks and large-scale SY…

Transformers’ self-attention mechanism, while powerful, suffers from rank collapse with increasing depth. This paper reveals that while masked attention still leads to exponential collapse, sparse att…

Graph Neural Networks (GNNs) struggle with heterophilic link prediction; this paper introduces formal definitions, theoretical analysis, improved designs, and real-world benchmarks to address this cha…

This paper introduces a novel notion of intrinsic similarity between language encoders, based on affine homotopy, and demonstrates its strong correlation with extrinsic similarity (downstream task per…

RNNs trained on dual-agent path integration develop distinct internal representations compared to single-agent models, exhibiting weaker grid cell responses and enhanced border/band cell activity, wit…

Non-Euclidean Mixture Model (NMM-GNN) outperforms existing methods by using spherical and hyperbolic spaces to model homophily and social influence in social network embedding, improving link predicti…

Neural Persistence Dynamics learns collective behavior from topological features, accurately predicting parameters of governing equations without tracking individual entities.

Researchers introduce a novel approach to mutual information (MI) estimation using normalizing flows, providing accurate estimates even in high dimensions.

S3F: a novel multi-scale model achieves state-of-the-art protein fitness prediction by integrating protein sequence, structure, and surface features.