↓Skip to main content

🏢 RIKEN AIP

SLTrain: a sparse plus low rank approach for parameter and memory efficient pretraining
·4422 words·21 mins· loading · loading
AI Generated Natural Language Processing Large Language Models 🏢 RIKEN AIP
SLTrain: Sparsity+low-rank pretraining boosts LLM efficiency by up to 73% memory reduction without performance loss!
On the Comparison between Multi-modal and Single-modal Contrastive Learning
·455 words·3 mins· loading · loading
AI Generated Multimodal Learning Vision-Language Models 🏢 RIKEN AIP
Multi-modal contrastive learning surpasses single-modal by leveraging inter-modal correlations to improve feature learning and downstream task performance, as demonstrated through a novel theoretical …
Generalized Tensor Decomposition for Understanding Multi-Output Regression under Combinatorial Shifts
·1493 words·8 mins· loading · loading
Machine Learning Multi-Output Regression 🏢 RIKEN AIP
This paper proposes Functional t-SVD and ERM-DS to solve multi-output regression under Combinatorial Distribution Shift (CDS), providing robust performance guarantees.
A Framework for Bilevel Optimization on Riemannian Manifolds
·1520 words·8 mins· loading · loading
Machine Learning Meta Learning 🏢 RIKEN AIP
This paper introduces a novel framework for bilevel optimization on Riemannian manifolds, providing efficient hypergradient estimation strategies and convergence analysis, with successful applications…