🏢 University of Oxford
Bayesian Optimization of Functions over Node Subsets in Graphs
·3183 words·15 mins·
loading
·
loading
AI Theory
Optimization
🏢 University of Oxford
GraphComBO efficiently optimizes functions defined on node subsets within graphs using Bayesian Optimization. It tackles challenges posed by combinatorial complexity and computationally expensive fun…
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts
·1807 words·9 mins·
loading
·
loading
Natural Language Processing
Large Language Models
🏢 University of Oxford
BAM! Efficiently upcycles pre-trained models into powerful Mixture-of-Experts (MoE) models, achieving state-of-the-art performance with reduced computational costs.
Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
·1937 words·10 mins·
loading
·
loading
Machine Learning
Reinforcement Learning
🏢 University of Oxford
Reinforcement learning agents achieve emergent cultural accumulation by balancing social and independent learning, outperforming single-lifetime agents.
An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem
·2679 words·13 mins·
loading
·
loading
Machine Learning
Deep Learning
🏢 University of Oxford
A novel multilinear model analytically explains the emergence and scaling laws of skills in the multitask sparse parity problem, accurately predicting skill emergence in neural networks.
Amortized Active Causal Induction with Deep Reinforcement Learning
·3383 words·16 mins·
loading
·
loading
AI Generated
Machine Learning
Reinforcement Learning
🏢 University of Oxford
CAASL: An amortized active intervention design policy trained via reinforcement learning, enabling adaptive, real-time causal graph inference without likelihood access.
Almost Surely Asymptotically Constant Graph Neural Networks
·1976 words·10 mins·
loading
·
loading
AI Theory
Generalization
🏢 University of Oxford
Many graph neural networks (GNNs) surprisingly converge to constant outputs with increasing graph size, limiting their expressiveness.
Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps
·2522 words·12 mins·
loading
·
loading
AI Generated
Machine Learning
Reinforcement Learning
🏢 University of Oxford
Adam-Rel: A novel optimizer for RL, dramatically improves performance by resetting Adam’s timestep to 0 after target network updates, preventing large, suboptimal changes.
A General Protocol to Probe Large Vision Models for 3D Physical Understanding
·4012 words·19 mins·
loading
·
loading
AI Generated
Computer Vision
3D Vision
🏢 University of Oxford
Researchers developed a lightweight protocol to probe large vision models’ 3D physical understanding by training classifiers on model features for various scene properties (geometry, material, lightin…