AI Applications

RAD: 3DGS-based RL advances autonomous driving, achieving a 3x lower collision rate!

ARM4R pre-trains autoregressive robotic models using low-level 4D representations from human videos, achieving efficient transfer learning and improved task performance across various environments.

HUMANUP: A novel two-stage reinforcement learning framework enables real-world humanoid robots to autonomously recover from falls on various terrains.

FLAG-TRADER fuses LLMs & RL for enhanced financial trading, achieving superior performance compared to traditional methods by efficiently integrating multimodal data and adapting to market dynamics.

V2V-LLM leverages multi-modal LLMs for safer cooperative autonomous driving by fusing perception data from multiple vehicles, answering driving-related questions, and improving trajectory planning.

DexTrack achieves highly generalizable neural tracking control for dexterous robot manipulation by iteratively training a controller using high-quality demonstrations refined via homotopy optimization…

HMA: a novel approach for generating high-quality robotic videos 15x faster, enabling real-time policy evaluation and data augmentation for scaling robot learning.

Current pathology foundation models struggle with center variations; this paper introduces a robustness index to quantify this, revealing model biases and advancing robust model development.

FAST: A novel action tokenization method using discrete cosine transform drastically improves autoregressive vision-language-action models’ training and performance, enabling dexterous and high-freque…

EnerVerse: A novel framework seamlessly integrates convolutional and attention mechanisms to generate embodied future spaces for enhanced robotic manipulation, mitigating data scarcity with a generati…

Android Agent Arena (A3): A novel evaluation platform for mobile GUI agents offering diverse tasks, flexible action space, and automated LLM-based evaluation, advancing real-world AI agent research.

SWE-Gym, a novel environment for training real-world software engineering agents using 2,438 real-world Python task instances, achieves new state-of-the-art performance and is publicly available.

LearnLM enhances Gemini for education by training it to follow pedagogical instructions, leading to significant preference improvements over GPT-40, Claude 3.5, and Gemini 1.5 Pro in diverse learning …

MoDE makes AI for robot control faster and more efficient.

From language models to action models: building AI that does things.

TidyBot++: Low-cost, open-source holonomic mobile base makes robot learning easier.

CARP: A novel visuomotor policy learning paradigm achieves high accuracy and 10x faster inference than state-of-the-art by combining autoregressive efficiency and diffusion model precision through a c…

Moto: Bridging language for robot manipulation using latent motion tokens, achieving superior performance with limited data.

Code-as-Monitor (CaM) uses vision-language models and constraint-aware visual programming to achieve both reactive and proactive robotic failure detection in real-time, improving success rates and red…

WildLMa enables robots to perform complex, long-horizon manipulation tasks in unstructured environments by combining language-conditioned imitation learning, a whole-body controller for efficient tele…