🏢 Microsoft Research

VASA-1: Real-time, lifelike talking faces generated from a single image and audio!

Researchers unveil how multi-modal LLMs process information, revealing that early layers are key for storage, and introduce MULTEDIT, a model-editing algorithm for correcting errors and inserting new …

Training-free guidance revolutionizes diffusion models by enabling zero-shot conditional generation, but suffers from misaligned gradients and slow convergence. This paper provides theoretical analysi…

Trace: Automating AI workflow design with LLMs.

FleVRS: One unified model masters standard, promptable, and open-vocabulary visual relationship segmentation, outperforming existing methods.

TEdit: a novel diffusion model edits existing time series to meet specified attribute targets, preserving other properties, solving limitations of prior synthesis methods.

Slot-VLM generates semantically decomposed video tokens using an Object-Event Slots module, improving video-language model performance.

VQGAN-LC massively scales VQGAN’s codebook to 100,000 entries while maintaining a 99% utilization rate, significantly boosting image generation and downstream task performance.

IBProtector shields LLMs from harmful outputs via prompt compression, selectively preserving essential information using a trainable extractor.

PCformer boosts Transformer performance by using a predictor-corrector learning framework and exponential moving average coefficient learning for high-order prediction, achieving state-of-the-art resu…

Boosting AI instruction following, Language Feedback Models (LFMs) leverage Large Language Models (LLMs) to identify desirable behaviors from visual trajectories, significantly improving task completi…

Physically consistent multi-task learning bridges heterogeneous molecular data by directly leveraging physical laws to improve predictions, enhancing accuracy beyond the limitations of individual data…

This paper introduces a novel information-theoretic framework, showing how to convert offline into online estimation algorithms efficiently, impacting interactive decision-making.

AI-generated preference data improves text-to-image alignment.

Multi-Head Mixture-of-Experts (MH-MoE) drastically boosts large language model efficiency by activating almost all expert networks, achieving superior performance compared to existing Sparse Mixture-o…

LLMs’ spatial reasoning abilities are boosted by visualizing their thought processes via ‘Visualization-of-Thought’ prompting, significantly improving performance on navigation and tiling tasks.

Offline imitation learning achieves surprisingly strong performance, matching online methods’ efficiency under certain conditions, contradicting prior assumptions.

SpatialEval benchmark reveals that current vision-language models struggle with spatial reasoning, highlighting the need for improved multimodal models that effectively integrate visual and textual in…

Deep Equilibrium Models (DEQs) infused into DFT Hamiltonian prediction achieves self-consistency, accelerating large-scale materials simulations.

Context-aware preference modeling improves language model alignment by resolving ambiguity through a two-step process: context selection followed by context-specific preference evaluation. The approa…