Video Understanding

4D LangSplat learns 4D language fields for dynamic scenes using multimodal large language models, enabling time-sensitive open-vocabulary queries.

TPDiff accelerates video diffusion by progressively increasing frame rates during diffusion, optimizing computational efficiency with a novel stage-wise training strategy.

Reangle-A-Video generates synchronized multi-view videos from a single video via video-to-video translation, surpassing existing methods without specialized 4D training.

Open-Sora 2.0: A commercial-level video generation model trained for only $200k, achieving comparable results to state-of-the-art models.

Unified framework reveals and mitigates error sources in autoregressive video diffusion models.

SynCoS: Synchronized sampling generates high-quality & coherent long videos from text, without extra training!

QuoTA: Task-aware token assignment boosts long video comprehension in LVLMs via query-decoupled processing, without extra training!

SBD: Self-supervised skill discovery from unsegmented videos!

AnyMoLe: Generate character motion in-between frames for diverse characters by video diffusion models without external data. Code: project page.

DreamRelation: Personalize videos by customizing relationships between subjects, generalizing to new domains.

DropletVideo: A dataset and approach to explore integral spatio-temporal consistent video generation.

VideoPainter: Edit any video, any length, with user-guided instructions!

TrajectoryCrafter: Precisely control camera movement in monocular videos with a novel diffusion model for coherent 4D content generation.

MagicInfinite: Infinite talking videos from words and voice!

AnyAnomaly: LVLM for customizable zero-shot video anomaly detection, adapting to diverse environments without retraining.

Mobius generates seamless looping videos from text using latent shift, repurposing pre-trained models without training.

X-Dancer: Expressive dance video generation from music and a single image!

VideoGrain: Fine-grained video editing via space-time attention!

Personalizing video models for dynamic concepts is now achievable with Set-and-Sequence: enabling high-fidelity generation, editing, and composition!

MaskGWM: Improves driving world models by using video mask reconstruction for better generalization.