🏢 Chinese University of Hong Kong

VideoPainter: Edit any video, any length, with user-guided instructions!

TrajectoryCrafter: Precisely control camera movement in monocular videos with a novel diffusion model for coherent 4D content generation.

Dyve: A novel dynamic process verifier boosts LLM reasoning accuracy by cleverly combining fast, immediate checks with deeper, slower analyses for complex steps, achieving significant performance gain…

Lumina-Video: Efficient and flexible video generation using a multi-scale Next-DiT architecture with motion control.

MotionCanvas lets users design cinematic video shots with intuitive controls for camera and object movements, translating scene-space intentions into video animations.

CMOE efficiently transforms dense LLMs into sparse MoE architectures via expert carving, enabling fast inference without extensive retraining.

Large language models (LLMs) are rapidly evolving, yet often struggle to adapt to human preferences quickly. This paper introduces Test-Time Preference Optimization (TPO), an innovative framework that…

Dispider: A novel system enabling real-time interaction with video LLMs via disentangled perception, decision, and reaction modules for efficient, accurate responses to streaming video.

NILE framework significantly boosts LLM performance by aligning instruction-tuning datasets with pre-trained internal knowledge, achieving up to 68.5% gains.

Neural LightRig uses multi-light diffusion to accurately estimate object normals and materials from a single image, outperforming existing methods.

Imagine360: Generating immersive 360° videos from perspective videos, improving quality and accessibility of 360° content creation.

Video-3D LLM masters 3D scene understanding by cleverly fusing video data with 3D positional encoding, achieving state-of-the-art performance.