Video Understanding

MTGS: a unified framework jointly predicts gaze and social gaze (shared attention, mutual gaze) for multiple people in videos, achieving state-of-the-art results using a temporal transformer model and…

MooG: Self-supervised video model learns off-the-grid representations, enabling consistent scene element tracking even with motion; outperforming grid-based baselines on various vision tasks.

MotionCraft: Physics-based zero-shot video generation creates realistic videos with complex motion dynamics by cleverly warping the noise latent space of an image diffusion model using optical flow fr…

Motion Graph unleashes efficient and accurate video prediction by transforming video frames into interconnected graph nodes, capturing complex motion patterns with minimal computational cost.

Boosting video diffusion: Motion Consistency Model (MCM) disentangles motion and appearance learning for high-fidelity, fast video generation using few sampling steps.

MoTE: A novel framework harmonizes generalization and specialization for visual-language video knowledge transfer, achieving state-of-the-art results.

MECD: A new task and dataset unlocks multi-event causal discovery in videos, enabling a novel framework that outperforms existing models by efficiently identifying causal relationships between chronol…

MambaSCI: Efficient, novel deep learning model reconstructs high-quality quad-Bayer video from compressed snapshots, surpassing existing methods.

Long-range feedback spiking network (LoRaFB-SNet) surpasses other models in capturing dynamic and static visual cortical representations under movie stimuli, advancing our understanding of visual syst…

TURTLE: a novel video restoration framework that learns a truncated causal history model for efficient and high-performing video restoration, achieving state-of-the-art results on various benchmark ta…

HOI-Swap: a novel diffusion model flawlessly swaps objects in videos while intelligently preserving natural hand interactions, producing high-quality edits.

GoMatching, a novel video text spotting baseline, enhances tracking efficiency while maintaining strong recognition by integrating long- and short-term matching via a Transformer-based module and a re…

GenRec: One diffusion model to rule both video generation and recognition!

Generalizable Implicit Motion Modeling (GIMM) revolutionizes video frame interpolation by accurately predicting optical flows at any timestep, surpassing existing methods and achieving state-of-the-ar…

FreeLong: Generate high-fidelity long videos without retraining using spectral blending of global and local video features!

FIFO-Diffusion generates infinitely long, high-quality videos from text prompts using a pretrained model, solving the challenge of long video generation without retraining.

Streamlined Inference, a novel training-free framework, dramatically reduces the computation and memory costs of video diffusion models without sacrificing quality, enabling high-resolution video gene…

FactorizePhys leverages Non-negative Matrix Factorization for a novel multidimensional attention mechanism (FSAM) to improve remote PPG signal extraction from videos.

Long-video masked autoencoders (LVMAE) achieve state-of-the-art performance by using an adaptive masking strategy that prioritizes important video tokens, enabling efficient training on 128 frames.

Exo2Ego-V generates realistic egocentric videos from sparse exocentric views, significantly outperforming state-of-the-art methods on a challenging benchmark.