Video Understanding

Optical-flow-free video semantic segmentation excels in adverse weather by merging adjacent frame information via a fusion block and a novel temporal-spatial teacher-student learning strategy.

BaFormer: a novel boundary-aware Transformer network achieves efficient and accurate temporal action segmentation by using instance and global queries for segment classification and boundary predictio…

EEG2Video reconstructs dynamic videos from EEG signals, achieving 79.8% accuracy in semantic classification and 0.256 SSIM in video reconstruction.

E-Motion: Predicting future motion with unprecedented accuracy using event cameras and video diffusion models.

DreamScene4D generates realistic 3D dynamic multi-object scenes from monocular videos via novel view synthesis, addressing limitations of existing methods with a novel decompose-recompose approach.

Run-Length Tokenization (RLT) dramatically speeds up video transformer training and inference by efficiently removing redundant video tokens, matching baseline model performance with significant time …

This paper introduces a novel differentiable framework for learning task graphs from video demonstrations of procedural activities. By directly optimizing the weights of a task graph’s edges, the mod…

DeltaDEQ accelerates deep equilibrium model inference by 73-84% via a novel ‘heterogeneous convergence’ exploitation technique, maintaining accuracy.

CV-VAE: A compatible video VAE enabling efficient, high-quality latent video generation by bridging the gap between image and video latent spaces.

COVE: Consistent high-quality video editing achieved by leveraging diffusion feature correspondence for temporal consistency.

Collaborative Video Diffusion (CVD) generates multiple consistent videos from various camera angles using a novel cross-video synchronization module, significantly improving consistency compared to ex…

bit2bit reconstructs high-quality videos from sparse, binary quanta image sensor data using self-supervised photon location prediction, significantly improving resolution and usability.

Beyond Euclidean spaces, Dual-Space Representation Learning (DSRL) enhances weakly supervised video violence detection by cleverly integrating Euclidean and hyperbolic geometries for superior discrimi…

CPDTrack: Human-like Visual Search Boosts Object Tracking!

AverNet: All-in-one video restoration defying time-varying unknown degradations.

ActAnywhere, a novel video diffusion model, seamlessly integrates foreground subjects into new backgrounds by generating realistic video backgrounds tailored to subject motion, significantly reducing …

A novel motion-aware spatio-temporal graph model surpasses existing methods in video salient object ranking by jointly optimizing multi-scale spatial and temporal features, thus accurately prioritizin…

4Real: Photorealistic 4D scene generation from text prompts using video diffusion models, exceeding object-centric approaches for higher realism and efficiency.