Multimodal Learning

PCRIL, a novel prompt contrastive recovery approach, significantly boosts cross-modal hashing accuracy, especially when dealing with incomplete labels by progressively identifying promising positive c…

EvolveDirector trains competitive text-to-image models using publicly available data by cleverly leveraging large vision-language models to curate and refine training datasets, dramatically reducing d…

Researchers introduce object-aware backdoors in Vision-and-Language Navigation, enabling malicious behavior upon encountering specific objects, demonstrating the vulnerability of real-world AI agents.

Frolic: A label-free framework boosts zero-shot vision model accuracy by learning prompt distributions and correcting label bias, achieving state-of-the-art performance across multiple datasets.

DEMO framework enhances text-to-video generation by decomposing text encoding and conditioning into content and motion components, resulting in videos with significantly improved motion dynamics.

Vision-Augmented Prompting (VAP) boosts LLM reasoning by automatically generating images from textual problem descriptions, incorporating visual-spatial clues to significantly improve accuracy across …

Large foundation models empower visible-infrared person re-identification by enriching infrared image representations with automatically generated textual descriptions, significantly improving cross-m…

Emotion-LLaMA: A new multimodal large language model excels at emotion recognition and reasoning, outperforming existing models and leveraging a newly created dataset, MERR.

E2, a novel regional contrastive learning method, enhances vision-language models for expressive fashion representations by explicitly attending to fashion details with minimal additional parameters, …

Dual Risk Minimization (DRM) improves fine-tuned zero-shot models’ robustness by combining empirical and worst-case risk minimization, using LLMs to identify core features, achieving state-of-the-art …

Dual Prototype Evolving (DPE) significantly boosts vision-language model generalization by cumulatively learning multi-modal prototypes from unlabeled test data, outperforming current state-of-the-art…

DigiRL: Autonomous RL trains robust in-the-wild device-control agents by offline-to-online RL, surpassing prior methods.

Diffusion-Inspired Truncated Sampler (DITS) revolutionizes text-video retrieval by progressively aligning embeddings and enhancing CLIP embedding space structure, achieving state-of-the-art results.

DiffusionPID unveils the secrets of text-to-image diffusion models by decomposing text prompts into unique, redundant, and synergistic components, providing insights into how individual words and thei…

DiffPano generates scalable, consistent, and diverse panoramic images from text descriptions and camera poses using a novel spherical epipolar-aware diffusion model.

Boosting multimodal LLMs, the Dense Connector efficiently integrates multi-layer visual features for significantly enhanced performance.

Vision-language models (VLMs) memorize training data, impacting generalization. This paper introduces ‘déjà vu memorization,’ a novel method measuring this, revealing significant memorization even in…

DeepStack: Stacking visual tokens boosts LMMs efficiency and performance!

Deep Correlated Prompting enhances large multimodal models’ robustness against missing data by leveraging inter-layer and cross-modality correlations in prompts, achieving superior performance with mi…

DARNet: a dual attention network for auditory attention detection surpasses current state-of-the-art models, especially in short decision windows, achieving this with a 91% reduction in parameters.