Vision-Language Models

ChatCam empowers users to control cameras via natural language, using CineGPT for text-conditioned trajectory generation and an Anchor Determinator for precise placement, enabling high-quality video r…

Cambrian-1: Open, vision-centric multimodal LLMs achieve state-of-the-art performance using a novel spatial vision aggregator and high-quality data.

CALVIN: Instruction tuning boosts contextual video captioning, achieving state-of-the-art results!

Calibrated Self-Rewarding (CSR) significantly improves vision-language models by using a novel iterative approach that incorporates visual constraints into the self-rewarding process, reducing halluci…

SWAB bridges modality and capability gaps in Vision-Language Model selection using optimal transport, enabling accurate prediction of VLM performance without images.

PCNet boosts weakly-supervised referring image segmentation by progressively processing textual cues, mimicking human comprehension, and significantly improving target localization.

TransCLIP significantly boosts vision-language model accuracy by efficiently integrating transduction, a powerful learning paradigm that leverages the structure of unlabeled data.

MLLMs enhance text-to-video generation by providing 135k fine-grained video preferences, creating VIDEOPREFER, and a novel reward model, VIDEORM, boosting video quality and alignment.

This research introduces a novel framework for post-unlearning in text-to-image generative models, optimizing model updates to ensure both effective forgetting and maintained text-image alignment.

BoostAdapter enhances vision-language model test-time adaptation by combining instance-agnostic historical samples with instance-aware boosting samples for superior out-of-distribution and cross-domai…

Black-Box Forgetting achieves selective forgetting in large pre-trained models by optimizing input prompts, not model parameters, thus enabling targeted class removal without requiring internal model …

This research introduces a novel two-phase approach to improve AI model trustworthiness by ensuring both correct predictions and correct rationales. A new dataset with structured rationales and a rat…

BEND-VLM: A novel, efficient test-time debiasing method for vision-language models, resolving bias without retraining.

AWT: a novel framework boosts vision-language model’s zero-shot capabilities by augmenting inputs, weighting them dynamically, and leveraging optimal transport to enhance semantic correlations.

Automated Multi-level Preference (AMP) framework significantly improves multimodal large language model (MLLM) performance by using multi-level preferences during training, reducing hallucinations and…

AttnDreamBooth: A novel approach to text-to-image generation that overcomes limitations of prior methods by separating learning processes, resulting in significantly improved identity preservation and…

Atlas3D enhances text-to-3D generation by integrating physics-based simulations, producing self-supporting 3D models for seamless real-world applications.

This paper introduces AAA, a novel three-stage decision-based black-box targeted attack against image-to-text models. AAA efficiently generates semantically consistent adversarial examples by asking …

MMStar benchmark tackles flawed LVLMs evaluation by focusing on vision-critical samples, minimizing data leakage, and introducing new metrics for fair multi-modal gain assessment.

Leveraging vision-language models, this research introduces a novel unsupervised zero-shot audio captioning method that achieves state-of-the-art performance by aligning audio and image token distribu…