Vision-Language Models

LeHaCE: a novel framework for evaluating object hallucination in LVLMs, improving evaluation stability and fairness by accounting for instruction-induced image description length variations.

TFG: A unified, training-free framework for boosting diffusion model performance by efficiently searching its algorithm-agnostic design space.

Hassle-Free Textual Training (HFTT) uses only textual data to effectively remove unwanted visual data from AI training datasets, significantly reducing human annotation needs.

Ti-FAD: a novel zero-shot temporal action detection model outperforms state-of-the-art methods by enhancing text-related visual focus and foreground awareness.

Text-Guided Attention for Zero-Shot Robustness (TGA-ZSR) significantly improves vision-language model robustness against adversarial attacks by aligning and constraining text-guided attention, achievi…

Text-DiFuse: A novel interactive multi-modal image fusion framework leverages text-modulated diffusion models for superior performance in complex scenarios.

Text-Aware Diffusion for Policy Learning (TADPoLe) uses pretrained diffusion models for zero-shot reward generation, enabling natural language-driven policy learning without manual reward design.

This work presents statistically grounded methods to rank semantic concept importance in black-box models, using conditional independence testing for both global and local interpretations.

RaTSG network tackles Temporal Sentence Grounding with Relevance Feedback (TSG-RF) by discerning query relevance at multiple granularities before selectively grounding segments.

Tactile DreamFusion: High-resolution tactile sensing enhances 3D generation, creating realistic geometric details previously unattainable.

TMEA: A novel approach significantly boosts multi-modal entity alignment accuracy by effectively handling uncertain correspondences between modalities, improving data integration for diverse knowledge…

TabPedia: a novel large vision-language model, achieves superior visual table understanding by seamlessly integrating diverse tasks via a concept synergy mechanism and a new benchmark.

T2V-Turbo breaks the quality bottleneck of video consistency models by integrating mixed reward feedback during consistency distillation, enabling high-quality video generation with significantly fast…

Synergistic Dual Spatial-aware Generation boosts image-to-text and text-to-image accuracy using a novel 3D scene graph and dual learning framework.

A-Harmonic reward function and Reward Preference Optimization (RPO) improve subject-driven text-to-image generation by enabling faster training and state-of-the-art results with a simpler setup.

ZAF: a novel replay-free continual learning method for vision-language models, significantly reduces forgetting by stabilizing zero-shot predictions.

SpatialRGPT enhances Vision-Language Models’ spatial reasoning by integrating 3D scene graphs and depth information, achieving significant performance gains on spatial reasoning tasks.

SpatialPIN boosts vision-language models’ spatial reasoning by cleverly combining prompting techniques with 3D foundation models, achieving zero-shot performance on various spatial tasks.

SocialGPT cleverly leverages Vision Foundation Models and Large Language Models for zero-shot social relation reasoning, achieving competitive results and offering interpretable outputs via prompt opt…

SlowFocus significantly improves fine-grained temporal understanding in video LLMs by using mixed-frequency sampling and a novel multi-frequency attention mechanism.