Large Language Models

One-shot dualization aligns large language models with safety constraints efficiently, eliminating iterative primal-dual methods for improved stability and reduced computational burden.

VideoLISA: A video-based multimodal large language model enabling precise, language-instructed video object segmentation with superior performance.

This research introduces Cluster-guided Sparse Experts (CSE), enabling pretrained language models to effectively learn long-tail domain knowledge without domain-specific pretraining, thus achieving su…

LLMs’ performance drastically varies depending on prompt phrasing; this paper introduces ROBUSTAL-PACAEVAL to evaluate lower-bound performance via worst-case prompt analysis, revealing model inconsist…

Auto-Regressive Decision Trees (ARDTs) surprisingly outperform Transformers on language tasks!

MIDAS: A novel training method improves language model reasoning by efficiently stacking middle layers, surprisingly boosting downstream task performance without increasing pretraining perplexity.

Softmax-DPO boosts LM-based recommender performance by directly optimizing for personalized ranking using a novel loss function that incorporates multiple negative samples, significantly outperforming…

Weak LLMs can accurately supervise strong LLMs via debate, outperforming simpler consultancy methods, especially in information-asymmetric tasks.

Effortlessly boost large language model performance by dynamically fusing knowledge from smaller, task-specific models – achieving near full fine-tuning results with minimal computational cost!

OccamLLM: LLMs now perform accurate arithmetic in a single step!

Researchers predict language model performance by observing existing models, bypassing costly training, revealing surprising predictability in complex scaling phenomena.

RHO-1, a novel language model, uses selective pretraining focusing on high-value tokens, achieving state-of-the-art results with significantly less data than existing models.

By cleverly integrating per-example gradient norm calculations during the backward pass of LayerNorm layers, this research enables efficient and accurate gradient noise scale estimation in Transformer…

NoMAD-Attention achieves up to 2x speedup in 4-bit quantized LLaMA inference on CPUs by replacing computationally expensive multiply-add operations with ultra-low-latency in-register lookups.

NoiseGPT uses multi-modal LLMs to detect & fix noisy image labels by identifying probability curvature differences between clean and noisy examples.

This paper introduces InfoNCA and NCA, novel frameworks for language model alignment using noise contrastive estimation, enabling direct optimization from both explicit rewards and pairwise preference…

LLM watermarking faces inherent trade-offs; this paper reveals simple attacks exploiting common design choices, proposing guidelines and defenses for more secure systems.

Neuro-symbolic framework generates high-quality mathematical datasets, enhancing LLMs’ mathematical reasoning capabilities and surpassing state-of-the-art counterparts.

NEST, a novel semi-parametric language model, significantly boosts LLM generation quality, provides accurate source attribution, and achieves a 1.8x speedup in inference time by cleverly incorporating…

Researchers discover ‘safety basins’ in LLMs, proposing a new metric (VISAGE) to quantify finetuning risks and visualize how these basins protect against safety compromise during model training.