Spotlight Large Language Models

XLSTM: Extended Long Short-Term Memory, introduces exponential gating and novel memory structures to overcome LSTM limitations, achieving performance comparable to state-of-the-art Transformers and St…

User personas significantly impact the safety of large language models, bypassing safety filters more effectively than direct prompting methods.

LLM watermarking leaves detectable traces in subsequently trained models, enabling detection of synthetic data usage—a phenomenon termed ‘radioactivity’.

Unlocking tight generalization bounds for massive LLMs using a novel token-level approach.

Compute-optimal protein language models are trained efficiently using scaling laws derived from a massive dataset, improving performance while optimizing compute budgets.

Moderation Using LLM Introspection (MULI) leverages the first response token’s logits from LLMs to create a highly accurate toxicity detector, surpassing state-of-the-art methods with minimal overhead…

TOPA: Extending LLMs for video understanding using only text data.

Time-reversed language models provide unsupervised feedback for improving LLMs, offering a cost-effective alternative to human feedback and enhancing LLM safety.

Stacking Your Transformers accelerates LLM pre-training by leveraging smaller, pre-trained models to efficiently train larger ones, achieving significant speedups and improved performance.

SEQUOIA: A novel algorithm boosts Large Language Model (LLM) inference speed by up to 9.5x using a scalable and robust speculative decoding approach!

Certified selective generation controls language model hallucinations by leveraging textual entailment and a novel semi-supervised algorithm, guaranteeing a controlled false discovery rate.

Revolutionizing LLM training: Constant learning rate with cooldown replaces cosine schedule, enabling cost-effective scaling experiments!

LLMs struggle with out-of-distribution (OOD) generalization. This research introduces ‘rule extrapolation’ using formal languages to rigorously evaluate OOD behavior in various LLM architectures, rev…

Robust Prompt Optimization (RPO) creates robust LLM defenses against jailbreaking attacks by optimizing a transferable suffix, achieving state-of-the-art robustness.

New research resolves discrepancies in language model scaling laws, revealing three key factors driving the differences and improving accuracy in predicting optimal model size based on compute budget.

Boost LLM reliability by adding redundancy! This paper uses a communication theory framework to show that generating multiple LLM outputs and reranking them significantly reduces errors, even with imp…

ReFT: Revolutionizing language model finetuning by directly manipulating hidden representations, achieving superior efficiency and performance compared to existing methods.

QTIP: Ultra-high dimensional LLM quantization using trellis codes for faster, higher-quality inference.

PiSSA, a novel parameter-efficient fine-tuning method, surpasses LoRA by initializing adapter matrices using the principal components of the original model, achieving faster convergence and enhanced p…

One-shot dualization aligns large language models with safety constraints efficiently, eliminating iterative primal-dual methods for improved stability and reduced computational burden.