Large Language Models

ZipCache: Efficient KV cache quantization for LLMs using salient token identification, achieving 4.98x compression with minimal accuracy loss!

Zero-Shot Tokenizer Transfer (ZeTT) detaches language models from their tokenizers via a hypernetwork, enabling efficient on-the-fly tokenizer swapping without retraining, significantly improving LLM …

YOCO: A decoder-decoder architecture for LLMs dramatically reduces memory usage and improves inference speed by caching key-value pairs only once.

XLSTM: Extended Long Short-Term Memory, introduces exponential gating and novel memory structures to overcome LSTM limitations, achieving performance comparable to state-of-the-art Transformers and St…

WorldCoder: an LLM agent builds world models via code generation and interaction, proving highly sample-efficient and enabling knowledge transfer.

WizardArena simulates offline chatbot arena battles to efficiently post-train LLMs, dramatically reducing costs and improving model performance.

WISE, a novel dual-memory architecture, solves the impossible triangle of reliability, generalization, and locality in lifelong LLM editing by employing a side memory for knowledge updates and a route…

User personas significantly impact the safety of large language models, bypassing safety filters more effectively than direct prompting methods.

LLMs learn tasks via in-context learning, but the task recognition location is unknown. This paper reveals that LLMs transition from task recognition to task performance at specific layers, enabling s…

RLbreaker uses deep reinforcement learning to efficiently create highly effective jailbreaking prompts, outperforming existing methods against multiple state-of-the-art LLMs and defenses.

LLM fine-tuning made easy! This paper reveals how analyzing weight vector angles in RoPE positional embeddings helps optimize LLMs, reducing parameter count and improving efficiency.

Safety fine-tuning for LLMs is shown to minimally transform weights, clustering inputs based on safety, but is easily bypassed by adversarial attacks.

Align LLMs efficiently via test-time search using smaller models!

SPLAT, a new benchmark using situation puzzles, effectively evaluates and elicits lateral thinking in LLMs through a multi-turn player-judge framework, revealing significant performance improvements o…

WaterMax: a novel LLM watermarking scheme achieving high detectability and preserving text quality by cleverly generating multiple texts and selecting the most suitable one.

LLM watermarking leaves detectable traces in subsequently trained models, enabling detection of synthetic data usage—a phenomenon termed ‘radioactivity’.

LLM-based agents are vulnerable to diverse backdoor attacks that manipulate their reasoning and outputs, highlighting the urgent need for targeted defenses.

WAGLE: A novel weight attribution-guided LLM unlearning framework boosts unlearning performance by strategically identifying and manipulating influential model weights, achieving a better balance betw…

LLMLIFT: An LLM-powered approach builds verified lifting tools for DSLs, outperforming prior symbolic methods in benchmark transpilation and requiring less development effort.

VeLoRA: Train massive LLMs efficiently by compressing intermediate activations!