Large Language Models

This paper presents a novel method for large language model (LLM) unlearning, enabling LLMs to ‘forget’ undesirable behaviors by using only negative examples. This computationally efficient approach o…

Large language models (LLMs) achieve lossless gradient compression, surpassing existing methods by up to 17.2%, thereby advancing distributed learning efficiency.

Language models struggle with hierarchical information. This work introduces Hierarchy Transformer Encoders (HITs), a novel method to retrain transformer encoders using hyperbolic geometry and special…

LACIE: Listener-aware finetuning improves LLM confidence calibration, reducing incorrect answers accepted by human listeners by 47% while maintaining correct answer acceptance.

KVQuant achieves <0.1 perplexity degradation with 3-bit quantization in LLMs by using per-channel key quantization, pre-RoPE quantization, and non-uniform quantization, enabling 10M context length inf…

Boost LLM inference speed 1.4-3.5x by using Coupled Quantization (CQ) to compress KV cache down to 1 bit per channel, while preserving model accuracy.

Kraken: A new Transformer architecture boosts multi-device inference speed by 35.6% by cleverly overlapping communication with computation.

KptLLM: A novel multimodal model leverages LLMs for superior keypoint comprehension, outperforming existing methods in various benchmarks.

Researchers unveil ‘knowledge circuits’ within LLMs, revealing how knowledge is collaboratively encoded and utilized, leading to improved LLM design and interpretations of model behavior.

KG-FIT boosts knowledge graph embedding by smartly integrating open-world knowledge from LLMs, achieving significant performance gains.

Kangaroo: Double early exiting boosts LLM speed!

JiuZhang3.0 efficiently enhances LLMs’ mathematical reasoning by training a small model to synthesize high-quality training data, drastically reducing costs.

New benchmark and jailbreak method exposes vulnerabilities of LLM moderation, achieving significantly higher success rates than existing methods.

Iterative Reasoning Preference Optimization boosts large language model reasoning by iteratively refining preferences between generated reasoning steps, achieving significant accuracy gains on benchma…

Researchers reveal how Chain-of-Thought reasoning emerges in transformers via specialized ‘iteration heads’, improving LLM performance and offering insights into mechanistic interpretability.

Large Language Models (LLMs) surprisingly improve the challenging task of Programming by Example (PBE) when fine-tuned on problem-specific data, outperforming classic symbolic methods and even surpass…

IRCAN tackles LLM knowledge conflicts by identifying and reweighting context-aware neurons, significantly improving context-sensitive outputs.

InversionView unveils neural network inner workings by decoding information from activations. It identifies inputs producing similar activations, revealing the information content. Case studies on v…

Mat2Seq revolutionizes crystal structure generation using language models by creating unique, invariant 1D sequences from 3D crystal structures, enabling accurate and efficient crystal discovery with …

Researchers developed methods to measure and interpret the divergence between learned feedback patterns (LFPs) in LLMs and human preferences, helping minimize discrepancies between LLM behavior and tr…