Large Language Models

FlashAttention-3: Achieves 1.5-2x faster attention on H100 GPUs using asynchrony and low-precision, reaching 1.3 PFLOPs/s.

FLAME: A novel alignment method enhances large language model factuality by addressing hallucination in supervised fine-tuning and reinforcement learning, resulting in more accurate and helpful AI ass…

Researchers crack the code of in-context learning in Transformers, revealing how architecture, low-rank parameters, and data correlations influence model optimization and generalization.

Prompt Adversarial Tuning (PAT) defends against LLM jailbreaking by training a protective prompt prefix. PAT uses adversarial and benign prompts to optimize this prefix, significantly reducing succes…

FlexLoRA: Efficient Federated Fine-tuning of LLMs for Heterogeneous Tasks and Resources.

Speculative Rejection: A novel algorithm boosts Large Language Model (LLM) alignment by speeding up inference-time alignment by 16-32x!

LLM-Infused Diffuser boosts text-to-image generation by smartly integrating LLMs, surpassing existing models in prompt understanding and image quality.

Researchers extended large language models’ context windows by training-free methods via analyzing and manipulating positional vectors, improving long-text processing.

LLM quantization, while improving efficiency, creates a security risk: attackers can craft seemingly benign models that exhibit malicious behavior only when quantized.

D2DMoE boosts Transformer efficiency by up to 60% via smart activation sparsity and dynamic expert selection, outperforming existing methods.

This paper introduces a model-agnostic algorithm that uses natural language predicates to make statistical model parameters directly interpretable, significantly improving explainability.

New metrics reveal that generative models often possess surprisingly incoherent world models, despite seemingly accurate next-token predictions. This incoherence leads to fragility in solving related …

New method estimates hallucination rates in generative AI’s in-context learning, improving model reliability.

ESPACE: A novel LLM compression technique achieving 50% model size reduction with minimal accuracy loss by cleverly projecting activations onto principal components.

LLM4EA: A novel framework efficiently merges knowledge graphs using LLMs, overcoming noisy annotations and high costs via active learning and unsupervised label refinement, boosting accuracy and effic…

DEEPEN: a training-free LLM ensemble framework fusing probability distributions in a relative space to overcome vocabulary misalignment, improving performance consistently across benchmarks.

Boosting AI reasoning! New research enhances LLMs’ logical abilities via a principled synthetic logic corpus, achieving substantial improvements across logic, math, and coding benchmarks.

Boosting LLM trustworthiness, researchers introduce Sparse Activation Control, a training-free method that concurrently enhances safety, factuality, and bias mitigation by selectively controlling atte…

LLMs struggle with complex, long-form text. This paper introduces ‘context structurization,’ transforming unstructured text into a structured format to enhance LLM comprehension. Experiments across …

Adaptive tokenizers enhance LLMs by dynamically optimizing vocabulary during training, improving accuracy without increasing vocabulary size.