Large Language Models

DenseFormer enhances transformers by adding a depth-weighted averaging step, improving data efficiency and outperforming baselines in memory and inference time without increasing model size.

Large language models struggle to generalize knowledge when facing seemingly simple reversals, a phenomenon termed the ‘reversal curse.’ This study reveals that this limitation is strongly linked to t…

Delta-CoMe: Training-free mixed-precision delta compression boosts LLM deployment efficiency.

BAL-PM, a novel active learning approach, drastically reduces human feedback in LLM preference modeling by leveraging both model uncertainty and prompt distribution diversity, achieving 33%-68% fewer …

Multi-objective decoding (MOD) efficiently aligns language models to diverse user needs by decoding the next token from a weighted combination of predictions from multiple base models trained on indiv…

New framework reveals LLMs’ human-like decision-making tendencies but highlights significant variations and biases influenced by demographic factors, underscoring ethical deployment needs.

DDK: Dynamically Distilling Domain Knowledge for efficient LLMs.

This paper introduces dataset decomposition (DD), a novel approach to accelerate LLM training while enhancing performance. DD significantly reduces training time by decomposing datasets into buckets …

ISED: a novel, data-efficient algorithm learns neural programs by sampling from neural predictions to estimate gradients of black-box components, outperforming baselines on various benchmarks.

Researchers uncover hidden training data secrets of large language models by analyzing their byte-pair encoding tokenizers, revealing the proportions of different languages and domains.

DART-Math tackles LLM limitations in mathematical problem-solving by introducing Difficulty-Aware Rejection Tuning, a novel method that generates high-quality, bias-reduced datasets, resulting in supe…

DARG dynamically evaluates LLMs via adaptive reasoning graphs, revealing performance drops with increased complexity and exposing model biases.

DAPE: A novel data-adaptive positional encoding method dynamically adjusts positional information based on input context, improving transformer performance and length generalization.

DAGER: Exact gradient inversion for LLMs; recovers full input text batches precisely.

D-LLM dynamically allocates computing resources during LLM token processing, reducing computational costs and memory usage by up to 50% without sacrificing accuracy.

New D-CPT Law optimizes continual pre-training for LLMs by predicting optimal data mixture ratios, drastically cutting training costs.

Instance-wise LoRA (iLoRA) boosts LLM sequential recommendation accuracy by customizing model parameters for each user, mitigating negative transfer and improving performance.

CulturePark, a novel multi-agent communication framework, generates high-quality cross-cultural data to fine-tune LLMs, significantly reducing cultural bias and boosting cross-cultural understanding.

CultureLLM, a new approach, effectively incorporates cultural nuances into LLMs using semantic data augmentation, significantly outperforming existing models.

One-time training improves multiple LLMs using a tiny portable model, drastically reducing costs and resource needs for model enhancement.