Large Language Models

VB-LoRA achieves extreme parameter efficiency in fine-tuning LLMs by sharing parameters globally via a vector bank, outperforming state-of-the-art PEFT methods while maintaining comparable or better p…

Vaccine: a novel technique safeguards LLMs against harmful fine-tuning attacks by creating invariant hidden embeddings.

UQE: A novel query engine uses LLMs for efficient and accurate unstructured data analytics, surpassing existing methods.

SalientLoRA unveils optimal LoRA ranks by analyzing rank salience via time-series analysis, improving fine-tuning efficiency and performance significantly.

Transformers learn complex tasks surprisingly well through in-context learning, but the mechanism remains unclear. This paper proves that a two-layer transformer trained on n-gram Markov chain data co…

LLMs struggle with genuine causal reasoning; new benchmark CausalProbe-2024 reveals limitations, and G2-Reasoner method improves causal reasoning by integrating general knowledge and goal-oriented pro…

This study disentangles best practices for learning from preference feedback in LLMs, revealing that data quality, algorithm choice, and reward model significantly impact performance.

Unlocking tight generalization bounds for massive LLMs using a novel token-level approach.

Reasoning Boundary Framework (RBF) quantitatively assesses and optimizes chain-of-thought (CoT) in LLMs, offering novel metrics and optimization strategies validated across various models and tasks.

Unlocking intermediate layers in MLLMs improves referring expression generation by enhancing accuracy and detail while reducing hallucinations.

Fully recurrent neural networks can be universal in-context approximators, achieving the same capabilities as transformer models by cleverly using prompts.

UniBias unveils and mitigates LLM bias by identifying and eliminating biased internal components (FFN vectors and attention heads), significantly improving in-context learning performance and robustne…

LLMs’ inner workings remain elusive. This study uses N-gram statistics to approximate transformer predictions, revealing how LLMs learn from simple to complex statistical rules, and how model variance…

Unifying framework reveals hidden connections between attention, recurrent, and state-space models, boosting foundation model efficiency.

Deep learning scaling laws are explained by novel approximation and estimation theories for transformers on low-dimensional data, resolving discrepancies between theory and practice.

Linear probing then fine-tuning (LP-FT) significantly improves language model fine-tuning; this paper uses Neural Tangent Kernel (NTK) theory to explain why.

Researchers unveil how multi-modal LLMs process information, revealing that early layers are key for storage, and introduce MULTEDIT, a model-editing algorithm for correcting errors and inserting new …

Language model emergent abilities aren’t exclusive to large models; they emerge when pre-training loss falls below a threshold, irrespective of model or data size.

New methods minimize outlier features in transformer training, improving quantization and efficiency without sacrificing convergence speed.

Researchers developed SCAV, a novel framework to effectively reveal safety risks in LLMs by accurately interpreting their safety mechanisms. SCAV-guided attacks significantly improve attack success r…