Large Language Models

Boosting AI instruction following, Language Feedback Models (LFMs) leverage Large Language Models (LLMs) to identify desirable behaviors from visual trajectories, significantly improving task completi…

PiSSA, a novel parameter-efficient fine-tuning method, surpasses LoRA by initializing adapter matrices using the principal components of the original model, achieving faster convergence and enhanced p…

New pipeline parallelism framework achieves up to 55% higher throughput and 50% less memory usage in large language model training by systematically controlling activation memory.

PhyloGen uses a genomic language model to generate and optimize phylogenetic trees, offering faster and more accurate evolutionary analysis than traditional methods.

Bi-directional Preference Optimization (BiPO) generates superior steering vectors for personalized LLM control, improving upon existing methods by directly influencing the generation probability of hu…

PerpCorrect: Robust LLM alignment despite noisy human preferences, achieved via perplexity-based noisy preference detection and correction.

This study reveals that large language models struggle with semi-open-ended questions, often hallucinating or providing insufficient answers. Researchers explored this by creating a new dataset of su…

PediatricsGPT: a novel Chinese pediatric LLM assistant trained on a large, high-quality dataset (PedCorpus) outperforms existing models, paving the way for improved pediatric healthcare.

PCB-MERGING: A training-free model merging technique boosts performance by intelligently balancing parameter competition across multiple tasks.

DeltaNet, a linear transformer boosting associative recall, now trains efficiently via a novel algorithm, scaling to large language models and outperforming existing linear baselines.

Panacea: a novel LLM alignment method achieving Pareto optimality via online preference adaptation using a single model.

PaCE, a novel activation engineering framework, efficiently aligns LLMs by removing undesirable concepts from activations using sparse coding, achieving state-of-the-art performance while preserving l…

Over-parameterized Distillation Framework (OPDF) boosts knowledge distillation by efficiently over-parameterizing student models via tensor decomposition, significantly improving performance without i…

Set-Based Prompting guarantees order-independent LLM outputs by modifying input representations, eliminating unwanted inconsistencies without fine-tuning.

Orchid: a novel deep learning architecture using data-dependent convolution achieves quasilinear scalability and outperforms attention-based models on various sequence modeling tasks.

LLMs boost tabular data prediction by generating optimized features via decision tree reasoning, outperforming existing methods.

Open LLMs outperform closed alternatives for private data adaptation, offering superior privacy, performance, and lower costs.

This paper proposes a novel, reward-free RLHF framework using a general preference oracle, surpassing existing reward-based approaches in efficiency and generalizability.

MAC: Efficiently updates large language models (LLMs) using a memory of compressed contexts for improved real-time knowledge retention and adaptation.

OneBit achieves surprisingly good performance in 1-bit quantized LLMs by using a novel 1-bit parameter representation method and an effective parameter initialization method.