Large Language Models

FILM-7B, trained with Information-Intensive (IN2) training, significantly overcomes the ’lost-in-the-middle’ problem in long-context LLMs, enabling robust information retrieval from all context positi…

MagR: a novel preprocessing technique boosts post-training quantization of LLMs by reducing weight magnitudes without inference overhead, achieving state-of-the-art performance.

MAGNET, a novel adaptive gradient-based tokenization method, tackles multilingual language model bias by employing language-specific boundary predictors to achieve equitable segmentation across divers…

MAGIS: A novel LLM-based multi-agent framework significantly boosts GitHub issue resolution by leveraging agent collaboration for planning and coding, achieving an eight-fold performance increase comp…

Multi-Agent System for Condition Mining (MACM) dramatically boosts large language model accuracy in complex math problem-solving, exceeding existing methods by achieving higher accuracy and better gen…

LT-Defense: a searching-free backdoor defense for language models leveraging the long-tailed effect of poisoned data. It achieves 98% accuracy across 1440 models with less than 1% time cost of existin…

LSH-MoE accelerates Mixture-of-Experts training by 1.28x-2.2x via Locality-Sensitive Hashing, significantly reducing communication costs.

LoRA-GA: A novel initialization method dramatically speeds up low-rank adaptation (LoRA) for LLMs, achieving convergence rates comparable to full fine-tuning while improving performance.

LoQT enables efficient large language model training on consumer hardware via quantized weights and low-rank weight updates, overcoming memory limitations.

LLMs often generate factually inaccurate long-form text. This work introduces LongFact, a new benchmark dataset of 2280 fact-seeking prompts, and SAFE, a novel automated evaluation method that outperf…

Loki: Low-rank Keys for Efficient Sparse Attention accelerates attention mechanisms in LLMs by exploiting the low-dimensionality of key vectors. It dynamically selects key tokens based on approximate…

LOFIT: Localized fine-tuning boosts LLMs’ performance by selectively training only a small subset of attention heads, achieving comparable accuracy to other methods while using significantly fewer par…

Localized Zeroth-Order Prompt Optimization (ZOPO) efficiently finds high-performing local optima for prompt optimization in black-box LLMs, outperforming existing global optimization methods.

TEA-GLM leverages LLMs for zero-shot graph learning by aligning GNN representations with LLM token embeddings, achieving state-of-the-art performance on unseen datasets and tasks.

LLMDFA: A novel LLM-powered framework performs compilation-free and customizable dataflow analysis, achieving high accuracy in bug detection by decomposing the task into sub-problems and mitigating L…

LLM-Check efficiently detects LLM hallucinations in a single response, using internal model analysis, improving real-time applications.

LLM Processes leverage LLMs to create probabilistic regression models guided by natural language, enabling seamless integration of expert knowledge and improving prediction accuracy.

LLMs show self-preference bias in evaluations, favoring their own outputs. This study reveals that LLMs surprisingly recognize their own generations, and this self-recognition directly causes the self…

LLM dataset inference reliably detects if a dataset was used in training, overcoming limitations of existing membership inference attacks.

LLM circuit analyses remain consistent across model scales and extensive training, enabling more efficient interpretability research.