Natural Language Processing

SHED, a Shapley value-based framework, efficiently refines instruction-tuning datasets for LLMs, producing high-performing subsets, only 10% of original size, that transfer well across different model…

SGLang: A new system boosts LLM program execution speed by up to 6.4x, simplifying complex LLM application programming.

Transformers and RNNs show contrasting representational capabilities: Transformers excel at tasks requiring associative recall, while RNNs are better suited for hierarchical language processing. This …

SEMCODER: A novel 6.7B parameter code LLM surpasses GPT-3.5-turbo’s performance on code generation and execution reasoning by employing ‘monologue reasoning’—training the model to verbally explain cod…

Emergent communication protocols are surprisingly inconsistent; this paper proves reconstruction-based objectives yield semantically consistent protocols, unlike discrimination-based ones, highlightin…

Learning-based semantic routing, a scalable approach to route planning using rich user queries, is introduced, accompanied by a large-scale public benchmark and a proof-of-concept model demonstrating …

SelfCodeAlign is a novel self-alignment method for code generation LLMs that surpasses existing methods by avoiding reliance on expensive human annotation or proprietary LLMs. The method achieves thi…

STAR, a novel unsupervised adaptation framework, drastically improves automatic speech recognition (ASR) robustness across diverse domains using only unlabeled data and outperforms existing self-train…

SAMI: Self-Supervised Alignment with Mutual Information, effectively teaches language models to follow principles without human preference labels by maximizing the mutual information between principle…

Self-Retrieval revolutionizes information retrieval by unifying indexing, retrieval, and reranking within a single large language model, achieving significantly improved performance.

Self-play adversarial language game boosts LLM reasoning!

LLMs excel at reasoning tasks, but their application to combinatorial problems (CPs) is underexplored. This paper introduces Self-Guiding Exploration (SGE), a novel prompting strategy that significan…

LLMs self-discover optimal reasoning structures for complex problems, boosting performance by up to 32% compared to existing methods.

Enhance Transformer models via Selective Self-Attention (SSA), a principled context control method that boosts accuracy and efficiency.

SelectIT leverages LLMs’ intrinsic uncertainty to efficiently select high-quality instruction tuning data, enhancing model performance without extra resources.

This paper presents novel statistical methods to reliably watermark and segment LLMs-generated text, ensuring source traceability even after user modifications.

Training-free architecture search finds optimal subnets in LLMs, boosting inference speed and slashing memory needs without retraining.

SDP4Bit achieves up to 4.08x speedup in LLM training by quantizing weight differences and gradients to ~4 bits, maintaining accuracy.

Researchers dramatically improved sign language translation by scaling up data, model size, and the number of languages, achieving state-of-the-art results.

Massive language models improve with bigger datastores at inference time. A 1.4 trillion-token datastore, MASSIVEDS, shows that retrieval-based LMs outperform larger, solely-trained models on knowled…