Natural Language Processing

SAGE: Precise RAG via semantic segmentation, adaptive chunking, and LLM feedback, boosting QA accuracy & cost-efficiency.

Liger: LLMs linearized to gated recurrent models, enabling efficient deployment via key matrix repurposing and LoRA fine-tuning.

RDS+ is the unsung hero for scaling instruction tuning data selection!

Transformers get forgetful! This paper introduces the Forgetting Transformer (FoX), incorporating a forget gate into the attention mechanism for improved sequence modeling.

Shakti SLMs: Fine-tuning compact language models for efficient, domain-specific AI on edge devices.

CROWDSELECT boosts instruction tuning by cleverly selecting synthetic data using multi-LLM wisdom, enhancing model performance across diverse tasks.

CodeArena: Collective evaluation for LLM code generation.

SemViQA: A new approach to boost Vietnamese fact-checking with semantic understanding and efficient evidence retrieval.

DuoDecoding: Accelerating LLM inference by strategically deploying draft & target models on CPU & GPU for parallel decoding and dynamic drafting.

Babel: An open multilingual LLM supports over 90% of global speakers, filling the language coverage gap and setting new performance standards.

R1-T1: RL-driven framework incentivizing translation capability in LLMs via reasoning learning, achieving superior performance in multiple languages & domains.

MoR: Adaptive knowledge retrieval by fusing structural and textual data for better question answering.

LongRoPE2: Extends LLM context windows while preserving performance and reducing training costs!

NeoBERT: A new encoder that enhances bidirectional language understanding with cutting-edge architecture, data, and training, achieving SOTA results with only 250M parameters.

TokenSwift: Accelerate LLM ultra-long sequence generation up to 100K tokens with >3x speedup and lossless accuracy!

Rewriting method is critical to conversational assistant effectiveness.

CoD: LLMs think faster by writing less! A novel prompting strategy cuts costs and latency while maintaining reasoning accuracy.

GOAT: Adaptively boosts LoRA with SVD & MoE alignment, closing the gap with Full FT.

Test-time scaling isn’t a universal solve-all for multilingual math reasoning, unlike pre-training scaling, shows MCLM benchmark.

CTM: A new benchmark for assessing temporal reasoning in LLMs across Chinese dynastic history.