Natural Language Processing

Sketch-of-Thought(SoT) reduces LLM token usage by up to 76% while maintaining (or improving) accuracy via cognitive-inspired sketching.

R1-Searcher: RL enhances LLMs by incentivizing autonomous search, outperforming RAG methods, even GPT-4o-mini!

Linear-MoE: Integrates Linear Sequence Modeling with Mixture-of-Experts, achieving efficiency gains and competitive performance in large language models.

TinyR1-32B-Preview: A novel branch-merge distillation approach that significantly enhances model accuracy and reduces computational costs for LLMs.

Time to focus on LLM’s long-form outputs! This paper advocates for research on generating high-quality, long, and coherent text.

More documents can hurt RAG performance, even with same length!

LLMs show translationese due to supervised training biases. Polishing references and filtering unnatural instances can mitigate this issue.

IFIR: a new benchmark for instruction-following retrieval in expert domains, revealing current model limitations.

FuseChat-3.0: Heterogeneous model fusion boosts LLM performance via preference optimization, creating efficient and powerful language models.

Task-aware KV cache compression enables efficient knowledge reasoning in LLMs.

This paper explores and improves R1-like reasoning models through RL and tool manipulation, achieving significant accuracy gains.

Process-based Self-Rewarding advances LLMs, surpassing human reasoning in math by step-wise self-evaluation.

LLMs modestly affect Wikipedia, subtly altering content and potentially skewing NLP benchmarks.

QE4PE: Word-level QE’s impact on MT post-editing with 42 pro-editors across English-Italian/Dutch is investigated. Usability&accuracy challenges in professional workflows are underlined.

Mask-DPO: Fine-grained Factuality Alignment improves LLMs’ factuality by masking sentence-level errors during DPO training for enhanced knowledge alignment.

LINGOLY-TOO: A new benchmark to disentangle memorization from reasoning in LLMs using linguistic templatization and orthographic obfuscation.

IVO: Iterative Value Function Optimization for Guided Decoding

LLMs primarily rely on word form, unlike humans, when reconstructing semantics, indicating a need for context-aware mechanisms to enhance LLMs’ adaptability.

This paper investigates when LLMs are apprehensive and when their uncertainty is justified.

SampleMix: Sample-wise Pre-training Data Mixing by Coordinating Data Quality and Diversity