Natural Language Processing

ARWKV: A novel RNN-attention-based language model, distilled from a larger model, achieves strong performance using significantly fewer resources, opening a new path in efficient language model develo…

RealCritic: A new benchmark effectively evaluates language models’ critique abilities using a closed-loop methodology, showcasing advanced reasoning models’ superiority in self and iterative critique.

Humanity’s Last Exam (HLE): a groundbreaking multi-modal benchmark pushing the boundaries of large language model (LLM) capabilities, revealing a significant gap between current LLMs and human experts…

CoRAG, a novel Chain-of-Retrieval Augmented Generation model, dynamically refines queries for improved accuracy in multi-hop question answering, achieving state-of-the-art performance.

SIGMA, a novel large language model, achieves up to 33.36% faster inference speeds by using DiffQKV attention, which differentially optimizes query, key, and value components in the attention mech…

Low-rank adapters combined with neural architecture search revolutionize LLM compression, enabling efficient fine-tuning and significantly reduced memory footprint.

Large language models (LLMs) are rapidly evolving, yet often struggle to adapt to human preferences quickly. This paper introduces Test-Time Preference Optimization (TPO), an innovative framework that…

Pairwise RM, a novel reward model with knockout tournaments, significantly boosts large language model accuracy in test-time scaling by comparing solution pairs, eliminating arbitrary scoring inconsis…

O1-Pruner efficiently prunes long-thought reasoning in LLMs by harmonizing reasoning length and accuracy via fine-tuning, significantly reducing inference time without sacrificing performance.

Kimi K1.5: A Multimodal LLM trained with RL achieves state-of-the-art reasoning by scaling long context RL training and improving policy optimization.

DeepSeek-R1 significantly improves LLM reasoning by using reinforcement learning, achieving performance comparable to OpenAI’s top models while addressing previous challenges of poor readability and l…

Revolutionizing large language models, Autonomy-of-Experts (AoE) empowers individual expert modules to autonomously select inputs, eliminating routers and boosting both efficiency and accuracy.

MMVU: a new benchmark pushes multimodal video understanding to expert level, revealing limitations of current models and paving the way for more advanced AI.

Debate-enhanced weak supervision boosts AI alignment by combining strong and weak models, enabling safer and more reliable AI systems.

This research proposes principles and a framework to tackle redundancy in MLLM benchmarks, enhancing efficiency and guiding future development.

Democratizing advanced reasoning in AI, this blueprint introduces a modular framework for building Reasoning Language Models (RLMs), simplifying development and enhancing accessibility.

Mobile-Agent-E: A self-evolving mobile assistant conquering complex tasks with hierarchical agents and a novel self-evolution module, significantly outperforming prior approaches.

Agent-R: A novel self-training framework enables language model agents to learn from errors by dynamically constructing training data that corrects erroneous actions, resulting in significantly improv…

IntellAgent: a novel open-source framework automating diverse conversational AI evaluation via policy-driven graph modeling, event generation, and user-agent simulations, enabling fine-grained diagnos…

STEP-KTO: A novel training framework boosts LLMs’ mathematical reasoning by providing binary feedback on both intermediate steps and final answers. This ensures logical reasoning trajectories and impr…