Natural Language Processing

MAGA reformulates existing corpora to massively expand LLM pretraining data, boosting performance across various model sizes while maintaining quality.

Llasa, a novel single-Transformer TTS model, achieves state-of-the-art performance by scaling both training and inference compute, improving naturalness, prosody and emotional expressiveness.

Language models surprisingly exhibit linear relationships when composing knowledge; this linearity, resilient to fine-tuning, predicts compositional generalization and hallucination.

CMOE efficiently transforms dense LLMs into sparse MoE architectures via expert carving, enabling fast inference without extensive retraining.

BOLT bootstraps Long Chain-of-Thought reasoning in LLMs without distillation, achieving impressive results across various benchmarks.

Researchers jointly optimize prompt content and format to significantly boost Large Language Model (LLM) performance.

Boosting language model reasoning: A novel hybrid approach using latent tokens drastically shortens reasoning traces, improving model performance and efficiency.

LLMs learn to critique and refine their output via reinforcement learning, significantly improving code generation.

LIMO: Few examples unlock complex reasoning in LLMs, challenging assumptions about data-hungry models and achieving state-of-the-art results with minimal training.

AlphaGeometry2 surpasses average IMO gold medalists in solving geometry problems!

DreamDPO: Revolutionizing text-to-3D generation by directly aligning outputs with human preferences via innovative preference optimization.

Researchers unveil a data-free method to visualize and control feature flow in LLMs, enhancing interpretability and enabling targeted model steering.

Satori: A novel 7B LLM achieves state-of-the-art mathematical reasoning via autoregressive search.

QLASS boosts language agent inference by using Q-values to guide a stepwise search, improving efficiency and performance even with limited data.

Language model distillation suffers from ’teacher hacking’, where student models over-optimize flawed teacher models, degrading true performance. This paper identifies this issue and offers effective…

LLMs struggle with complex logical reasoning; ZebraLogic benchmark reveals a ‘curse of complexity’, highlighting inherent limitations and guiding future research.

Direct alignment algorithms are a blur, but this paper shows how a simple SFT phase and a scaling parameter significantly improve alignment quality, regardless of the specific reward function used.

PRIME (Process Reinforcement through IMplicit rEwards) revolutionizes LLM training by efficiently using implicit process rewards from online policy rollouts and outcome labels, significantly boosting …

PlotGen: A novel multi-agent LLM framework automates accurate scientific data visualization via multimodal feedback, boosting novice productivity and improving visualization accuracy.

New benchmark challenges LLMs with general knowledge puzzles, revealing reasoning gaps and suggesting improvements for future models.