Natural Language Processing

Predicting emergent LLM capabilities is now possible by finetuning smaller models; this approach shifts the emergence point, enabling accurate predictions of future model performance, even with up to …

Simple distillation from OpenAI’s API, combined with fine-tuning, surprisingly surpasses OpenAI’s O1-preview on complex mathematical reasoning, urging transparency in AI research.

MH-MoE: A novel implementation of Multi-Head Mixture-of-Experts achieves superior performance in large language models by enhancing efficiency without sacrificing model size or computational cost.

LLMs are revolutionizing AI evaluation by offering nuanced judgments surpassing traditional methods. This paper provides a taxonomy, benchmark, and future directions for LLM-as-a-judge.

A novel LLM-based transpiler, CRT, efficiently converts x86 assembly to ARM and RISC-V assembly, achieving high accuracy and significant performance improvements over existing virtualization methods.

AI-powered robots now understand and execute complex natural language commands, adapting seamlessly to dynamic environments thanks to a new architecture integrating LLMs, perception, and planning.

MolReFlect achieves state-of-the-art molecule-text alignment by using a teacher-student LLM framework that generates fine-grained alignments, improving accuracy and explainability.

UnifiedCrawl efficiently harvests massive monolingual datasets for low-resource languages from Common Crawl, enabling affordable LLM adaptation via QLoRA, significantly improving performance.

Marco-01: a novel large reasoning model surpasses existing LLMs by using Chain-of-Thought, Monte Carlo Tree Search, and reflection mechanisms to excel in open-ended problem-solving, particularly in co…

LLMs’ hallucinations stem from entity recognition: SAEs reveal model ‘self-knowledge’, causally affecting whether it hallucinates or refuses to answer. This mechanism is even repurposed by chat finet…

AnchorAttention enhances long-context LLMs by mitigating BFloat16’s disruptive effects on RoPE, improving performance and speeding up training.

Boosting Large Language Model (LLM) reasoning without massive datasets: A novel training method encourages ‘patient’ reasoning, improving accuracy by up to 6.7% on benchmark tasks.

ORID framework leverages organ-regional information to boost radiology report generation, achieving state-of-the-art accuracy by integrating multi-modal data and reducing noise from unrelated organs.

Hymba: Hybrid-head architecture boosts small language model performance by 11.67x cache size reduction and 3.49x throughput, surpassing existing models.

BALROG benchmark rigorously evaluates LLMs’/VLMs’ abilities in complex games, revealing their strengths and weaknesses in long-term planning and decision-making, highlighting the need for improved vis…

New data-free methodology creates effective, generalizable LLMs guardrails against off-topic prompts, significantly improving LLM safety and responsible use.

UltraMem, a novel ultra-sparse memory network, drastically speeds up LLM inference by 6x compared to MoE while maintaining performance, paving the way for efficient large-scale model deployment.

RedPajama, two massive open-source datasets, are released for training LLMs, improving transparency and facilitating the development of high-performing open-source models.

SUTRA tokenizer outperforms other LLMs in Indian languages, improving efficiency and facilitating better model performance.

Verifier engineering: A new post-training paradigm for foundation models using automated verifiers to provide effective supervision signals, enhancing capabilities beyond traditional data-centric meth…