Spotlight Large Language Models

Researchers predict language model performance by observing existing models, bypassing costly training, revealing surprising predictability in complex scaling phenomena.

Bayesian Optimization Model Fusion (BOMF) significantly boosts language model fine-tuning by optimizing both loss and metrics through multi-objective Bayesian optimization, yielding considerable perfo…

Researchers taught a large language model (LLM) a three-word ‘Knowledge Graph Language’ (KGL) to improve knowledge graph (KG) completion, drastically reducing errors compared to other methods.

MInference 1.0 accelerates LLM pre-filling via dynamic sparse attention, achieving up to 10x speedup on an A100 GPU while maintaining accuracy.

MaskLLM learns efficient semi-structured sparsity in LLMs via end-to-end training, achieving significant speedup and memory reduction without sacrificing performance.

Scaling up in-context learning using thousands of examples significantly boosts Large Language Model (LLM) performance, particularly for complex tasks. Novel training methods mitigate reliance on hum…

Localized Zeroth-Order Prompt Optimization (ZOPO) efficiently finds high-performing local optima for prompt optimization in black-box LLMs, outperforming existing global optimization methods.

Learn-To-be-Efficient (LTE) trains LLMs to achieve structured sparsity, boosting inference speed by 25% at 50% sparsity without sacrificing accuracy.

Restricted models often outperform full-featured models when training data is limited. This paper introduces Induced Model Matching (IMM), a novel technique that uses a restricted model as a guide to…

Adam’s superior performance on language models stems from its resilience to heavy-tailed class imbalance, unlike SGD, which struggles with infrequent word losses.

HaloScope leverages unlabeled LLM outputs to accurately detect AI hallucinations without human annotation, significantly outperforming existing methods.

GREATS: a novel online batch selection method significantly speeds up LLM training by greedily selecting high-quality data batches in every iteration, improving both convergence and generalization per…

Graph Uncertainty boosts LLM factuality by 6.8% using graph centrality to estimate claim-level uncertainty and a novel uncertainty-aware decoding process.

LLM-powered prototype refinement boosts few-shot 3D point cloud segmentation accuracy.

FlashAttention-3: Achieves 1.5-2x faster attention on H100 GPUs using asynchrony and low-precision, reaching 1.3 PFLOPs/s.

Researchers extended large language models’ context windows by training-free methods via analyzing and manipulating positional vectors, improving long-text processing.

New metrics reveal that generative models often possess surprisingly incoherent world models, despite seemingly accurate next-token predictions. This incoherence leads to fragility in solving related …

DEEPEN: a training-free LLM ensemble framework fusing probability distributions in a relative space to overcome vocabulary misalignment, improving performance consistently across benchmarks.

Boosting LLM robustness against attacks efficiently: Continuous adversarial training in embedding space outperforms discrete methods, achieving improved robustness with less computation.

Discrete Flow Matching (DFM) revolutionizes discrete data generation by introducing a novel flow paradigm that surpasses existing methods. DFM leverages flexible probability paths, enabling efficient …