Natural Language Processing

IBProtector shields LLMs from harmful outputs via prompt compression, selectively preserving essential information using a trainable extractor.

LoPA: a novel parameter-efficient fine-tuning method matches state-of-the-art performance while requiring no server-side adapters, improving upon traditional prompt tuning.

LLMs’ in-context learning, though effective, exhibits surprisingly irregular decision boundaries, hindering generalization; this paper reveals this issue and proposes methods to improve smoothness via…

ChatGPT amplifies gender bias in job applications, revealing AI’s potential to worsen labor market inequality.

Pretrained transformers surprisingly learn low-dimensional nonlinear functions efficiently from few in-context examples, outperforming baseline algorithms.

Despite common belief, state-of-the-art preference learning algorithms for LLMs achieve surprisingly low ranking accuracy, highlighting significant flaws in current alignment techniques.

PCformer boosts Transformer performance by using a predictor-corrector learning framework and exponential moving average coefficient learning for high-order prediction, achieving state-of-the-art resu…

This paper presents a novel statistical framework for ranking LLMs using pairwise comparisons, accounting for the uncertainty introduced when using an LLM instead of human preferences. The framework …

Foundation model OOD performance prediction is reliably achieved via ensemble diversity, especially through random linear head initialization, enabling precise estimations without extensive OOD labels…

Pre-trained LLMs surprisingly use Fourier features to perform addition, with MLP layers approximating magnitude and attention layers handling modular arithmetic; this mechanism requires pre-training.

Boosting AI instruction following, Language Feedback Models (LFMs) leverage Large Language Models (LLMs) to identify desirable behaviors from visual trajectories, significantly improving task completi…

Plan-on-Graph (PoG) revolutionizes KG-augmented LLMs with a self-correcting adaptive planning paradigm, enabling more efficient and accurate reasoning over knowledge graphs by dynamically adjusting ex…

New pipeline parallelism framework achieves up to 55% higher throughput and 50% less memory usage in large language model training by systematically controlling activation memory.

PhyloGen uses a genomic language model to generate and optimize phylogenetic trees, offering faster and more accurate evolutionary analysis than traditional methods.

Bi-directional Preference Optimization (BiPO) generates superior steering vectors for personalized LLM control, improving upon existing methods by directly influencing the generation probability of hu…

PerpCorrect: Robust LLM alignment despite noisy human preferences, achieved via perplexity-based noisy preference detection and correction.

This study reveals that large language models struggle with semi-open-ended questions, often hallucinating or providing insufficient answers. Researchers explored this by creating a new dataset of su…

PediatricsGPT: a novel Chinese pediatric LLM assistant trained on a large, high-quality dataset (PedCorpus) outperforms existing models, paving the way for improved pediatric healthcare.

PCB-MERGING: A training-free model merging technique boosts performance by intelligently balancing parameter competition across multiple tasks.

DeltaNet, a linear transformer boosting associative recall, now trains efficiently via a novel algorithm, scaling to large language models and outperforming existing linear baselines.