Large Language Models

This paper introduces ActiveACRE, a model that uses LLMs and probabilistic inference to infer natural language rules through online experimentation, demonstrating higher accuracy than existing methods…

DoFIT: A novel domain-aware framework significantly reduces catastrophic forgetting in federated instruction tuning by finely aggregating overlapping weights and using a proximal perturbation initiali…

LLMs’ reasoning abilities are assessed via a novel framework that leverages probabilities of causation, revealing that while capable, their understanding of causality falls short of human-level reason…

LLMs’ fact retrieval is easily manipulated by context, highlighting their associative memory behavior; this paper studies this with transformers, showing how self-attention and value matrices support …

LLMs prioritize task completion over full world-state understanding by using goal-oriented abstractions.

DALD: A novel framework for black-box LLM text detection, achieving state-of-the-art performance without relying on source model logits, by aligning surrogate model distributions.

FUNCODER: a novel code generation framework that uses a divide-and-conquer approach with functional consensus to generate code that meets complex requirements.

Language models struggle with social/emotional intelligence and physical commonsense, unlike human brains. Fine-tuning models on these aspects improves their brain response prediction accuracy.

LLMs are aligned to human preferences distributionally using Optimal Transport, achieving state-of-the-art performance.

DISP-LLM: A novel dimension-independent structural pruning method for LLMs achieves accuracy similar to semi-structural pruning while improving flexibility and efficiency, outperforming state-of-the-a…

Discrete Flow Matching (DFM) revolutionizes discrete data generation by introducing a novel flow paradigm that surpasses existing methods. DFM leverages flexible probability paths, enabling efficient …

COAT leverages LLMs to identify high-level causal factors from unstructured data, enabling causal discovery in real-world scenarios where well-defined variables are lacking.

DSA, a novel automated framework, discovers optimal sparsity allocation for layer-wise LLM pruning, achieving significant performance gains across various models and tasks.

LLMs discover novel offline preference optimization algorithms, achieving state-of-the-art performance on various tasks.

Diffusion-of-Thought (DoT) boosts reasoning in diffusion language models by enabling parallel reasoning steps, outperforming larger autoregressive models in speed and accuracy.

Diff-eRank: A novel rank-based metric assessing LLMs’ efficiency in eliminating redundant information during training, showing improved correlation with model size and performance.

Decoupled-Head Attention (DHA) drastically cuts LLM inference costs by adaptively sharing key/value heads, achieving 97.6% of original performance with only 0.25% pre-training.

DeTikZify: AI synthesizes publication-ready scientific figures from sketches and existing figures, automatically generating semantically-preserving TikZ code.

DeTeCtive: a novel multi-task contrastive learning framework, achieves state-of-the-art AI-generated text detection by distinguishing diverse writing styles instead of simple binary classification.

DETAIL: A novel attribution method reveals the impact of individual demonstrations in in-context learning, boosting interpretability and improving transformer-based model performance.