Natural Language Processing

Unified Spoken Dialog Model (USDM) directly generates coherent spoken responses with natural prosody, surpassing cascaded baselines and enhancing natural conversation in speech-enabled LLMs.

Panacea: a novel LLM alignment method achieving Pareto optimality via online preference adaptation using a single model.

PaDeLLM-NER massively accelerates LLM-based NER inference by up to 10x, enabling near real-time performance without accuracy loss.

PaCE, a novel activation engineering framework, efficiently aligns LLMs by removing undesirable concepts from activations using sparse coding, achieving state-of-the-art performance while preserving l…

Over-parameterized Distillation Framework (OPDF) boosts knowledge distillation by efficiently over-parameterizing student models via tensor decomposition, significantly improving performance without i…

Set-Based Prompting guarantees order-independent LLM outputs by modifying input representations, eliminating unwanted inconsistencies without fine-tuning.

Orchid: a novel deep learning architecture using data-dependent convolution achieves quasilinear scalability and outperforms attention-based models on various sequence modeling tasks.

LLMs boost tabular data prediction by generating optimized features via decision tree reasoning, outperforming existing methods.

Open LLMs outperform closed alternatives for private data adaptation, offering superior privacy, performance, and lower costs.

This paper proposes a novel, reward-free RLHF framework using a general preference oracle, surpassing existing reward-based approaches in efficiency and generalizability.

MAC: Efficiently updates large language models (LLMs) using a memory of compressed contexts for improved real-time knowledge retention and adaptation.

OneBit achieves surprisingly good performance in 1-bit quantized LLMs by using a novel 1-bit parameter representation method and an effective parameter initialization method.

VideoLISA: A video-based multimodal large language model enabling precise, language-instructed video object segmentation with superior performance.

This research introduces Cluster-guided Sparse Experts (CSE), enabling pretrained language models to effectively learn long-tail domain knowledge without domain-specific pretraining, thus achieving su…

LLMs’ performance drastically varies depending on prompt phrasing; this paper introduces ROBUSTAL-PACAEVAL to evaluate lower-bound performance via worst-case prompt analysis, revealing model inconsist…

Auto-Regressive Decision Trees (ARDTs) surprisingly outperform Transformers on language tasks!

MIDAS: A novel training method improves language model reasoning by efficiently stacking middle layers, surprisingly boosting downstream task performance without increasing pretraining perplexity.

Softmax-DPO boosts LM-based recommender performance by directly optimizing for personalized ranking using a novel loss function that incorporates multiple negative samples, significantly outperforming…

Weak LLMs can accurately supervise strong LLMs via debate, outperforming simpler consultancy methods, especially in information-asymmetric tasks.

Effortlessly boost large language model performance by dynamically fusing knowledge from smaller, task-specific models – achieving near full fine-tuning results with minimal computational cost!