Deep Learning

LogQuant: 2-bit quantization for KV cache, superior accuracy!

CURA: Verbal process supervision improves coding agents.

DeLoRA: Decoupling angles and strength in low-rank adaptation for robust & efficient finetuning of large models!

Gumbel-Softmax Flow Matching enables controllable biological sequence generation with straight-through guidance, scaling efficiently to high-dimensional simplices.

UAE-3D: A unified latent space approach for efficient & high-quality 3D molecular generation, outperforming existing methods in accuracy and speed.

Frac-Connections: An efficient alternative to Hyper-Connections that divides hidden states into fractions.

Transformers can achieve state-of-the-art performance without normalization layers via Dynamic Tanh (DyT), offering a simpler and more efficient alternative.

KArAt: Can Learnable Attention Beat Standard Attention in Vision Transformers?

Navigating millions of models is hard. This paper charts Hugging Face, revealing model relationships and attribute predictions.

Rimer: RWKV-7 empowers superior time series modeling, offering a simple yet effective alternative to Transformers with fewer parameters.

LoRACode enhances code embeddings using LoRA, achieving SOTA in code retrieval with minimal computational cost.

This paper reviews AI4SE benchmarks, introduces BenchScout for benchmark discovery, and proposes BenchFrame for benchmark enhancement, demonstrated via HumanEvalNext.

KODCODE: A new synthetic coding dataset with verified solutions and tests, enabling state-of-the-art performance for coding LLMs.

PQI: Accurately identify sensitive weights in post-quantization to enhance LLM compression & performance!

SoS1: O1 and R1-Like Reasoning LLMs are Sum-of-Square Solvers.

Stable-SPAM stabilizes 4-bit LLM training, outperforming Adam.

f-distill: One-step diffusion models through f-divergence minimization, outperforming reverse-KL with better mode coverage and lower variance.

MONSTER: Large datasets for time series classification!

S*: Hybrid test-time scaling for code generation, boosting both coverage and selection accuracy.

ReQFlow: Efficiently generate high-quality protein backbones with rectified quaternion flow, outperforming existing methods in speed and designability.