🏢 Baichuan Inc.
Base of RoPE Bounds Context Length
·3176 words·15 mins·
loading
·
loading
AI Generated
Natural Language Processing
Large Language Models
🏢 Baichuan Inc.
LLM long-context ability is fundamentally limited by RoPE’s base parameter, which determines an absolute lower bound for achievable context length.