🏢 Research Center for Social Computing and Information Retrieval,Harbin Institute of Technology
OneBit: Towards Extremely Low-bit Large Language Models
·2001 words·10 mins·
loading
·
loading
Natural Language Processing
Large Language Models
🏢 Research Center for Social Computing and Information Retrieval,Harbin Institute of Technology
OneBit achieves surprisingly good performance in 1-bit quantized LLMs by using a novel 1-bit parameter representation method and an effective parameter initialization method.