Chaeyeon Kwak

Fine-tuning bitnet-b1.58-2B-4T-bf16 on Korean dataset results in high loss (~3.3–3.6)

2

Hi, and thanks for the great work on BitNet! I'm trying to fine-tune `microsoft/bitnet-b1.58-2B-4T-bf16` using a Korean dataset (`nlpai-lab/kullm-v2`) with SFTTrainer. However, during training, the loss remains around **3.3 to...