Moris Johnson

icon indicating a website link morisjohnson.in icon indicating an email [email protected]

icon location Sivagangai icon location I'm just your average Python coder trying to make it in this cruel, cruel world. I've been slaving away at this whole programming thing for a few years.

Results 1 comments of Moris Johnson

Fine-tuning bitnet-b1.58-2B-4T-bf16 on Korean dataset results in high loss (~3.3–3.6)

@kwak513 Not sure if it'll help, I was trying to replicate [onebitllms](https://github.com/tiiuae/onebitllms/blob/main/examples/sft.py) script for a custom text data, faced the same problem where loss doesn't decrease. I changed the training...