Ankit Pal

> Was wondering when would this be merged? Too busy to work on this :P

Any progress on this?

I have added one example for fft via this pr https://github.com/OpenAccess-AI-Collective/axolotl/pull/1580

> @jaywongs , did the above solve it for you? I find this issue dependent on machine. It may also be bitsandbytes issue. Yes, it solved for me!

Thank you, @rasbt , for your scratch implementations. I am really enjoying them. Yes, You got it, I am confused about line 121's linear layer transformation. Please forgive my naive...

This is interesting, I was also looking same, this is my config, do I need to change anything else? optim: "adamw_8bit" ? ```yaml # Model configuration model: name: "meta-llama/Meta-Llama-3.1-8B" max_seq_length:...

MMLU dataset is available on hugginface now, Please refer : https://huggingface.co/datasets/lukaemon/mmlu