Daniel Han

@kiddyboots216 Ohh no so what we're doing is correct. It seems like you're not using mixed precision for training (fp16 = True, bf16 = True)

@kiddyboots216 For training, dY is in bfloat16. LoRA A and B must be in float32. This is for mixed precision training. The code you provided will not run at all,...

@BrunoBSM Wait so does normal Unsloth work on V100s? T4s work for now. @world2vec Apologies on the delay - this got lost! When dropout = 0, Unsloth will call the...

Ye so a dropout = 0 is optimized , but anything else is not - it still runs correct. @world2vec sadly I'm unsure why your RTX 4090 isn't working sorry...

@gotzmann Unsloth itself can load adapters, but unsure on Llama-Factory @hiyouga might be able to help

@hellangleZ Apologies on the delay - I'm assuming removing `dataclasses` could help?

@Qinolion Are you using multiple GPUs?

@erwe324 The current OSS here won't exactly work for multi GPU, but Llama Factory's Unsloth integration does have a slow somewhat functional multi GPU offering. Accelerate only partially works, and...

Thanks for the PR again!

I added this into nightly for now! Will do some final changes! Thanks @mahiatlinux !