Daniel Han

Oh it seems like Discord members have shown Unsloth can do 97% "partial" full finetuning, albeit the layernorms are not updated.

Working on a fix! Sorry on the issue again!

Wait maybe GTX 1080s might not work anymore, although I cannot confirm. You first need to successfully install bitsandbytes in a standalone way first ie: `pip install bitsandbytes` then run...

@yangjianxin1 Oh wait does Qwen2 not have that weird alternating sliding window & normal attention thingo?

Thanks for the PR again! I streamlined Qwen2 to call `FastMistralModel` (since I think it's an exact replica right?)

Are you training on `embed_tokens` and `lm_head`?

Oh no sorry guys - i will take a look

Thanks for the code repro - will test this out - sorry on the issue again!

@pyrotank41 Whoops my bad - just fixed it!! You'll have to restart Colab / Kaggle or at the very top of the cell with ``` pip uninstall unsloth -y pip...

Try installing bitsandbytes itself then calling `python -m bitsandbytes`