Daniel Han

@gracehubai I just added chat templates! It supports Zephyr (the one TInyLlama uses), ChatML, Vicuna etc - https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing

@abstrcode You might need to restart the Runpod instance, and just run `pip install "unsloth[cu121] @ git+https://github.com/unslothai/unsloth.git"` without doing anything else. If you're using A100s or Ampere GPUs, `pip install...

@abstrcode I don' think the GPU matters - V100 is supported. It's mainly bitsandbytes being unable to be installed. If you have conda, try using the conda method to install...

@RonanKMcGovern Yes! We had a chat on our Discord server about this! It looks very promising, and it removes lora_alpha (finally!!!) 1 less hyperparameter! Love how it looks very similar...

@namednil Oh nice catch on the bug - will solve this

@gottlike Will take a look!

Hmmm Ill see what I can do but hmmm

Hmm an issue is Deberta is an encoder decoder BERT type model right? I might add support for other BERT type models in a future release :)

More than happy to welcome contributions for Mamba!!

@jeromeku I will get to reviewing GPTQ - sorry on the delay!! * The VRAM reductions are from Unsloth's optims :) Ie Triton kernels, making memory copies go away, FA2,...