unsloth
unsloth copied to clipboard
unslothai
Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! 🦥
So I finetuned a model using a custom dataset. The output should be in JSON format. All the keys are the same for each output, i.e. structure of the response...
I am new to fine-tuning. I ran through this very helpful notebook and got it to run locally: https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing#scrollTo=EWGFqAo5Q2me I assume that after the dpo_trainer.train() is finished, the "model" object...
I followed the conda installation instructions in the README: ``` conda create --name unsloth_env python=3.10 conda activate unsloth_env conda install pytorch cudatoolkit torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia conda...
Hi, I'm getting the following error when I try to run unsloth on Windows with WSL, any idea what causes it? Python 3.10 ~~~ model, tokenizer = FastLanguageModel.from_pretrained(mistralai/Mistral-7B-Instruct-v0.2) FastLanguageModel.for_inference(model) output_ids...
Xformers only supports torch 2.2.0 and not torch 2.2.1: **UPDATE** Xformers supports 2.2.1!! Replace `conda` with `mamba` if you have it. Much much faster. ```bash conda create --name unsloth_env python=3.10...
@danielhanchen In the unsloth Gemma intro [blogpost](https://unsloth.ai/blog/gemma), you mention VRAM increase due to larger `MLP` size in `Gemma` compared to `Llama` and `Mistral`, and show a [graph](https://unsloth.ai/cgi/image/VRAM_usage_(extrapolated)_YwIoc66yMei-LpOGYqftM.svg?width=1920&quality=80&format=auto) demonstrating decreased memory...
Add a support for deberta model "MoritzLaurer/deberta-v3-large-zeroshot-v1.1-all-33". This model is heavily used for text classification.
Hi, I am facing a new error now while trying to run the alpaca fine-tuning notebook in kaggle-  Complete logs are attached- [logs_kaggle.txt](https://github.com/unslothai/unsloth/files/14446951/logs_kaggle.txt)
