notebooks
notebooks copied to clipboard
unslothai
100+ Fine-tuning Tutorial Notebooks on Google Colab, Kaggle and more.
The two cases provided in the document involve different dataset formats. Which one should we choose when we do fine-tuning without CoT?  **1、[Qwen3 (14B) Reasoning + Conversational notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(14B)-Reasoning-Conversational.ipynb) (recommended)**...
`tokenizer` is undefined, we should use `processor` from that cell instead: ``` model, processor = FastModel.from_pretrained( model_name = "unsloth/csm-1b", max_seq_length= 2048, # Choose any for long context! dtype = None,...
I use the `Gemma 3 (1B)`, pick some question from the `openai/gsm8k`, the result is pretty bad compare `Gemma 3 (1B)` without fine-tune. I run it with `llama-cli -m ./gemma-3-finetune.Q8_0.gguf`,...
Set tokenize=False in tokenizer.apply_chat_template It won't run otherwise since the tokenizing is happening twice. I faced this minor issue when running today. ```python def apply_chat_template(examples): texts = tokenizer.apply_chat_template(examples["conversations"], tokenize =...
from trl import SFTTrainer, SFTConfig trainer = SFTTrainer( model = model, tokenizer = tokenizer, train_dataset = dataset, eval_dataset = None, # Can set up evaluation! args = SFTConfig( dataset_text_field =...
vanilla Llama3.1_(8B)-Alpaca.ipynb throws in the second codeblock ``` :1: UserWarning: WARNING: Unsloth should be imported before trl, transformers, peft to ensure all optimizations are applied. Your code may run slower...
The original regex didn’t correctly match newlines inside the `reasoning` and `answer` tags. As a result, `soft_format_reward_func` always returned 0, since `strict_format_reward_func` required exactly two newlines inside each tag.
Add tool calling notebooks for the following models: - [Qwen-2.5-Instruct example](https://colab.research.google.com/github/oliveirabruno01/unsloth-challenge/blob/main/Qwen2_5_1_5B_Tool_Calling.ipynb) - [Llama-3.1-Instruct example](https://colab.research.google.com/github/oliveirabruno01/unsloth-challenge/blob/main/Llama3_1_%288B%29_Tool_Calling.ipynb)
when opening for more than 3 chunks: import time # Process 3 chunks for now -> can increase but slower! for filename in filenames[:5]: !synthetic-data-kit \ -c synthetic_data_kit_config.yaml \ create...
About instruction&question padding. I saw in some SFT notebook demos, e.g. Gemma3_(4B).ipynb, we enable padding these parts -100, while some others like Qwen2.5_(7B)-Alpaca.ipynb we do not. Is it necessary or...
