Daniel Han
Daniel Han
Hey there! First, fabulous work on speeding up Sklearn! I'm the author of Hyperlearn (https://github.com/danielhanchen/hyperlearn), where I also tried investigating how to speed up algos on the CPU (all the...
Hey @mlabonne! Actually found this repo via Linkedin! :) Happy New Year! Had a look through your notebooks - they look sick! Interestingly I was trying myself to run `axolotl`...
Just a few more Gemma fixes :) Currently checking for more as well! Related PR: https://github.com/huggingface/transformers/pull/29285, which showed RoPE must be done in float32 and not float16, causing positional encodings...
Xformers only supports torch 2.2.0 and not torch 2.2.1: **UPDATE** Xformers supports 2.2.1!! Replace `conda` with `mamba` if you have it. Much much faster. ```bash conda create --name unsloth_env python=3.10...
Kaggle also got updated. Please change the install instructions at the top to: ```bash %%capture !pip install -U xformers --index-url https://download.pytorch.org/whl/cu121 !pip install "unsloth[kaggle-new] @ git+https://github.com/unslothai/unsloth.git" import os os.environ["WANDB_DISABLED"] =...
Colab recently got updated. Please change the top cell's (replace entirely) install instructions with: ```bash %%capture import torch major_version, minor_version = torch.cuda.get_device_capability() # Must install separately since Colab has torch...
From Discord bugs: Adding `lm_head` and `embed_tokens` for training is broken, since they are not upcast to float32 for mixed precision training
Bug report from Discord Will support this in a future release.
Fully supported! Scroll down on our latest Mistral notebook: https://colab.research.google.com/drive/1Dyauq4kTZoLewQ1cApceUQVNcnnNTzg_?usp=sharing For 16bit merging: ``` model.save_pretrained_merged("dir", tokenizer, save_method = "merged_16bit") model.push_to_hub_merged("dir", tokenizer, save_method = "merged_16bit") ``` For GGUF merging: ``` model.save_pretrained_gguf("dir",...