unsloth
unsloth copied to clipboard
Published
20 hours ago •
unslothai
unslothai
Error patching {trainer_name} when importing FastLanguageModel
I am doing a fresh install on a new machine with this hardware and cuda installed;
nvidia-smi
Thu Feb 13 15:55:03 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.15 Driver Version: 570.86.15 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA H100 NVL Off | 00000000:21:00.0 Off | 0 |
| N/A 33C P0 61W / 400W | 1MiB / 95830MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA H100 NVL Off | 00000000:81:00.0 Off | 0 |
| N/A 33C P0 63W / 400W | 1MiB / 95830MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
I just ran the snippet from here; https://github.com/unslothai/unsloth?tab=readme-ov-file#conda-installation on an empty anaconda environment with no other envs installed.
conda create --name unsloth_env \
python=3.11 \
pytorch-cuda=12.1 \
pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers \
-y
conda activate unsloth_env
pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
pip install --no-deps trl peft accelerate bitsandbytes
import os
os.environ["UNSLOTH_DISABLE_PATCHING"] = "1"
from unsloth import FastLanguageModel
...:
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
🦥 Unsloth Zoo will now patch everything to make training faster!
---------------------------------------------------------------------------
SyntaxError Traceback (most recent call last)
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/tokenizer_utils.py:1061
1060 try:
-> 1061 exec(trainer_text, globals())
1062 except:
SyntaxError: invalid syntax (<string>, line 4)
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
Cell In[1], line 1
----> 1 from unsloth import FastLanguageModel, is_bfloat16_supported
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/__init__.py:212
209 raise ImportError("Unsloth: Please install unsloth_zoo via `pip install unsloth_zoo`")
210 pass
--> 212 from .models import *
213 from .save import *
214 from .chat_templates import *
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/models/__init__.py:16
1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 16 from .granite import FastGraniteModel
17 from .loader import FastLanguageModel, FastVisionModel
18 from .llama import FastLlamaModel
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/models/granite.py:15
1 # Copyright 2023-present Daniel Han-Chen & the Unsloth team. All rights reserved.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 15 from .llama import *
16 import os
17 from ._utils import __version__
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/models/llama.py:36
32 from transformers.modeling_attn_mask_utils import (
33 _prepare_4d_causal_attention_mask_for_sdpa,
34 )
35 from ..kernels import *
---> 36 from ..tokenizer_utils import *
37 if HAS_FLASH_ATTENTION:
38 from flash_attn import flash_attn_func
File ~/anaconda3/envs/unsloth_env/lib/python3.11/site-packages/unsloth/tokenizer_utils.py:1063
1061 exec(trainer_text, globals())
1062 except:
-> 1063 raise RuntimeError(f"Unsloth: Please file a bug report! Error patching {trainer_name}")
1064 exec(f"trl.trainer.{trainer_name} = Unsloth{trainer_name}", globals())
1065 pass
RuntimeError: Unsloth: Please file a bug report! Error patching SFTTrainer
pip list
Package Version
------------------------ ------------
accelerate 1.3.0
aiohappyeyeballs 2.4.6
aiohttp 3.11.12
aiosignal 1.3.2
annotated-types 0.7.0
asttokens 3.0.0
attrs 25.1.0
bitsandbytes 0.45.2
certifi 2025.1.31
charset-normalizer 3.4.1
click 8.1.8
cut-cross-entropy 25.1.1
datasets 3.2.0
decorator 5.1.1
diffusers 0.32.2
dill 0.3.8
docker-pycreds 0.4.0
docstring_parser 0.16
einops 0.8.1
executing 2.2.0
filelock 3.13.1
flash_attn 2.7.4.post1
frozenlist 1.5.0
fsspec 2024.9.0
gitdb 4.0.12
GitPython 3.1.44
gmpy2 2.2.1
hf_transfer 0.1.9
huggingface-hub 0.28.1
idna 3.10
importlib_metadata 8.6.1
ipython 8.32.0
jedi 0.19.2
Jinja2 3.1.5
markdown-it-py 3.0.0
MarkupSafe 3.0.2
matplotlib-inline 0.1.7
mdurl 0.1.2
mpmath 1.3.0
multidict 6.1.0
multiprocess 0.70.16
networkx 3.4.2
ninja 1.11.1.3
numpy 2.2.2
nvidia-cublas-cu12 12.4.5.8
nvidia-cuda-cupti-cu12 12.4.127
nvidia-cuda-nvrtc-cu12 12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.2.1.3
nvidia-curand-cu12 10.3.5.147
nvidia-cusolver-cu12 11.6.1.9
nvidia-cusparse-cu12 12.3.1.170
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.4.127
packaging 24.2
pandas 2.2.3
parso 0.8.4
peft 0.14.0
pexpect 4.9.0
pillow 11.1.0
pip 25.0.1
platformdirs 4.3.6
prompt_toolkit 3.0.50
propcache 0.2.1
protobuf 3.20.3
psutil 6.1.1
ptyprocess 0.7.0
pure_eval 0.2.3
pyarrow 19.0.0
pydantic 2.10.6
pydantic_core 2.27.2
Pygments 2.19.1
python-dateutil 2.9.0.post0
pytz 2025.1
PyYAML 6.0.2
regex 2024.11.6
requests 2.32.3
rich 13.9.4
safetensors 0.5.2
sentencepiece 0.2.0
sentry-sdk 2.21.0
setproctitle 1.3.4
setuptools 75.8.0
shtab 1.7.1
six 1.17.0
smmap 5.0.2
stack-data 0.6.3
sympy 1.13.1
tokenizers 0.21.0
torch 2.5.0
torchvision 0.20.1
tqdm 4.67.1
traitlets 5.14.3
transformers 4.48.3
triton 3.1.0
trl 0.15.0
typeguard 4.4.1
typing_extensions 4.12.2
tyro 0.9.14
tzdata 2025.1
unsloth 2025.2.4
unsloth_zoo 2025.2.3
urllib3 2.3.0
wandb 0.19.6
wcwidth 0.2.13
wheel 0.45.1
xformers 0.0.28.post2
xxhash 3.5.0
yarl 1.18.3
zipp 3.21.0
Cannot import from unsloth import FastLanguageModel
