unsloth
unsloth copied to clipboard
Error while importing "from unsloth import FastLanguageModel"
Trace -
File "/home/ec2-user/SageMaker/mistral-finetune-unsloth/multi-run-compare/run_model_qwen.py", line 1, in <module> from unsloth import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/__init__.py", line 174, in <module> from .models import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/__init__.py", line 15, in <module> from .loader import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/loader.py", line 16, in <module> from .llama import FastLlamaModel, logger File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/llama.py", line 31, in <module> from ..kernels import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/__init__.py", line 15, in <module> from .cross_entropy_loss import ( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/cross_entropy_loss.py", line 33, in <module> def _cross_entropy_forward( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 882, in jit return decorator(fn) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 871, in decorator return JITFunction( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 717, in __init__ self.src = self.src[re.search(r"^def\s+\w+\s*\(", self.src, re.MULTILINE).start():] AttributeError: 'NoneType' object has no attribute 'start'
unsloth version - 2024.11.7
triton - 3.0.0
torch - 2.4.0
xformers - 0.0.27.post2
Tried uninstalling everything -
pip uninstall -y accelerate bitsandbytes captum deepspeed flash-attn peft torchaudio torchdata torchtext torchvision unsloth_zoo xformers unsloth
Then installing -
pip install --upgrade setuptools packaging
pip install torch==2.4.0 torchvision torchaudio
pip install "unsloth @ git+https://github.com/unslothai/unsloth.git"
pip install "transformers @ git+https://github.com/huggingface/transformers.git"
pip install --no-deps "xformers==0.0.27.post2"
pip install git+https://github.com/microsoft/DeepSpeed
pip install git+https://github.com/huggingface/accelerate
pip install git+https://github.com/huggingface/peft
pip install trl bitsandbytes datasets protobuf sentencepiece scikit-learn flash_attn unsloth-zoo
But still doesn't work