unsloth icon indicating copy to clipboard operation
unsloth copied to clipboard

Error while importing "from unsloth import FastLanguageModel"

Open thesillystudent opened this issue 3 months ago • 6 comments

Trace -

File "/home/ec2-user/SageMaker/mistral-finetune-unsloth/multi-run-compare/run_model_qwen.py", line 1, in <module> from unsloth import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/__init__.py", line 174, in <module> from .models import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/__init__.py", line 15, in <module> from .loader import FastLanguageModel File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/loader.py", line 16, in <module> from .llama import FastLlamaModel, logger File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/models/llama.py", line 31, in <module> from ..kernels import * File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/__init__.py", line 15, in <module> from .cross_entropy_loss import ( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/unsloth/kernels/cross_entropy_loss.py", line 33, in <module> def _cross_entropy_forward( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 882, in jit return decorator(fn) File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 871, in decorator return JITFunction( File "/home/ec2-user/anaconda3/envs/JupyterSystemEnv/lib/python3.10/site-packages/triton/runtime/jit.py", line 717, in __init__ self.src = self.src[re.search(r"^def\s+\w+\s*\(", self.src, re.MULTILINE).start():] AttributeError: 'NoneType' object has no attribute 'start'

unsloth version - 2024.11.7
triton - 3.0.0
torch - 2.4.0
xformers - 0.0.27.post2

Tried uninstalling everything - pip uninstall -y accelerate bitsandbytes captum deepspeed flash-attn peft torchaudio torchdata torchtext torchvision unsloth_zoo xformers unsloth

Then installing -

 pip install --upgrade setuptools packaging
 pip install torch==2.4.0 torchvision torchaudio
 pip install "unsloth @ git+https://github.com/unslothai/unsloth.git"
 pip install "transformers @ git+https://github.com/huggingface/transformers.git"
 pip install  --no-deps "xformers==0.0.27.post2"
 pip install git+https://github.com/microsoft/DeepSpeed
 pip install git+https://github.com/huggingface/accelerate
 pip install git+https://github.com/huggingface/peft
 pip install trl bitsandbytes datasets protobuf sentencepiece scikit-learn flash_attn unsloth-zoo

But still doesn't work

thesillystudent avatar Nov 15 '24 08:11 thesillystudent