huggingface-vscode-endpoint-server
huggingface-vscode-endpoint-server copied to clipboard
LucienShui
ValueError: Could not load model bigcode/starcoder with any of the following classes
pip freeze:
accelerate==0.19.0 anyio==3.6.2 boto3==1.26.135 botocore==1.29.135 certifi==2023.5.7 charset-normalizer==3.1.0 click==8.1.3 fastapi==0.95.2 filelock==3.12.0 fsspec==2023.5.0 h11==0.14.0 huggingface-hub==0.14.1 idna==3.4 jmespath==1.0.1 numpy==1.24.3 nvidia-cublas-cu11==11.10.3.66 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cudnn-cu11==8.5.0.96 packaging==23.1 psutil==5.9.5 pydantic==1.10.7 python-dateutil==2.8.2 pytorch-pretrained-bert==0.6.2 PyYAML==6.0 regex==2023.5.5 requests==2.30.0 s3transfer==0.6.1 six==1.16.0 sniffio==1.3.0 starlette==0.27.0 tokenizers==0.13.3 torch==1.13.1 tqdm==4.65.0 transformers==4.28.1 typing_extensions==4.5.0 urllib3==1.26.15 uvicorn==0.22.0 `
python main.py --host=0.0.0.0 --port=7088 --pretrained="bigcode/starcoder"
Traceback (most recent call last):
File "/data/sd-webui/7099/main.py", line 38, in
Have you checked your model's hash? It looks like something goes wrong when loading config or weights. Perhaps problem from system maybe.
Have you checked your model's hash? It looks like something goes wrong when loading config or weights. Perhaps problem from system maybe.
yes, how to check whether the hash is right or not?
What does your git status command display? If there is any diff, that means there is something wrong maybe.
Are you using an AMD GPU?
If yes, continue reading, if not, just ignore my comment.
I have a 6800XT and managed to make it work, but eventually it ran out of VRAM after loading the model files. I suggest just give up now before wasting more time, unless you have a AMD GPU which have at least 32GB of VRAM.
And: Traceback of TorchScript (most recent call last): RuntimeError: HIP out of memory. Tried to allocate 224.00 MiB (GPU 0; 15.98 GiB total capacity; 15.36 GiB already allocated; 156.00 MiB free; 15.80 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_HIP_ALLOC_CONF
You need to use ROCM supported PyTorch.
Use pip list |grep rocm to determine if you have correct pytorch installed.
Expected:
If you have a similar result, run this simple test command in python to determine your torch worked with rocm.
import torch
torch.cuda.is_available()
The result must be True. If not your amd driver install steps is incomplete.
I have the same problem (on windows) and was trying to get starcode to run using different ways, i.e. downloading the model seperatly and also let the endpoint-server download the model. It wasn't succesful. So I tried a small model (to save some time)
python main.py --pretrained="bigcode/tiny_starcoder_py"
and that seems to work fine.
So, I'm am wildly guessing, that there is something wrong with the bigcode/starcoder in conjunction with the endpoint-server so that it wouldn't find the model. Also, maybe it's only a windows problem?
I have the same problem (on windows) and was trying to get starcode to run using different ways, i.e. downloading the model seperatly and also let the endpoint-server download the model. It wasn't succesful. So I tried a small model (to save some time)
python main.py --pretrained="bigcode/tiny_starcoder_py"and that seems to work fine.So, I'm am wildly guessing, that there is something wrong with the bigcode/starcoder in conjunction with the endpoint-server so that it wouldn't find the model. Also, maybe it's only a windows problem?
Thanks, this worked for me, do you have the information for other ones? how to add your own? like Llama2?
Also, maybe it's only a windows problem?
Not a Windows only problem, I just ran into it on Ubuntu 22.04
