inference icon indicating copy to clipboard operation
inference copied to clipboard

Got an error when try to launch GLM4 model in docker container

Open HDhuangdi opened this issue 8 months ago • 1 comments

System Info / 系統信息

ValueError: [address=0.0.0.0:43891, pid=429] The checkpoint you are trying to load has model type glm4 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command pip install --upgrade transformers. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command pip install git+https://github.com/huggingface/transformers.git

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

  • [x] docker / docker
  • [ ] pip install / 通过 pip install 安装
  • [ ] installation from source / 从源码安装

Version info / 版本信息

doker image tag "v1.5.0.post2" transformers(in docker container) version 4.50.3 vllm(in docker container) version 0.8.4

The command used to start Xinference / 用以启动 xinference 的命令

docker run --shm-size=20g -d --name xinference -v /data/models:/models -e XINFERENCE_HOME=/models -p 19997:9997 --gpus all xprobe/xinference:v1.5.0.post2 xinference-local -H 0.0.0.0

Reproduction / 复现过程

None

Expected behavior / 期待表现

None

HDhuangdi avatar Apr 24 '25 03:04 HDhuangdi

同样的问题

Halflifefa avatar Apr 25 '25 03:04 Halflifefa

容器内升级transforemrs>=4.51.3,然后就会变另外一个报错

Image

Docum-J avatar Apr 30 '25 03:04 Docum-J

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] avatar May 07 '25 19:05 github-actions[bot]

This issue was closed because it has been inactive for 5 days since being marked as stale.

github-actions[bot] avatar May 13 '25 19:05 github-actions[bot]