ConniePK

vllm多卡推理遇到的问题

10

### Reminder - [X] I have read the README and searched the existing issues. ### System Info absl-py 2.1.0 accelerate 0.32.0 aiofiles 23.2.1 aiohttp 3.9.1 aiosignal 1.3.1 altair 5.3.0 annotated-types...

量化会卡住，Issues里很多人遇到了同样的问题，但都没有解决方案

3

### Reminder - [X] I have read the README and searched the existing issues. ### System Info - `llamafactory` version: 0.8.3.dev0 - Platform: Linux-5.4.54-1.0.0.std7c.el7.2.x86_64-x86_64-with-glibc2.29 - Python version: 3.8.10 - PyTorch...