FastChat
FastChat copied to clipboard
BUG for ImportError: cannot import name 'AsyncLLMEngine' from 'vllm' (unknown location)
trafficstars
python -m fastchat.serve.vllm_worker --model-path "/home/incar/newdata2/tms/llm/chatglm3-6b-32k" --trust-remote-code
Traceback (most recent call last):
File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/site-packages/fastchat/serve/vllm_worker.py", line 15, in <module>
from vllm import AsyncLLMEngine
ImportError: cannot import name 'AsyncLLMEngine' from 'vllm' (unknown location)
I test in vllm and it's ok!so,I think something would be wrong in fastchat.