FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

BUG for ImportError: cannot import name 'AsyncLLMEngine' from 'vllm' (unknown location)

Open tms2003 opened this issue 1 year ago • 3 comments
trafficstars

python -m fastchat.serve.vllm_worker --model-path "/home/incar/newdata2/tms/llm/chatglm3-6b-32k" --trust-remote-code


Traceback (most recent call last):
  File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/incar/miniconda3/envs/chatlain/lib/python3.10/site-packages/fastchat/serve/vllm_worker.py", line 15, in <module>
    from vllm import AsyncLLMEngine
ImportError: cannot import name 'AsyncLLMEngine' from 'vllm' (unknown location)

I test in vllm and it's ok!so,I think something would be wrong in fastchat.

tms2003 avatar Nov 24 '23 09:11 tms2003