swift icon indicating copy to clipboard operation
swift copied to clipboard

部署yi-vl模型报错cannot import name 'VllmGenerationConfig' from 'swift.llm.utils' (/home/ubuntu/project/swift/swift/llm/utils/__init__.py)

Open Sui-Xing opened this issue 4 months ago • 1 comments

在部署yi-vl-6b-chat模型时报错

CUDA_VISIBLE_DEVICES=0 swift deploy --model_type="yi-vl-6b-chat" --model_cache_dir="/home/ubuntu/project/Models/01ai/Yi-VL-6B/"

报错

Traceback (most recent call last):
  File "/home/ubuntu/project/swift/swift/utils/import_utils.py", line 71, in _get_module
    return importlib.import_module('.' + module_name, self.__name__)
  File "/home/ubuntu/miniconda3/envs/modelscope_env/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/ubuntu/project/swift/swift/llm/deploy.py", line 14, in <module>
    from .utils import (ChatCompletionRequest, ChatCompletionResponseChoice,
ImportError: cannot import name 'VllmGenerationConfig' from 'swift.llm.utils' (/home/ubuntu/project/swift/swift/llm/utils/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ubuntu/project/swift/swift/cli/deploy.py", line 2, in <module>
    from swift.llm import deploy_main
  File "<frozen importlib._bootstrap>", line 1075, in _handle_fromlist
  File "/home/ubuntu/project/swift/swift/utils/import_utils.py", line 60, in __getattr__
    module = self._get_module(self._class_to_module[name])
  File "/home/ubuntu/project/swift/swift/utils/import_utils.py", line 73, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import swift.llm.deploy because of the following error (look up to see its traceback):
cannot import name 'VllmGenerationConfig' from 'swift.llm.utils' (/home/ubuntu/project/swift/swift/llm/utils/__init__.py)

Sui-Xing avatar Feb 27 '24 10:02 Sui-Xing