api-for-open-llm
api-for-open-llm copied to clipboard
xusenlinzy
💡 [REQUEST] - 请问如何支持 Qwen/Qwen-VL-Chat
起始日期 | Start Date
No response
实现PR | Implementation PR
No response
相关Issues | Reference Issues
No response
摘要 | Summary
请问如何支持 Qwen/Qwen-VL-Chat
基本示例 | Basic Example
File "server.py", line 2, in <module> from api.models import EMBEDDED_MODEL, GENERATE_MDDEL, app, VLLM_ENGINE File "/root/api-for-open-llm/api/models.py", line 140, in <module> VLLM_ENGINE = create_vllm_engine() if (config.USE_VLLM and config.ACTIVATE_INFERENCE) else None File "/root/api-for-open-llm/api/models.py", line 98, in create_vllm_engine engine = AsyncLLMEngine.from_engine_args(engine_args) File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/engine/async_llm_engine.py", line 232, in from_engine_args engine = cls(engine_args.worker_use_ray, File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/engine/async_llm_engine.py", line 55, in __init__ self.engine = engine_class(*args, **kwargs) File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 101, in __init__ self._init_workers(distributed_init_method) File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 133, in _init_workers self._run_workers( File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/engine/llm_engine.py", line 470, in _run_workers output = executor(*args, **kwargs) File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/worker/worker.py", line 67, in init_model self.model = get_model(self.model_config) File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/model_executor/model_loader.py", line 57, in get_model model.load_weights(model_config.model, model_config.download_dir, File "/usr/local/miniconda3/lib/python3.8/site-packages/vllm/model_executor/models/qwen.py", line 308, in load_weights param = state_dict[name] KeyError: 'transformer.visual.positional_embedding'
缺陷 | Drawbacks
Qwen/Qwen-VL-Chat
未解决问题 | Unresolved questions
No response
