modelscope
目前支持多模态模型部署了吗
Describe the bug
目前使用以下脚本部署internvl-v15显示部署成功：
CUDA_VISIBLE_DEVICES=0 swift deploy
--model_type internvl-chat-v1_5
--model_id_or_path /home/weight/internvl-chat-v1-5/
--use_flash_attn true
--dtype bf16
使用openai格式脚本调用，可以获取模型名称，然后就报错：
from openai import OpenAI
import base64
import time
client = OpenAI(
api_key="EMPTY",
base_url="http://127.0.0.1:8000/v1/",
)
model_name = client.models.list().data[0].id
print(model_name)
st = time.time()
print_stream = True
stream = client.chat.completions.create(
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "请描述一下图像"},
{
"type": "image_url",
"image_url": {
"url": 'https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg'
},
},
],
},
],
model=model_name,
stream=print_stream,
max_tokens=512,
top_p=0.2,
)
if not print_stream:
print(stream)
else:
for part in stream:
print(part.choices[0].delta.content or "", end="", flush=True)
print("process_time:{}".format(round(time.time() - st, 3)))
服务端显示信息： INFO: 127.0.0.1:40132 - "GET /v1/models HTTP/1.1" 200 OK INFO: 127.0.0.1:40132 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity
Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等) swift版本： ms-swift 2.1.0.dev0
Additional context Add any other context about the problem here(在这里补充其他信息)
已经支持多模态部署：https://github.com/modelscope/swift/blob/main/docs/source/Multi-Modal/MLLM%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3.md
