使用以下命令对qwen_vl_chat多模态模型量化失败,请问目前支持多模态模型的量化吗?
##############################
CUDA_VISIBLE_DEVICES=6 swift export
--model_type qwen-vl-chat
--quant_bits 4
--dataset coco-en-mini
--quant_method gptq
--quant_output_dir output/quant
#############################
报错信息
############################
Traceback (most recent call last):
File "/miniconda/lib/python3.9/site-packages/swift/cli/export.py", line 5, in
export_main()
File "/miniconda/lib/python3.9/site-packages/swift/utils/run_utils.py", line 27, in x_main
result = llm_x(args, **kwargs)
File "/miniconda/lib/python3.9/site-packages/swift/llm/export.py", line 113, in llm_export
gptq_quantizer = gptq_model_quantize(model, template.tokenizer)
File "/miniconda/lib/python3.9/site-packages/swift/llm/export.py", line 87, in gptq_model_quantize
gptq_quantizer.quantize_model(model, tokenizer)
File "/miniconda/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/miniconda/lib/python3.9/site-packages/optimum/gptq/quantizer.py", line 436, in quantize_model
model(**data)
File "/miniconda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/miniconda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 856, in forward
transformer_outputs = self.transformer(
File "/miniconda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/miniconda/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/Qwen-VL-Chat/modeling_qwen.py", line 557, in forward
assert (bos_pos[0] == eos_pos[0]).all()
RuntimeError: The size of tensor a (7) must match the size of tensor b (6) at non-singleton dimension 0