swift icon indicating copy to clipboard operation
swift copied to clipboard

internvl-chat-v1.5-int8 推理时报错,应该如何处理

Open wlg-tt opened this issue 1 month ago • 3 comments

Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图) 可以正常启动 CUDA_VISIBLE_DEVICES=0 swift infer --model_type internvl-chat-v1_5-int8 --model_id_or_path /home/tione/notebook/community/scan/InternVL-Chat-V1-5-int8/ --dtype bf16 但是推理时报错 image internvl-chat-v1_5可以正常启动和推理 CUDA_VISIBLE_DEVICES=0 swift infer --model_type internvl-chat-v1_5 --model_id_or_path /home/tione/notebook/community/scan/InternVL-Chat-V1-5/ --dtype bf16

Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等) torch2.1 py3.10 cuda12.1

Additional context Add any other context about the problem here(在这里补充其他信息)

wlg-tt avatar May 17 '24 06:05 wlg-tt