wang tianyi

icon indicating a website link https://www.bupt.edu.cn/

icon company Beijing University of Posts and Telecommunications icon location Beijing, China icon location My name is Wang tianyi and I am studying at the School of Computer Science, Beijing University of Posts and Telecommunications.

Results 1 issues of wang tianyi

swfit deploy qwen2.5vl 32b awq模型推理乱码

1 comment

**Describe the bug** 通过下面的代码部署vllm server CUDA_VISIBLE_DEVICES=4,6 swift deploy \ --model "/hdd/wangty/model/Qwen2.5-VL-32B-Instruct-AWQ" \ --infer_backend vllm \ --served_model_name Qwen2.5-VL-32B-Instruct-AWQ \ --vllm_max_model_len 8192 \ --vllm_gpu_memory_utilization 0.9 \ --vllm_tensor_parallel_size 2 \ --vllm_enforce_eager true \...