dukelee111

Running vLLM service benchmark(4xARC770) with Qwen1.5-32B-Chat model failed

1

Environment: Platform: 6548N+4ARC770 Docker Image: intelanalytics/ipex-llm-serving-xpu:2.1.0 servicing script:  Error info: 1.With Dtype SYM_INT4 could succeed. 2.With Dtype FP8 failed with concurrency>=4. No error for concurrency 1 and 2. 2.GPU...

Running benchmark/all-in-one with GLM-4-9B-Chat model report "AutoTP not support for models"

1

Please help to confirm if the GLM-4-9B-Chat is supported , thanks so much. Docker images：intelanalytics/ipex-llm-serving-vllm-xpu-experiment Tag：2.1.0b2 Image ID：0e20af44ad46 step: cd /benchmark/all-in-one edit config.yaml bash run-deepspeed-arc.sh Attached the error trace details:...

Running vLLM service benchmark(1xARC770) with Qwen1.5-14B-Chat model failed(compression weight:SYM_INT4).

3

Environment: Platform: 6548N+1 ARC770 Docker Image:  servicing script:  Error info: 1.With compression weight SYM_INT4 failed. 2.Has tried the parameter "gpu-memory-utilization" from 0.65 to 0.95 with step 0.05 could...