Shawn Zhao
Shawn Zhao
is there any update ?
Do you have the docker image pushed on dockerhub? it is easy to docker pull from the docker hub
Thanks for help @andrei-kochin @gkrivor could you please give a short answer ? is it a low hanging fruit or a long journey? Thank you!
运行 Model: DeepSeek-R1-Distill-Qwen-32B 数据精度: INT4 or FP8 or FP16 Task: AIME24 会遇到同样得问题
Step1: docker run --rm -dit \ --privileged \ --net=host \ --device=/dev/dri \ --name=lighteval-b21 \ -v /home/shawn:/llm/shawn \ -e no_proxy=localhost,127.0.0.1 \ -e http_proxy=$http_proxy \ -e https_proxy=$http_proxy \ --shm-size="32g" \ --entrypoint /bin/bash...
> You can use image `intelanalytics/ipex-llm-serving-xpu 0.8.3-b22` to test again. This problem is not encountered on b22 because the sdpa method update on qwen2.5-vl. Yes , it works intelanalytics/ipex-llm-serving-xpu 0.8.3-b22...