Wang, Jian4
Wang, Jian4
It seems that there is nothing wrong with your test related configuration and OS kernel. Maybe this error is caused by the newer driver: ```bash Driver related package version: ii...
@flekol Maybe this issue is caused by the 1ccl don't work normally. In your dockerfile, you use `RUN . /opt/intel/1ccl-wks/setvars.sh` to instead of `source /opt/intel/1ccl-wks/setvars.sh`. I think it may not...
This issue is caused by bmg card 0xe211, maybe we need to upgrade [compute-runtime](https://github.com/intel/compute-runtime/releases) to latest in container to get the normal preformance.
This issue will be fixed by [pr-10678](https://github.com/intel-analytics/ipex-llm/pull/10678)
Currently, neither XPU nor CUDA support using float32 dtype when running weight as fp16. .
https://github.com/vllm-project/vllm/pull/5358 GLM-4V-9B model is not yet supported by vllm now.
Could you provide more details on how to reproduce it? Provide the bash script on how to start vllm service and how to use Lighteval to run AIME24 case.
> export USE_XETLA=OFF > export SYCL_CACHE_PERSISTENT=1 > export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=2 > export FI_PROVIDER=shm > export TORCH_LLM_ALLREDUCE=0 > export CCL_WORKER_COUNT=1 > export CCL_ATL_TRANSPORT=ofi > export CCL_ZE_IPC_EXCHANGE=sockets > export CCL_ATL_SHM=1 > export CCL_SAME_STREAM=1...
目前vllm 0.9.2-b22还不支持跑GLM-4.1V
I can't reproduce it in our ENV on desktop with CPU Intel(R) Core(TM) i9-14900K and kernel 6.5.0-28-generic, and we don't have a UHD Graphics on it. It looks like a...