ModuleNotFoundError: No module named 'ipex_llm.vllm.xpu' while using docker and installation

[sachinchandra]

followed installation of Vllm via this link

tried running via docker too: Here is the image

Still getting the no module named ipex_llm.vllm.xpu

Using currently on Arc A770

python -m ipex_llm.vllm.xpu.entrypoints.openai.api_server \
  --served-model-name $served_model_name \
  --port 8000 \
  --model $model \
  --trust-remote-code \
  --gpu-memory-utilization 0.75 \
  --device xpu \
  --dtype float16 \
  --enforce-eager \
  --load-in-low-bit sym_int4 \
  --max-model-len 4096 \
  --max-num-batched-tokens 10240 \
  --max-num-seqs 12 \
  --tensor-parallel-size 2

``
/usr/local/lib/python3.11/dist-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
[2024-05-24 19:06:41,339] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to xpu (auto detect)
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  async_io: please install the libaio-dev package with apt
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
2024-05-24 19:06:41,508 - root - INFO - intel_extension_for_pytorch auto imported
/usr/bin/python: Error while finding module specification for 'ipex_llm.vllm.xpu.entrypoints.openai.api_server' (ModuleNotFoundError: No module named 'ipex_llm.vllm.xpu')

[glorysdj]

We refactored the vLLM code today, the new code and image are in the CI/CD progress, while the doc is updated first. Please try python -m ipex_llm.vllm.entrypoints.openai.api_server for today's ipex-llm build. And with the nightly build tomorrow, please follow the doc.