eugeooi

vLLM offline_inference.py failed to run on CPU inference

1

Failed to run python `offline_inference.py` from [link](https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/example/CPU/vLLM-Serving) for vLLM offline inference on CPU. It seems that `llm.py` has been removed in the previous version.

Error: Failed to load the llama dynamic library. Segmentation fault

6

platform：Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz os: Suse 13 model：mistralai/Mistral-7B-Instruct-v0.2 ipex-llm：2.1.0b20240515 transformers: 4.37.0 ldd: 2.22 gcc/g++: 11.1.0 After Loading checkpoint shards 100%, it shows: `Error: Failed to load the...