Lu Fang

Results 87 comments of Lu Fang

[Installation]: error: can't copy 'build/lib.linux-x86_64-3.10/vllm/_core_C.abi3.so': doesn't exist or not a regular file

The root cause is actually cmake 3.26.0, upgrading to 3.26.1 or newer version should solve the problem.

[Installation]: error: can't copy 'build/lib.linux-x86_64-3.10/vllm/_core_C.abi3.so': doesn't exist or not a regular file

Same as https://github.com/vllm-project/vllm/issues/18748

[Bug]: using qwen-8B , LLVM ERROR: Failed to compute parent layout for slice layout

Try --dtype float32?

[Bug]: vllm 0.8.3 v1 startup time is too long when using multi lora

Sorry about this issue. Currently we are looking into the LoRA support for Llama4. cc: @frank-wei

[V1] Support MP Executor for multi node distributed inference

Yes, please review.

Make key optional in ipex.llm.functional.rotary_embedding

format, also wondering how do we test here?

[Feature]: torch compile for llama4 unsloth

You can try to remove the torch.compile cache and see if it causes any difference. Or try VLLM_DISABLE_COMPILE_CACHE=1 to disable torch compile cache. Likely it's not due torch.compile, but another...

Add `pt_load_map_location` to allow loading to cuda

Could you check if the failed test is related or not? Like if fails without PR or not locally?

Add SM120 to the Dockerfile

What's the new wheel size? :-)

[Feature]: Context Parallelism

We will try to pick this up.