Roger Wang comments

Results 132 comments of


                                            Roger Wang

uv pip install failes for wheel URLs with placeholder

A few of us from the vLLM project are addressing this issue so that you can use `uv` to install nightly wheels - See https://github.com/vllm-project/vllm/pull/11404

[Feat] Support XPU Backend in vLLM-Omni

@faaany Hey thanks for your contribution! I just want to emphasize that the project is still in a fast-iterating and developing phase and IMO it's not in a shape where...

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

Just an update to the thread - sorry for the delay but this bug hasn't been fixed yet but we plan to get it fixed by the end of Q3!

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

@Chen-Song Unfortunately we haven't got to fix this due to our bandwidth - but we already have a tracking PR for it https://github.com/vllm-project/vllm/pull/16229

[Bug]: When using the VLLM framework to load visual models, CPU memory overflow occurs while continuously processing data with images.

@woshiwanlei1 Can you try specifying `disable_mm_preprocessor_cache=True` and see if the host memory overflow issue still persists?

Roger Wang

uv pip install failes for wheel URLs with placeholder

[Feat] Support XPU Backend in vLLM-Omni

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

[Bug]: qwen2-vl 7b, on vllm 0.8.1 & 0.8.2, sometimes (not deterministically but depends on data) I got: ValueError: Attempted to assign 702 = 702 multimodal tokens to 703 placeholders

[RFC]: Sparse KV cache management framework

[Misc] Separate hf dataset sampling function from benchmark_serving.py

[Chore] Bump Python version and apply formatting changes for v0.12 release

[Bug]: vllm 0.6.3 generates incomplete/repeated answers for long length (over 8k) input

[RFC]: Offload KV cache to CPU in V1

[Bug]: When using the VLLM framework to load visual models, CPU memory overflow occurs while continuously processing data with images.