Harry Mellor

icon company @huggingface @vllm-project icon location ML Engineer @huggingface

Results 298 comments of Harry Mellor

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

@BiEchi do you still see thisn issue with the latest vLLM?

[Bug]: Gemma-3 (27B) can't load save_pretrained() checkpoint: AssertionError: expected size 5376==2560, stride 1==1 at dim=0

Thanks for confirming, I'll close this as complete.

[Bug]: Non-coherent output from DeepSeek-R1 671B on H200 SXM

Can you please tell us the commit hash of the snapshot of R1 you have downloaded? DeepSeek have been making small changes to the `json` files so it'd be good...

[Bug]: Can't run offline inference (example script) in OpenVINO: TypeError: OpenVINOCausalLM.forward() missing 2 required positional arguments: 'kv_caches' and 'attn_metadata'

@dernosmirc I can't seem to install OpenVINO on my dev machine because it has CUDA devices in it. Can you confirm if #14271 solves the issue?

[Bug]: Can't run offline inference (example script) in OpenVINO: TypeError: OpenVINOCausalLM.forward() missing 2 required positional arguments: 'kv_caches' and 'attn_metadata'

I know I should be able to, but I get: ``` RuntimeError: Only one platform plugin can be activated, but got: ['openvino', 'cuda'] ``` When I try and run anything.

[Bug]: Can't run offline inference (example script) in OpenVINO: TypeError: OpenVINOCausalLM.forward() missing 2 required positional arguments: 'kv_caches' and 'attn_metadata'

Sorry it was probably too early to ask you to try it. The error you saw was legitimate. If you have time to test the latest version of that PR...

Support FP8 Quantization and Inference Run on Intel Gaudi (HPU) using INC (Intel Neural Compressor)

Looks good from a docs persoective now https://vllm--12010.org.readthedocs.build/en/12010/features/quantization/inc.html

[Bug]:Lora Adapters with num-scheduler-steps doesn't work in version 0.7.2, even with VLLM_USE_V1=0

Does it work with `VLLM_WORKER_MULTIPROC_METHOD=fork`?

[Kernel][Model] PagedAttention: Support custom attention bias for T5 model (1/2)

It ultimately came down to architectural complexity in vLLM. Whisper is still supported and multimodal models that do not use cross attention are still supported.

[Model] Support MiMo-7B inference with MTP

FYI, this model already works with vLLM ```console vllm serve XiaomiMiMo/MiMo-7B-Base --model-impl transformers --trust-remote-code ```