Xingkai Yu comments

Results 9 comments of


                                            Xingkai Yu

NoneType check

The modification is not need because `i in range(self.experts_start_idx, self.experts_end_idx)`.

A potential problem with GPU memory allocation

Yes, this could indeed cause problems. A more robust approach would be to: 1. Calculate the maximum number of blocks each GPU can allocate independently based on its own available...

Implement support for Qwen2.5vl

Thank you for your interest in extending nano-vllm! As nano-vllm is primarily designed for educational purposes to demonstrate the core concepts of LLM serving, we're currently keeping the scope limited...

ERROR while installing with pip

Thanks for reporting this issue. I suspect you might be trying to install triton on Windows which has limited platform support. This is a known issue that's been discussed in...

rope argument inconsistent

You're right, the Qwen3 model doesn't implement RoPE scaling, so we `asserts rope_scaling is None`. The parameter was included in `Qwen3Attention` for potential future use. For YaRN or other scaling...

fix: modify input when input is fp32

@philhuan Your observation is valid. The project implicitly restricts dtype to bfloat16 or float16 due to Flash Attention constraints. But your PR is correct overall. One small suggestion: could you...

fix: recompute scheduled tokens after prefix-cache allocation

Thank your advice. We fixed it in newest PR #218

Xingkai Yu

Test