nano-vllm icon indicating copy to clipboard operation
nano-vllm copied to clipboard

Fix bug in block manager's may_append

Open yue-zhang-2025 opened this issue 5 months ago • 1 comments

Hi nano-vllm team, this is Yue, nice to meet you, I'm just a fan of this repo and learning it! As I learn the code of block_manager, IIUC the current implementation of can_append and may_append is assuming seq already includes the upcoming token_id - this seems a bug to me as the upcoming token_id will be added in scheduler's postprocess which comes after the schedule(). I saw an existed PR fix the can_append() function, I'm creating this PR to fix the may_append() function.

I might be wrong if I miss any context, please correct me if I'm wrong :) Thanks for review!

yue-zhang-2025 avatar Jul 03 '25 07:07 yue-zhang-2025