vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[BugFix] get_and_reset only when scheduler outputs are not empty

Open mzusman opened this issue 7 months ago • 2 comments

Following #4115 , Thanks to https://github.com/vllm-project/vllm/pull/4115#discussion_r1670888574 , Noticed that scheduler.finished_requests_ids could reset without sending the finished requests ids back to the workers. This PR resets the finished_requests_ids only when the scheduler outputs are not empty.

mzusman avatar Jul 09 '24 18:07 mzusman