lkchen comments

Results 12 comments of


                                            lkchen

[Misc] Add type alias `ReqId` and `EngineId` for better readability

`distributed-tests-4-gpus` already failing on [nightly build](https://buildkite.com/vllm/ci/builds/22323) and unrelated to this PR @njhill could you take a quick review?

[V1][Help Wanted] Porting missing sampling parameters to V1

As #13376 is merged, I think all TODOs are done, and this issue can be closed?

WIP [P/D] Use ThreadPoolExecutor to do handshake for each P-D pair

close in favor of #19836

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

`distributed-tests-4-gpus` already failing on [nightly build](https://buildkite.com/vllm/ci/builds/22323) `v1-test` already failing on [main](https://buildkite.com/vllm/ci/builds/22358#_) This PR should be mergable @njhill

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

@njhill v1-test passed https://buildkite.com/vllm/ci/builds/22459

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

Hi @NickLucche @wseaton and I both observed it [as mentioned in slack](https://vllm-dev.slack.com/archives/C08MCDXAS8Y/p1750352335005179?thread_ts=1750198629.534519&cid=C08MCDXAS8Y) I don't really know a stable setup that can reproduce it in real run, but only mimic it...

lkchen

[Misc] Add type alias `ReqId` and `EngineId` for better readability

[V1][Help Wanted] Porting missing sampling parameters to V1

WIP [P/D] Use ThreadPoolExecutor to do handshake for each P-D pair

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

[BugFix][P/D] Fix for cases where _recving_transfers can be cleaned up when all transfer done

[Bug]: Vllm 0.8.2 + Ray 2.44 (Ray serve deployment) fallbacks to V0 Engine

[Bug]: Vllm 0.8.2 + Ray 2.44 (Ray serve deployment) fallbacks to V0 Engine

[ V0 ][ sample ] improve sample performance when using guide decoding