Woosuk Kwon

icon indicating a website link https://woosuk.me icon indicating an email [email protected]

icon company University of California, Berkeley icon location Berkeley, CA icon location CS PhD student at UC Berkeley, building @vllm-project

Results 284 comments of Woosuk Kwon

[V1][Spec Decode] Apply torch.compile & cudagraph to EAGLE3

@luyuzhe111 @zixi-qi I've updated the PR. Thanks for catching the bug! Will merge once it passes the CI.

[Bugfix][V1][Spec Dec] Add generator to request even when no seed is provided.

Closed by #17929

[Bug]: [V1][Core] Structured decoding - FSM update and matching blocked by request status

Hi @rishitdholakia13, thanks for reporting the bug. V1 doesn't support structured decoding yet: We are actively working on it at https://github.com/vllm-project/vllm/pull/12388, which will hopefully be merged by EoW.

[Bug]: [V1][Core] Structured decoding - FSM update and matching blocked by request status

@rishitdholakia13 Please add a comment directly to the PR. @russellb and @mgoin took over the PR, so they should know about it.