Woosuk Kwon

icon indicating a website link https://woosuk.me icon indicating an email [email protected]

icon company University of California, Berkeley icon location Berkeley, CA icon location CS PhD student at UC Berkeley, building @vllm-project

Results 284 comments of Woosuk Kwon

[v1] Support allowed_token_ids in v1 Sampler

cc @njhill Could you please take a look?

[Bug]: vLLM crashes with larger context sizes on TPUs

This must be fixed by #9350.

Support non-attention path operators in Triton

Hi @JackChuang, thanks for the PR. I think it's a cool idea, and we do want to reduce the CUDA kernels in our repo. However, our solution to this is...

[Minor] Add benchmark for activation layer

Hi @esmeetu, thanks for the PR! Could you elaborate more on the benefit of this change? I think we can keep the current version for better compatibility with HF, if...

[Minor] Add benchmark for activation layer

> I found that nn.GELU(approximate="tanh") is identical to NewGELU() Could you provide a reference to this?

[Kernel] Add ModelOpt FP4 Checkpoint Support

@mgoin thanks for your review. can you please take another look?

[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs

@robertgshaw2-redhat Can you please take a look?

[Feature][Hardware][AMD] Enable Scaled FP8 GEMM on ROCm

> Going to summarize comments into a single PR @robertgshaw2-neuralmagic Is there any update?

[Sampler] Adapt to FlashInfer 0.2.3 sampler API

@mgoin +1. Let's update this. @abmfy Sorry for the late review. Could you please add an accuracy test for the new kernel?

[Kernel] [Triton] [AMD] Add Triton implementation of awq_gemm

@robertgshaw2-neuralmagic @tlrmchlsmth could you please take a look?