Woosuk Kwon
Woosuk Kwon
> [rank0]: File "/data/woosuk/workspace/vllm/vllm/engine/output_processor/multi_step.py", line 88, in process_outputs > [rank0]: assert valid_samples @SolitaryThinker Huge thanks for the PR! QQ: I got the above error when running benchmark scripts with num_scheduler_steps...
> I have been testing this against latest pytorch nightly @drisspg Is torch nightly required? I'm now seeing reasonable outputs with torch v2.7.0.
@drisspg Could you please check the failed CI tests and rebase the PR? Will merge once the CI gets green. :)
cc @hongxiayang @mawong-amd Could you please take a look?
Hmm this makes `v1-test` much longer: from 20 mins to 2.5 hours (timeout).
@hongxiayang @mawong-amd Are we ready to upgrade the ROCm and Ubuntu versions?
Hi @bigPYJ1151, can you please rebase the PR and resolve merge conflicts?
@ruisearch42 @comaniac @youkaichao Can you please take a final look by any chance?
@bigPYJ1151 I've just started the CI test. Will merge once it becomes green.
Hi @bao231, V1 does not support T4 or older-generation GPUs since the kernel libraries used in V1 (e.g., flash-attn) do not support them.