Simon Mo
Simon Mo
This sounds good. Two comment: * I created a `keep-open` label. However, not everyone has the ability to attach label. Can this be done by some other way? * We...
@cakeng can you merge the latest main branch to pick up some fixes for test failures? Thank you!
@cakeng a test failure in CI https://buildkite.com/vllm/ci/builds/14038#019534e8-1eef-4be1-8e89-3f718502ddb4/2582-4097 ``` [2025-02-23T23:32:34Z] FAILED models/test_initialization.py::test_can_initialize[QuantMixtralForCausalLM] - AttributeError: 'FusedMoE' object has no attribute 'num_experts' ```
Sorry I believe the `other` argument changes the return value of tl.load? With a mask, if other is None, the returned value if all elements where mask is true. If...
Can you demonstrate this issue with a sample program?
Hi @gauravkr2108, sorry about the delay, please address @Yard1's comment.
This sounds great! In general in vLLM we want to ensure we are compatible with open format for load balancing and observability. This helps people actually run vLLM in production....
This will be ready to merge once example JSON is posted. ideally on two different TPU machine.
Added your email to our buildkite org.
Can you add an example in the documentation for such plugin? so we have some examples people can leverage and knowing how to use it.