Simo Lin
Simo Lin
/tag-and-rerun-ci
lgtm, @key4ng please also take a look
> Hi @slin1237, can I pick up this task? 100% Thank you so much. I should be able to finish task 4 today so u can use it for testing
I don't think PV and pvc support for base models are properly implemented. I will address that Thanks for raising this up
depending on the vendor out of the box, it uses k8s round robin, in our runtimes we use sgl router for better load balancing as well as for pd load...
thanks for the contribution, please fix the CI
most of these CI commands are in make file ``` make tidy; make fmt; make vet; make test ``` you can verify those changes locally
GenAI-bench requires tokenizer That tokenizer is currently only available in GPU nodes the problem is probably how we handled getting the tokenizer benchmark controller