Justin
Justin
> [@justinSmileDate](https://github.com/justinSmileDate) Thanks for the interest. By default we take advantage of the k8s service to distribute the inference traffic. Additionally we do have a more complicated design of and...
> depending on the vendor out of the box, it uses k8s round robin, in our runtimes we use sgl router for better load balancing as well as for pd...