Nitin Kedia

Hi @yipengLeo, there is indeed a bug here. However, the impact is relatively little as the `add_time` is a small percentage of the time. For all the models supported right...

Hi @nba556677go, There is a `vidur` branch in [sarathi-serve](https://github.com/microsoft/sarathi-serve) repo. That would be closest baseline for Vidur. Next closest is the `main` branch. vLLM has undergone tremendous amount of development...

Hi @mayuqing111, it is wonderful to know that Vidur is providing great value in your work. It is indeed feasible to support dynamic replica count adjustment. We don't have plans...