Varun Gupta comments

Results 87 comments of


                                            Varun Gupta

Documentation is not clearly defined on how to set the RateLimiting and how to measure the token consumption and how to enable the authentication for different users

@vivekrsintc Right now features you have listed are not present. Before implementation, I need more understanding of the requirement. I could not find on [aibrix slack channel](https://vllm-dev.slack.com/archives/C08EQ883CSV). Can you ping...

v0.4.1: fatal error: concurrent map writes in prefix cache eviction

@gangmuk PTAL

In PD separation scenarios, model access requests are directed to non-master nodes.

Can you describe the DO and D1, and share the label key/values. I want to check the value of pod-group-index label.

In PD separation scenarios, model access requests are directed to non-master nodes.

It is missing pod-group-index label, can you use latest release or main branch for controller.

In PD separation scenarios, model access requests are directed to non-master nodes.

@wangchuanfang @ying2025 Above scenario has TP=16, and nnodes=2. This is not a 2P2D scenario, but a 1P1D scenario. For router to identify master prefill and decode nodes, pod group size...

In PD separation scenarios, model access requests are directed to non-master nodes.

@XiaobinZhao Is the issue fixed now?

stormservice deploy use mutiple node

@ying2025 Is the issue still relevant?