Jiaxin Shan comments

Results 742 comments of


                                            Jiaxin Shan

[RFC] Support different inference engines like vLLM, SGLang, TensorRT-LLM

@Belyenochi We build storm service to support advanced orchestration scenario and provide production grade rolling upgrade and roll out strategies. We plan **not** to use LWS for such cases. If...

aibrix cant find correct router

> If I understand correctly, the pd routing-strategy no longer uses the http route crd for routing. 🤔 > Is there more info? cc @Jeffwan yes, that's true, if we...

aibrix cant find correct router

@googs1025 We can append the podSet_NAME in podGroupSize != 1 case. This is the compatibility issue. I suggest to have a fix.

aibrix cant find correct router

@fungaren what's your stormservice version?

v0.4.1: fatal error: concurrent map writes in prefix cache eviction

/cc @varungup90

Scope Kubeflow components in given namespace

> https://github.com/kubeflow/pipelines/blob/ec721fe94dbcaa054b1057e5503e4f9823fdf2a5/manifests/kustomize/base/cache-deployer/cluster-scoped/cache-deployer-clusterrole.yaml#L21 > > is a cluster-scoped resource and RBAC for cluster-scoped resource can only be granted via `ClusterRole`: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#role-and-clusterrole. > That was the reason we included those I see....

Jiaxin Shan

[RFC] Support different inference engines like vLLM, SGLang, TensorRT-LLM

aibrix cant find correct router

aibrix cant find correct router

aibrix cant find correct router

v0.4.1: fatal error: concurrent map writes in prefix cache eviction

Scope Kubeflow components in given namespace

Scope Kubeflow components in given namespace

Scope Kubeflow components in given namespace

In PD separation scenarios, model access requests are directed to non-master nodes.

In PD separation scenarios, model access requests are directed to non-master nodes.