SYing
SYing
> My guess is they are caused by the proxy service. Could you access code-server directly to confirm? > > I do not believe there is a list of non-core...
> Those are static resources so should be no issue in mirroring them via CDN. Why is the loading speed of the `static/out/vs/workbench/workbench.web.main.js` file slower when hosted on a content...
> That might be a question better asked to your CDN provider. It could be any number of reasons, but ultimately it sounds like your connection to wherever code-server is...
I think multi cluster unified routing, and multi cluster PodAutoscaler also need to support.
> [@ying2025](https://github.com/ying2025) on using routing-strategy, a target pod is selected and does not use httproute for request forwarding. For now, can you try request without routing-strategy, it will use the...
@varungup90 Otherwise, I set envoy circuit-breakers. `https://www.envoyproxy.io/docs/envoy/latest/configuration/upstream/cluster_manager/cluster_stats#circuit-breakers-statistics`, and the configs is as follows.It also appears to have not taken effect. ``` "circuit_breakers": { "thresholds": [ { "max_connections": 50, "max_pending_requests": 100,...
I change the httproute as follow, the target pod port is fixed by `https://github.com/vllm-project/aibrix/blob/main/pkg/types/router_context.go#L96`. I think it should firstly judge the pod label `model.aibrix.ai/port`, if not set the default port.Otherwise...
> [@ying2025](https://github.com/ying2025) the orchestration part is for launching instances in scale easily and it won't offer inference performance gains. Performance gains mainly comes from other features like routing policies. I...
@Jeffwan At present, there is a 6-card A100, with plans to add 1-2 4090 or A30 or 4-6 K80 and some T4,K40 machines. The vllm version is v0.7.2.Otherwise, Provide a...
@Jeffwan I am looking forward to these perf benchmark as soon as possible, and also hope to have routing strategies to improve the performance gains in single machine and multi...