Peter Pan

exactly, me too. it happens now and then. We suffered from it very well.... ``` Cloning repository https://github.com/RackHD/on-core.git > git init /var/lib/jenkins/jobs/on-core/workspace/build # timeout=10 Fetching upstream changes from https://github.com/RackHD/on-core.git >...

although it works in theory, but actually, there will be flood of StorageClass which are hard to be managed.. and each time , team 1 want to share/public something ,...

a reference below, to help us to understand the situation better ( as well, below states an implementation by Alluxio/Fluid) https://www.alibabacloud.com/blog/fluid-sharing-dataset-across-kubernetes-namespaces_599739

> then you could use a root directory in storage class, e.g. `server: 10.20.20.1:/nfs/` instead of `server: 10.20.20.1:/nfs/team-1` it breaks the isolation rule between teams (say, Team-3 will access all...

seems V0 not working now ``` cd vllm python3 examples/others/lmcache/cpu_offload_lmcache.py -v v0 ``` ``` ERROR 07-20 07:14:23 [registry.py:389] Error in inspecting model architecture 'MistralForCausalLM' ... ERROR 07-20 07:14:23 [registry.py:389] File...

> Thank you @panpan0000 for pushing the PR. > > The Dockerfile now use `buildarg` for `vllm-openai` repo and tag. See https://github.com/LMCache/LMCache/blob/dev/docker/Dockerfile#L8-L9 for more details. > > As a consequence,...

I see, very clear. thank you @hickeyma . By the way , very good job for your CICD work for LMCache.

其实最典型的场景还是公有云。 公有云的VM的IP都是私网，全局管理的reverse-proxy需要是公网ELB的IP。 有2种思路： （1）安装器传入新参数为：根据公有云类型switch-case。不同的公有云的做法不同。比如华为是给LB类型svc加annotation（预先建好ELB pool），之后k8s get svc就能拿到ELB IP了。 （2）安装器传入新参数为：上文所谓的NAT地址，或者事先建好的ELB IP。之后把这个IP写入全局管理的reverse-proxy 列入本月的spring task中

linked to internal tracker : https://gitlab.daocloud.cn/ndx/engineering/infrastructure/dce5-installer/-/issues/298

we also see 3 token/s on 16x H20 with TP=8,PP=2