[SUPPORT] how to skip huggingface model download and directly use a local model

[XiaobinZhao]

My environment does not have access to the external network，Of course, cannot access huggingface.

Question

1. How to skip huggingface model download and directly use a local model？
2. How to delete resources without deleting existing model files (kubectl delete-f ./config/models/moonshotai/Kimi-K2-Instruct.yaml)
3. When deleting the resource ClusterBaseModel, it gets stuck and cannot be deleted

apiVersion: ome.io/v1beta1
kind: ClusterBaseModel
metadata:
  name: kimi-k2-instruct
spec:
  vendor: moonshotai
  modelFormat:
    name: safetensors
  disabled: false
  version: "1.0.0"
  storage:
    storageUri: hf://moonshotai/Kimi-K2-Instruct-0905
    path: /llm/ome-model

What did you try?

git clone https://github.com/sgl-project/ome
# Step 1: Install OME CRDs
helm upgrade --install ome-crd ./charts/ome-crd --namespace ome
# Step 2: Install OME core resources
helm upgrade --install ome ./charts/ome-resources --namespace ome
# 注册kimi k2
kubectl apply -f ./config/models/moonshotai/Kimi-K2-Instruct.yaml
# 安装 Kimi K2
kubectl apply -f ./config/runtimes/srt/kimi-k2-pd-rt.yaml
# 部署
kubectl apply -f ./config/samples/isvc/moonshotai/kimi-k2-pd.yaml

Environment

• OME version: ome main(commit 906589b2cd2dc6f0f0b0f7635aa4eac03252e5fe Mon Oct 27 12:20:04 2025) /model-agent:v0.1.3 / ome-manager:v0.1.3
• Kubernetes version: v1.26.11
• Runtime being used (SGLang etc.): sglang-0.4.10
• Model being served (if applicable): moonshotai/kimi-k2-instruct-0905

Additional context

My environment does not have access to the external network，Of course, cannot access huggingface

@pallasathena92 could you help me ?

[pallasathena92]

for question 1: https://github.com/sgl-project/ome/blob/main/pkg/modelagent/gopher.go#L382 we supported local storage path. In your helm chart, you can update storageUri: hf://moonshotai/Kimi-K2-Instruct-0905 to "local://"

for question 2: don’t quite follow. Could you clarify what you mean?

for question 3: do you have an active inference service using this model now?

[XiaobinZhao]

@pallasathena92 i fllow the article: Deploying Kimi K2 with PD Disaggregation and Large-Scale Expert Parallelism on 128 H200 GPUs

1. I saw the 'local' case in the code, but it's on the master branch and hasn't been released yet. I'll give it a try.
2. when i delete the ClusterBaseModel with the command kubectl delete -f ./config/models/moonshotai/Kimi-K2-Instruct.yaml , the model file will be deleted; I don't want the model file to be deleted.

apiVersion: ome.io/v1beta1
kind: ClusterBaseModel
metadata:
  name: kimi-k2-instruct
spec:
  vendor: moonshotai
  modelFormat:
    name: safetensors
  disabled: false
  version: "1.0.0"
  storage:
    storageUri: hf://moonshotai/Kimi-K2-Instruct-0905
    path: /llm/moonshotai/Kimi-K2-Instruct-0905/

3. no active inference service using this model; i just want to clear all the resource/instances of ome; but the ClusterBaseModel cant be deleted even after i uninstall the ome/ome-crd.