How to load a model offline

[fawpcmhgung162]

When I execute openllm start for the first time, the model will be downloaded to the local and then started, but it seems that it will also send a network request to huggingface (although it will not be downloaded again) before it can be started.

I tried setting HF_DATASETS_OFFLINE=1 TRANSFORMERS_OFFLINE=1 but it didn't work

[zhangxinyang97]

same question here, need a fully local mode

[aarnphm]

In 0.5, you can save it to bento model store first with

with bentoml.models.create("your-model") as model:
  ...

then you can start with openllm start your-model.