Can the basemodel be loaded directly from the local system without connecting to Hugging Face? It cannot be launched because there is no connection.

[Loovelj]

Model description

model=Qwen/Qwen-7B-Chat volume=/hub_models/Qwen-7B-Chat/ docker run --gpus=1 --shm-size 1g -p 8080:80 -v $volume:/data \ ghcr.nju.edu.cn/predibase/lorax:latest --model-id $model

if load the local base model directly will be very well

Open source status

• [ ] The model implementation is available
• [ ] The model weights are available

Provide useful links for the implementation

No response

[jeffreyftang]

Hi @Loovelj, can you provide the exact error message(s) you're running into?

It also looks like you're setting $model to be a HF-like model name. Can you try setting it to an absolute local path and seeing if that helps?

[Loovelj]

Referen It didn't work, model must repo_name huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '

[nirvitarka]

I am facing the same situation, need to load model from local directory. Is any solution found for this?