sglang icon indicating copy to clipboard operation
sglang copied to clipboard

Switch to non gated models

Open tom-doerr opened this issue 10 months ago • 1 comments

When running

python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000

I get

Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
Access to model meta-llama/Llama-2-7b-chat-hf is restricted. You must be authenticated to access it.

I assume it's the same for most people. How about switching it to a non gated model, e.g. Mistral or a quantized 8B Llama 3?

tom-doerr avatar Apr 23 '24 16:04 tom-doerr

Mistral is also gated.

image

KMouratidis avatar May 03 '24 10:05 KMouratidis