sglang
sglang copied to clipboard
Switch to non gated models
When running
python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000
I get
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json.
Access to model meta-llama/Llama-2-7b-chat-hf is restricted. You must be authenticated to access it.
I assume it's the same for most people. How about switching it to a non gated model, e.g. Mistral or a quantized 8B Llama 3?
Mistral is also gated.