sglang icon indicating copy to clipboard operation
sglang copied to clipboard

[Bug] `served_model_name` arguement in the server_arg.py is useless

Open zhaochenyang20 opened this issue 5 months ago • 3 comments

Checklist

  • [X] 1. I have searched related issues but cannot get the expected help.
  • [X] 2. The bug has not been fixed in the latest version.
  • [X] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
  • [X] 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
  • [X] 5. Please use English, otherwise it will be closed.

Describe the bug

No matter how you set this argument while initializing this parameter, you can pass in anything in the model arguments when sending requests.

python -m sglang.launch_server --model-path Alibaba-NLP/gte-Qwen2-7B-instruct \
--port 30000 --api-key sk-None-hello-world --host 0.0.0.0 --is-embedding --served-model-name whatever
curl http://localhost:30000/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-None-hello-world" \
  -d '{
    "model": "Nothing",
    "input": "Once upon a time"
  }'

Reproduction

Easy to reproduce.

Environment

Easy to reproduce and it's just a software-level bug.

zhaochenyang20 avatar Sep 08 '24 05:09 zhaochenyang20