vllm
vllm copied to clipboard
vllm-project
[Bug]: 400 Bad Request
Your current environment
使用docker run --gpus all -v D:\docker\huggingface:/root/.cache/huggingface --env "HUGGING_FACE_HUB_TOKEN=****************" -d -p 8000:8000 --ipc=host --name vllm vllm/vllm-openai:latest --model nvidia/Llama3-ChatQA-1.5-8B --api-key vllm-abc123456 --gpu-memory-utilization 0.9部署成功了server端
客户端运行下面的代码报错： from openai import OpenAI
Set OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "vllm-abc123456" openai_api_base = "http://192.168.2.6:8000/v1"
client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, )
chat_response = client.chat.completions.create( model="nvidia/Llama3-ChatQA-1.5-8B", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me a joke."}, ], extra_body = { "guided_choice": ["positive", "negative"] } ).choices[0].message.content print("Chat response:", chat_response)
错误：
D:\docker\vllm\venv\Scripts\python.exe D:\docker\vllm\llm_client.py
Traceback (most recent call last):
File "D:\docker\vllm\llm_client.py", line 14, in Your browser sent a request that this server could not understand. D:\docker\vllm\venv\Scripts\python.exe D:\docker\vllm\llm_client.py
Traceback (most recent call last):
File "D:\docker\vllm\llm_client.py", line 14, in Your browser sent a request that this server could not understand.Bad Request
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
🐛 Describe the bug
Bad Request
Reason: You're speaking plain HTTP to an SSL-enabled server port.
Instead use the HTTPS scheme to access this URL, please.
Have you tried using HTTPS instead of HTTP according to the error message?
- openai_api_base = "http://192.168.2.6:8000/v1"
+ openai_api_base = "https://192.168.2.6:8000/v1"
Have you tried using HTTPS instead of HTTP according to the error message?
- openai_api_base = "http://192.168.2.6:8000/v1" + openai_api_base = "https://192.168.2.6:8000/v1"
我是本地部署的，没有https只有http
I'm seeing a 400 bad request with 0.4.2 (vllm.entrypoints.openai.api_server) that does not occur with 0.4.1 with nothing else changing.... It's strange because if I submit my normal test CURL request, it works fine, but my application using axios returns the 400... Reverting to 0.4.1 and everything works. This has to be a regression.
The issue I mentioned above still occurs with the latest, 0.5.0-post1. I'm stuck on 0.4.1 to have a functioning openai api server.
I'm seeing a 400 bad request with 0.4.2 (vllm.entrypoints.openai.api_server) that does not occur with 0.4.1 with nothing else changing.... It's strange because if I submit my normal test CURL request, it works fine, but my application using axios returns the 400... Reverting to 0.4.1 and everything works. This has to be a regression.
I encountered this issue as well. I think it's caused by the current default output being set to streaming. As long as you set 'stream': false, you should be able to get the correct response.
In my case, it was items in the messages array containing extra fields... date, and _id... This is the core of the breaking change that occurred after 0.4.1. The API is extremely rigid now in validating the input. I've worked around it in my app by stripping out the fields before submission, but not excited about that.
In my case, it was items in the messages array containing extra fields... date, and _id... This is the core of the breaking change that occurred after 0.4.1. The API is extremely rigid now in validating the input. I've worked around it in my app by stripping out the fields before submission, but not excited about that.
This was an intentional change in response to #4339. Sorry for the late reply!
same error , please fix
Please provide more information about this.