vllm-project
Error: Invalid character 'u' looking for beginning of value
🐛 Describe the bug
We deployed latest stable version of AIBRIX and the following error is coming
curl -v http://localhost:8888/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "deepseek-r1-distill-llama-8b", "messages": [{"role": "user", "conten t": "Say this is a test!"}], "temperature": 0.7}
"error":{"code":500,"message":"invalid character 'u' looking for beginning of value"
Steps to Reproduce
- Deploy default AIBRIX
- Deploy the Deepseek v1 Llama 8B model on CPU
- Port-foward the envoygateway
- curl -v http://localhost:8888/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "deepseek-r1-distill-llama-8b", "messages": [{"role": "user", "conten t": "Say this is a test!"}], "temperature": 0.7}
You will end up with the above error
Expected behavior
Expected behavior proper result to come
Environment
- AIBrix latest
@vivekrsintc Thanks for reporting the issue. We've fixed a few issues and now it exposes the accurate messages. I think only one case now will return such error which is missing service for the deployment. Could you help check the pod status and whether you created the service?
My POD Status were fine. Gateway plugin service binded properly with Gateway POD IP. But even though we are getting this error
@vivekrsintc Can you share the output of kubectl describe httproute -A and kubectl describe envoyextensionpolicy -A
They both look good.
I could not find you on aibrix slack channel, can you ping me on slack.
@varungup90, how can I join the Aibrix Slack workspace? (Here is my email: [email protected]; I'd be grateful for an invitation or something similar.)
I faced same error when i call v1/models But when i call v1/chat/compilations its work i use 0.2.1v stable so how to fix this can anyone help me
@MohamedHAlzhrani could you try v0.3.0 instead? This issue has been resolved.
I will close this task, please create a new one for current release v0.3.0.