aibrix
aibrix copied to clipboard
jinja2.exceptions.UndefinedError: 'bos_token' is undefined
🐛 Describe the bug
Steps to Reproduce
deploy the models
vllm serve Qwen/Qwen2.5-Coder-7B-Instruct --enable-lora --lora-modules model-1=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-2=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-3=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-4=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c --max-lora-rank 64
send the request
curl -v "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "model-1",
"messages": [{"role": "user", "content": "help me run a python randome generator method"}],
"temperature": 0.7
}'
Expected behavior
No response
Environment
No response
I change to absolute path and it's still not working. the lora adapter issue