aibrix icon indicating copy to clipboard operation
aibrix copied to clipboard

jinja2.exceptions.UndefinedError: 'bos_token' is undefined

Open Jeffwan opened this issue 11 months ago • 1 comments

🐛 Describe the bug

Image

Steps to Reproduce

deploy the models

vllm serve Qwen/Qwen2.5-Coder-7B-Instruct     --enable-lora     --lora-modules model-1=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-2=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-3=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c model-4=VERSIL91/10627788-942b-4b44-b5f5-167c4b543f2c     --max-lora-rank 64

send the request

curl -v "http://localhost:8000/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
     "model": "model-1",
     "messages": [{"role": "user", "content": "help me run a python randome generator method"}],
     "temperature": 0.7
   }'

Expected behavior

No response

Environment

No response

Jeffwan avatar Jan 22 '25 22:01 Jeffwan

Image

I change to absolute path and it's still not working. the lora adapter issue

Jeffwan avatar Jan 22 '25 22:01 Jeffwan