Qwen2.5
Qwen2.5 copied to clipboard
[Bug]: 使用vllm进行推理时,设置parallel_tool_calls似乎不生效。我想实现单工具调用,应该怎么设置呢
Model Series
Qwen2.5
What are the models used?
Qwen2.5-72B-Instruction
What is the scenario where the problem happened?
vllm
Is this a known issue?
- [X] I have followed the GitHub README.
- [X] I have checked the Qwen documentation and cannot find an answer there.
- [X] I have checked the documentation of the related framework and cannot find useful information.
- [X] I have searched the issues and there is not a similar one.
Information about environment
vllm>0.0.0
Log output
curl --location --request POST '***' \
--header 'User-Agent: Apifox/1.0.0 (https://apifox.com)' \
--header 'Content-Type: application/json' \
--data-raw '{
"model": "Qwen2.5",
"stream": true,
"parallel_function_calls":false,
"messages": [
{
"role": "user",
"content": "查一下西安和北京的天气"
}
],
"stream_options":{"include_usage": true},
"tools": [
{
"type": "function",
"function": {
"name": "查天气",
"description": "根据城市名查询天气",
"parameters": {
"properties": {
"city": {
"type": "string",
"description": "城市名"
}
},
"type": "object"
}
}
}
]
}'
I got 2 call rather than 1.
Description
Steps to reproduce
This happens to Qwen2.5-72B-Instruct
The problem can be reproduced with the following steps:
curl --location --request POST '*****'
--header 'User-Agent: Apifox/1.0.0 (https://apifox.com)'
--header 'Content-Type: application/json'
--data-raw '{
"model": "Qwen2.5",
"stream": true,
"parallel_function_calls":false,
"messages": [
{
"role": "user",
"content": "查一下西安和北京的天气"
}
],
"stream_options":{"include_usage": true},
"tools": [
{
"type": "function",
"function": {
"name": "查天气",
"description": "根据城市名查询天气",
"parameters": {
"properties": {
"city": {
"type": "string",
"description": "城市名"
}
},
"type": "object"
}
}
}
]
}'
Expected results
The results are expected to be call one tools
Attempts to fix
I have tried several ways to fix this, including:
- make parallel_tool_calls usable