Qwen2.5 icon indicating copy to clipboard operation
Qwen2.5 copied to clipboard

[Bug]: 使用vllm进行推理时,设置parallel_tool_calls似乎不生效。我想实现单工具调用,应该怎么设置呢

Open RayneSun opened this issue 3 months ago • 5 comments

Model Series

Qwen2.5

What are the models used?

Qwen2.5-72B-Instruction

What is the scenario where the problem happened?

vllm

Is this a known issue?

  • [X] I have followed the GitHub README.
  • [X] I have checked the Qwen documentation and cannot find an answer there.
  • [X] I have checked the documentation of the related framework and cannot find useful information.
  • [X] I have searched the issues and there is not a similar one.

Information about environment

vllm>0.0.0

Log output

curl --location --request POST '***' \
--header 'User-Agent: Apifox/1.0.0 (https://apifox.com)' \
--header 'Content-Type: application/json' \
--data-raw '{
  "model": "Qwen2.5",
  "stream": true,
  "parallel_function_calls":false,
  "messages": [
    {
      "role": "user",
      "content": "查一下西安和北京的天气"
    }
  ],
  "stream_options":{"include_usage": true},
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "查天气",
        "description": "根据城市名查询天气",
        "parameters": {
          "properties": {
            "city": {
              "type": "string",
              "description": "城市名"
            }
          },
          "type": "object"
        }
      }
    }
  ]
}'

I got 2 call rather than 1.

Description

Steps to reproduce

This happens to Qwen2.5-72B-Instruct The problem can be reproduced with the following steps: curl --location --request POST '*****'
--header 'User-Agent: Apifox/1.0.0 (https://apifox.com)'
--header 'Content-Type: application/json'
--data-raw '{ "model": "Qwen2.5", "stream": true, "parallel_function_calls":false, "messages": [ { "role": "user", "content": "查一下西安和北京的天气" } ], "stream_options":{"include_usage": true}, "tools": [ { "type": "function", "function": { "name": "查天气", "description": "根据城市名查询天气", "parameters": { "properties": { "city": { "type": "string", "description": "城市名" } }, "type": "object" } } } ] }'

Expected results

The results are expected to be call one tools

Attempts to fix

I have tried several ways to fix this, including:

  1. make parallel_tool_calls usable

RayneSun avatar Nov 19 '24 03:11 RayneSun