vllm icon indicating copy to clipboard operation
vllm copied to clipboard

[Feature]: Support "required" option in tool_choice

Open terrytangyuan opened this issue 10 months ago • 1 comments

🚀 The feature, motivation and pitch

The documentation for tool calling mentioned:

The tool_choice option "required" is not yet supported but on the roadmap.

This issue is used to track the support for the "required" option.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • [x] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

terrytangyuan avatar Feb 10 '25 03:02 terrytangyuan

Can confirm this would be great, it's how pydantic-ai enforces result type check.

arty-hlr avatar Mar 06 '25 15:03 arty-hlr

hey guys, this is crucial for agentic applications, why this is not in yet?

jorodg avatar Mar 27 '25 21:03 jorodg

How is this not done yet? This is literally your #1 issue if you want vllm to be seriously used for any kind of agent tasks. (This is a similar problem for AG2, which is only as of like 0.8.4 from 4 days ago supporting "required")

Y'all can find time to merge the most insane MLA kernels to speedup deepseek but you don't have time to implement the literal most important feature of the year? Maybe sglang will be faster to implement this - and if they are, vllm will lose its relevancy as an inference engine for most open source agent AI companies

Hellisotherpeople avatar Apr 01 '25 15:04 Hellisotherpeople

i will look into this

paolovic avatar Apr 01 '25 21:04 paolovic