Add support for correct tool names

[cyril-k]

Feature request

Current implementation 1.4.5 does not return the correct tool name when using function calling. When returning a ToolCall, tool name is hardcoded to tools: https://github.com/huggingface/text-generation-inference/blob/4ee0a0c4010b6e000f176977648aa1749339e8cb/router/src/server.rs#L950

The name of the tool should be inferred by the model and parsed together with parameters to return the ToolCall with correct tool name.

Motivation

The name of the tool in the ToolCall is used in popular LLM frameworks (such as Langchain and LLamaIndex) to dispatch the underlying function calls. Currently, the corresponding function should be named "tools" for this to function properly. This modification will improve integration with open source LLM frameworks and allow to use multiple tools with correct names.

Your contribution

Need to verify if tool's name is handled properly by the tool_grammar and make sure the name field is valid in gen_text_value: https://github.com/huggingface/text-generation-inference/blob/4ee0a0c4010b6e000f176977648aa1749339e8cb/router/src/server.rs#L934-L943

[Jofthomas]

@drbh, this is an issue that is currently holding us back. Do you think it can be fixed "easily" ?

[cyril-k]

It appears that this issue is going to be fixed in the related pull request.

[drbh]

resolved with https://github.com/huggingface/text-generation-inference/pull/1693