Keno Teppris

First thanks for your amazing work! I actually run in the same Issue but from a different Use Case: We are running our own Text Generation Inference API of Huggingface...

We encounter the Issue, as pointed out here: https://github.com/langchain-ai/langchain/discussions/26342 I think the parsing of the Tool Calling is the main issue as @drbh already pointed out, the chat prompt template...

Same issue here. After we got excited that the tool call returns now a string, like OpenAIs API, we found that the model keeps calling the tool with the same...

We completely switch to vLLM now and with some additional settings now Llama 3.3 Tool Calling works exactly as expected. I can imagine changing TGI backend to vLLM could potentially...

I guess this is related to #2480. As meta describes, for passing back the ToolCall Message we need to use their new Role ipython: https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_1 Maybe that is causing the...