haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Google GenAI: use `finish_reason.tool_calls` when the model produces a tool call?

Open anakin87 opened this issue 4 months ago • 0 comments

Haystack's finish_reason is used both in StreamingChunk and ChatMessage.meta["finish_reason"] to indicate why the text generation stopped.

Google GenAI uses STOP as a general-purpose finish reason, for both normal text completions and tool calls. This differs from other providers like OpenAI, which return more specific reasons (e.g., tool_calls or similar) when a tool call is involved.

In Google GenAI integration, Haystack mirrors Google's convention by passing through STOP even for tool calls. One alternative would be to implement a logic to detect tool calls in the response and remap the finish_reason accordingly.

Initially discussed in https://github.com/deepset-ai/haystack-core-integrations/pull/2142#discussion_r2251062617

anakin87 avatar Aug 04 '25 10:08 anakin87