deepset-ai
Check if the util method `_convert_chat_completion_chunk_to_streaming_chunk` works as expected for integrations that inherit `OpenAIChatGenerator`
Now that we have merged https://github.com/deepset-ai/haystack/issues/9358 it would be good to double check if the logic inside of _convert_chat_completion_chunk_to_streaming_chunk applies well to our core-integrations that inherit from the OpenAIChatGenerator.
Specifically, I'm concerned the logic outlined here https://github.com/deepset-ai/haystack/blob/54c5057e0bdb8d0f85ecb8f427c2c839ebb28697/haystack/components/generators/chat/openai.py#L571-L577 will not hold for other LLM providers like Mistral, OpenRouter, MetaLlama, etc. This mostly depends on if those providers allow text + tool calls to come in as a single completion. So far OpenAI does not do this but it's possible these other integrations work differently.
So I think it would be good to go through some of our other integrations to observe whether they allow text + tool calls to be sent in one completion and if the resulting StreamingChunks have the correct attributes. E.g. index and start are correctly set for a single stream.
The impacted integrations are:
- MistralChatGenerator
- STACKITChatGenerator
- MetaLlamaChatGenerator
- OpenRouterChatGenerator
