haystack-core-integrations
haystack-core-integrations copied to clipboard
feat: Update our ChatGenerators in integrations to use the new StreamingChunk fields
We have expanded StreamingChunk to include new fields such as
content: str
meta: Dict[str, Any] = field(default_factory=dict, hash=False)
component_info: Optional[ComponentInfo] = field(default=None)
# All fields below here are new
index: Optional[int] = field(default=None)
tool_calls: Optional[List[ToolCallDelta]] = field(default=None)
tool_call_result: Optional[ToolCallResult] = field(default=None)
start: bool = field(default=False)
finish_reason: Optional[FinishReason] = field(default=None)
so we should update our existing ChatGenerator integrations to use these new fields. This will standardize the streaming in Haystack making it easy to use one streaming_callback like our print_streaming_chunk for all ChatGenerators.
This really emphasizes the interoperability of Haystack and will make it very straightforward to switch out to different ChatGenerators from different providers in a pipeline or when using a ChatGenerator within the Agent component.
When working on this I'd suggest:
- Testing with
haystack.components.generators.utils.print_streaming_chunkto make sure that the output looks clean. Ideally use the ChatGenerator in an Agent and try to get the Agent to output at least two tool calls in one round. You can refer to the PR description of this PR you can see example outputs ofprint_streaming_chunk - Refer to the OpenAIChatGenerator implementation to get inspiration:
_handle_stream_responsewhich loops through each stream object from OpenAI and then converts it into aStreamingChunk_convert_chat_completion_chunk_to_streaming_chunkwhich converts each individual stream object from OpenAI and converts it into a singularStreamingChunk_convert_streaming_chunks_to_chat_messagewhich is a general utility method that converts a list ofStreamingChunks into a HaystackChatMessage