haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

feat: Update our ChatGenerators in integrations to use the new StreamingChunk fields

Open sjrl opened this issue 5 months ago • 0 comments

We have expanded StreamingChunk to include new fields such as

    content: str
    meta: Dict[str, Any] = field(default_factory=dict, hash=False)
    component_info: Optional[ComponentInfo] = field(default=None)
# All fields below here are new
    index: Optional[int] = field(default=None)
    tool_calls: Optional[List[ToolCallDelta]] = field(default=None)
    tool_call_result: Optional[ToolCallResult] = field(default=None)
    start: bool = field(default=False)
    finish_reason: Optional[FinishReason] = field(default=None)

so we should update our existing ChatGenerator integrations to use these new fields. This will standardize the streaming in Haystack making it easy to use one streaming_callback like our print_streaming_chunk for all ChatGenerators.

This really emphasizes the interoperability of Haystack and will make it very straightforward to switch out to different ChatGenerators from different providers in a pipeline or when using a ChatGenerator within the Agent component.

When working on this I'd suggest:

  • Testing with haystack.components.generators.utils.print_streaming_chunk to make sure that the output looks clean. Ideally use the ChatGenerator in an Agent and try to get the Agent to output at least two tool calls in one round. You can refer to the PR description of this PR you can see example outputs of print_streaming_chunk
  • Refer to the OpenAIChatGenerator implementation to get inspiration:

sjrl avatar Jul 08 '25 15:07 sjrl