haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Published 20 hours ago • verified publisher icon deepset-ai

VertexAIGeminiChatGenerator streaming_callback implementation (or streaming response)

Open sataycat opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Similar to this ticket, the VertexAIGeminiChatGenerator does have streaming_callback implementation, even though the underlying GenerativeModel does support stream=True https://github.com/deepset-ai/haystack/issues/7836

The underlying model supports an iterable chat response https://github.com/googleapis/python-aiplatform/blob/main/vertexai/generative_models/_generative_models.py#L522

Describe the solution you'd like Similar to this https://github.com/deepset-ai/haystack/pull/8054, by @Amnah199 :)

sataycat avatar Aug 01 '24 08:08 sataycat