haystack-core-integrations icon indicating copy to clipboard operation
haystack-core-integrations copied to clipboard

Published 20 hours ago • verified publisher icon deepset-ai

Support streaming responses in `SagemakerGenerator`

Open ZanSara opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. Sagemaker added support for streaming responses https://aws.amazon.com/about-aws/whats-new/2023/09/sagemaker-real-time-inference-response-streaming/

Describe the solution you'd like We should support streaming the responses in SagemakerGenerator.

Describe alternatives you've considered n/a

Additional context n/a

ZanSara avatar Jan 29 '24 13:01 ZanSara