ValueError: Streaming is not supported for batched prompts. Let us know if you need this feature.

[radna0]

[Balakrishna-Chennamsetti]

Hi @radna0 ,

Thanks for bringing this to our attention, the Gemma models are accepts the community contributions. Please raise the PR for your requested changes, once it's got reviewed and adhere to complains the contributions the PR will going to be merged. Your contribution is really appreciated.

To know more about contribution guidelines please visit the following page.

Thanks.

[XBastille]

Hi @Balakrishna-Chennamsetti ,

I've implemented streaming support for batched prompts! PR: #446

The feature now allows:

sampler.sample(['prompt1', 'prompt2'], stream=True)

[Balakrishna-Chennamsetti]

@XBastille , thank you so much for your contribution.