streaming_callback with async functions

[Filocava99]

Is your feature request related to a problem? Please describe. Currently the streaming_callback for chat generators do not support async functions as callbacks, limiting the potential of backend solutions made, for instance, with FastAPI.

Describe the solution you'd like Make streaming_callback accept both Optional[Callable[[StreamingChunk], None]] and Optional[(chunk: StreamingChunk) -> Coroutine[Any, Any, None]

Describe alternatives you've considered As of now the alternative is to run the pipeline on a different thread and then use appropriate data structures to concurrently add data from the callback and retrieve them from FastAPI main thread. This way the code is clunky and not optimal

[Filocava99]

Bump

[WissamAntoun]

Anyone figured out how we can stream tokens from the generator thru fastapi back to the user?

[shadeMe]

Blocked by #6012