DRAFT for Feedback - Support for token streaming for more dynamic UX

[jspv]

Why are these changes needed?

ChatCompletionClient nicely supports token level streaming via create_stream, but this method is currently not accessible in the AssistantAgent. This proposed change adds an option to pass a token_callback when instantiating AssistantAgent, if provided:

1. create_stream will be leveraged instead of create when calling on_messages_stream
2. the provided callback will be called with the returned token as the argument.

This will allow the calling application access to the returned tokens real-time. Nothing else is changed, the normal returns to on_messages_streams are not affected.

Example:

If folks feel this a good idea, I will make appropriate updates in documentation and tests.

Related issue number

Checks

• [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR.
• [ ] I've made sure all auto checks have passed.