use sockets rather than print messages to terminal

[ragyabraham]

Rather than print messages to the terminal, we will use sockets to push messages to a socket server. This will allow Autogen to utilise python-socketio library to emit messages via sockets

[ragyabraham]

@sonichi please assign to me

[victordibia]

Hi @ragyabraham ,

Thanks for getting started on this! I think its an important capability and will help improve the developer experience (and user experience) for building with Autogen.

I also took a quick look at your socket implementation and it looks great! I like how you introduce a use_socket flag and socket_client is passed to agent classes and then used to log intermediate agent messages.

Some things to consider:

• Generalization: Socket/Messaging implementations might vary .. e.g., pysocketio, gcppubsub, azure pubsub, rabbitmq, redis pubsub etc. Based on this, it might be valuable to pass in a use_agent_stream flag and a Callable function with some set signature that is called if the flag is true. This way, developers can implement their own streaming architecture that gets called.
• Supporting multiple streaming methods: Build example StreamHandler classes (which we can add in contrib for now) to show examples of streaming agent response e.g., a first example will be based on your implementation with pysocket.io PySockerIOStreamHandler . We can then write a notebook that shows how to use a StreamHandler and then add support for other streaming approaches incrementally.

What do you think?

[ragyabraham]

Hey @victordibia thank you for your feedback. I think both of those points are extremely valid.

• The generalisation is certainly a bit more tricky, but it's definitely doable. I will give it a crack.

• In terms of multiple streaming examples, I think that should be straightforward. We are already using this internally (see gif below), so I can add to contrib a function using socketio and, as you said, as we build up our library of messaging engines we can look to add to this collection

[victordibia]

The demo looks great, thanks for sharing!! Thanks for helping with this again, its a really valuable feature! Looking forward to testing as you make progress!

[maxim-saplin]

A dumb question... What are the use cases? Out of process agents that can be deployed remotely? The current AutoGen implementation is in-process only?

[juan-cnba96]

Will this cover user input as well? Or just the output of the print statements?

[ragyabraham]

@juan-cnba96 hopefully, both input and output

[juan-cnba96]

@ragyabraham I had to implement this feature for a project I'm working on. Tried to make it a tad generic in case I needed a different way of handling IO in the future. Sharing here in case there's anything in there that might help you out. https://github.com/microsoft/autogen/compare/main...juan-cnba96:autogen:5d54f2a2093fd7a6bf1a81d7f65d0e36c878c026

I believe I'm still missing two "print()" statements where code execution is happening.

[ragyabraham]

Hey @juan-cnba96, this is great! Thank you for this. I have a few questions:

1. Did you altogether remove the _print_received_message function? If so, I think the idea is to give users the choice to use sockets, so it might be worthwhile to have a flag that indicates that an agent should use sockets and, if so send messages via sockets rather than print
2. @victordibia had some thoughts about making sockets generic so that users can dictate which messaging framework they want to use (see comment above).

[juan-cnba96]

@ragyabraham the way I thought about it is that the code that was originally inside _print_received_message should be the "default" way of outputting information. Same with get_human_input which simply calls input() by default.

Following this line of thought, I created two generic classes to handle IO that I put in io_utils.py: HumanInputHandler and OutputHandler. These two classes define the interface for both synchronous and asynchronous IO.

Then I grabbed the content of the current _print_received_message and get_human_input and implemented children of the handlers inside conversable_agent.py. This way, nothing changes for current users. But, if you want to change IO to sockets, or anything else, all you need to do is implement the handlers (new classes inherited from HumanInputHandler and OutputHandler) and pass them as parameters to the agents constructor.

These two lines inside ConversableAgent take care of establishing the right default behavior: self.output_handler = PrintOutputHandler(self) if output_handler is None else output_handler self.input_handler = HumanInputHandler() if input_handler is None else input_handler

This approach "should" be a step in the direction @victordibia was suggesting. I've added two sample handler implementations inside io_utils.py that I'm currently using in my Django app to handle IO using channels.generic.websocket.AsyncWebsocketConsumer.

Hope this helps!

[pq-dong]

I'm building a Browser/Server mode web system based on autogen, and I really need to use sockets rather than print messages to terminal. I think @victordibia idea is great, just like what @juan-cnba96 did. Is it possible to unify opinions? , merge into the main branch as early as possible

[joshkyh]

Hi Everyone, I'll be working together with ragyabraham on this.

[ekzhu]

@joshkyh and @ragyabraham make sure to checkout the refactoring effort under way in for conversable agent #1240