langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Agent simulations

Open mbchang opened this issue 3 years ago • 1 comments

This post is about agent_simulations_v2.

This is a draft of what I am imagining the interfaces for a 2-agent system to look like. It is not completely generic, but just wanted to push to share the direction I am going. It is just in a jupyter notebook for easy sharing, but as we finalize the interfaces I can integrate the classes into the actual code.

Main things to pay attention to:

  • Entity base class
  • BaseSimulator base class
  • main loop (at the bottom of the notebook)

In order for us to make this more generic, I plan to do the following:

  1. Allow for arbitrary roles. Currently the ChatOpenAI model only allows for "system", "user", "assistant", but this will not allow us to expand to multiple agents with different roles. This means I would need to use an standard LLM Chain with a ConversationBufferWindowMemory instead of ChatOpenAI model.
  2. Have simulator.step() take in a dictionary of messages and return a dictionary of messages. Currently we have hardcoded it to take exactly 2 messages as input and return 2 messages as output.

Currently I am not implementing simulator.run() to allow the user to write print statements and break statements, as we can see from the example.

mbchang avatar Apr 24 '23 21:04 mbchang

This post is about agent_simulations_v3, which improves upon agent_simulations_v2 by using a generic RoundRobinSimulator rather than a CAMEL-specific simulator. If all we want is a simulator for two agents, then RoundRobinSimulator is a complete MVP.

What is new in this notebook:

  • A generic RoundRobinSimulator for running a simulation of an agent dialogue in a round-robin fashion. It takes an agents list and a leader_idx as input. The agents speak in a round-robin fashion, where the agent at index i speaks to the agent at index (i+1) % len(agents). The agent at index leader_idx speaks first.

Comments:

  • This uses the ChatOpenAI model, which expects chat roles to either be assistant or user. This restricts us to only two agents. Therefore, even though RoundRobinSimulator can support multiple agents, it really only makes sense to use it for two agents because the AIMessage that agent i sends is received as a HumanMessage by agent i+1.
  • If all we want is a simulator for two agents right now, the RoundRobinSimulator is a complete MVP.

mbchang avatar Apr 24 '23 23:04 mbchang