microsoft
[Bug]: 'last_msg' summary method for nested group chat doesn't return the last message in the group chat
Describe the bug
If you use the 'last_msg' summary_method for a nested group chat, the returned message is the last message between the speaker (agent who is registering the nested chat) and the group chat manager (the recipient defined in the nested chat).
Steps to reproduce
import autogen
from autogen import ConversableAgent, GroupChat, GroupChatManager, UserProxyAgent, AssistantAgent
llm_config = autogen.config_list_from_json("OAI_CONFIG_LIST")
agent_1 = ConversableAgent(
name="agent_1",
default_auto_reply="I hate orcas",
llm_config=False,
)
agent_2 = ConversableAgent(
name="agent_2",
default_auto_reply="I like orcas",
llm_config=False,
)
group_chat = GroupChat(
agents=[agent_1, agent_2],
messages=[],
allow_repeat_speaker=False,
speaker_selection_method="round_robin",
max_round=3,
)
group_chat_manager = GroupChatManager(
groupchat=group_chat,
llm_config=llm_config,
)
user = UserProxyAgent(
name="user",
description="user",
code_execution_config=False,
)
group_chat_proxy = AssistantAgent(
name="group_chat_proxy",
description="A simple proxy agent that passes messages to and from the user and the group chat",
system_message="You are a proxy agent that passes messages to and from the group_chat_manager and the user. Do not modify the received message.",
llm_config=llm_config,
)
group_chat_proxy.register_nested_chats([
{
"recipient": group_chat_manager,
"summary_method": "last_msg",
}
], trigger=user)
user.initiate_chat(message="hello, do you like orcas?", recipient=group_chat_proxy)
Model Used
gpt-4o
Expected Behavior
I would expect that the last_msg from the group chat is returned back to the user_proxy agent. The last message in the output should be I like orcas instead of hello, do you like orcas?.
Screenshots and logs
user (to group_chat_proxy):
hello, do you like orcas?
--------------------------------------------------------------------------------
********************************************************************************
Starting a new chat....
********************************************************************************
group_chat_proxy (to chat_manager):
hello, do you like orcas?
--------------------------------------------------------------------------------
Next speaker: agent_1
>>>>>>>> USING AUTO REPLY...
agent_1 (to chat_manager):
I hate orcas
--------------------------------------------------------------------------------
Next speaker: agent_2
>>>>>>>> USING AUTO REPLY...
agent_2 (to chat_manager):
I like orcas
--------------------------------------------------------------------------------
group_chat_proxy (to user):
hello, do you like orcas?
--------------------------------------------------------------------------------
Additional Information
I took a closer look at the implementation of _last_msg_summary in ConversableAgent and it looks like it calls recipient.last_message(sender)["content"] which will never include the messages from the group chat.
If you look at how _reflection_with_llm_as_summary is implemented it uses recipient.chat_messages_for_summary(sender) which does include the group chat messages. So a simple fix might be to use chat_messages_for_summary and the taking the last message in that list in the _last_msg_summary method.
Agree! I am currently using version 0.2.37 of autogen. After debugging, I found the following:
Although there is a method chat_messages_for_summary() in the ConversableAgent class, which is called in the _reflection_with_llm_as_summary() method, the GroupChatManager class overrides this method, and the values returned by the two are not the same.
My solution is to add a property：_agent_class='ConversableAgent'to the ConversableAgent class, and correspondingly set _agent_class='GroupChatManager' for the GroupChatManager. Based on the value of this property, we can make slight modifications to the _last_msg_as_summary() method in conversable_agent.py.
Below is the original code and the modified code for _last_msg_as_summary (note that different values for _agent_class should be set in ConversableAgent and GroupChatManager):
Original Code:
# Original code for _last_msg_as_summary
def _last_msg_as_summary(sender, recipient, summary_args) -> str:
"""Get a chat summary from the last message of the recipient."""
summary = ""
try:
content = recipient.last_message(sender)["content"]
.......
Modified Code:
# Modified code for _last_msg_as_summary
def _last_msg_as_summary(sender, recipient, summary_args) -> str:
"""Get a chat summary from the last message of the recipient."""
summary = ""
try:
#### self-adding
if recipient._agent_class=='ConversableAgent':
content = recipient.last_message(sender)["content"]
elif recipient._agent_class=='GroupChatManager':
content=recipient.chat_messages_for_summary(sender)[-1]['content']
......
Then you will receive the expected response.
user (to group_chat_proxy):
hello, do you like orcas?
--------------------------------------------------------------------------------
********************************************************************************
Starting a new chat....
********************************************************************************
group_chat_proxy (to chat_manager):
Context:[] .
User's latest reply:{"content": "hello, do you like orcas?", "role": "user", "name": "user"} .
lease provide the correct and concise answer.
--------------------------------------------------------------------------------
Next speaker: agent_1
>>>>>>>> USING AUTO REPLY...
agent_1 (to chat_manager):
I hate orcas
--------------------------------------------------------------------------------
Next speaker: agent_2
>>>>>>>> USING AUTO REPLY...
agent_2 (to chat_manager):
I like orcas
--------------------------------------------------------------------------------
group_chat_proxy (to user):
I like orcas
--------------------------------------------------------------------------------
Replying as user. Provide feedback to group_chat_proxy. Press enter to skip and use auto-reply, or type 'exit' to end the conversation:
Thanks @smallQQ0227 we will be addressing nested chat issues in our v0.4 API.
Latest update from v0.4 preview: the SocietyofMindAgent is a built-in approach to nested chat.
https://microsoft.github.io/autogen/dev/reference/python/autogen_agentchat.agents.html#autogen_agentchat.agents.SocietyOfMindAgent
You can refer to its implementation and implement your own nested chat agent using custom agent:
https://microsoft.github.io/autogen/dev/user-guide/agentchat-user-guide/tutorial/custom-agents.html
Will be released in the upcoming dev12 release.