autogen icon indicating copy to clipboard operation
autogen copied to clipboard

During Teachability dialogues, the agent does not trigger is_termination_msg.

Open kev1nzh37 opened this issue 1 year ago • 3 comments

Describe the bug

Although the is_termination_msg function is set for the agent in Teachability, it does not actually trigger when called.

def termination_msg(x):
       print(x)
        return (
            isinstance(x, dict)
            and "TERMINATE" == str(x.get("content", ""))[-9:].upper()
        )
teachability = Teachability(
        verbosity=1,
        reset_db=True,
        path_to_db_dir="./tmp/notebook/teachability_db",
        recall_threshold=1.5,
    )
    
user_proxy = autogen.UserProxyAgent(
        name="USER"
        is_termination_msg=termination_msg,
        description="A human admin as well as the boss of the group, who ask questions, give tasks and provide information.",
        human_input_mode="NEVER",
        code_execution_config=False,
    )
    
teachable_agent = AssistantAgent(
            name="Assistant",
            is_termination_msg=termination_msg,
            llm_config={
                "config_list": llm_list,
                "timeout": 60,
                "cache_seed": None,
            },
        )
teachability.add_to_agent(teachable_agent)

user.initiate_chats(
            [
                {
                    "recipient": teachable_agent,
                    "message": MSG,
                    "max_turns": 1,
                },
            ]
        )

Result:


********************************************************************************
Starting a new chat....

********************************************************************************
kevaaa (to Assistant):

call me kev

--------------------------------------------------------------------------------
# call me kev (is_termination_msg)
Seed_Assistant_B (to kevaaa):

Sure, Sager! How can I assist you today?

# Here, is_termination_msg does not trigger and there are no print statements.

Steps to reproduce

No response

Model Used

No response

Expected Behavior

No response

Screenshots and logs

No response

Additional Information

No response

kev1nzh37 avatar May 14 '24 02:05 kev1nzh37

What happens if you just comment out the line that calls add_to_agent and run the same test again?

rickyloynd-microsoft avatar May 14 '24 04:05 rickyloynd-microsoft

What happens if you just comment out the line that calls add_to_agent and run the same test again?如果您只是注释掉调用 add_to_agent 的行并再次运行相同的测试,会发生什么?

Thank you. After testing, it seems that the issue is not with Teachability. When I set max_turns to 2 or 3, I found that only the last response does not call the is_termination_msg function.

If I want the last response of each chat to also call the is_termination_msg function, what should I do?

kev1nzh37 avatar May 14 '24 04:05 kev1nzh37

If I want the last response of each chat to also call the is_termination_msg function, what should I do?

I suspect that's not possible because the max_turns condition triggers first. But @ekzhu would know better.

rickyloynd-microsoft avatar May 14 '24 13:05 rickyloynd-microsoft

should both max_turns and is_termination_msg both trigger in this case?

jtoy avatar May 17 '24 15:05 jtoy

To be more exact, either condition (max_turns or is_termination_msg) will cause the termination here, so it's just a question of which condition is checked first in the code.

rickyloynd-microsoft avatar May 17 '24 15:05 rickyloynd-microsoft

If I want the last response of each chat to also call the is_termination_msg function, what should I do?

@kev1nzh37 if the chat is terminated, you will get the messages from the ChatResult object returned. You can run your termination check. It's a weird usage, could you elaborate on your scenario?

ekzhu avatar May 22 '24 00:05 ekzhu