autogen Some Team/Agent like SocietyOfMindAgent is trying to use multiple/separated system messages, but the current system does not allow that.

What happened?

Describe the bug Some Team/Agent like SocietyOfMindAgent or ~SelectorGroupChat~ tries to use multiple or separated system_messages, but the current system doesn't support that. As a result, non-OpenAI model backends may break or crash depending on how they handle system_messages.

Currently:

SelectorGroupChat manually downgrades the system_message to a user message unless the model is OpenAI-compatible.
Other agents (like SocietyOfMindAgent) do not have such fallback, and will fail silently or cause exceptions depending on the model vendor.

This issue is especially problematic for models like Anthropic, which only support multiple/separated system_messages when invoked via OpenAI-compatible APIs, but not via their native SDK.

This situation makes some teams or agents effectively OpenAI-only, which is not an intentional design.

To Reproduce Use SocietyOfMindAgent or any GroupChat-style agent that passes multiple/separated system_messages to sub-agents, and test against various model backends.

Example:

    from autogen_agentchat.agents import AssistantAgent, SocietyOfMindAgent
    from autogen_agentchat.teams import RoundRobinGroupChat
    from autogen_ext.models.anthropic import AnthropicChatCompletionClient  # native SDK
    from autogen_agentchat.ui import Console

    client = AnthropicChatCompletionClient(model="claude-3-5-haiku-20241022")

    agent1 = AssistantAgent(name="agent1", system_message="You are a writer.", model_client=client)
    agent2 = AssistantAgent(name="agent2", system_message="You are a critic.", model_client=client)

    group = RoundRobinGroupChat([agent1, agent2], max_turns=1)

    society = SocietyOfMindAgent(name="society", team=group, model_client=client)
    
    async def team_run():
        await Console(
            society.run_stream(
                task="Write a short story with critique."
            )
        )
    
    asyncio.run(team_run())

Depending on the SDK, this may throw an error or silently misbehave (e.g., ignore one of the system_messages).

Expected behavior Agents and teams that wish to use multiple/separated system_messages should:

Check model compatibility before applying that structure
Fallback gracefully (e.g., convert to user message) when unsupported
Not crash or behave incorrectly on non-OpenAI models

Additional context We propose the following as a longer-term structural fix:

Survey required
a. Which models (by SDK / backend) support multiple/separated system_messages
b. Which Team/Agent types want to make use of this feature (e.g., SocietyOfMindAgent, ~~SelectorGroupChat~~, custom GroupChats)
Introduce capability flags
A new field (e.g., supports_multi_system_message) should be added in model_info or similar, and exposed through the model registry.
Propagate model capabilities to agents
Agent constructors or transformers should receive this flag and adjust behavior accordingly.
Conditional behavior per agent
Each agent or team should gracefully fallback or reformat messages based on the capability flag.

These changes should be implemented across two or more separate PRs, and step 1a/1b will likely require community input — it's probably too much to handle alone.

However, this issue is critical because some agents/teams silently become OpenAI-only, and we need to address this explicitly in the design.

TODO...

Check list - is Agent/Team need to multi-system message?

Agents

[x] Name : assistant_agent Need it? : X
[x] Name : base_chat_agent Need it? : X
[x] Name : code_executor_agent Need it? : X
[x] Name : society_of_mind_agnet Need it? : O
[x] Name : user_proxy_agent Need it? : X

Teams

[?] Name : magentic_one Need it? : ?
[x] Name : base_group_chat Need it? : X
[x] Name : round_robin_group_chat Need it? : X
[x] Name : selector_group_chat Need it? : X
[?] Name : swarm_group_chat Need it? : ?

Model SDK

[x] Name : anthropic Could Multi-System message : X
[ ] Name : azure Could Multi-System message : ?
[ ] Name : cache Could Multi-System message : ?
[ ] Name : llama_cpp Could Multi-System message : ?
[ ] Name : ollama Could Multi-System message : ?
[ ] Name : replay Could Multi-System message : ?
[ ] Name : semantic_kernel Could Multi-System message : ?
[x] Name : openai Could Multi-System message : go to down

OpenAI API aware model

[x] Name : openAI Could Multi-System message : O
[x] Name : anthropic Could Multi-System message : O
[x] Name : gemini Could Multi-System message : X

Which packages was the bug in?

Python AgentChat (autogen-agentchat>=0.4.0)

AutoGen library version.

Python dev (main branch)

Other library version.

No response

Model used

No response

Model provider

None

Other model provider

No response

Python version

None

.NET version

None

Operating system

None

Apr 13 '25 07:04 SongChiYoung

Thanks for the comprehensive survey. I think for this issue a quick fix would be appreciated. Namely:

Check model compatibility before applying that structure Fallback gracefully (e.g., convert to user message) when unsupported Not crash or behave incorrectly on non-OpenAI models

This can be addressed with the same type of fix as in SelectorGroupChat.

Let's use a separate issue to address the broader usage case of single shot prompting with system message.

Apr 15 '25 06:04 ekzhu

Hi everyone, I’d appreciate your help on this issue — please consider filling out the survey above!

If you have any confirmed findings or model-specific behavior to share, feel free to leave a comment. I’ll incorporate those into the solution as we go.

As a heads-up: this issue is not directly related to my primary work, so my availability during working hours may be limited. I’ll do my best to check things when I can.

@ekzhu
As a quick path forward, I’d like to propose adding a multiple_system_message: bool field to model_info.

I will start by setting this flag to True only for the models I’ve already confirmed (e.g., OpenAI, Claude with OpenAI api, etc.),
and then apply the same merging/filtering strategy used in SelectorGroupChat.

This should help resolve immediate compatibility issues while we wait for broader survey results.
Once the survey is complete, I’ll revisit and update the implementation accordingly.

If you're okay with this direction, I’ll begin implementation — it should take no more than 1–2 hours to complete.

Apr 17 '25 00:04 SongChiYoung

Let's make an optional field and by default set it to False?

Apr 17 '25 05:04 ekzhu

Let's make an optional field and by default set it to False?

Yes, that's what I want.

Apr 17 '25 05:04 SongChiYoung

@ekzhu I found at blame https://github.com/microsoft/autogen/commit/59e392cd0f5e6075d2f6f5b527a0914825049286 That SelectorGroupChat downgrade SystemMessage to UserMessage, It not for multiple system message issue.

So, Now only SocietyOfMindAgent has multiple group chat.

--> however, when still SelectorGroupChat need to downgrade SystemMessage to UserMessage for like that bug

openai.BadRequestError: Error code: 400 - {'error': {'code': 'invalid_request_error', 'message': 'At least one non-system, non-developer message is required.', 'type': 'invalid_request_error', 'param': None}}

Apr 17 '25 13:04 SongChiYoung