.Net: Streaming Issue with AggregatorAgent in Nested Mode in version 1.19.0-alpha

[SlimKtari]

Describe the bug When using AggregatorAgent in Nested mode with Semantic Kernel version 1.19.0-alpha, the AgentGroupChat streaming functionality with InvokeStreamingAsync does not work. The inner agents of the AggregatorAgent (which are ChatCompletionAgents inside an AgentGroupChat) are not triggered, and the method does not iterate over the agents as expected.

However, when the AggregatorAgent is switched to Flat mode, or when the outer AgentGroupChat containing the AggregatorAgent is removed, the InvokeStreamingAsync method works as expected, returning responses from the inner agents.

To Reproduce Steps to reproduce the behavior:

1. Update Semantic Kernel to version 1.19.0-alpha.
2. Use the InvokeStreamingAsync method on an AgentGroupChat containing an AggregatorAgent in Nested mode.
3. The AggregatorAgent contains another AgentGroupChat with agents.
4. Note that the InvokeStreamingAsync function returns nothing. await foreach (var response in groupChat.InvokeStreamingAsync()) { if (string.IsNullOrEmpty(response.Content)) { continue; } yield return response.Content; }

Expected behavior I expected the InvokeStreamingAsync method to work with AggregatorAgent in Nested mode.

Platform

• OS: Windows - OS: [e.g. Windows, Mac]
• IDE: Visual Studio
• Language: C#
• Source: NuGet package version 1.19.0-alpha

[crickman]

@slimktari - Thank you for reporting this issue. I'll dig into what's happening.

[anshul-rohilla]

Facing similar issue function not invoked when using InvokeStreamingAsync. Any workaround? I am using a AgentGroupChat with 2 agents, @slimktari can you please help with the work around to set the mode to Flat.

[SlimKtari]

@anshul-rohilla With an AgentGroupChat and two agents, the InvokeStreamingAsync method worked for me. Maybe we’re not using the same type of agents. In my case, my agents are of type ChatCompletionAgent. For reference, I use a selection and termination strategy of type KernelFunctionSelectionStrategy, which might make a difference.

The flat mode is specific to the AggregatorAgent. Essentially, you need an outer AgentGroupChat with the AggregatorAgent in flat mode and an inner AgentGroupChat with your two agents. You can see an example at this link, you would just need to change Nested to Flat.

But honestly, if the two agents directly in the AgentGroupChat don’t work, I think even with the AggregatorAgent, you’ll get the same result. Feel free to share the outcome if you do more tests in this direction.

[SlimKtari]

I just updated to Semantic Kernel version 1.20, and I still have the same issue.

[jonepo]

I tested this with the latest 1.21.1 and the situation is the same. The method InvokeStreamingAsync does not return anything. I'm using the method call with AggregatorAgent and CancellationToken arguments. And I have only used the nested mode.

[crickman]

@slimktari / @jonepo:

Ok, the root cause is no message is being yielded for the nested case. The complication is that identifying the final streamed content to provide is entirely ambiguous. I wonder if an acceptable middle ground would be to yield the final message as streamed content...so at least there isn't an absence of content.

We could attempt to do something more fancy to simulate streaming the final message, but at the point this would occur the final message is fully formed.