.Net: Python: [New Group Chat Orchestration] LLM Prompt usage not available in metadata of the orchestration result

[sushaanttb]

[SK version - 1.32.0] Unlike general metadata field with LLM prompts usage info which is available in ChatMessageContent objects, same is not available in the final response from the new group chat orchestration result.

Please suggest how to get the LLM prompt usage metadata for the final response from orchestration?

This is required for observability of my orchestration component.

[SuvitKumar003]

Hi @markwallace-microsoft thanks for tagging this for triage!

I’m exploring this issue further and noticed that while ChatMessageContent retains prompt usage metadata, the final_response from orchestration_result.get() doesn't expose it directly — as shown in the debug logs.

Is this an intentional design, or is support for exposing such metadata in final_response planned in future updates?

Also, I’d be happy to help contribute a fix or improvement if this is something the team would be open to. Just let me know if there's any specific direction or constraint you'd suggest following.

Thanks again!

[sushaanttb]

Hi @moonbox3 Would appreciate an update for this.

[moonbox3]

This isn't intentional to not include usage metadata, looks like it was simply a miss. @SuvitKumar003 please feel free to work on this if you have time.

[TaoChenOSU]

Hi @sushaanttb,

Thank you for posting the issue!

Are you looking for the token usage of just the final response or the total toke usage of the entire run of the orchestration?

[TaoChenOSU]

Hi @sushaanttb,

In Python, the result of the group chat orchestration is produced by the manager's filter_results method: https://learn.microsoft.com/en-us/semantic-kernel/frameworks/agent/agent-orchestration/group-chat?pivots=programming-language-python#customize-the-group-chat-manager.

The return value of the filter_results method is a MessageResult. If you are using the default round robin manager, you should get the message generated by the last agent who spoke, which should contain the metadata of the response. If you are customizing your own group chat manager, please make sure you are including the metadata in the ChatMessageContent.

[crickman]

@TaoChenOSU - This is flagged for both Python and .NET analysis.

[TaoChenOSU]

@TaoChenOSU - This is flagged for both Python and .NET analysis.

Will look at the .Net side too

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.