microsoft
.Net: New Feature: Add usage information to chat completion response when using Amazon Bedrock
Some connectors (for example OpenAI and Mistral) return token usage in ChatMessageContent.Metadata, which might be used later in application's logic (usually for request/usage limiting).
Currently Amazon Bedrock connector does not do that, even though that would be trivial to implement:
Index: dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs
<+>UTF-8
===================================================================
diff --git a/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs b/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs
--- a/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs (revision 71ea643dee10486fd1fff93ade09cab85f6b44b0)
+++ b/dotnet/src/Connectors/Connectors.Amazon/Bedrock/Core/Clients/BedrockChatCompletionClient.cs (date 1747666877685)
@@ -130,7 +130,11 @@
{
Role = BedrockClientUtilities.MapConversationRoleToAuthorRole(message.Role.Value),
Items = CreateChatMessageContentItemCollection(message.Content),
- InnerContent = response
+ InnerContent = response,
+ Metadata = new Dictionary<string, object?>
+ {
+ { "Usage", response.Usage }
+ }
}
];
}
I can volunteer to add this and update tests.
Notes:
I don't like how currently token usage is stored in ChatMessageContent - it is hidden in Metadata under Usage key and it leaks model specific usage structure (for example OpenAI.Chat.ChatTokenUsage or Microsoft.SemanticKernel.Connectors.MistralAI.Client.MistralUsage), but that could be separate discussion and larger change.
@dziedrius Thanks for creating the issue. Would you be interested in contributing this functionality to Semantic Kernel?
