Azure
[FEATURE REQ] Azure.AI.OpenAI `include_usage` in chat completion options
Library name
Azure.AI.OpenAI
Please describe the feature.
Add include_usage options for streaming chat completions. Relevant OpenAI docs below.
https://platform.openai.com/docs/api-reference/completions/create
include_usage
Request:
Respose:
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Updated to the new v2 beta2 client. This feature seems to be switched to true by default in the OpenAI client but doesn't seem to be working.
https://github.com/openai/openai-dotnet/issues/103
Are there any updates on this?
@Mako217 on https://github.com/openai/openai-dotnet/issues/103 I was told the backend doesn't have the feature yet. Generally Azure have been terrible at communicating what the state of these features are. There are at least two other features we are waiting for were the backend state is unclear.
Seems it is now supported. Tested working on chat completion with OpenAI npm SDK with Azure credential, with ChatGPT-4o-0806, API version 2024-08-01-preview.
It never works in previous versions.
yes, confirmed for the save model and rest API version against eastus and sweden deployments. streaming + include_usage works for me with Postman.
We are not on v2.1 of the dotnet SDK and this issue still has not been fixed.
As documented in https://github.com/openai/openai-dotnet/issues/103 this is a bug in the client. The backend is confirmed to support this feature.
Can somebody on the SDK team at least acknowledge this is an issue cc @jpalvarezl @trrwilson .