Add additional token calculation information to the Limit Azure OpenAI API token usage API Management policy

[robinmanuelthiel]

Add additional token calculation information to the Limit Azure OpenAI API token usage API Management policy

How does the new APIM policy “Limit Azure OpenAI API token usage” estimate-prompt-tokens feature work? How does it estimate the tokens? Only of the prompt or also of the response? What changes, when I activate/deactivate it? The documentation was way too minimal on this.

This change adds some additional information about the impact of setting estimate-prompt-tokens to false.

This is important, because it might cause the probably unexpected behavior of sending requests to the LLM backend, which should have been filtered out by the policy.

Please check for correctness by the Product Team.

[prmerger-automator[bot]]

@robinmanuelthiel : Thanks for your contribution! The author(s) have been notified to review your proposed change.

[learn-build-service-prod[bot]]

Learn Build status updates of commit 4c9b755:

:white_check_mark: Validation status: passed

File	Status	Preview URL	Details
articles/api-management/azure-openai-token-limit-policy.md	:white_check_mark:Succeeded

For more details, please refer to the build report.

For any questions, please:

• Try searching the learn.microsoft.com contributor guides
• Post your question in the Learn support channel

[Court72]

@dlepow

Can you review the proposed changes?

Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.

#label:"aq-pr-triaged" @MicrosoftDocs/public-repo-pr-review-team

[dlepow]

@robinmanuelthiel - Thanks for the PR. Let me check with the engineering team about this first before we merge the update.

[Court72]

@dlepow, For convenience, I committed your suggestions. Please add a #sign-off comment if this PR is ready to merge. Thanks!

[learn-build-service-prod[bot]]

Learn Build status updates of commit b218d18:

:white_check_mark: Validation status: passed

File	Status	Preview URL	Details
articles/api-management/azure-openai-token-limit-policy.md	:white_check_mark:Succeeded

For more details, please refer to the build report.

For any questions, please:

• Try searching the learn.microsoft.com contributor guides
• Post your question in the Learn support channel

[robinmanuelthiel]

LGTM

[dlepow]

Thanks all! #sign-off