Export-AzRuleData may not export all data if throttled

[BernieWhite]

Description of the issue

For high volume subscriptions with many or complex resources, calls to Export-AzRuleData request may get throttled.

Get-AzResource: TooManyRequests : The request is being throttled as the limit has been reached for operation type -
Read_ObservationWindow_00:05:00

Currently PSRule for Azure doesn't specifically handle throttling gracefully and may miss resources.

Expected behaviour

We may need some command line options to configure the behaviour on throttling as timely export of resource may be more important.

However we should provide the ability to export as resources observing throttling window, and take steps to avoid throttling all together if any additional tweaks can be made.

Module in use and version:

• Module: PSRule.Rules.Azure
• Version: 1.13.4

Additional reference

https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling

[ArmaanMcleod]

Maybe for this we could provide -MaximumRetryCount and -RetryIntervalSec parameters?

I know for the 429 error code, the response does include a Retry-After header indicating how long you should wait until making another request.

[abelal83]

This is a huge problem when a subscription contains hundreds of resources. There's not much we can do as the call to Azure is handled by Export-AzRuleData.

[BernieWhite]

@abelal83 Thanks for your interest in this issue.