ocean icon indicating copy to clipboard operation
ocean copied to clipboard

[GCP] Updated Retry Mechanism

Open mk-armah opened this issue 1 year ago • 0 comments

Description

What:

  • Implemented a custom retry mechanism to replace GCP's AsyncRetry due to reliability issues.
  • Added rate limiting for "Search All Resources" requests, configurable via the SEARCH_ALL_RESOURCES_PER_MINUTE_QUOTA environment variable.

Why:

  • The default GCP AsyncRetry mechanism exhibited reliability issues during operations, leading to the need for a more robust custom implementation.
  • Rate limiting was introduced to ensure compliance with GCP's quota restrictions, allowing for configurable limits on the number of "Search All Resources" requests made per minute.

How:

  • Developed and integrated a custom retry mechanism tailored to handle specific scenarios more effectively than GCP's default.
  • Introduced a rate limiting feature, controlled by the SEARCH_ALL_RESOURCES_PER_MINUTE_QUOTA environment variable, to manage API request flow.

Type of change

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] New Integration (non-breaking change which adds a new integration)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [x] Non-breaking change (fix of existing functionality that will not change current behavior)
  • [ ] Documentation (added/updated documentation)

Screenshots

Screenshot 2024-08-19 at 7 37 37 PM

mk-armah avatar Aug 19 '24 19:08 mk-armah