ocean
ocean copied to clipboard
[GCP] Updated Retry Mechanism
Description
What:
- Implemented a custom retry mechanism to replace GCP's AsyncRetry due to reliability issues.
- Added rate limiting for "Search All Resources" requests, configurable via the
SEARCH_ALL_RESOURCES_PER_MINUTE_QUOTAenvironment variable.
Why:
- The default GCP AsyncRetry mechanism exhibited reliability issues during operations, leading to the need for a more robust custom implementation.
- Rate limiting was introduced to ensure compliance with GCP's quota restrictions, allowing for configurable limits on the number of "Search All Resources" requests made per minute.
How:
- Developed and integrated a custom retry mechanism tailored to handle specific scenarios more effectively than GCP's default.
- Introduced a rate limiting feature, controlled by the
SEARCH_ALL_RESOURCES_PER_MINUTE_QUOTAenvironment variable, to manage API request flow.
Type of change
- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] New Integration (non-breaking change which adds a new integration)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [x] Non-breaking change (fix of existing functionality that will not change current behavior)
- [ ] Documentation (added/updated documentation)