Concurrency for fetching and processing of GKE data

[mikouaj]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritise the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

In a current implementation, clusters are processed one by one. Such approach does not scale. It is not suitable i.e. for a customers with large number of clusters, that would like to use cluster-discovery and serverless approaches.

We need to introduce concurrency for fetching and processing GKE cluster data. This should be done with a Cloud Run Jobs limitations in mind.

References

N/A