dftimewolf
dftimewolf copied to clipboard
gcp_logging_collect recipe fails silently when it hits Cloud Logging API limits
When running the gcp_logging_collect recipe, the collection will fail silently if it exceeds the cloud logging API quota limit.
Based on my testing, this will first issue a 'google_api_exceptions.TooManyRequests' exception. Following this, all subsequent queries from the same logging client will be returned with a 'google_api_exceptions.StopIteration' exception. This occurs even when I implemented a 2 minute delay between requests after the first 'TooManyRequests' exception.
Due to this, I believe the previous logging retries fix from https://github.com/log2timeline/dftimewolf/commit/32e39780d808cc506c19580aea68e3b1c1810ede may not achieve any successful retries, as a new logging client is not generated after hitting the API limit.