cycloidio
Gracefully handle throttling
Received the error:
could not import from AWS: Error retrieving EC2 tags for CloudFront Distribution [omitted] Throttling: Rate exceeded
and got a stack trace, with no output going to the output file.
There should be some way to see all of the complete resources that have been fetched so that even with throttling all resources can be fetched over time.
Sorry for the delay I was on vacations :sun_with_face:
That's a good idea :).
I wanted to do that with fully support by passing a "half" state/hcl and ignoring the data already fetched for example.
The other thing is that I've improved the Throttling in a pending PR which I think I'll split to just add the better support for it directly without waiting for the full PR :smile:
Right now we give a small delay when a Throttling, a handmade retry basically to be able to maybe avoid it in a while haha, but sometimes it still has it after a while of retrying.
The cause of the retry is not displayed until after all retries. So, I have to wait for 90s until I get the error. Not great.
Also, some errors are not errors (I'm raising a separate bug report for that).
