docker-volume-hetzner icon indicating copy to clipboard operation
docker-volume-hetzner copied to clipboard

Published 20 hours ago • verified publisher icon costela

Improvement: Add Backoff for Rate Limits

Open s4ke opened this issue 4 months ago • 7 comments

Currently it can happen that for whatever reason the requests to the Hetzner Cloud API are going a bit crazy, causing 429 errors.

Currently, once a rate limit is reached, we keep hammering the rate API and we never get out of the rate limit issue.

s4ke avatar Feb 27 '24 16:02 s4ke

To me it is still a bit unclear whether we want to just communicate back the error to the docker engine directly or whether we want to wrap the calling code in https://github.com/cenkalti/backoff to automatically do the backoff.

s4ke avatar Feb 27 '24 17:02 s4ke

Prototype here: https://github.com/costela/docker-volume-hetzner/compare/master...neuroforgede:docker-volume-hetzner:master

Currently evaluating.

s4ke avatar Feb 27 '24 18:02 s4ke

@costela What are your thoughts on this?

s4ke avatar Mar 17 '24 22:03 s4ke

@costela ping

s4ke avatar Apr 22 '24 06:04 s4ke

sorry, I'm bit swamped, as always :sweat:

I understand the motivation, but I'm not sure about the particular solution. Why not something a bit more general like WithBackoffFunc?

costela avatar Apr 23 '24 12:04 costela

No worries.

If I read the docs for WithBackoffFunc correctly this is used for automatic retrying of requests. We don't want automatic retries though, right? I'd rather have the volume creation fail?

s4ke avatar Apr 23 '24 20:04 s4ke

ah, you mean the fact that the retries are endless? Yes, that's a bit of a problem. But I'd suggest making that improvement at the hcloud-go side? Adding a way to stop retrying?

That feels like the better way to put this?

costela avatar Apr 24 '24 06:04 costela