interledger-deprecated
Consider different forms of backoff
Currently, we use exponential backoff. But jittering backoff may be better.
It's unlikely to matter much for our use case, but it's also a pretty simple change.
I don't understand what jitter backoff would buy us -- as I understand it, it's meant for cases involving resource contention.
Couldn't there be contention around a PUT /notifications/:id endpoint? Let's say you're running an FX service and a large number of payments executes at once. (E.g. because they were all triggered by some common event.)
The sudden spike of /notifications crashes your backend database. However, the retries all happen at the same (exponentially increasing) intervals, so every time the notifications are retried, your database crashes again. End result is that a lot of notifications time out on each retry and it takes much longer than necessary to process the backlog.
Plus jitter isn't any more complex to implement than exponential backoff - it's just a different one-liner. So even if we consider the above scenario fairly remote, there doesn't seem to be much of a reason not to use jitter just in case. (For testing we can mock Math.random to get back determinism.)
Ok, so reading that article again, the best performing algorithms were Full Jitter and Decorrelated Jitter. Either one would be fine I think. Full Jitter seems a tad more natural to me and easier to understand, so that's the one I propose that we use:
sleep = random_between(0, min(cap, base * 2 ** attempt))
In the context of our code:
let retries = notification.retry_count = (notification.retry_count || 0) + 1
let delay = Math.random() * Math.min(120, Math.pow(2, retries))
notification.retry_at = new Date(Date.now() + 1000 * delay)
It's literally just adding the Math.random() in the second line.
@sentientwaffle @naoitoi Agreed?
