lo icon indicating copy to clipboard operation
lo copied to clipboard

Add AttemptWithDeadline and AttemptWhileWithDeadline / ETA-based retry strategy

Open samber opened this issue 3 years ago • 3 comments

This library already provides 4 "retry" helpers:

  • Attempt
  • AttemptWhile
  • AttemptWithDelay
  • AttemptWhileWithDelay

But a use case is missing: retrying until a deadline is exceeded.

In microservice architectures, we can have service-1, requesting service-2, requesting service-3, requesting service-4... When designing retry strategies, service-4 should retry until service-1 exceeds its intrinsic timeout contract. If the end-to-end timeout is 1s, service-4 must not retry 10 times with a 200ms interval.

I suggest the following prototype: lo.AttemptWithDeadline(maxAttempts, untilDate, func(index int, duration time.Time) error { ... }). If maxAttempts is equal to 0, the callback repeats until untilDate is reached. I'm not sure we need a delay between retries.

WDYT?

samber avatar Nov 28 '22 10:11 samber

Usually, developers may use context.WithTimeout() or context.WithDeadline() when they are involved with deadline stuff, also, for most of the libraries such as gRPC client/server codes that generated from protos, and github.com/go-redis, go.mongodb.org/mongo-driver libraries, they will force the caller to pass context.Context as a parameter in.

So, what about we introduce the variant functions for Attempt* like Attempt*WithContext, but with a additional ctx context.Context parameters required?

I think we could also introduce some utilities functions about context.Context (such as ensuring context.Context as a parameter) to help out the later implementations with context.Context involved? If so, we may build variant functions for Attempt* based on these utilities.

nekomeowww avatar Nov 30 '22 04:11 nekomeowww

I was looking for this generic helper. some func from external library do not accept a context, or you cannot trust that it will not ever block, so you have to wrap their usage with your own context + select to avoid blocking the entire flow. I think that could be made generic

serbrech avatar Jan 07 '25 01:01 serbrech

I was looking for this generic helper. some func from external library do not accept a context, or you cannot trust that it will not ever block, so you have to wrap their usage with your own context + select to avoid blocking the entire flow.

For now, you can use my utility: https://pkg.go.dev/github.com/nekomeowww/fo#Invoke

nekomeowww avatar Jan 08 '25 13:01 nekomeowww