noleme-flow icon indicating copy to clipboard operation
noleme-flow copied to clipboard

[API] Retry

Open eledhwen opened this issue 3 years ago • 0 comments

Migrated over from https://github.com/lumio-medical/lumio-flow/issues/6

Add the ability to make nodes retry-able.

Nodes should be made retry-able if certain conditions are met such as

  • a Predicate matching over the return value
  • an interruption error being thrown
  • a blocking error being thrown (actor signed exceptions or unchecked exceptions)

This could look like the following snippet and should be rather straightforward to implement.

Let's assume the following function for clarity's sake, we'll use it as a transformer:

public static HttpResponse<InputStream> sendRequest(URL url) throws TransformationException
{
    try {
        HttpClient http = HttpClient.newBuilder().build();
        return http.send(request, HttpResponse.BodyHandlers.ofInputStream());
    }
    catch (InterruptedException | IOException e) {
        throw new TransformationException("An error occurred while attempting to send the request.", e);
    }
}

Then we could do this to make the above code retry as long as the predicate isn't matched.

Recipient<InputStream> flow = Flow
    .from(() -> "https://github.com")
    .pipe(HttpTransformers::asURL) //this is a lumio-etl feature, don't mind it, it simply transforms a string into a URL
    .pipe(Main::sendRequest).retryIf(response -> (response.statusCode() < 200 || response.statusCode() >= 300))
    .collect(HttpResponse::body)
;

We could make it retry if a specific exception is caught:

Recipient<InputStream> flow = Flow
    //[...] same as above
    .pipe(Main::sendRequest).retryIfError(SomeException.class)
    //[...] same as above
;

We could make it retry if an interruption notice is sent:

Recipient<InputStream> flow = Flow
    //[...] same as above
    .pipe(Main::sendRequest).retryIfInterrupted()
    //[...] same as above
;

We could add more fine-grained options, some ideas:

  • how many retries ?
  • how much time spent retrying ?
  • sleep in between retries ?
  • fallback transformer with different parameters ? possibly a bi-transformer leveraging the previous output ?
  • inspection of exceptions if applicable ?

eledhwen avatar Mar 25 '21 00:03 eledhwen