conduit icon indicating copy to clipboard operation
conduit copied to clipboard

Design: Handling invalid data/errors

Open lovromazgon opened this issue 3 years ago • 0 comments

Feature description

We need to come up with a good design for handling invalid data / errors. If an error happens while processing a message the pipeline currently stops entirely. We need to allow the user to choose a different behavior, e.g. ignore errors, log and continue, reroute to dead letter queue. The design should define how the user specifies the behavior in the API or config file (if we have pipeline config files by then) and how it works behind the scenes.

lovromazgon avatar Mar 16 '22 16:03 lovromazgon