Handling retries gracefully without logging error from `eachMessage`

[sdomeshok]

We are using the consumer with eachMessage and our processing semantics assume that we can throw from anywhere in our handler with the assumption that the message will just replay. We specifically use this to handle failures in optimistic locking when writing to our data store.

However, because eachMessage insists on logging on error when handle throws, we end up with a lot of clutter in our logs from errors that are actually totally benign and expected.

See https://github.com/tulios/kafkajs/blob/master/src/consumer/runner.js#L152

Is there a better way to be handling retries? We could of course retry explicitly, but that seems less graceful and more likely to put us at risk of bumping into session timeouts, rather than just yielding back to the runtime.

We do note that we could just wrap our exception as a KafkaJSError but we don't have a clue what the other consequences of that might be.

[wong-codaio]

@sdomeshok - Agree that there might be a better way to handle it.

As a temporary workaround to the log flood, I wonder if you can override the default logger to ignore/by-pass the message for the time being.