redpanda-data
Option to chill error logs on cache processor
Hey all! Really appreciate your work on benthos and really excited to see where the project goes in the future!
Jumping in from the slack channel from a few days ago:
Would it be possible to provide an option that reduces some of the error logs that come out of the cache processor?
We have a pipeline that is regularly checking the cache for the existence of a given key, and we can safely handle the returned error from the operation.
However, this results in the benthos log being full of "errors" that look like this:
ERRO Branch error: processors failed: operator failed for key 'some-long-unique-key-string': key does not exist @service=benthos label=cache_retrieval path=root.pipeline.processors.2
Which seems to originate from the component, rather than a log statement here.
In my experience, checking a cache for the existence of a given key is a reasonably common operation that may happen frequently(depending on the pipeline) and might not be worth logging a "not found" every time.
Just for clarity, setting the handle-able error state from the event is still very valuable here.
I would appreciate this feature, some way to not consider key not found as an error. It is really annoying.
I checked out the code and removing the error message seems straightforward. Logging an error for an operationally common case does seem verbose. But It seems to me though that using error handling on get/add to check for the presence of a cache entry is a very complicated way of saying exists.
Some preliminary code examination shows that creating a boolean exists function should be fairly easy and it should simplify the usage of the cache processor.
+1. I am doing KV lookups with kafka messages and it trashes my error-logs in loki.
