slf4j-timbre
slf4j-timbre copied to clipboard
fzakaria
High log volume can cause serious performance degredation
When using this library in a project, an upstream library was using apache.http.*, which has significant trace logging. It looks like on every log message this library calls .getStackTrace, which causes a really significant spike in CPU usage, even if our log level isn't at trace.
Does putting apache.http.* in timbre's :ns-blacklist resolve the issue?
I realise this isn't a proper solution but the blacklist should be checked before the call to getStackTrace occurs.
We were scared that there was unknown impact--any time any new upstream dependency is added we're nervous about the potential for new spats of trace logging that causes a performance degredation that's opaque unless you're explicitly looking for it.
Yes I understand your concern. Thanks for confirming the blacklist works.
Please could you now check timbre's :ns-log-level works?
Put :ns-log-level [["apache.http.*" :error]] in your config, and check the calls to getStackTrace are skipped
Is this still an issue? Should we check if we have "apache.http.*" logging in our setup? Is there a better/more reliable way to check without "watching the CPU usage" ?
We (Amperity) eventually wound up writing a new logging library that combines some of the concepts in timbre with a direct SLF4J implementation, so it doesn't suffer from some of the handoff pain this ticket is an example of.
@greglook : thanks for the suggestion. Will check it out in depth.
We have a big app that uses libraries with all sorts of logging. As of right now, we can't filter logging from other logging frameworks. This is kind of a deal breaker.