kit
kit copied to clipboard
go-kit
How to leverage kit/log logging in several .go files/libraries without producing an overwhelming amount of output?
I have experience with log4cxx and log4net that support hierarchical logging. I'd like to include a range of logging in my libraries and .go files but I'm not sure what the best approach is to keep the console output down when doing so.
If I have an issue I'll raise the level to debug to see the details. Even though I know the issue is likely in a single or handful of .go files, or maybe in a library, raising the level means seeing the debug output from everywhere I'm using logging that is part of the execution path. I could comment those log entries out but that's a lot of extra time.
Is there a generally accepted way of including logging in all of your .go files and methods without overwhelming yourself with output that you don't need to enable unless debugging that particular file?
If you add a key that identifies the subsystem that is the origin of a log event, then you can filter on that.
On Thu, Jul 6, 2017, 4:06 PM Chris Morgan [email protected] wrote:
I have experience with log4cxx and log4net that support hierarchical logging. I'd like to include a range of logging in my libraries and .go files but I'm not sure what the best approach is to keep the console output down when doing so.
If I have an issue I'll raise the level to debug to see the details. Even though I know the issue is likely in a single or handful of .go files, or maybe in a library, raising the level means seeing the debug output from everywhere I'm using logging that is part of the execution path. I could comment those log entries out but that's a lot of extra time.
Is there a generally accepted way of including logging in all of your .go files and methods without overwhelming yourself with output that you don't need to enable unless debugging that particular file?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/go-kit/kit/issues/565, or mute the thread https://github.com/notifications/unsubscribe-auth/ABAvZ-xt-GPtHmiKxlxiiGl7tccrDm5Zks5sLT4ngaJpZM4OQHst .
I think I kind of recall reading that somewhere but wasn't sure if there was an example. If not I could always contribute one and some docs if that is a gap. It's something that wasn't clear and could see that being quite helpful to other people.
I'd be happy to look at some ideas for examples. I will say in advance that I am likely to be rather conservative in what I would want to accept into the package itself, at least until it has been somewhat proven to be a good idea.
On a more supportive note, though, I would also be happy to brainstorm or review ideas for how to best use the Go Kit log packages to meet specific use cases. Based on the capabilities you mentioned in your first post above, for example, it would be interesting to explore the idea of a Logger that filters log events by inspecting the call site data recorded by log.Caller for things like the source package, file, or function.
My typical use case with log4cxx and log4net is to name the logger instance, the thing that you can control from the top level, after the class name (which by convention matches the file name). If Go Kit log could control log output based on a given level in a given package name, function or file I think that would provide the same functionality without the additional level of having to manually add a key to each log message.
So I quite like the idea of being able to filter, maybe psuedo code like:
Display(InfoOrAbove) AlsoDisplayPackage(packageNameHere, DebugOrAbove)
I'm new to Golang but was under the impression that package names had to be unique? If so that could ease the use case of filtering by package name or file. (Imo it was rare to want function level of enable/disable but someone may find that handy depending on the side of their codebase or complexity of functions.)
@ChrisHines I'm looking at the go-kit log code.
Should the approach be syntax like:
logger = level.NewFilter(logger, level.AllowError())
logger = log.WithPackage(logger, "package name here")
And after reading more of the code, should I be looking more towards a package valuer akin to DefaultCaller or one of the other valuers? That seems like a better approach.
@ChrisHines any comments on direction? I'd like to hack something together but would appreciate some direction before starting.
@chmorgan
I don't mean to cut into this issue, but for sure the "subsystem" approach to the logger is nice. Typically you would want a key to your value, or "subsystem"/tag, you want to add to the logger. This can be done something like this ...
// logger is some global log.Logger with more logging configurations or what have you
logger = level.NewFilter(logger, level.AllowError())
// appends the 'method=Create' key-value pair when using this logger
logger = log.With(logger, "method", "Create")
Typically you can just keep adding key-value pairs and pass the logger to a specific service, in go-kit designs, or endpoint logging. It's all very flexible to however you choose to use it.
@jtaylor32 that's why I was wondering if something like DefaultCaller but for the current package might work. I'm also not sure if those kvps will show up in the final output, it might be ok, or even helpful to have them there.
I'm just not experienced enough with go and go-kit/log to want to dig in without some clear direction.
I was speaking with a colleague who wanted something very similar to this: a component that they could wire into their logger stack, that would allow or deny certain key=value pairs based on API calls. They wanted to wire the API into an HTTP handler, to dynamically enable/disable logging on a per-package (or per-level, or whatever other) basis.
Strawman API:
filter := log.NewFilter()
logger := log.NewLogfmtLogger(os.Stdout)
logger = filter.Decorate(logger)
logger = log.With(logger, "ts", log.DefaultTimestampUTC())
http.Handle("/debug/log/", http.StripPrefix("/debug/log/", filter))
