Roadmap for 1.x

[Petr-Hlavenka]

Hi, I often use the Logging infrastructure that is part of 1.0 stdlib. I'd like to ask what is the purpose of MicroLogging.jl in the context of Logging being part of stdlib?

• is it a way to enhance the logging functionality? For example, I like the progress logging...
• or is it going to fade-out as the stdlib will get more functionality from here so that MicroLogging will be abandoned in the future?

Also, I'm using a private VisualLogger as a tool, see discussion in discourse. The motivation is, if one writes nice "robust" reusable functionality, e.g. search for peak maxims in an array or image, where the user needs to fine-tune some input parameters that are used somewhere buried inside the function. To see why the algorithm is not satisfied with the provided inputs (e.g. thresholds, ...) the best way is to attach some array intermediates (or directly plots) to a @debug block and after the function is executed with VisualLogger the VisualLogger collects all the objects and those can be reviewed by the user.

Is it possible to host such a functionality in MicroLogging?

[c42f]

MicroLogging was the prototype for the logging frontend which went into Base as of julia-0.7. Right now, it's a prototype for various backend functionality which is desirable in Base, but which I haven't had time/motivation to fully flesh out.

There's now various helper libraries which provide useful logging backend utilities

• @oxinabox's LoggingExtras.jl which provides reusable components for log routing
• @adamslc's ProgressLogging.jl. (Adam, you should check out my StickyMessages utility :-) )
• @PhilipVinc's TensorBoardLogger.jl
• The stdlib Test log testing infrastructure originally prototyped in MicroLogging
• @timholy's logging utilities in Revise.jl
• @wookay's utilities in Bukdu.jl

At some stage we should consolidate some common ideas from the tooling which has been emerging and do a design iteration to make the Logging stdlib more composable with the tooling people want to build.

Currently I've changed jobs and I don't have a strong need for production grade logging which I admit has sapped my motivation to drive this development along! I'm definitely still interested though and would gladly collaborate to review pull requests here and elsewhere.

[Petr-Hlavenka]

Thank you for the great summary.

I really like the basic logging functionality included in the stdlib. Also, I'm fully happy with the @debug @info ... implementation and I don't support any activities in the direction of having any ProgressLogingBase package as suggested in ProgressLogging.jl - I think the progress "keyword" of MicroLogging is the right way.

On the other way, what would be nice to have is the composability of loggers.

Id like to combine a

• InteractiveLogger (nice human readable output) with a
• ResultStorageLoger (that is actually a object persistence container, currently my unpublished VisualLogger) to collect intermediate results for ex-situ post processing, with a
• ModuleFilterLogger that activates logging only for MyModule1 and TheirModule4 and ignores everything else, with an
• IterationFilterLogger that logs e.g. only first 3 and last 3 iterations and at the same time having
• hierarchical progress bar by ProgressLogger also activated by the Logging infra. With a simple composable syntax

lg= @logger InteractiveLogger(color = true, ...) ∘ ResultStorageLoger() ∘ ModuleFilterLogger(whitelist = [MyModule1, TheirModule4]) ∘ IterationFilterLogger([1:3] ,[end-3:end]) ∘ ProgressLogger()
whith_logger(lg) do
...
end

So, yes, this definitely needs an architecture refactor (in a way fof maybe @oxinabox's LoggingExtras.jl ) and some amount of work to do. On the other hand, I think there is no need to touch the stdlib and all the desired composability is possible on the level of packages, with the enabling infrastructure in e.g. MicroLogging.

[oxinabox]

This can definately be done with LoggingExtras. Most of that is similar to stuff in the examples You probably don't combose it as a single stream It would be a tree with demux loggers at each step. Since in LoggingExtras seperates sinks (loggers that display things) form filters (or other loggers that pass on things)

ResultStorageLoger is a pure sink logger that I have been wanting for a while. ProgressLogger is also a pure sink I would like. I used to have one in https://github.com/oxinabox/OhMyLog.jl then that got merged into OhMyRepl, then it got removed cos it turns out to cause a bunch of issues. But with more care it could be done right.

PR to add that those would be welcomed. I think everything else is very simiilar to stuff in the LoggingExtra's readme