Convergence Logging

[mileslucas]

Right now logging in the convergence sampling mode is almost useless. The @logprogress requires passing a fraction from 0 to 1, which isn't conducive to convergence logging at all. I realize this is backed up into the design of ProgressLogging.jl, which is even further backed up into the current logging backends, some parts end up tied directly into base julia. I've tried circumventing the issue by printing out my own statements but they actually get gobbled up due to the @withprogresslogging context.

I'm not sure the best path forward, other than re-introducing the ProgressMeter.jl dependency only for those sampling routines.

[devmotion]

Do you have an example for what you tried and how it failed?

[mileslucas]

Inside my default convergence method:

# logging; don't even try to print every time
if progress && iszero(iteration % 10)
    print("dlogz threshold: $(dlogz_current) --> $(dlogz)\r")
end

I also tried printing to stderr instead of stdout, either way, this is all I ever see:

Convergence sampling 100%|███████████████████████████████████████████████████████████████████████████████████| Time: 0:00:00

(this is not different during the sampling, either)

I've also tried switching the carriage return to the beginning of the string instead of the end, with no improvement. I have tested using a basic for-loop that printing with a carriage return works for "logging", so I'm fairly confident the context is killing it.

[devmotion]

Sorry, it's still unclear to me where exactly you put these lines of code and how you called them. Do you have a MWE or a link to a repo?

[mileslucas]

https://github.com/TuringLang/NestedSamplers.jl/blob/59191565d0163be14341fd1ab2f2d52c6674f7f3/src/NestedSamplers.jl#L309-L330

This is a function that is being passed to is_done in the sampling routine.

[devmotion]

Ah OK, and do you have a MWE that I can run to reproduce the issue?

[mileslucas]

using NestedSamplers, StatsBase

priors = [Uniform(0, 1)]
model = NestedModel(p->rand(), priors)
samples = sample(model, Nested(10), dlogz=1e-5, chain_type=Array) 

It should "freeze" a little to show both during sampling and after sampling outputs

[mileslucas]

See also: https://github.com/JunoLab/ProgressLogging.jl/issues/27

[devmotion]

Thanks for the MWE, I'll have a look at it. In principle, I think the right approach is to add support for convergence based logging upstream in ProgressLogging, as you already suggested, in a similar way as also the possibility to print additional information should preferably be added upstream (see https://github.com/JunoLab/ProgressLogging.jl/issues/23).