initial parameters / NUTS step size

[chriswaudby]

Hi everyone,

I’ve just started using Turing, and have a question about setting initial parameters, and the behaviour of the NUTS sampler. I’m quite new to this area so forgive me if this should be obvious. I found the documentation a bit unclear on setting initial parameters, but I was able to find a bit of example code that seems to be working:

model = nmr(t,yobs)
varinfo = Turing.VarInfo(model)
model(varinfo, Turing.SampleFromPrior(), Turing.PriorContext(θ0))
initθ = varinfo[Turing.SampleFromPrior()]
chain = sample(model, NUTS(), 100, init_theta=initθ)

However, I’ve noticed that the NUTS sampler runs a number of cycles from a random starting point first to set the step size, before my initial parameters are used (below). This generates quite a few warnings about numerical errors, which don’t occur once I have a sensible starting point. Is this something to be concerned about?

┌ Warning: The current proposal will be rejected due to numerical error(s).
│   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)
└ @ AdvancedHMC ~/.julia/packages/AdvancedHMC/P9wqk/src/hamiltonian.jl:47
...
┌ Info: Found initial step size
└   ϵ = 0.00078125
┌ Info: Using passed-in initial variable values
│   init_theta =
│    24-element Array{Float64,1}:
│      1.1333897198974898
│     -0.14345687626868692
│      ⋮
└      0.0

Thanks!

[trappmartin]

cc: @xukai92

[xukai92]

I'm a bit off from the status of the AbstractMCMC interface therefore I cannot infer what's the order now. But I believe the correct order should be (1) set initial parameters and (2) call find_good_stepsize. @devmotion can you help me check if this is the case or not?

[devmotion]

Probably it is better to check the implementation in https://github.com/TuringLang/DynamicPPL.jl/pull/150/ instead of the current one in Turing. The implementation in https://github.com/TuringLang/DynamicPPL.jl/pull/150/files#diff-2c85169e13f65836353350667ef4fa08dc52d726a6c20f6c1b4869bd099e4498 first sets the initial parameters, either specified by the user or sampled with SampleFromUniform, and then calls find_good_stepsize in the initial step (implemented as initialstep) if the stepsize was not provided by the algorithm (i.e. if iszero(spl.alg.ϵ)).

[xukai92]

Cool. Thanks for the confirmation David! So it's currently implemented in that PR. This issue should be fixed once the upstream package is updated.