nsiccha

Well, I have no idea what's going on, but things diverge (occasionally) at least as early as [after the first line search in Optim.jl](https://github.com/JuliaNLSolvers/Optim.jl/blob/adaa8db69ae43466f0abad911a0574b443a73e8d/src/multivariate/solvers/first_order/l_bfgs.jl#L223). As for why, I have no...

Looks like it's due to the way the [default optimizer reuses global objects](https://github.com/mlcolab/Pathfinder.jl/blob/20dd77e5514c2d5d65a2a5edfb2cf72e1dfb9758/src/Pathfinder.jl#L26-L37).

Sounds like this is very bad. ALL THREADS will repeatedly and concurrently empty and overwrite `DEFAULT_LINE_SEARCH`'s cache.

I agree 100%. This bug has gone from "affects reproducibility" to "affects correctness".

Sorry that I haven't gotten around to fixing this yet, will probably only manage to do so next week. In the meantime, the obvious workaround is to construct one optimizer...

I'm guessing this makes a bunch of tests fail which rely on the previous behavior?

Ah, okay! Will do. The new test would probably have to check that constructing a `default_optimizer` pre and post a pathfinder run results in identical-in-value but different-in-memory objects, if that...

> My guess is that these are currently passing because the log-density is so trivial that very little time is spent in the linesearch, so the runs don't interfere with...

@sethaxen, right, so sorry, I've been a lot busier than expected. I'll try to do it now :)

Ah, I've been trying to construct a test that fails for `multipathfinder`, but was unable to. The reason being you already catching the `multipathfinder`+stateful optimizer case, see https://github.com/mlcolab/Pathfinder.jl/blob/f4ca90dc3d91f077f479d13904a2b6bf99e8ee25/src/multipath.jl#L181-L182 Which is...