How to interpret `Warning: Unrecognized stop reason: ABNORMAL_TERMINATION_IN_LNSRCH . Defaulting to ReturnCode.Default.`

[TorkelE]

I have a profiling problem (generating it is quite extensive, will see if I can generate a somewhat small example, but uncertain about it). When I do

method = OptimizationProfiler(optimizer = Optimization.LBFGS(), stepper = FixedStep(; initial_step=0.25))
prof = profile(plprob, method; idxs = [1])

I get a

┌ Warning: Unrecognized stop reason: ABNORMAL_TERMINATION_IN_LNSRCH                              . Defaulting to ReturnCode.Default.
└ @ Optimization ~/.julia/packages/Optimization/qX4vR/src/utils.jl:93
┌ Warning: Solver returned Default retcode at profile point x = 1.2322235864144397. Profiling is interrupted.
└ @ LikelihoodProfiler ~/.julia/dev/LikelihoodProfiler/src/profiler_state.jl:170
  1.251671 seconds (10.81 M allocations: 720.561 MiB, 13.58% gc time)

and the profile ends up not covering the full range. I am not sure what exactly ABNORMAL_TERMINATION_IN_LNSRCH is, any idea of how to interpret this? Or is it the optimiser itself that is doing something weird?

Using an IntegrationProfiler I do not get this problem (however, it does have very long runtimes).

Partially also asking in case this is some weird edge case that is not checked somewhere.

[ivborissov]

It seems to be related to Optimization.LBFGS() and the error retcodes produced by underlying LBFGS optimizer, which may not be handled correctly by Optimization.jl. Can try another solver, for example, the same LBFGS from NLopt ?

using Optimization, OptimizationNLopt

method = OptimizationProfiler(optimizer = NLopt.LD_LBFGS(), stepper = FixedStep(; initial_step=0.25))

Also, you can play with integrator_opts in the IntegrationProfiler, like set larger dtmin to force the integrator make bigger steps or change the integration alg (integrator). We are currently testing different models to come up with some defaults for these arguments and general recommendations on how to set them.

[TorkelE]

using

using OptimizationNLopt
method = OptimizationProfiler(optimizer = NLopt.LD_LBFGS(), stepper = FixedStep(; initial_step=0.25))
prof = profile(plprob, method; idxs = [1])

gives a different warning type:

┌ Warning: NLopt failed to converge: FAILURE
└ @ OptimizationNLopt ~/.julia/packages/OptimizationNLopt/YE3fr/src/OptimizationNLopt.jl:299
┌ Warning: Solver returned Failure retcode at profile point x = 0.8775524087988034. Profiling is interrupted.
└ @ LikelihoodProfiler ~/.julia/dev/LikelihoodProfiler/src/profiler_state.jl:170
┌ Warning: NLopt failed to converge: FAILURE
└ @ OptimizationNLopt ~/.julia/packages/OptimizationNLopt/YE3fr/src/OptimizationNLopt.jl:299
┌ Warning: Solver returned Failure retcode at profile point x = 0.8775524087988034. Profiling is interrupted.
└ @ LikelihoodProfiler ~/.julia/dev/LikelihoodProfiler/src/profiler_state.jl:170

Currently playing around with the integrator option. Changing dtmin sounds like a good idea, haven't tried that. Will also try the implicit solvers once available.

[ivborissov]

For some reason optimizers are failing with certain parameter values. You can try to get these values and run optimization without profiling to get some idea, what can cause it. You can also try smth gradient-free, like NLopt.LN_NELDERMEAD()

[TorkelE]

tring NLopt.LN_NELDERMEAD() seems useful. It works without errors, although the result seem a bit dependent on the initial step size that I set.