Still an infinite loop in bisect

[marius311]

Sorry to sound like a broken record and I know we discussed in https://github.com/Jutho/OptimKit.jl/pull/10 but there's still an infinite loop path in bisect. I confirm because I run into it sporadically, and editing this line to read if c.dϕ >= 0 || numfg>10 to break out if it tries more than 10 bisect steps fixes it for me.

No doubt I have some numerically noisy gradients causing this, and I'm happy if the package sends me a warning or even errors when bisect fails due to that, but just infinite looping really kills my analysis (eg I run 100 sims and step away from my computer for a while, only to come back and find sim 3 got stuck in an infinite loop and it never went further).

[Jutho]

I am happy to accept a PR, but I would just make a separate branch in the if for the case numfg>10 and print out a warning indeed.

[ArtemStrashko]

Hi, it seems I have exactly the same issue and in my case it would be useful if the optimizer returned something (e.g., just an initial guess x_0). Is there some simple way to force it to stop bisecting after a few attempts and give up (just returning an initial condition)? Thank you.

[Jutho]

I will try to look at this soon; the linesearch logic should be changed somewhat to allow for a graceful return indeed.

[maartenvd]

out of curiosity, is there a small minimal example somewhere of bisect looping?

[ArtemStrashko]

Oh, I am afraid not really right now. I am developing an MPS-like architecture for image classification and when I optimize MPS tensors one by one, sometimes optimizations just gets stuck on some tensor and does not proceed to the next tensors.

[ArtemStrashko]

Hi, if I just download OptimKit.jl and then add this condition numfg>10 as suggested by @marius311, could you please suggest how I then can use it? If I just include a line include("../../OptimKit/src/OptimKit.jl"), I then get an error UndefVarError: ConjugateGradient not defined. But if I also add another line include("../../OptimKit/src/gd.jl"), I get another error LoadError: UndefVarError: AbstractLineSearch not defined. Finally, if I include the following lines

using Printf
include("../../OptimKit/src/OptimKit.jl")
include("../../OptimKit/src/linesearches.jl")
include("../../OptimKit/src/gd.jl")

I get this error: LoadError: UndefVarError: OptimizationAlgorithm not defined (and I always get WARNING: replacing module OptimKit, even though I ran Pkg.rm("OptimKit.jl) before). Thank you.

[marius311]

One hack is just to do

using OptimKit
@eval OptimKit function bisect(iter::HagerZhangLineSearchIterator, a::LineSearchPoint, b::LineSearchPoint)
  # modified body here...
end

Another is you could pkg> dev --local OptimKit and edit the files directly.

[ArtemStrashko]

Thank you @marius311! I use the first approach (with @eval). Sorry for my ignorance, but how should I proceed with the second approach? I mean, I execute pkg> dev --local OptimKit, but what's next? One part of an output of the previous command is OptimKit v0.3.1 ⇒ v0.3.2 dev/OptimKit, but when I cd to \dev, I cannot find OptimKit there. Thank you!