IPNewton / ConstrainedOptim TODOs

[anriseth]

This was originally at https://github.com/JuliaNLSolvers/ConstrainedOptim.jl/issues/4

Here's a list of things that I believe we should focus on next. If anyone is interested in helping out here that would be very much appreciated.

Interface

• [ ] Enable value! et al. for the constraints in NLSolversBase top priority
• [ ] Add a quasi-Newton algorithm
• [x] ~Update MathProgBase interface~ Add MathOptInterface.jl functionality
• [ ] Add a Newton-Krylov scheme

Interior / IPNewton

• [ ] Support sparse Hessian and sparse constraint Jacobian top priority
• [ ] Figure out how to enable the autodiff-tests in constraints.jl again
• [ ] Move "show_linesearch` to Optim.options
• [ ] Rethink how to deal with allow_f_increases for constrained problems (it should "always" be allowed)
• [ ] There are lots of functions from Tim's PR that are not used anywhere. Figure out what to delete.

Linesearch

• [ ] Figure out how Tim inteded to use backtrack_constrained (only backtrack_constrained_grad works at the moment)
• [ ] Move line searches to LineSearches.jl (after we change the interface in LineSearches)

Testing

• [ ] Add more constrained optimization tests to OptimTestProblems
• [ ] Update OptimTests to work with ConstrainedOptim (We need sparse Hessian + Jacobian support before going all in on CUTEst problems)
• [ ] Compare against Ipopt

Docs

• [x] Add ConstrainedOptim's README as docs/tutorial for IPNewton (Maybe we can use some Literate.jl approach to also test the examples in CI?)

[kkmann]

Hey, just wanted to bump this since I just recently used the constrained optimization functionality in Optim.jl - thanks a lot for implementing this! I wanted to use ForwardDiff and followed the IPNewton tutorial as close as possible. I.e. I implemented all gradients, Jacobians, and Hessians myself via ForwardDiff - is there a similar functionality like autodiff = :forward for non-constrained problems (if not, that would be cool ;))? Also, I had a hard time figuring out that the mu parameter is quite important - had to chose it manually to get a solution that respected the constraints.

[anriseth]

Happy to hear that you've used this:)

There is an automatic differentiation convenience call for first order, but not for second order. See https://github.com/JuliaNLSolvers/NLSolversBase.jl/blob/master/src/objective_types/constraints.jl#L122

If you want to extend it to TwiceDifferentialbleConstraints that would be very helpful; PRs welcome :)

When I looked at convenience functions for constraint differentiation I stopped at first order because (1) I wasn't sure at the time what the best way to evaluate the Hessian part of the constraints, and (2) my view is that most cases where we don't have explicit second-order functions it's better to just LBFGS.

Unfortunately I never got around to implement interior point (L)BFGS. I've started working in industry now and don't user Julia anymore, so I won't be fixing any of the to-dos myself.

[anriseth]

Regarding ~my~mu: I'm glad you figured out how to set it manually. Can you please add a section on fixing constraint violations in the IPNewton docs, here https://github.com/JuliaNLSolvers/Optim.jl/blob/master/docs/src/algo/ipnewton.md

If you don't know how, please let us know and we'll give further instructions :)

[antoine-levitt]

I've started working in industry now and don't user Julia anymore, so I won't be fixing any of the to-dos myself.

On behalf of both academia and the Julia ecosystem, sorry to hear that! Best of luck :-)

[anriseth]

On behalf of both academia and the Julia ecosystem, sorry to hear that! Best of luck :-)

Thanks for saying that, much appreciated! I'll probably follow the developments in the Julia-verse and hope I get to come back to it someday :)

[kkmann]

Okay, It's been a while since I fiddled with Documenter - your make.jl complains about a missing site name on my system. Are there any special twists and whistles to your Documenter setup?

Concerning the 2nd order autodiff, I'll have a look at it. Might not be an ideal solution but I got it working for my problem already :)

[anriseth]

I used to set up Documenter to use format = :html when testing locally. You can follow the setup at https://github.com/JuliaNLSolvers/LineSearches.jl/blob/master/docs/make.jl#L15 Just include the relevant docs page in the pages keyword.