Fast resolves for NLP

[mlubin]

https://github.com/JuliaOpt/MathOptInterface.jl/pull/202. Needs more infrastructure outside JuMP first (i.e. MOIT, MOIU, solver wrappers). We'll have to think about how we want to test this on the JuMP side.

[mlubin]

This is mostly covered by https://github.com/JuliaOpt/JuMP.jl/pull/1223 except for duals and deciding what to do about incremental solves.

[mlubin]

Duals are working. The only remaining issue is re-implementing fast resolves which I'll drop from the 0.19 milestone.

Under 0.18 and prior, if only parameter values changed between solves, then JuMP would skip much of the initial setup after the first solve. I don't know if anyone used this feature. Please ping or :+1: this thread if it's important.

[mlubin]

Dropping this from the 1.0 milestone. I've yet to see anyone who has said that they care and have benefited from fast NLP resolves.

[odow]

I took a look at this and started hacking some things. But it's actually a little tricky, because now that we abstract the AD from the Nonlinear.Model object, there's no guarantee that updating the parameter values in the model will be reflected in the NLP evaluator.

And even if JuMP doesn't set a new NLPBlockData, solvers like Ipopt call MOI.initialize in their optimize! call anyway, https://github.com/jump-dev/Ipopt.jl/blob/a1502e6bc410ee4531e07cc792932a7b0453e73b/src/MOI_wrapper.jl#L1085, which is the cause of the initial overhead.

I'll also note that this has had no comments since 2019, and only one thumbs up, so

I've yet to see anyone who has said that they care and have benefited from fast NLP resolves.

is probably still true. The overhead of setting up the AD system obviously isn't that big of a deal.

An alternative approach moving forward (that would require a fair bit more work) is probably to allow solvers like Ipopt to accept a Nonlinear.Model object directly, and then they can just check if the nonlinear model object is === to the last one they received.

I vote we close this. If someone asks in future, we can revisit.

[blegat]

Maybe now with https://github.com/odow/SymbolicAD.jl it makes more sense to care about it ?

[odow]

Not sure. It'd still take some changes in Ipopt.jl to implement properly. I think we should wait to see someone complain that it is a bottleneck. Premature optimization and all that.

[odow]

We discussed this on the monthly nonlinear call.

[odow]

@ccoffrin wants to properly benchmark on an AC power flow problem.

[odow]

I talked with the folks at https://github.com/LAMPSPUC yesterday, and this is a problem for them with PowerModels. So yes, I've now seen it in the wild. (They want to solve a sequence of 15-minute power-flow problems over a 4 year time horizon.)

It's the exact problem from the benchmark in https://github.com/jump-dev/Ipopt.jl/pull/321.

[ccoffrin]

Indeed. This repeated AC Power Flow problem crops up many places in power systems research and there are similar use-cases in all infrastructure sectors and I would guess MPC as well. The at the moment frameworks like SIIP and PowerModels have built non-JuMP alternatives to hit required performance targets for these work flows.

[odow]

Running @ccoffrin's (private) examples, but with https://github.com/lanl-ansi/nlp-jump-examples/blob/main/pf/nlparameter.jl updated to use the new nonlinear syntax and parameters, we now get:

julia> include("pf/base.jl"); @time bench_pf_base(case="pglib_opf_case2000_goc.m")
...
179.875041 seconds (1.05 G allocations: 62.591 GiB, 26.58% gc time)

julia> include("pf/nlsolve.jl"); @time bench_pf_nlsolve(case="pglib_opf_case2000_goc.m")
...
 25.218796 seconds (56.25 M allocations: 4.750 GiB, 4.83% gc time, 0.02% compilation time)

julia> include("pf/nlparameter.jl"); @time bench_pf_nlparameter(case="pglib_opf_case2000_goc.m")
...
 42.277532 seconds (40.22 M allocations: 1.928 GiB, 2.84% gc time)

So we're much better than before but still a little bit left on the table. I'd need to profile where exactly the remaining time is spent, but it's no longer JuMP's fault of rebuilding the evaluator every time.

I don't know if there's an easy way to test this at the JuMP level, so we could consider just closing.

[ccoffrin]

That's a really nice improvement, yey!

[odow]

Closing because there's nothing left to do here. The key bottleneck was that JuMP rebuild the nonlinear evaluator on every call to optimize!. That doesn't happen anymore, so the decision is up to the solver.