experiment with converting quadratic functions to nonlinear, to simplify/streamline code

[chriscoey]

Pavito is harder to maintain, debug, and read/understand while the quadratic functions are handled separately from the more general nonlinear functions.

after ScalarNonlinearFunction is finalized and https://github.com/jump-dev/MathOptInterface.jl/pull/2059 is in, I think we should think about perhaps a bridge from ScalarQuadraticFunction to ScalarNonlinearFunction in MOI, or if not a bridge then a more manual conversion inside Pavito.

@odow any thoughts? I'd like to have a shot at this when the time is right.

[chriscoey]

Hmm I wonder if https://github.com/jump-dev/Ipopt.jl/pull/320 slash https://github.com/jump-dev/MathOptInterface.jl/issues/1397 may help here.

[blegat]

Yes, I agree that having a bridge from ScalarQuadraticFunction to ScalarNonlinearFunction would be ideal and would simplify Pavito but it seems to be hard to handle constraints one by one, see https://github.com/jump-dev/MathOptInterface.jl/pull/2059#issuecomment-1355618965

[chriscoey]

I'm a little confused about that point. if our problem has a bunch of ScalarQuadraticFunctions constraints and ScalarNonlinearFunction constraints, and we bridge the SQFs to SNFs, then we have a problem with all SNFs. and that pure nonlinear model can be handled with a single NLP block - so all grad, Hess etc oracles can be done in one go

[blegat]

Yes but it might be more efficient to create the grad/Hess oracles directly from the SQF and not from the SNF where we have lost the special quadratic structure. Otherwise, it could be bridged indeed, @odow is more familiar with the AD internals.

[chriscoey]

Ok, I'd be surprised if it made much difference in practice, but yeah @odow would have a better idea than me of course.

[ccoffrin]

One thing to keep an eye out for is derivative performance on very large problems. Jacobian and Hessian computations can be much faster for the subset of linear and quadratic expressions than general nonlinear expressions.

On Mon, Jan 16, 2023 at 7:34 AM Chris Coey @.***> wrote:

Ok, I'd be surprised if it made much difference in practice, but yeah @odow https://github.com/odow would have a better idea than me of course.

— Reply to this email directly, view it on GitHub https://github.com/jump-dev/Pavito.jl/issues/67#issuecomment-1383989448, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIN5KW75A6RHIZUOUFIP7TWSU533ANCNFSM6AAAAAAT32OEVY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[chriscoey]

This would be an issue for the nonlinear solver, so we would want to preserve quadraticness there. but for deriving cuts from gradients for the OA mip solver, it shouldn't matter because the costs would surely be dwarfed.

[odow]

I think we should think about perhaps a bridge from ScalarQuadraticFunction to ScalarNonlinearFunction in MOI

Yes. That's reasonable once we have an ScalarNonlinearExpression in MOI.

Jacobian and Hessian computations can be much faster for the subset of linear and quadratic expressions than general nonlinear expressions.

The default JuMP AD has support for detecting linear expressions, but not quadratic.

but for deriving cuts from gradients for the OA mip solver, it shouldn't matter because the costs would surely be dwarfed.

Yeah. For the trade-off, the overhead of Pavito going back and forward to the MIP solver, etc, is probably larger than the AD cost of explicit quadratic versus an NLP version of quadratic.

[odow]

8 months later... we should take another look at this

[chriscoey]

Yeah! I won't have time in the next few months I think, but happy if anyone else wants to try it