Add support for other solvers and OSQP

[bamos]

There are a few technical details to be discussed/worked out here and we should decide on the best interface for this, but just quickly filing this for now to get things started and to have something to point to. It would be really powerful to provide an option to add in a custom solver for the forward pass that we then map to a cone program and differentiate through the conic form for the backward pass so this doesn't need to be explicitly provided. One extremely compelling use case of this is that it would also enable us to easily add in an OSQP solver for the QP forward pass.

I can post some proposals for an interface here later, but for now the one main technical issue that I'm not immediately sure how to work out is how to map from the optimal primal solution of the original problem form to the optimal primal/dual solutions of the conic form for the backward pass

\cc @bstellato

[SteveDiamond]

Random update here, there's been a lot of progress towards adding the necessary canonicalization for OSQP to cvxpy, on the param_qp branch.

[zcyang]

Hi, want to check if the OSQP solver is ready to use? I checked the branches and didn't find the one named param_qp. Thanks!

[mariohsouto]

Does cvxpylayers currently supports any other solver rather than SCS? For my application SCS keeps mistakenly identifying some sub-problems as infeasible. Let me know if there is such capability or if it is a feature that should be released in future versions of cvxpylayers.

[aadharna]

I just wanted to follow-up on this because I noticed that the only solver is still SCS: https://github.com/cvxgrp/cvxpylayers/blob/483e9220ff34a8eea31d80f83a5cdc930925925d/cvxpylayers/torch/cvxpylayer.py#L114

Has there been any work on integrating MIP solvers e.g. GLPK_MI?

[sinemuysal]

Hello, first of all thanks for the amazing library!

I wanted to follow-up on this issue as well. SCS is not very reliable, and increasing number of iterations or decreasing eps doubles the run time in my problem. It would be great to have QP solver option. Thank you very much!

[akshayka]

I just wanted to follow-up on this because I noticed that the only solver is still SCS:

https://github.com/cvxgrp/cvxpylayers/blob/483e9220ff34a8eea31d80f83a5cdc930925925d/cvxpylayers/torch/cvxpylayer.py#L114

Has there been any work on integrating MIP solvers e.g. GLPK_MI?

There has been no work on integrating MIP solvers. Unfortunately, there is no plan on integrating MIP solvers soon, though we have had vague discussions on how this might be done (with Michal Rolinek) in the past. It's a bit out there because the derivative of the solution mapping of integer programs is zero almost everywhere.

You might try Michal's package for MIPs, here: https://github.com/martius-lab/blackbox-backprop

[akshayka]

Hello, first of all thanks for the amazing library!

You're welcome!

I wanted to follow-up on this issue as well. SCS is not very reliable, and increasing number of iterations or decreasing eps doubles the run time in my problem. It would be great to have QP solver option. Thank you very much!

If you want to solve just QPs, you can try ECOS instead. We have not yet documented how to use ECOS, but all you need to do is pass

solver_args={'solve_method':'ECOS'}

to the layer when you call it.

For example,

layer(args, solver_args={'solve_method':'ECOS'})

[aadharna]

I just wanted to follow-up on this because I noticed that the only solver is still SCS: https://github.com/cvxgrp/cvxpylayers/blob/483e9220ff34a8eea31d80f83a5cdc930925925d/cvxpylayers/torch/cvxpylayer.py#L114

Has there been any work on integrating MIP solvers e.g. GLPK_MI?

There has been no work on integrating MIP solvers. Unfortunately, there is no plan on integrating MIP solvers soon, though we have had vague discussions on how this might be done (with Michal Rolinek) in the past. It's a bit out there because the derivative of the solution mapping of integer programs is zero almost everywhere.

You might try Michal's package for MIPs, here: https://github.com/martius-lab/blackbox-backprop

Makes sense that in the MIP b/c we're constrained to a lattice that would force the derivative to zero in most places and therefore make the problem much harder. I've seen similar behaviour in RL were b/c the reward function is zero for most behaviours, it can be very hard to find a good policy. That's clear intuition as to why the problem would be hard.

Thanks for the quick response and the link to the blackbox-backprop repo!

[sinemuysal]

Hello, first of all thanks for the amazing library!

You're welcome!

I wanted to follow-up on this issue as well. SCS is not very reliable, and increasing number of iterations or decreasing eps doubles the run time in my problem. It would be great to have QP solver option. Thank you very much!

If you want to solve just QPs, you can try ECOS instead. We have not yet documented how to use ECOS, but all you need to do is pass

solver_args={'solve_method':'ECOS'}

to the layer when you call it.

For example,

layer(args, solver_args={'solve_method':'ECOS'})

Hi Akshay,

Thanks for the response, and I tried using ECOS but it gives an implementation error. 'NotImplementedError: Exponential cones not supported yet.'

[akshayka]

Thanks for the response, and I tried using ECOS but it gives an implementation error. 'NotImplementedError: Exponential cones not supported yet.'

@sinemuysal Can you give a complete code snippet that we can run to reproduce that? The error makes it sound like your problem is not a QP, but an exponential cone program (meaning you might have atoms like a log, or exp). We haven't implemented support for the exponential cone yet in ECOS, so if your program did use atoms like those, then that error would be expected. But it's possible there's a bug on our end.

[sinemuysal]

Thanks for the response, and I tried using ECOS but it gives an implementation error. 'NotImplementedError: Exponential cones not supported yet.'

@sinemuysal Can you give a complete code snippet that we can run to reproduce that? The error makes it sound like your problem is not a QP, but an exponential cone program (meaning you might have atoms like a log, or exp). We haven't implemented support for the exponential cone yet in ECOS, so if your program did use atoms like those, then that error would be expected. But it's possible there's a bug on our end.

Yes, I have cp.log in constraints. Probably that's the reason for error. I will stick with SCS for now then. Thanks!

[yanshen0]

It would be great if other commercial solvers, such as MOSEK, could be supported. I assume it needs help and collaboration from the MOSEK team, hope they can pull it off.