Returning and differentiating through the dual

[bamos]

With diffcp, we can use adjoint_derivative to differentiate through the primal and dual solutions w.r.t. the coefficients. It would be convenient to expose the dual here too and the only part that's not immediately clear to me if it'll be easy or not is that we'll need to map from the cone program's solution back to the dual variables of the CVXPY constraints.

@sbarratt @akshayka -- I remember you mentioning this at some point, have you given any more thought to it?

[sbarratt]

CVXPY already extracts the dual variables associated with CVXPY constraints: https://github.com/cvxgrp/cvxpy/blob/715e7ad1b1e32de7a718225edbae2c96ea6dd00f/cvxpy/reductions/solvers/conic_solvers/scs_conif.py#L218. Since this mapping is also linear, we could surely differentiate this. Should this be a part of cvxpylayers, though, or a part of the cvxpy differentiation interface?

[bamos]

Hmm, would it be easy to add something like split_dual and split_dual_adjoint to ParamConeProg in CVXPY using most of the existing code here and then expose that in CvxpyLayer here in a clean way?

One idea on the interface is that we could add a constraints input to the CvxpyLayer class that returns the dual variables associated with those constraints as the output, in addition to the primals. Using a default of constraints=[] here would be backwards-compatible and exactly the same as the current API.

[mhr]

Any update on this?

[holmesco]

I am also interested in this. Its especially important because setting certain problems up using the dual problem is more efficient in CVXPY (see this example with SDPs, although more control over how CVXPY canonicalizes would help here too). Exposing the dual vars would allow us to formulate using the dual, then use still use the primal variables.