cvxpy
Support complex parameters in DPP problems
I am trying to parameterize a CVXPY program as I need to repeatedly solve the problem, but I noticed that when my parameters are complex numbers, CVXPY models the problem in each iteration.
<img src="https://latex.codecogs.com/svg.image?\underset{\mathbf&space;x}{\text{minimize}}&space;\Vert&space;\mathbf&space;A&space;\mathbf&space;x&space;-&space;\mathbf&space;b\Vert&space;+&space;&space;|\boldsymbol{\gamma}&space;\mathbf&space;x|,&space;" title="\underset{\mathbf x}{\text{minimize}} \Vert \mathbf A \mathbf x - \mathbf b\Vert + |\boldsymbol{\gamma} \mathbf x|, " />
minimizeπ±βππ±βπβ+|πΈπ±|,
Please note: I don't need to solve this problem repeatedly, but I am forcefully doing so to check whether I can consider πΈ as a parameter or not?
import cvxpy as cp
import numpy as np
m = 4
n = 3
A = np.random.randn(n,m)+1j*np.random.randn(n,m)
b = np.random.randn(n)+1j*np.random.randn(n)
x = cp.Variable((m,),complex=True)
gamma = cp.Parameter((1,m),complex=True)
obj = cp.norm(A@x-b)+cp.abs(gamma@x)
prob = cp.Problem(cp.Minimize(obj))
for iIter in range(5):
gamma.value = np.random.randn(1,m)+1j*np.random.randn(1,m)
prob.solve(verbose=True)
What I see from verbose is that CVXPY first models my problem in each iteration and then solves it. On the other hand, if I assume πΈββ1Γ4, it bypasses the modeling from the next iteration. Why is this so? Are we not allowed to create complex-valued parameters?
Hi @vkumar-ucd, thanks for your question and sorry for not getting back to you earlier. I've slightly modified your code to make our investigation easier:
import cvxpy as cp
import numpy as np
m = 20
n = 10
A = np.random.randn(n,m)+1j*np.random.randn(n, m)
b = np.random.randn(n)+1j*np.random.randn(n)
x = cp.Variable((m,), complex=True)
complex = True # or complex = False
gamma = cp.Parameter((1,m), complex=complex)
obj = cp.norm(A@x-b)+cp.abs(gamma@x)
prob = cp.Problem(cp.Minimize(obj))
for iIter in range(5):
gamma.value = np.random.randn(1, m)
if complex:
gamma.value = gamma.value + 1j*np.random.randn(1, m)
prob.solve(verbose=True)
When I use that code with complex = False I get the following output at the first call to prob.solve().
===============================================================================
(CVXPY) Feb 22 09:07:56 PM: Your problem has 20 variables, 0 constraints, and 20 parameters.
(CVXPY) Feb 22 09:07:56 PM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Feb 22 09:07:56 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
Compilation
-------------------------------------------------------------------------------
(CVXPY) Feb 22 09:07:56 PM: Compiling problem (target solver=MOSEK).
(CVXPY) Feb 22 09:07:56 PM: Reduction chain: Complex2Real -> Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> MOSEK
(CVXPY) Feb 22 09:07:56 PM: Applying reduction Complex2Real
(CVXPY) Feb 22 09:07:56 PM: Applying reduction Dcp2Cone
(CVXPY) Feb 22 09:07:56 PM: Applying reduction CvxAttr2Constr
(CVXPY) Feb 22 09:07:56 PM: Applying reduction ConeMatrixStuffing
(CVXPY) Feb 22 09:07:56 PM: Applying reduction MOSEK
(CVXPY) Feb 22 09:07:56 PM: Finished problem compilation (took 1.429e-02 seconds).
(CVXPY) Feb 22 09:07:56 PM: (Subsequent compilations of this problem, using the same arguments, should take less time.)
Then, calls to prob.solve() after updating gamma's value results in output like:
===============================================================================
(CVXPY) Feb 22 09:07:56 PM: Your problem has 20 variables, 0 constraints, and 20 parameters.
(CVXPY) Feb 22 09:07:56 PM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Feb 22 09:07:56 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
Compilation
-------------------------------------------------------------------------------
(CVXPY) Feb 22 09:07:56 PM: Using cached ASA map, for faster compilation (bypassing reduction chain).
(CVXPY) Feb 22 09:07:56 PM: Finished problem compilation (took 2.218e-03 seconds).
The compilation log mentions that it bypasses the reduction chain in this case.
Compare to when complex = True, you get output like the following at every solve. (Note that there is no reference to using a cached ASA map.)
(CVXPY) Feb 22 09:12:11 PM: Your problem has 20 variables, 0 constraints, and 20 parameters.
(CVXPY) Feb 22 09:12:11 PM: It is compliant with the following grammars: DCP, DQCP
(CVXPY) Feb 22 09:12:11 PM: CVXPY will first compile your problem; then, it will invoke a numerical solver to obtain a solution.
-------------------------------------------------------------------------------
Compilation
-------------------------------------------------------------------------------
(CVXPY) Feb 22 09:12:11 PM: Compiling problem (target solver=MOSEK).
(CVXPY) Feb 22 09:12:11 PM: Reduction chain: EvalParams -> Complex2Real -> Dcp2Cone -> CvxAttr2Constr -> ConeMatrixStuffing -> MOSEK
(CVXPY) Feb 22 09:12:11 PM: Applying reduction EvalParams
(CVXPY) Feb 22 09:12:11 PM: Applying reduction Complex2Real
(CVXPY) Feb 22 09:12:11 PM: Applying reduction Dcp2Cone
(CVXPY) Feb 22 09:12:11 PM: Applying reduction CvxAttr2Constr
(CVXPY) Feb 22 09:12:11 PM: Applying reduction ConeMatrixStuffing
(CVXPY) Feb 22 09:12:11 PM: Applying reduction MOSEK
(CVXPY) Feb 22 09:12:11 PM: Finished problem compilation (took 2.157e-02 seconds).
It seems that although compilation produces correct results in both cases, the potential performance benefit of using Parameter objects disappears when the Parameter is complex. @SteveDiamond @akshayka it would be good if one of you could look into this.
Our implementation for extracting the ASA maps doesn't support complex paramters:
https://github.com/cvxpy/cvxpy/blob/master/cvxpy/reductions/solvers/solving_chain.py#L171-L172
We should document this here: https://www.cvxpy.org/tutorial/advanced/index.html#disciplined-parametrized-programming
@akshayka @rileyjmurray Thanks for your comments. What I am confused that if using complex-valued parameters doesn't support ASA maps, what is the need for allowing parameters to be complex-valued?
Eventually complex-valued parameters will support ASA maps. Adding that functionality was more than we could handle at the time ASA was introduced.
@SteveDiamond Makes sense. Thanks a lot for the clarification. Hope that this new feature will be supported soon.
@vkumar-ucd we'll do our best! I've re-opened this issue so it's clear that this is something we would like to do in the future.
Is there still an interest in caching compilations for problems with complex parameters?
Upon inspecting the canonicalization chain, one solution I see is to subclass Parameter to take in a parent parameter and a transformation function. Accessing subclass's .value attr should apply the transformation fn to the value of the parent param.
Then, during complex2real parameter canon, we can simply split a complex parameter into two, one with np.real and the other with np.imag as transformation fns.
Creating a new class with an unconstrained transformation fn is perhaps less than ideal but appears cleaner/easier than passing around problem metadata linking new params to their original. Plus, it looks like this could potentially be adapted to simplify the log(param) logic of solving DGP problems? Any feedback/thoughts on this would be great!
Also, I'm not too familiar with the gradient features of cvxpy and it's admittedly been a while since complex analysis for me, but iirc the only holomorphic real-valued functions are constant -- so there shouldn't be huge use cases for differentiating objective wrt complex params? But an exception might be finding the derivative of the solution vars wrt complex param?
If there's any interest/if this is even viable at all I can turn these changes into a PR -- thanks!
Hi Jeff,
Many thanks for your email. Owing to the fact that complex-valued parameters are used mostly in wireless communications, adding the new feature will be of great help to researchers and practitioners in the domain. However, I am not sure how much this be helpful in other domains like computer science and controls.
As far as I am concerned, this is still of great interest to me.
Cheers, Vaibhav
On Mon, Jan 23, 2023 at 3:06 AM Jeffrey Chen @.***> wrote:
Is there still an interest in caching compilations for problems with complex parameters?
Upon inspecting the canonicalization chain, one solution I see is to subclass Parameter to take in a parent parameter and a transformation function. Accessing subclass's .value attr should apply the transformation fn to the value of the parent param.
Then, during complex2real parameter canon, we can simply split a complex parameter into two, one with np.real and the other with np.imag as transformation fns.
Creating a new class with an unconstrained transformation fn is perhaps less than ideal but appears cleaner/easier than passing around problem metadata linking new params to their original. Plus, it looks like this could potentially be adapted to simplify the log(param) logic of solving DGP problems? Any feedback/thoughts on this would be great!
Also, I'm not too familiar with the gradient features of cvxpy and it's admittedly been a while since complex analysis for me, but iirc the only holomorphic real-valued functions are constant -- so there shouldn't be huge use cases for differentiating objective wrt complex params? But an exception might be finding the derivative of the solution vars wrt complex param?
If there's any interest/if this is even viable at all I can turn these changes into a PR -- thanks!
β Reply to this email directly, view it on GitHub https://github.com/cvxpy/cvxpy/issues/1666#issuecomment-1399731536, or unsubscribe https://github.com/notifications/unsubscribe-auth/AN442A33CZKD4AXTS7JIMK3WTXYS7ANCNFSM5O3IT2DQ . You are receiving this because you were mentioned.Message ID: @.***>
I would say weβre always interested in having different CVXPY features work seamlessly together :)
On Tue, Jan 24, 2023 at 5:11 AM vkumar-ucd @.***> wrote:
Hi Jeff,
Many thanks for your email. Owing to the fact that complex-valued parameters are used mostly in wireless communications, adding the new feature will be of great help to researchers and practitioners in the domain. However, I am not sure how much this be helpful in other domains like computer science and controls.
As far as I am concerned, this is still of great interest to me.
Cheers, Vaibhav
On Mon, Jan 23, 2023 at 3:06 AM Jeffrey Chen @.***> wrote:
Is there still an interest in caching compilations for problems with complex parameters?
Upon inspecting the canonicalization chain, one solution I see is to subclass Parameter to take in a parent parameter and a transformation function. Accessing subclass's .value attr should apply the transformation fn to the value of the parent param.
Then, during complex2real parameter canon, we can simply split a complex parameter into two, one with np.real and the other with np.imag as transformation fns.
Creating a new class with an unconstrained transformation fn is perhaps less than ideal but appears cleaner/easier than passing around problem metadata linking new params to their original. Plus, it looks like this could potentially be adapted to simplify the log(param) logic of solving DGP problems? Any feedback/thoughts on this would be great!
Also, I'm not too familiar with the gradient features of cvxpy and it's admittedly been a while since complex analysis for me, but iirc the only holomorphic real-valued functions are constant -- so there shouldn't be huge use cases for differentiating objective wrt complex params? But an exception might be finding the derivative of the solution vars wrt complex param?
If there's any interest/if this is even viable at all I can turn these changes into a PR -- thanks!
β Reply to this email directly, view it on GitHub https://github.com/cvxpy/cvxpy/issues/1666#issuecomment-1399731536, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AN442A33CZKD4AXTS7JIMK3WTXYS7ANCNFSM5O3IT2DQ
. You are receiving this because you were mentioned.Message ID: @.***>
β Reply to this email directly, view it on GitHub https://github.com/cvxpy/cvxpy/issues/1666#issuecomment-1401926268, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACRLIFEQJ3UCK3LM5GAS2A3WT7IG3ANCNFSM5O3IT2DQ . You are receiving this because you modified the open/close state.Message ID: @.***>
Hey @jlchen0 are you still working on this? I am trying to tackle this issue. PS: any help is appreciated :)
I found some interesting discussions on a similar implementation for julia. Automatic Differentiation for Julia Zygote.jl ForwardDiff.jl