cvxgrp
Performance issues with SCS 2.1.4 vs 2.1.2
Specifications
- OS: Zorin OS 16 (Ubuntu 20.04.3 LTS)
- SCS Version: 2.1.4
- Compiler:
Description
Using SCS 2.1.4 with cvxpy is significantly slower than with 2.1.2, and sometimes even fails.
Additional information
My particular use case requires pretty intense calculations with cvxpy. ECOS was selected by default but I had to switch to SCS because it couldn't handle some of the larger tasks.
While switching from Windows to an Ubuntu-based OS, I ran a reproducibility test to check if everything worked as expected. However, the part of my script that calls cvxpy was significantly slower, and failed at some point. I investigated further, and I noticed than when reinstalling cvxpy through conda, it automatically installed the latest version of SCS, while my previous environment used SCS 2.1.2. Switching back to that version solved the issue.
As this issue is probably specific to the problem and data I'm working with, I am just wondering if there any release notes or summary of what changed between those two versions ? I have tried looking at PRs and issues, but the only thing I could find was comparing files between the two branches, and trying to understand that kind of code is well above my abilities.
Edit : I looked a bit further into CPU usage, running a specific task with SCS 2.1.2 makes the CPU spike at about 55%, while running the same task with SCS 2.1.4 makes the CPU max out at 99 to 100%, constantly. I am using an Intel i7-10810U.
That's very strange because the main changes between those two tags are to the GPU solver (which I presume you're not using here) and adding the cmake install option (which doesn't affect python). No core part of SCS was changed. Can you run cvxpy with verbose=True in both cases and put the output here so I can take a look?
Here is the list of files changed between those two tags:
=> git diff v2.1.2 v2.1.4 --stat
.appveyor.yml | 31 ----
.bumpversion.cfg | 3 +-
.github/ISSUE_TEMPLATE.md | 28 +++
.github/workflows/build.yml | 21 +++
.travis.yml | 36 ----
CMakeLists.txt | 273 ++++++++++++++++++++++++++++
README.md | 66 ++++++-
cmake/AddInstallRPATHSupport.cmake | 237 +++++++++++++++++++++++++
cmake/AddUninstallTarget.cmake | 70 ++++++++
cmake/InstallBasicPackageFiles.cmake | 677 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
docs/scs_logo.png | Bin 0 -> 153979 bytes
include/glbopts.h | 2 +-
linsys/gpu/gpu.c | 61 +++++--
linsys/gpu/gpu.h | 31 +++-
linsys/gpu/indirect/private.c | 140 +++++++++++++--
linsys/gpu/indirect/private.h | 5 +
src/rw.c | 2 +-
17 files changed, 1572 insertions(+), 111 deletions(-)
Here's the output with 2.1.2 :
SCS v2.1.2 - Splitting Conic Solver
(c) Brendan O'Donoghue, Stanford University, 2012
Lin-sys: sparse-direct, nnz in A = 35307 eps = 1.00e-04, alpha = 1.50, max_iters = 5000, normalize = 1, scale = 1.00 acceleration_lookback = 10, rho_x = 1.00e-03 Variables n = 7850, constraints m = 11775 Cones: linear vars: 7845 soc vars: 3930, soc blks: 2 Setup time: 6.94e-03s
Iter | pri res | dua res | rel gap | pri obj | dua obj | kap/tau | time (s)
0| 5.82e+01 5.61e+01 1.00e+00 -5.49e+06 1.25e+04 6.44e-10 2.22e-03
100| 7.55e-05 1.00e-04 4.85e-01 -9.35e-01 7.26e-03 2.09e-13 1.31e-01 200| 3.68e-05 4.52e-05 3.31e-01 -4.90e-01 4.61e-03 8.64e-12 2.49e-01 300| 2.62e-05 3.51e-05 2.42e-01 -3.14e-01 4.68e-03 1.40e-12 3.65e-01 400| 1.91e-05 2.82e-05 1.68e-01 -1.98e-01 4.86e-03 6.15e-12 4.81e-01 500| 1.51e-05 2.08e-05 1.24e-01 -1.37e-01 4.18e-03 6.71e-12 6.04e-01 600| 1.15e-05 1.38e-05 9.17e-02 -9.64e-02 4.53e-03 5.02e-12 7.22e-01 700| 8.79e-06 1.19e-05 6.39e-02 -6.39e-02 4.35e-03 2.32e-11 8.40e-01 800| 7.83e-06 1.13e-05 4.82e-02 -4.62e-02 4.42e-03 1.04e-11 9.58e-01 900| 7.10e-06 8.95e-06 3.98e-02 -3.69e-02 4.46e-03 1.52e-11 1.07e+00 1000| 5.55e-06 7.55e-06 3.08e-02 -2.70e-02 4.78e-03 8.99e-12 1.20e+00 1100| 5.82e-06 8.30e-06 2.57e-02 -2.14e-02 4.89e-03 8.28e-12 1.32e+00 1200| 5.55e-06 7.52e-06 2.31e-02 -1.88e-02 4.87e-03 1.39e-11 1.45e+00 1300| 5.13e-06 7.20e-06 1.84e-02 -1.40e-02 4.82e-03 2.11e-11 1.59e+00 1400| 6.54e-06 8.76e-06 1.87e-02 -1.46e-02 4.48e-03 4.36e-13 1.72e+00 1500| 5.61e-06 7.04e-06 1.59e-02 -1.14e-02 4.76e-03 7.82e-13 1.85e+00 1600| 5.46e-06 7.92e-06 1.35e-02 -8.71e-03 4.95e-03 2.56e-11 1.98e+00 1700| 5.02e-06 6.54e-06 1.17e-02 -6.95e-03 4.86e-03 6.84e-12 2.11e+00 1800| 4.79e-06 7.19e-06 9.47e-03 -4.94e-03 4.63e-03 4.17e-12 2.24e+00 1900| 4.78e-06 6.11e-06 9.11e-03 -4.48e-03 4.72e-03 7.62e-13 2.37e+00 2000| 5.05e-06 7.01e-06 8.26e-03 -3.55e-03 4.77e-03 6.43e-12 2.51e+00 2100| 4.54e-06 5.38e-06 6.99e-03 -2.43e-03 4.61e-03 6.58e-12 2.63e+00 2200| 4.39e-06 6.51e-06 6.15e-03 -1.64e-03 4.54e-03 3.90e-12 2.75e+00 2300| 4.28e-06 5.66e-06 6.35e-03 -1.64e-03 4.75e-03 1.72e-11 2.88e+00 2400| 4.49e-06 6.62e-06 5.68e-03 -1.03e-03 4.68e-03 1.10e-11 3.00e+00 2500| 4.10e-06 5.51e-06 4.89e-03 -4.59e-04 4.45e-03 2.54e-12 3.12e+00 2600| 4.68e-06 6.14e-06 4.79e-03 -3.76e-04 4.44e-03 7.47e-12 3.24e+00 2700| 4.38e-06 6.21e-06 4.57e-03 -2.91e-04 4.30e-03 1.87e-12 3.36e+00 2800| 4.18e-06 5.56e-06 4.25e-03 1.47e-04 4.41e-03 4.41e-12 3.48e+00 2900| 4.50e-06 6.25e-06 3.92e-03 1.77e-04 4.12e-03 1.26e-13 3.62e+00 3000| 3.91e-06 5.73e-06 4.04e-03 2.91e-04 4.35e-03 2.53e-11 3.76e+00 3100| 4.66e-06 5.49e-06 3.59e-03 5.91e-04 4.20e-03 1.14e-11 3.88e+00 3200| 4.00e-06 5.99e-06 3.49e-03 5.81e-04 4.08e-03 9.86e-12 4.01e+00 3300| 3.84e-06 5.32e-06 3.51e-03 6.45e-04 4.17e-03 4.57e-12 4.14e+00 3400| 3.59e-06 4.74e-06 3.17e-03 9.05e-04 4.10e-03 8.48e-12 4.27e+00 3500| 4.05e-06 5.32e-06 3.04e-03 8.69e-04 3.92e-03 2.13e-12 4.39e+00 3600| 3.52e-06 5.08e-06 3.21e-03 9.10e-04 4.13e-03 5.40e-12 4.52e+00 3700| 3.69e-06 4.78e-06 2.84e-03 9.72e-04 3.82e-03 2.21e-11 4.65e+00 3800| 3.75e-06 4.52e-06 2.89e-03 8.68e-04 3.77e-03 1.32e-11 4.77e+00 3900| 3.70e-06 5.31e-06 2.93e-03 1.01e-03 3.95e-03 7.67e-12 4.90e+00 4000| 3.71e-06 5.38e-06 2.51e-03 1.08e-03 3.60e-03 7.31e-13 5.03e+00 4100| 3.94e-06 5.34e-06 2.90e-03 9.40e-04 3.85e-03 9.56e-12 5.15e+00 4200| 4.05e-06 4.75e-06 2.55e-03 1.08e-03 3.63e-03 2.51e-11 5.28e+00 4300| 3.82e-06 5.09e-06 2.42e-03 1.08e-03 3.51e-03 5.68e-12 5.40e+00 4400| 3.60e-06 5.27e-06 2.75e-03 1.01e-03 3.77e-03 8.18e-12 5.53e+00 4500| 3.29e-06 5.06e-06 2.36e-03 1.18e-03 3.55e-03 2.45e-12 5.65e+00 4600| 3.31e-06 4.49e-06 2.64e-03 9.60e-04 3.61e-03 1.18e-11 5.78e+00 4700| 3.63e-06 4.29e-06 2.46e-03 1.11e-03 3.58e-03 8.18e-12 5.90e+00 4800| 3.49e-06 4.57e-06 2.34e-03 1.08e-03 3.43e-03 1.83e-11 6.03e+00 4900| 3.40e-06 5.04e-06 2.48e-03 9.85e-04 3.47e-03 1.62e-11 6.15e+00 5000| 3.59e-06 4.71e-06 2.61e-03 8.78e-04 3.50e-03 7.99e-12 6.28e+00
Status: Solved/Inaccurate Hit max_iters, solution may be inaccurate, returning best found solution. Timing: Solve time: 6.28e+00s Lin-sys: nnz in L factor: 62774, avg solve time: 2.17e-04s Cones: avg projection time: 1.13e-05s Acceleration: avg step time: 7.30e-04s
Error metrics: dist(s, K) = 4.5765e-15, dist(y, K*) = 4.3368e-19, s'y/|s||y| = 2.0987e-18 primal res: |Ax + s - b|_2 / (1 + |b|_2) = 3.3716e-06 dual res: |A'y + c|_2 / (1 + |c|_2) = 5.0109e-06 rel gap: |c'x + b'y| / (1 + |c'x| + |b'y|) = 2.3090e-03
c'x = 0.0011, -b'y = 0.0034
And here it is with 2.1.4 :
SCS v2.1.4 - Splitting Conic Solver (c) Brendan O'Donoghue, Stanford University, 2012
Lin-sys: sparse-direct, nnz in A = 35307 eps = 1.00e-04, alpha = 1.50, max_iters = 5000, normalize = 1, scale = 1.00 acceleration_lookback = 10, rho_x = 1.00e-03 Variables n = 7850, constraints m = 11775 Cones: linear vars: 7845 soc vars: 3930, soc blks: 2 Setup time: 7.55e-03s
Iter | pri res | dua res | rel gap | pri obj | dua obj | kap/tau | time (s)
0| 5.82e+01 5.61e+01 1.00e+00 -5.49e+06 1.25e+04 6.44e-10 1.39e-03
100| 7.55e-05 1.00e-04 4.85e-01 -9.35e-01 7.26e-03 1.79e-11 2.33e-01 200| 3.68e-05 4.52e-05 3.31e-01 -4.90e-01 4.61e-03 8.67e-12 4.87e-01 300| 2.62e-05 3.51e-05 2.42e-01 -3.14e-01 4.68e-03 5.37e-14 7.78e-01 400| 1.91e-05 2.82e-05 1.68e-01 -1.98e-01 4.86e-03 1.29e-11 1.25e+00 500| 1.51e-05 2.08e-05 1.24e-01 -1.37e-01 4.18e-03 3.42e-12 1.71e+00 600| 1.15e-05 1.38e-05 9.17e-02 -9.64e-02 4.53e-03 1.69e-12 2.30e+00 700| 8.79e-06 1.19e-05 6.39e-02 -6.39e-02 4.35e-03 2.18e-11 2.70e+00 800| 7.83e-06 1.13e-05 4.82e-02 -4.62e-02 4.42e-03 1.04e-11 3.09e+00 900| 7.10e-06 8.95e-06 3.98e-02 -3.69e-02 4.46e-03 8.13e-12 3.46e+00 1000| 5.55e-06 7.55e-06 3.08e-02 -2.70e-02 4.78e-03 8.54e-12 3.80e+00 1100| 5.82e-06 8.30e-06 2.57e-02 -2.14e-02 4.89e-03 1.21e-11 4.17e+00 1200| 5.55e-06 7.52e-06 2.31e-02 -1.88e-02 4.87e-03 2.00e-11 4.52e+00 1300| 5.13e-06 7.20e-06 1.84e-02 -1.40e-02 4.82e-03 1.08e-11 4.91e+00 1400| 6.54e-06 8.76e-06 1.87e-02 -1.46e-02 4.48e-03 3.15e-12 5.35e+00 1500| 5.61e-06 7.04e-06 1.59e-02 -1.14e-02 4.76e-03 1.65e-12 5.77e+00 1600| 5.46e-06 7.92e-06 1.35e-02 -8.71e-03 4.95e-03 1.48e-11 6.23e+00 1700| 5.02e-06 6.54e-06 1.17e-02 -6.95e-03 4.86e-03 7.91e-12 6.55e+00 1800| 4.79e-06 7.19e-06 9.47e-03 -4.94e-03 4.63e-03 4.54e-12 7.00e+00 1900| 4.78e-06 6.11e-06 9.11e-03 -4.47e-03 4.72e-03 1.42e-13 7.38e+00 2000| 5.05e-06 7.01e-06 8.26e-03 -3.55e-03 4.77e-03 5.52e-13 7.71e+00 2100| 4.54e-06 5.38e-06 6.99e-03 -2.43e-03 4.61e-03 2.29e-11 8.11e+00 2200| 4.39e-06 6.51e-06 6.15e-03 -1.64e-03 4.54e-03 5.15e-12 8.48e+00 2300| 4.28e-06 5.66e-06 6.35e-03 -1.64e-03 4.75e-03 9.89e-12 8.87e+00 2400| 4.49e-06 6.62e-06 5.68e-03 -1.03e-03 4.68e-03 1.07e-11 9.23e+00 2500| 4.10e-06 5.51e-06 4.89e-03 -4.59e-04 4.45e-03 2.20e-11 9.62e+00 2600| 4.68e-06 6.14e-06 4.79e-03 -3.76e-04 4.44e-03 1.37e-11 9.97e+00 2700| 4.38e-06 6.21e-06 4.57e-03 -2.91e-04 4.30e-03 2.53e-11 1.03e+01 2800| 4.18e-06 5.56e-06 4.25e-03 1.47e-04 4.41e-03 1.33e-11 1.08e+01 2900| 4.50e-06 6.25e-06 3.92e-03 1.77e-04 4.12e-03 9.64e-12 1.13e+01 3000| 3.91e-06 5.73e-06 4.04e-03 2.91e-04 4.35e-03 1.50e-11 1.17e+01 3100| 4.66e-06 5.49e-06 3.59e-03 5.92e-04 4.20e-03 1.31e-11 1.21e+01 3200| 4.00e-06 5.99e-06 3.49e-03 5.81e-04 4.08e-03 2.64e-12 1.24e+01 3300| 3.84e-06 5.32e-06 3.51e-03 6.45e-04 4.17e-03 8.48e-12 1.28e+01 3400| 3.59e-06 4.74e-06 3.17e-03 9.05e-04 4.10e-03 1.13e-11 1.31e+01 3500| 4.05e-06 5.32e-06 3.04e-03 8.69e-04 3.92e-03 1.32e-11 1.35e+01 3600| 3.52e-06 5.08e-06 3.21e-03 9.10e-04 4.13e-03 1.89e-11 1.39e+01 3700| 3.69e-06 4.78e-06 2.84e-03 9.72e-04 3.82e-03 1.11e-11 1.42e+01 3800| 3.75e-06 4.52e-06 2.89e-03 8.68e-04 3.77e-03 1.25e-11 1.45e+01 3900| 3.70e-06 5.31e-06 2.93e-03 1.01e-03 3.95e-03 4.26e-12 1.48e+01 4000| 3.71e-06 5.38e-06 2.51e-03 1.08e-03 3.60e-03 8.84e-12 1.53e+01 4100| 3.94e-06 5.34e-06 2.90e-03 9.40e-04 3.85e-03 1.95e-11 1.56e+01 4200| 4.05e-06 4.75e-06 2.55e-03 1.08e-03 3.63e-03 2.21e-12 1.60e+01 4300| 3.82e-06 5.09e-06 2.42e-03 1.08e-03 3.51e-03 2.35e-11 1.63e+01 4400| 3.60e-06 5.27e-06 2.75e-03 1.01e-03 3.77e-03 1.30e-11 1.66e+01 4500| 3.29e-06 5.06e-06 2.36e-03 1.18e-03 3.55e-03 9.70e-12 1.70e+01 4600| 3.31e-06 4.49e-06 2.64e-03 9.59e-04 3.61e-03 4.11e-12 1.74e+01 4700| 3.63e-06 4.29e-06 2.46e-03 1.11e-03 3.58e-03 1.55e-11 1.77e+01 4800| 3.49e-06 4.57e-06 2.34e-03 1.07e-03 3.43e-03 6.16e-12 1.83e+01 4900| 3.40e-06 5.04e-06 2.48e-03 9.85e-04 3.47e-03 1.04e-11 1.88e+01 5000| 3.59e-06 4.71e-06 2.61e-03 8.78e-04 3.50e-03 4.22e-12 1.92e+01
Status: Solved/Inaccurate Hit max_iters, solution may be inaccurate, returning best found solution. Timing: Solve time: 1.92e+01s Lin-sys: nnz in L factor: 62774, avg solve time: 3.91e-04s Cones: avg projection time: 2.51e-05s Acceleration: avg step time: 2.88e-03s
Error metrics: dist(s, K) = 7.1054e-15, dist(y, K*) = 2.1684e-19, s'y/|s||y| = -3.4633e-17 primal res: |Ax + s - b|_2 / (1 + |b|_2) = 3.3716e-06 dual res: |A'y + c|_2 / (1 + |c|_2) = 5.0109e-06 rel gap: |c'x + b'y| / (1 + |c'x| + |b'y|) = 2.3093e-03 c'x = 0.0011, -b'y = 0.0034
To my eye those outputs look identical (except for timing etc). Could it be something changed inside cvxpy?
I investigated a bit further, and it might indeed be because of cvxpy. I thought I had 1.1.7 installed in both of my environment, but in fact what is listed in my conda environment isn't what is actually used. It seems I cannot use scs 2.1.2 with both cvxpy 1.1.7 and 1.1.15, or at least I cannot install 2.1.4 on 1.1.7. My 2.1.4 environment probably updated cvxpy as well without me noticing.
By the way, I noticed that installing scs through pip didn't work, as in the installation is marked as completed but running a script using SCS freezes because "SCS is not installed". I ran into the same problem when installing cvxpy with pip, where the install fails on installing SCS. Using conda works otherwise.