PyPSA
Non-Gurobi Solvers not finding solution
Checklist
- [x] I am using the current
masterbranch - [x] I am running on an up-to-date
pypsa-usaenvironment. Update viaconda env update -f envs/environment.yaml
The Issue
When solving, if I use Gurobi an optimal solution is found on a simple electricity only model. However, if I use other solvers (Ive tested with Highs and CPLEX) I get an unknown termination condition. See below for the outputs from each solver. This error happens for sector networks as well
Linopy writes the LP okay, with the following output
Edit: I think the CPLEX error has to do with the python path, and its defaulting to the community edition then just getting killed cause the problem is too big. But gotta double check that.
linopy log
INFO:linopy.model:Solver options:
- threads: 4
- lpmethod: 4
- solutiontype: 2
- barrier.convergetol: 1e-05
- feasopt.tolerance: 1e-06
INFO:linopy.io:Writing objective.
Writing constraints.: 100%|█████████████████████████████████████████| 31/31 [00:01<00:00, 19.83it/s]
Writing continuous variables.: 100%|██████████████████████████████████| 8/8 [00:00<00:00, 38.21it/s]
INFO:linopy.io: Writing time: 1.93s
Gurobi Output
46 5.20600204e+09 5.20600200e+09 1.90e-07 7.84e-05 5.67e-05 8s
47 5.20600201e+09 5.20600201e+09 4.49e-08 3.86e-05 7.85e-07 8s
Barrier solved model in 47 iterations and 7.68 seconds (11.75 work units)
Optimal objective 5.20600201e+09
INFO:linopy.constants: Optimization successful:
Status: ok
Termination condition: optimal
Solution: 344624 primals, 890676 duals
Objective: 5.21e+09
Solver model: available
Solver message: 2
CPLEX Output
INFO:linopy.solvers:Unable to save solution file. Raised error: CPLEX Error 1217: No solution exists.
WARNING:linopy.solvers:Solution status unknown. Trying to parse solution.
WARNING:linopy.constants:Optimization failed:
Status: unknown
Termination condition: unknown
Solution: 0 primals, 0 duals
Objective: nan
Solver model: available
Solver message: Unknown status value
WARNING:__main__:Solving status 'unknown' with termination condition 'unknown'
highs Output
55 9.24e-12 1.96e-08 6.65831968e+09 6.65550984e+09 3.91e+00 741s
56 9.26e-12 6.11e-09 6.65773024e+09 6.65668218e+09 1.46e+00 750s
57* 1.46e-11 3.73e-09 6.65746563e+09 6.65700661e+09 6.39e-01 759s
Summary
Runtime: 759.21s
Status interior point solve: optimal
Status crossover: not run
objective value: 6.65746563e+09
interior solution primal residual (abs/rel): 1.82e-11 / 1.74e-16
interior solution dual residual (abs/rel): 2.52e-05 / 4.54e-11
interior solution objective gap (abs/rel): 4.59e+05 / 6.90e-05
Ipx: IPM optimal
WARNING: LP solver residuals: primal = 62.5; dual = 1470.48 yield num/max/sum primal (126580/62.5/561308) and dual (46575/1470.48/2413.75) corrections
WARNING: Unwelcome IPX status of Unknown: basis is not valid; solution is valid; run_crossover is "off"
Model name : linopy-problem-hfebb0yy
Model status : Unknown
IPM iterations: 57
Objective value : 5.2139443572e+09
Relative P-D gap : 1.7754187089e+01
HiGHS run time : 759.97
Writing the solution to /tmp/linopy-solve-q1omjfnn.sol
WARNING:linopy.solvers:Solution status unknown. Trying to parse solution.
WARNING:linopy.constants:Optimization failed:
Status: unknown
Termination condition: unknown
Solution: 344624 primals, 890676 duals
Objective: 5.21e+09
Solver model: available
Solver message: unknown
WARNING:__main__:Solving status 'unknown' with termination condition 'unknown'
Steps To Reproduce
Im just running a small model with the following differences from the default config.
scenario:
interconnect: [western] #"usa|texas|western|eastern"
clusters: [4m]
simpl: [12]
opts: [REM-12h]
ll: [v1.0]
sector: "G" # G
planning_horizons: [2030] #(2018-2023, 2030, 2040, 2050)
foresight: 'perfect' # myopic, perfect
model_topology:
transmission_network: 'reeds' # [reeds, tamu]
topological_boundaries: 'reeds_zone' # [county, reeds_zone, state]
interface_transmission_limits: false
include: # mixed zone types not supported
# reeds_zone: []
reeds_state: ['CA']
# reeds_ba: []
aggregate: # eligible keys: [reeds_zone, trans_reg]
# trans_grp: []
# reeds_zone: []
Expected Behavior
No response
Error Message
Anything else?
Not sure if this relates to solution quality as mentioned in issue #633
If I set highs to solve via simplex, rather than ipm, I do arrive at a solution. Alternatively, if I allow crossover with ipm you arrive at a solution. But then plotting fails, so Im guessing something is still not being written out correctly.
highs-simplex:
threads: 4
solver: "simplex"
small_matrix_value: 1e-6
large_matrix_value: 1e9
primal_feasibility_tolerance: 1e-5
dual_feasibility_tolerance: 1e-5
ipm_optimality_tolerance: 1e-4
parallel: "on"
random_seed: 123
262149 5.1845545917e+09 Pr: 55605(2.19757e+09); Du: 0(0.00871426) 206s
268982 5.2042658295e+09 Pr: 51216(2.07586e+08); Du: 0(0.00903361) 212s
271799 5.2059703211e+09 Pr: 0(0); Du: 0(0.000642853) 214s
271799 5.2059703211e+09 Pr: 0(0); Du: 0(0.000642853) 214s
Using EKK primal simplex solver
Iteration Objective Infeasibilities num(sum)
271799 5.2059692795e+09 Pr: 0(0); Du: 602(0.122727) 214s
272411 5.2059703173e+09 Pr: 0(0); Du: 0(8.09872e-11) 214s
Solving the original LP from the solution after postsolve
Model name : linopy-problem-zmld1d4k
Model status : Optimal
Simplex iterations: 272411
Objective value : 5.2059703192e+09
Relative P-D gap : 1.2420185804e-13
HiGHS run time : 214.40
Writing the solution to /tmp/linopy-solve-8u2ym1ua.sol
INFO:linopy.constants: Optimization successful:
Status: ok
Termination condition: optimal
Solution: 344624 primals, 890676 duals
Objective: 5.21e+09
Solver model: available
Solver message: Optimal
Okay, for reference, CPLEX was my fault. I wasn't pulling in my licensed version, so the size was just exceeding problem limit. For future reference, since it took me FOREVER to figure out how to install the licensed version.
pip install cplex
pip install docplex
docplex config --upgrade ~/../../opt/ibm/ILOG/CPLEX_Studio2212/
More information at the bottom of the /opt/ibm/ILOG/CPLEX_Studio2212/cplex/readmeUNIX.html file