Crash when parsing CPLEX mipopt solutions

[gacheiro]

Details for the issue

What did you do?

I tried to solve an integer model with CPLEX_CMD with a time limit set. I noticed that for some cases PuLP crashes when parsing feasible integer solutions.

A hidden feature that I just found out is that for some reason PuLP tries to optimize it twice. However, this is what happens when no integer solution is found in the first optimization, but one is found in the second optimization:

Read time = 0.01 sec. (2.62 ticks)
CPLEX> New value for time limit in seconds: 15
CPLEX> Version identifier: 20.1.0.0 | 2020-11-10 | 9bedb6d68
CPXPARAM_TimeLimit                               15
Tried aggregator 1 time.
MIP Presolve eliminated 74 rows and 1096 columns.
MIP Presolve modified 27553 coefficients.
Reduced MIP has 1982 rows, 29416 columns, and 138726 nonzeros.
Reduced MIP has 29385 binaries, 31 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.13 sec. (108.17 ticks)
Tried aggregator 1 time.
Detecting symmetries...
Reduced MIP has 1982 rows, 29416 columns, and 138726 nonzeros.
Reduced MIP has 29385 binaries, 31 generals, 0 SOSs, and 0 indicators.
Presolve time = 0.09 sec. (76.76 ticks)
Probing fixed 12989 vars, tightened 58 bounds.
Probing changed sense of 1 constraints.
Probing time = 8.59 sec. (4415.97 ticks)
Clique table members: 7992.
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: deterministic, using up to 16 threads.
Root relaxation solution time = 0.97 sec. (576.62 ticks)

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

      0     0       29.8256   206                     29.8256      126

Zero-half cuts applied:  2

Root node processing (before b&c):
  Real time             =   15.05 sec. (12647.53 ticks)
Parallel b&c, 16 threads:
  Real time             =    0.00 sec. (0.00 ticks)
  Sync time (average)   =    0.00 sec.
  Wait time (average)   =    0.00 sec.
                          ------------
Total (root+branch&cut) =   15.05 sec. (12647.53 ticks)


MIP - Time limit exceeded, no integer solution.
Current MIP best bound =  2.9825581395e+01 (gap is infinite)
Solution time =   15.05 sec.  Iterations = 6414  Nodes = 0 (1)
Deterministic time = 12647.62 ticks  (840.54 ticks/sec)

CPLEX> CPLEX Error  1217: No solution exists.
No changes made.
CPLEX> Version identifier: 20.1.0.0 | 2020-11-10 | 9bedb6d68
CPXPARAM_TimeLimit                               15
MIP emphasis: balance optimality and feasibility.
MIP search method: dynamic search.
Parallel mode: deterministic, using up to 16 threads.

        Nodes                                         Cuts/
   Node  Left     Objective  IInf  Best Integer    Best Bound    ItCnt     Gap

*     0+    0                           47.0000       30.0000            36.17%
      0     0       30.1400   202       47.0000   ZeroHalf: 1    14379   30.80%

Zero-half cuts applied:  3

Root node processing (before b&c):
  Real time             =   15.03 sec. (20002.10 ticks)
Parallel b&c, 16 threads:
  Real time             =    0.00 sec. (0.00 ticks)
  Sync time (average)   =    0.00 sec.
  Wait time (average)   =    0.00 sec.
                          ------------
Total (root+branch&cut) =   15.03 sec. (20002.10 ticks)

Solution pool: 1 solution saved.

MIP - Time limit exceeded, integer feasible:  Objective =  4.7000000000e+01
Current MIP best bound =  3.2522329910e+01 (gap = 14.4777, 30.80%)
Solution time =   15.03 sec.  Iterations = 25946  Nodes = 0 (1)
Deterministic time = 20002.19 ticks  (1330.73 ticks/sec)

CPLEX> Incumbent solution written to file /tmp/83816d43922d4b12b3b0b36f2e00e638-pulp.sol'.
CPLEX> Traceback (most recent call last):
  ...
  File ".../venv/lib/site-packages/pulp/pulp.py", line 1737, in solve
    status = solver.actualSolve(self, **kwargs)
  File ".../venv/lib/site-packages/pulp/apis/cplex_api.py", line 101, in actualSolve
    status, values, reducedCosts, shadowPrices, slacks, solStatus = self.readsol(tmpSol)
  File ".../venv/lib/site-packages/pulp/apis/cplex_api.py", line 168, in readsol
    shadowPrices[name] = float(shadowPrice)
TypeError: float() argument must be a string or a number, not 'NoneType'

As you can see in the log above, one integer feasible solution is found during the second optimization, but PuLP crashes when parsing it.

What did you expect to see?

The integer solution found by CPLEX.

What did you see instead?

Nothing. PuLP crashes when trying to parse CPLEX mipopt solution.

Useful extra information

What operating system are you using?

• [ ] Windows: ( version: ___ )
• [x] Linux: ( _distro: ubuntu)
• [ ] Mac OS: ( version: ___ )
• [ ] Other: ___

I'm using python version:

• [ ] 2.7
• [ ] 3.4
• [ ] 3.5
• [ ] 3.6
• [x] Other: 3.9

I installed PuLP via:

• [x] pypi (python -m pip install pulp)
• [ ] github (python -m pip install -U git+https://github.com/coin-or/pulp)
• [ ] Other: ___ (conda?)

Did you also

• [x] Tried out the latest github version: https://github.com/coin-or/pulp
• [x] Searched for an existing similar issue: https://github.com/coin-or/pulp/issues?utf8=%E2%9C%93&q=is%3Aissue%20

[pchtsp]

I've seen this issue when using CPLEX. For some reason, and with some versions of CPLEX, if the solver does not find an integer solution in the timeLimit, the solver re-starts the search and continues. Sadly, if CPLEX does find an integer solution in that second run, the code it returns is weird and PuLP does not know what to do with it. So:

1. The root cause I think has to do with CPLEX itself (PuLP does not ask for a second resolution...).
2. I'm not sure if we can do anything in the PuLP side. I think at some point I used to do some try-catch somewhere outside pulp to detect this behavior.
3. Feel free to propose any changes, including a PR if you think there's something we can do to avoid it / deal with it.