Misleading `OPTIMAL` solution parsed by the Coin API

[wookenny]

Hi all.

I stumbled over a problem where Pulp is interpreting CBCs solution as Optimal whereas CBC itself logged Postprocessed model is infeasible - possible tolerance issue - try without preprocessing.

Digging into the problem, I found that CBC writes a solutions like this:

Optimal (within gap tolerance) - objective value 4030862607.65109825
      0 balanced_sales_constraint(0,_0)                               0          -1.9739301e+08
      1 balanced_sales_constraint(0,_1)                    2.220446e-16               -73504793
      2 balanced_sales_constraint(0,_10)                  -2.220446e-16              -8861044.1
      3 balanced_sales_constraint(0,_11)                              0              -5441152.5
      4 balanced_sales_constraint(0,_12)                   2.220446e-16               -11581374
      5 balanced_sales_constraint(0,_13)                              0              -9966135.6
      6 balanced_sales_constraint(0,_2)                               0               -63164426
      7 balanced_sales_constraint(0,_3)                               0               -13694117
**       8 balanced_sales_constraint(0,_4)                       2571.9571                       0
      9 balanced_sales_constraint(0,_5)                               0               -15212725
     10 balanced_sales_constraint(0,_6)                               0                       0

Even though the solution file contains 153 infeasibilities, CBC writes OPTIMAL as objective status. Pulp in version from 2.0 to 2.5 is parsing Optimal as status and ignoring the ** in the solution file.

These lines acknowledge that it is understood that the solution is infeasible but they do not check/change the status. https://github.com/coin-or/pulp/blob/ec7105f515c0dc8058493677061b6d8d8c4b5be8/pulp/apis/coin_api.py#L202-L204

I was trying different CBC versions. In particular, 2.9 in the lastest master showed a problem that is beyond being slightly off, the objective shown above should be -361,460,080 instead of 4,030,862,607. It would be very valuable to change the status to something differentiable to catch and handle such cases.

[pchtsp]

Thanks, maybe we should change that line.

We have a more general problem with status in pulp. Right now, we have status and sol_status. The second should give the status of the solution itself (no solution, no solution found, optimal solution found). Right now, the first one is a mixed between status of the solver and the solution. We should change the first one (or create another one) that correctly determines the reason for having stopped (time limit, memory limit).

In any case, you should be using sol_status for determining if it's optimal or not. Not sure what code should we return in your case though. is it infeasible? Or the optimal solution CBC though it had found was infeasible? I think when you have numeric issues it's hard to say anything about the status... and that's really your problem here.

[wookenny]

The code I use modified pulp a bit for my needs, I use both status and sol_status. Unfortunately, both were claiming an optimal solution was found.

CBC solved the model optimally, but after post-processing it is infeasible. The logs end like this:

Cgl0013I Postprocessed model is infeasible - possible tolerance issue - try without preprocessing
151 relaxed row infeasibilities - summing to 51635.8
Accuracy problem on post-processing - maybe try without pre-processing
151 relaxed row infeasibilities - summing to 51635.8
151 relaxed row infeasibilities - summing to 51635.8

Result - Optimal solution found (within gap tolerance)

Objective value:                -361413449.59429789
Lower bound:                    -361414578.397
Gap:                            0.00
Enumerated nodes:               0
Total iterations:               0
Time (CPU seconds):             0.36
Time (Wallclock seconds):       0.37

Total time (CPU seconds):       0.38   (Wallclock seconds):       0.39

Here the objective value is completely different from the one written in the solution file.

The problem is indeed numeric and I found the settings to solve this case. It took quite some time to find out that I have a problem which isn't optimally solved, so I thought another status code might help others to identify these cases quicker.

[tkralphs]

This looks more about Cbc than PuLP. Can I transfer the issue to Cbc?

[pchtsp]

Sure Ted, go ahead!

On Wed, Dec 30, 2020, 20:25 Ted Ralphs [email protected] wrote:

This looks more about Cbc than PuLP. Can I transfer the issue to Cbc?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/coin-or/pulp/issues/379#issuecomment-752731477, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ46FKMG4WHH6PLUQJYTSXN5AZANCNFSM4VNI2TAQ .

[wookenny]

Would you claim Pulp should not change the quoted code to ensure that the status is set correctly? My thought was that problems with CBC can happen but should be discovered and handled. This was one of thousands of MIPs I was solving and it took lots of effort to spot it.

Personally, I see an issue with CBC and Pulp.

[pchtsp]

The issue with PuLP (which is something we know we need to fix) is the way we report the solver status and the solution status which has never been clear / standardized between solvers. Right now the status property is standardized across solvers (=1 if a solution is found) but wrong (because the status=1 is called Optimal).

For more information: https://github.com/coin-or/pulp/discussions/382

On Mon, Jan 4, 2021 at 2:53 PM tgmath [email protected] wrote:

Would you claim Pulp should not change the quoted code to ensure that the status is set correctly? My thought was that problems with CBC can happen but should be discovered and handled. This was one of thousands of MIPs I was solving and it took lots of effort to spot it.

Personally, I see an issue with CBC and Pulp.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/coin-or/Cbc/issues/353#issuecomment-753986908, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJUZ46IOANJVT74Q7JVGH3SYHB5ZANCNFSM4VQRUZXQ .