message_ix LP status (22): dual objective limit exceeded

Code sample or context

Reading parameter(s) from "C:\path\for\model\files\cplex.opt":

>>  advind = 0
>>  lpmethod = 4
>>  threads = 4
>>  epopt = 1e-06
Finished reading from "C:\path\for\model\files\cplex.opt"
Reading data...
Starting Cplex...
Space for names approximately 2.28 Mb
Use option 'names no' to turn use of names off
CPXPARAM_Advance                                 0
CPXPARAM_LPMethod                                4
CPXPARAM_Threads                                 4
CPXPARAM_Parallel                                1
CPXPARAM_Barrier_Limits_Iteration                100000000
CPXPARAM_TimeLimit                               1000000
CPXPARAM_Tune_TimeLimit                          200000
Tried aggregator 1 time.
LP Presolve eliminated 17222 rows and 16299 columns.
Aggregator did 4392 substitutions.
Reduced LP has 3160 rows, 4341 columns, and 11584 nonzeros.
Presolve time = 0.06 sec. (26.05 ticks)
Parallel mode: using up to 4 threads for barrier.
Number of nonzeros in lower triangle of A*A\' = 11367
Using Approximate Minimum Degree ordering
Total time for automatic ordering = 0.00 sec. (2.05 ticks)
Summary statistics for Cholesky factor:
  Threads                   = 4
  Rows in Factor            = 3160
  Integer space required    = 7621
  Total non-zeros in factor = 37546
  Total FP ops to factor    = 574028
 Itn      Primal Obj        Dual Obj  Prim Inf Upper Inf  Dual Inf Inf Ratio
   0   9.7279927e+07  -7.8126256e+05  1.41e+05  1.86e+04  1.32e+05  1.00e+00
   1   9.3484805e+07  -7.4296856e+05  1.36e+05  1.79e+04  1.10e+05  1.30e-03
   2   7.0365533e+07  -8.6358996e+05  1.03e+05  1.35e+04  7.14e+04  4.96e-04
   3   2.8089196e+07  -3.4600013e+05  4.09e+04  5.39e+03  3.38e+04  4.75e-04
   4   1.8156132e+07   1.3207494e+05  2.55e+04  3.37e+03  2.55e+04  5.40e-04
   5   1.4626779e+07   5.7771989e+05  1.97e+04  2.60e+03  2.10e+04  6.09e-04
   6   1.2446796e+07   1.2926442e+06  1.58e+04  2.08e+03  1.61e+04  7.26e-04
   7   1.1413697e+07   2.1246119e+06  1.34e+04  1.77e+03  1.30e+04  8.17e-04
   8   1.7802544e+07   6.4896797e+06  1.57e+04  2.07e+03  1.51e+04  4.75e-04
   9   3.5932993e+07   1.8463734e+07  1.85e+04  2.44e+03  1.71e+04  2.08e-04
  10   6.0785712e+07   4.2326482e+07  1.82e+04  2.39e+03  1.68e+04  1.14e-04
  11   8.7816931e+07   1.5521879e+08  1.71e+04  2.26e+03  1.29e+04  7.75e-05
  12   4.6475114e+08   8.1249650e+08  1.54e+04  2.04e+03  1.07e+04  1.41e-05
  13   7.3186519e+09   1.0965018e+10  1.71e+04  2.26e+03  1.24e+04  8.81e-07
  14   2.2146343e+12   5.4715240e+12  5.55e+04  7.31e+03  8.08e+04  2.91e-09
  15   4.0681899e+12   4.7253654e+12  1.12e+04  1.47e+03  1.63e+04  1.34e-09
  16   4.3354377e+16   3.8826127e+16  1.57e+04  2.07e+03  2.49e+04  1.26e-13
  17   1.5659308e+20   6.0689768e+20  6.31e+04  8.32e+03  6.75e+05  3.48e-17
  *    3.5932993e+07   1.8463734e+07  1.85e+04  2.44e+03  1.71e+04  2.08e-04
 Barrier limit on dual objective exceeded.
 Infeasible barrier solution (dependent on objective limit).
Barrier time = 0.08 sec. (28.37 ticks)

Total time on 4 threads = 0.14 sec. (55.77 ticks)
LP status(22): dual objective limit exceeded
Cplex Time: 0.14sec (det. 55.84 ticks)

Solution aborted due to dual objective limit.
--- Restarting execution
--- MESSAGE_run.gms(4499) 15 Mb
--- Reading solution for model MESSAGE_LP
--- MESSAGE_run.gms(4481) 19 Mb
    +++ MESSAGEix did not solve to optimality - run is aborted, no output produced! +++
--- MESSAGE_run.gms(4481) 19 Mb 1 Error
*** Status: Execution error(s)
--- Job MESSAGE_run.gms Stop 05/14/24 17:27:20 elapsed 0:00:01.316

Expected result

Problem description

Hello, I would like to ask if the parameters of the solver need to be changed? I built a model of 31 nodes, without adding soft constraints, can run the result, but adding the capacity limit constraint will report an error, can not find whether it is related to the capacity limit data or the solver, I want to consult,

>> advind = 0
>> lpmethod = 4
>> threads = 4
>> epopt = 1e-06

Do I need to change these parameters?

Versions

Output of message-ix show-versions

<!--
Run one of the following and paste the results here:
- 'message-ix show-versions' in a terminal, or
- 'import ixmp; ixmp.show_versions()' in a Python interpreter.
-->

May 14 '24 10:05 meng25meng

Hi @meng25meng, I edited your answer to make it clearer. Please take a look and try to produce something similar next time.

As for your question, we will need more information to judge the situation and answer it. For starters, what do you mean by "capacity limit constraint"? There are many ways with which you can limit the capacity of technologies in our model, you can have fixed limits or dynamic limits on new capacity and total capacity (total capacity can just be fixed). Which one did you set? How does the rest of your model setup look? The scenario might be infeasible e.g. because with the "capacity limit" for the existing technologies, your demand could not be met.

You can of course play around with the parameters you mention, though I can't predict how successful this may be. Please share more information with us so that we can provide better help.

May 14 '24 10:05 glatterf42

Hi @glatterf42, I think I encountered a similar problem. I added the parameter bound_new_capacity_up and then met the error message Barrier limit on dual objective exceeded. Infeasible barrier solution (dependent on objective limit).

I asked ChatGPT and tried adding a GAMS parameter

DEFAULT_CPLEX_OPTIONS = {
    "advind": 0,
    "lpmethod": 4,
    "threads": 4,
    "epopt": 1e-6,
    "BarObjRng": 1e+20,  # <-- added this one
}

Then, the model could be solved.

Jun 11 '24 08:06 SongminYu

Per @SongminYu, here is the actual (not ChatGPT) documentation on the barobjrng setting of CPLEX: https://www.gams.com/latest/docs/S_CPLEX.html#CPLEXbarobjrng

barobjrng (real): maximum objective function

Determines the maximum absolute value of the objective function. The barrier algorithm looks at this limit to detect unbounded problems.

Default: 1.0e+20

Some thoughts:

Given the description, this could indeed be related to the "Barrier limit … exceeded".
However, the value that @SongminYu reports entering is the same as the default. It's unclear why giving the default value explicitly would change the behaviour of the solver. I would guess something else was also changed.
Without the exact contents of either @meng25meng or @SongminYu's models, it's impossible to guess why this limit is encountered.
However, from the description, I would guess that OBJ (or its dual) or something related ends up having a high value.
- In our typical models like MESSAGEix-GLOBIOM or Westeros, the objective values are 3.9e6 (here) or 1.6e5 (here) —i.e. lower than the limit by a factor of 1e14.
- Thus I would (again) guess that @meng25meng or @SongminYu's models have (probably in different ways) scaling issues due to choices of units that could be improved. For instance, entering a var_cost value as "1000000.0 EUR" instead of "1.0 mEUR" would increase the value of OBJ by 1e6. Several such choices could put OBJ over the limit.
- They could possibly use the .lp_diag tool to identify and fix these parameter values.
- But, since we haven't encountered this in our own workflows, we wouldn't be able to provide much help.

I would suggest this is wontfix. At most, we could:

Add a short tip to the docs, to the effect that seeing LP status 22 should prompt (a) a check of units and scaling and/or (b) use of barobjrng.
Encourage the users to share their complete model data via the GitHub discussions section for “MESSAGE modeling”, in case anyone has an interest to help. However, since as far as I can see there is no bug in message_ix per se.

Jun 11 '24 19:06 khaeru

Hi @khaeru, thanks for your detailed answer as always! Then I don't remember exactly if there is something else that I changed. I have made some changes in the model so I cannot replicate it, either. I will report if I see it again. I also just revised the units of my monetary parameters by 1e6. Thanks for your advice!

Jun 12 '24 04:06 SongminYu