HiGHS icon indicating copy to clipboard operation
HiGHS copied to clipboard
verified publisher icon ERGO-Code

Callback change from 1.10.0 to 1.11.0 has too much overhead

Open jhmenke opened this issue 5 months ago • 9 comments

I am solving an optimization problem in Julia using JuMP.jl and HiGHS.jl

After updating from HiGHS.jl 1.17.0 to 1.18.1 (which internally uses HiGHS 1.11.0 instead of 1.10.0), the solver needs much more time to find the solution. We tested on different computers (all running Windows), but the behaviour is consistent.

If one logs at the log output with the newest version, there seem to be multiple restarts (?) happening, resetting the optimization time as well. Look at Iteration "80306" in the new log output.

To determine if there are threading issues, i set "parallel" to "off" and "threads" to 1. However, the optimization is still much slower than in 1.10.0.

~~I have attached an .lp file (compressed) for testing. Thanks for taking a look!~~

~~model_debug.zip~~

edit: see new mps file in my comment reply

Log output before:

Running HiGHS 1.10.0 (git hash: fd8665394e): Copyright (c) 2025 HiGHS under MIT licence terms LP has 1205710 rows; 1333632 cols; 4661366 nonzeros Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 1064725 rows, 1168930 cols, 4170653 nonzeros 1s 660563 rows, 1091203 cols, 3315731 nonzeros 3s Dependent equations search running on 60582 equations with time limit of 1000.00s Dependent equations search removed 384 rows and 30283 nonzeros in 2.88s (limit = 1000.00s) 653491 rows, 1084729 cols, 3245666 nonzeros 10s Presolve : Reductions: rows 653491(-552219); columns 1084729(-248903); elements 3245666(-1415700) Solving the presolved LP WARNING: Number of threads available = 4 < 8 = Simplex concurrency to be used: Parallel performance may be less than anticipated Using EKK parallel dual simplex solver - SIP with concurrency of 8 Iteration Objective Infeasibilities num(sum) 0 -1.5979516828e+04 Ph1: 29721(1.01311e+07); Du: 253(15979.5) 12s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 17s 71923 3.7588240693e+11 Pr: 197902(6.95171e+07); Du: 0(4.69743e-05) 23s 110876 3.7594473904e+11 Pr: 202808(8.5837e+07); Du: 0(4.47227e-05) 28s 159940 3.9592446600e+11 Pr: 186975(5.48664e+07); Du: 0(3.60225e-05) 33s 219488 5.0956972385e+11 Pr: 154996(2.51965e+06); Du: 0(3.40521e-05) 38s 287207 5.0959634073e+11 Pr: 109118(1.38989e+06); Du: 0(3.40983e-05) 43s 355538 5.1703372314e+11 Pr: 57805(150545); Du: 0(3.00364e-05) 48s 420538 5.1710111257e+11 Pr: 5258(395.409); Du: 0(2.6219e-05) 54s 426825 5.1709668533e+11 Pr: 0(0); Du: 0(4.32917e-06) 59s 426825 5.1709668533e+11 Pr: 0(0); Du: 0(4.32917e-06) 59s 426825 5.1709668533e+11 Pr: 0(0); Du: 0(4.32917e-06) 59s Solving the original LP from the solution after postsolve Model status : Optimal Simplex iterations: 426825 Objective value : 5.1709668533e+11 Relative P-D gap : 8.0027269776e-14 HiGHS run time : 59.74

Log output after update:

Running HiGHS 1.11.0 (git hash: 364c83a51e): Copyright (c) 2025 HiGHS under MIT licence terms LP has 1205710 rows; 1333632 cols; 4661366 nonzeros Coefficient ranges: Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 1064725 rows, 1168930 cols, 4170653 nonzeros 1s 660563 rows, 1091203 cols, 3315731 nonzeros 4s Dependent equations search running on 60582 equations with time limit of 1000.00s Dependent equations search removed 384 rows and 30283 nonzeros in 2.88s (limit = 1000.00s) 653491 rows, 1084729 cols, 3245666 nonzeros 12s Presolve : Reductions: rows 653491(-552219); columns 1084729(-248903); elements 3245666(-1415700) Solving the presolved LP WARNING: Number of threads available = 4 < 8 = Simplex concurrency to be used: Parallel performance may be less than anticipated Using EKK parallel dual simplex solver - SIP with concurrency of 8 Iteration Objective Infeasibilities num(sum) 0 -1.5979516828e+04 Ph1: 29721(1.01311e+07); Du: 253(15979.5) 13s 2165 -1.5979516828e+04 Ph1: 28343(5.28809e+06); Du: 253(15979.5) 19s 4869 3.7543029042e+11 Pr: 115334(2.41663e+08); Du: 0(6.43793e-15) 26s 7255 3.7543059481e+11 Pr: 115354(2.11151e+08); Du: 0(6.43793e-15) 33s 9338 3.7544147877e+11 Pr: 123244(8.8585e+07); Du: 0(9.95755e-07) 39s 11860 3.7547934360e+11 Pr: 127981(7.16323e+07); Du: 0(1.13024e-06) 45s 14262 3.7550457135e+11 Pr: 141335(5.99416e+07); Du: 0(1.22994e-05) 52s 16594 3.7552720318e+11 Pr: 142791(5.56167e+07); Du: 0(1.23312e-05) 58s 18931 3.7563533564e+11 Pr: 143871(5.31062e+07); Du: 0(1.28747e-05) 63s 21276 3.7565762372e+11 Pr: 146856(5.33957e+07); Du: 0(1.40694e-05) 69s 23305 3.7572210406e+11 Pr: 159671(5.01672e+07); Du: 0(1.43519e-05) 75s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 151s 54478 3.7585548532e+11 Pr: 192936(1.95498e+08); Du: 0(4.00807e-05) 161s 58425 3.7586114145e+11 Pr: 194099(6.60104e+07); Du: 0(4.20528e-05) 171s 61496 3.7586632954e+11 Pr: 194785(6.73359e+07); Du: 0(4.47281e-05) 180s 65206 3.7587129598e+11 Pr: 196868(5.24737e+07); Du: 0(4.59631e-05) 190s 67939 3.7587721794e+11 Pr: 197277(1.1053e+08); Du: 0(4.54218e-05) 197s 71923 3.7588240693e+11 Pr: 197902(6.95171e+07); Du: 0(4.69743e-05) 208s 76325 3.7589304860e+11 Pr: 199767(6.57616e+07); Du: 0(4.82349e-05) 219s 80306 3.7589925659e+11 Pr: 201391(8.28603e+07); Du: 0(4.80013e-05) 229s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 151s 54478 3.7585548532e+11 Pr: 192936(1.95498e+08); Du: 0(4.00807e-05) 161s 58425 3.7586114145e+11 Pr: 194099(6.60104e+07); Du: 0(4.20528e-05) 171s 61496 3.7586632954e+11 Pr: 194785(6.73359e+07); Du: 0(4.47281e-05) 180s 65206 3.7587129598e+11 Pr: 196868(5.24737e+07); Du: 0(4.59631e-05) 190s 67939 3.7587721794e+11 Pr: 197277(1.1053e+08); Du: 0(4.54218e-05) 197s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 151s 54478 3.7585548532e+11 Pr: 192936(1.95498e+08); Du: 0(4.00807e-05) 161s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 151s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 81s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 88s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 95s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 102s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 110s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 119s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 129s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 140s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 151s 54478 3.7585548532e+11 Pr: 192936(1.95498e+08); Du: 0(4.00807e-05) 161s 58425 3.7586114145e+11 Pr: 194099(6.60104e+07); Du: 0(4.20528e-05) 171s 61496 3.7586632954e+11 Pr: 194785(6.73359e+07); Du: 0(4.47281e-05) 180s 65206 3.7587129598e+11 Pr: 196868(5.24737e+07); Du: 0(4.59631e-05) 190s 67939 3.7587721794e+11 Pr: 197277(1.1053e+08); Du: 0(4.54218e-05) 197s 71923 3.7588240693e+11 Pr: 197902(6.95171e+07); Du: 0(4.69743e-05) 208s 76325 3.7589304860e+11 Pr: 199767(6.57616e+07); Du: 0(4.82349e-05) 219s 80306 3.7589925659e+11 Pr: 201391(8.28603e+07); Du: 0(4.80013e-05) 229s 84359 3.7590395260e+11 Pr: 201742(6.05599e+07); Du: 0(4.88401e-05) 240s 88469 3.7591501440e+11 Pr: 203037(5.11959e+07); Du: 0(4.84729e-05) 250s 92479 3.7592290825e+11 Pr: 203662(6.41326e+07); Du: 0(4.80197e-05) 261s 96893 3.7592851813e+11 Pr: 204143(6.60302e+07); Du: 0(4.6596e-05) 272s 101893 3.7593632478e+11 Pr: 203290(5.74779e+07); Du: 0(4.53808e-05) 284s 105876 3.7594185606e+11 Pr: 203306(7.48051e+07); Du: 0(4.54618e-05) 295s 110876 3.7594473904e+11 Pr: 202808(8.5837e+07); Du: 0(4.47227e-05) 309s 115640 3.7595329456e+11 Pr: 202653(7.66767e+07); Du: 0(4.34125e-05) 321s 120161 3.7596040616e+11 Pr: 202439(6.60721e+07); Du: 0(4.17689e-05) 334s 124419 3.7596623780e+11 Pr: 200724(6.30341e+07); Du: 0(4.26813e-05) 345s 128877 3.7597108450e+11 Pr: 199074(5.16526e+07); Du: 0(4.26183e-05) 356s 133877 3.7626239077e+11 Pr: 196579(3.67015e+07); Du: 0(4.21808e-05) 368s 138151 3.7626635540e+11 Pr: 194996(3.45712e+07); Du: 0(4.21442e-05) 378s 142337 3.7772420588e+11 Pr: 193523(5.39973e+07); Du: 0(4.03742e-05) 389s 146688 3.8273183771e+11 Pr: 191803(2.50836e+07); Du: 0(4.04344e-05) 400s 150885 3.9591418424e+11 Pr: 190369(3.80035e+07); Du: 0(3.80137e-05) 410s 155192 3.9591865980e+11 Pr: 189234(5.27243e+07); Du: 0(3.60914e-05) 421s 159940 3.9592446600e+11 Pr: 186975(5.48664e+07); Du: 0(3.60225e-05) 433s 164940 4.0924941465e+11 Pr: 184532(1.25168e+08); Du: 0(3.51364e-05) 447s 169883 4.0925489102e+11 Pr: 182649(1.14487e+07); Du: 0(3.48135e-05) 459s ...

One can see that the results in every iteration seem to stay the same, but overall the process is incredibly slow now.

My settings (although changing them does not seem to improve the solve time):

    model = Model(HiGHS.Optimizer; add_bridges=false)
    set_optimizer_attribute(model, "primal_feasibility_tolerance", 1e-5)
    set_optimizer_attribute(model, "simplex_strategy", 2)
    set_optimizer_attribute(model, "simplex_scale_strategy", 0)
    set_optimizer_attribute(model, "simplex_primal_edge_weight_strategy", 1)
    set_optimizer_attribute(model, "simplex_dual_edge_weight_strategy", 2)
    set_optimizer_attribute(model, "run_crossover", "on")

Log output with "parallel" = "off".

Running HiGHS 1.11.0 (git hash: 364c83a51e): Copyright (c) 2025 HiGHS under MIT licence terms LP has 1205710 rows; 1333632 cols; 4661366 nonzeros Coefficient ranges: Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 1064725 rows, 1168930 cols, 4170653 nonzeros 1s 660563 rows, 1091203 cols, 3315731 nonzeros 4s Dependent equations search running on 60582 equations with time limit of 1000.00s Dependent equations search removed 384 rows and 30283 nonzeros in 2.99s (limit = 1000.00s) 653491 rows, 1084729 cols, 3245666 nonzeros 13s Presolve : Reductions: rows 653491(-552219); columns 1084729(-248903); elements 3245666(-1415700) Solving the presolved LP WARNING: Number of threads available = 1 < 8 = Simplex concurrency to be used: Parallel performance may be less than anticipated Using EKK parallel dual simplex solver - SIP with concurrency of 8 Iteration Objective Infeasibilities num(sum) 0 -1.5979516828e+04 Ph1: 29721(1.01311e+07); Du: 253(15979.5) 15s 2165 -1.5979516828e+04 Ph1: 28343(5.28809e+06); Du: 253(15979.5) 20s 4869 3.7543029042e+11 Pr: 115334(2.41663e+08); Du: 0(6.43793e-15) 26s 7255 3.7543059481e+11 Pr: 115354(2.11151e+08); Du: 0(6.43793e-15) 32s 9338 3.7544147877e+11 Pr: 123244(8.8585e+07); Du: 0(9.95755e-07) 37s 11860 3.7547934360e+11 Pr: 127981(7.16323e+07); Du: 0(1.13024e-06) 43s 14262 3.7550457135e+11 Pr: 141335(5.99416e+07); Du: 0(1.22994e-05) 49s 16594 3.7552720318e+11 Pr: 142791(5.56167e+07); Du: 0(1.23312e-05) 54s 18931 3.7563533564e+11 Pr: 143871(5.31062e+07); Du: 0(1.28747e-05) 60s 21276 3.7565762372e+11 Pr: 146856(5.33957e+07); Du: 0(1.40694e-05) 66s 23305 3.7572210406e+11 Pr: 159671(5.01672e+07); Du: 0(1.43519e-05) 71s 25691 3.7575755136e+11 Pr: 162875(5.054e+07); Du: 0(1.66323e-05) 77s 28358 3.7576960540e+11 Pr: 166647(4.50719e+07); Du: 0(2.47908e-05) 83s 30877 3.7577716334e+11 Pr: 172671(5.15514e+07); Du: 0(3.00849e-05) 90s 33673 3.7578739969e+11 Pr: 175942(4.45846e+07); Du: 0(3.14852e-05) 97s 36640 3.7579483029e+11 Pr: 181741(6.19912e+07); Du: 0(3.32855e-05) 104s 39872 3.7581246549e+11 Pr: 186500(5.91658e+07); Du: 0(3.3804e-05) 113s 43296 3.7582626958e+11 Pr: 187528(4.19892e+07); Du: 0(3.66546e-05) 122s 47197 3.7584127459e+11 Pr: 192245(3.71125e+07); Du: 0(3.99741e-05) 133s 51228 3.7584953660e+11 Pr: 191216(3.4081e+07); Du: 0(4.03838e-05) 142s 54478 3.7585548532e+11 Pr: 192936(1.95498e+08); Du: 0(4.00807e-05) 152s 58425 3.7586114145e+11 Pr: 194099(6.60104e+07); Du: 0(4.20528e-05) 162s 61496 3.7586632954e+11 Pr: 194785(6.73359e+07); Du: 0(4.47281e-05) 171s 65206 3.7587129598e+11 Pr: 196868(5.24737e+07); Du: 0(4.59631e-05) 181s 67939 3.7587721794e+11 Pr: 197277(1.1053e+08); Du: 0(4.54218e-05) 190s 71923 3.7588240693e+11 Pr: 197902(6.95171e+07); Du: 0(4.69743e-05) 200s 76325 3.7589304860e+11 Pr: 199767(6.57616e+07); Du: 0(4.82349e-05) 211s 80306 3.7589925659e+11 Pr: 201391(8.28603e+07); Du: 0(4.80013e-05) 222s 84359 3.7590395260e+11 Pr: 201742(6.05599e+07); Du: 0(4.88401e-05) 233s 88469 3.7591501440e+11 Pr: 203037(5.11959e+07); Du: 0(4.84729e-05) 243s 92479 3.7592290825e+11 Pr: 203662(6.41326e+07); Du: 0(4.80197e-05) 254s 96893 3.7592851813e+11 Pr: 204143(6.60302e+07); Du: 0(4.6596e-05) 265s 101893 3.7593632478e+11 Pr: 203290(5.74779e+07); Du: 0(4.53808e-05) 277s 105876 3.7594185606e+11 Pr: 203306(7.48051e+07); Du: 0(4.54618e-05) 288s 110876 3.7594473904e+11 Pr: 202808(8.5837e+07); Du: 0(4.47227e-05) 300s 115640 3.7595329456e+11 Pr: 202653(7.66767e+07); Du: 0(4.34125e-05) 312s 120161 3.7596040616e+11 Pr: 202439(6.60721e+07); Du: 0(4.17689e-05) 323s 124419 3.7596623780e+11 Pr: 200724(6.30341e+07); Du: 0(4.26813e-05) 334s 128877 3.7597108450e+11 Pr: 199074(5.16526e+07); Du: 0(4.26183e-05) 345s 133877 3.7626239077e+11 Pr: 196579(3.67015e+07); Du: 0(4.21808e-05) 358s

jhmenke avatar Jul 14 '25 10:07 jhmenke

I can't reproduce this slow solving with HiGHS 1.11.0 (git hash: 364c83a51) However, model_debug.zip does not appear to be the LP you used. Using model_debug.zip (AKA 2466.lp) I get

LP 2466 has 991849 rows; 1097468 cols; 3829286 nonzeros Coefficient ranges: Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 876085 rows, 961698 cols, 3427013 nonzeros 1s 543422 rows, 898115 cols, 2722868 nonzeros 2s Dependent equations search running on 49952 equations with time limit of 1000.00s Dependent equations search removed 316 rows and 24504 nonzeros in 0.25s (limit = 1000.00s) 537574 rows, 892738 cols, 2665877 nonzeros 5s Presolve : Reductions: rows 537574(-454275); columns 892738(-204730); elements 2665877(-1163409)

and

Objective value : 4.1820280416e+11

but you get

LP has 1205710 rows; 1333632 cols; 4661366 nonzeros Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 1064725 rows, 1168930 cols, 4170653 nonzeros 1s 660563 rows, 1091203 cols, 3315731 nonzeros 3s Dependent equations search running on 60582 equations with time limit of 1000.00s Dependent equations search removed 384 rows and 30283 nonzeros in 2.88s (limit = 1000.00s) 653491 rows, 1084729 cols, 3245666 nonzeros 10s Presolve : Reductions: rows 653491(-552219); columns 1084729(-248903); elements 3245666(-1415700)

and

Objective value : 5.1709668533e+11

Note also that if you write a model defined internally as a .lp file then, when it's read in, the order of the variables (and hence tied decisions in simplex) will change. The indices of variables are defined by the order in which their names appear in the .lp file. Hence, if your original variable 0 is named "col0", but does not appear in the objective then, after reading the .lp file, the index of variable "col0" will be greater than 0. It's much better to communicate an MPS file, since the order of the columns and rows is maintained.

jajhall avatar Jul 14 '25 23:07 jajhall

Sorry, i did not know that about .lp files. We do not use file-based storage usually.

I have exported the model as an .mps file and validated that the slowdown occurs even if i load this exact file. The slowdown occurs when switching from HiGHS.jl 1.17.0 to 1.18.0 (which updates HiGHS_jll 1.10.0 to 1.11.0).

import JuMP
import HiGHS

model = JuMP.read_from_file("debug_model.mps")
JuMP.set_optimizer(model, HiGHS.Optimizer)
JuMP.optimize!(model)

println("--- Results ---")
println("Termination Status: ", JuMP.termination_status(model))
println("Objective Value: ", JuMP.objective_value(model))

My results with the mps file.

1.10:

Model status : Optimal Simplex iterations: 446850 Objective value : 4.1820280416e+11 Relative P-D gap : 4.5243356212e-15 HiGHS run time : 133.14 --- Results --- Termination Status: OPTIMAL Objective Value: 4.182028041574081e11

1.11.0:

Model status : Optimal Simplex iterations: 446850 Objective value : 4.1820280416e+11 P-D objective error : 8.1000202251e-15 HiGHS run time : 917.32 --- Results --- Termination Status: OPTIMAL Objective Value: 4.182028041574032e11

debug_model.zip

jhmenke avatar Jul 15 '25 06:07 jhmenke

Could this be the increased precision change in the MPS file writer? I've got no clue if JuMP writes out some temporary representation @odow

Opt-Mucca avatar Jul 15 '25 06:07 Opt-Mucca

@jhmenke what is the output of versioninfo() and ] st -m for each of your environments?

Do you have the logs? Both models take 446850 simplex iterations?

If @jhmenke used write_to_file(model, "model.mps"), this is probably different to the representation that HiGHS gets because we don't guarantee the ordering of variables/constraints in a model (it's a very complicated topic; we order by constraint type and then constraints within a constraint type, but not the exact order as written by the user. It's something I would change now but which we didn't think about in 2019).

But regardless: the last example should give equivalent models to HiGHS.

odow avatar Jul 15 '25 07:07 odow

The effect happens if i build the JuMP model and solve without saving as an .mps file (which is how i noticed this issue) but also when using the .mps file and simply solving that in a different julia instance.

This is the log output for HiGHS 1.10.0 (HiGHS.jl 1.17.0):

Running HiGHS 1.10.0 (git hash: fd8665394e): Copyright (c) 2025 HiGHS under MIT licence terms LP has 991849 rows; 1097468 cols; 3829286 nonzeros Coefficient ranges: Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 876074 rows, 961701 cols, 3426946 nonzeros 1s 543435 rows, 898128 cols, 2723024 nonzeros 3s Dependent equations search running on 49965 equations with time limit of 1000.00s Dependent equations search removed 316 rows and 24487 nonzeros in 1.87s (limit = 1000.00s) 537606 rows, 892770 cols, 2666088 nonzeros 8s Presolve : Reductions: rows 537606(-454243); columns 892770(-204698); elements 2666088(-1163198) Solving the presolved LP Using EKK dual simplex solver - serial Iteration Objective Infeasibilities num(sum) 0 -8.1306717279e+02 Ph1: 24276(4.79758e+08); Du: 174(813.067) 10s 82948 3.5204077749e+11 Pr: 108132(2.59337e+09); Du: 0(1.15995e-05) 15s 139806 4.1386524513e+11 Pr: 105914(1.75352e+10); Du: 0(1.18632e-05) 20s 181120 4.1631569446e+11 Pr: 102932(5.86065e+10); Du: 0(1.37198e-05) 26s 221052 4.1832780990e+11 Pr: 91897(1.07374e+12); Du: 0(1.3871e-05) 31s 242079 4.1834255032e+11 Pr: 84872(9.85533e+10); Du: 0(1.30607e-05) 36s 250131 4.1834883532e+11 Pr: 83740(4.71013e+10); Du: 0(1.30512e-05) 42s 267980 4.1837001219e+11 Pr: 78442(2.14453e+09); Du: 0(1.24289e-05) 47s 307082 4.1841594670e+11 Pr: 64345(2.06726e+09); Du: 0(1.0861e-05) 53s 354099 4.1846333776e+11 Pr: 48402(1.85511e+08); Du: 0(1.073e-05) 58s 404536 4.1847811591e+11 Pr: 21643(1.7926e+07); Du: 0(9.90774e-06) 63s 431979 4.1820295991e+11 Pr: 0(0); Du: 13038(545.571) 79s 434395 4.1820287747e+11 Pr: 0(0); Du: 9724(4.77047) 90s 439395 4.1820281951e+11 Pr: 0(0); Du: 10308(593.899) 107s 444131 4.1820280474e+11 Pr: 0(0); Du: 6910(1.36408) 122s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 130s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 130s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 130s Using EKK primal simplex solver Iteration Objective Infeasibilities num(sum) 446820 4.1820280416e+11 Pr: 0(0); Du: 17(7.94029e-05) 131s 446850 4.1820280416e+11 Pr: 0(0); Du: 0(8.43066e-06) 131s Solving the original LP from the solution after postsolve Model status : Optimal Simplex iterations: 446850 Objective value : 4.1820280416e+11 Relative P-D gap : 4.5243356212e-15 HiGHS run time : 133.14 --- Results --- Termination Status: OPTIMAL Objective Value: 4.182028041574081e11

This is the log output for HiGHS 1.11.0:

Running HiGHS 1.11.0 (git hash: 364c83a51e): Copyright (c) 2025 HiGHS under MIT licence terms LP has 991849 rows; 1097468 cols; 3829286 nonzeros Coefficient ranges: Matrix [1e-05, 5e+03] Cost [1e-02, 3e+06] Bound [1e-04, 2e+07] RHS [3e-02, 9e+06] Presolving model 876074 rows, 961701 cols, 3426946 nonzeros 1s 543435 rows, 898128 cols, 2723024 nonzeros 3s Dependent equations search running on 49965 equations with time limit of 1000.00s Dependent equations search removed 316 rows and 24487 nonzeros in 1.92s (limit = 1000.00s) 537606 rows, 892770 cols, 2666088 nonzeros 8s Presolve : Reductions: rows 537606(-454243); columns 892770(-204698); elements 2666088(-1163198) Solving the presolved LP Using EKK dual simplex solver - serial Iteration Objective Infeasibilities num(sum) 0 -8.1306717279e+02 Ph1: 24276(4.79758e+08); Du: 174(813.067) 11s 3765 3.0902601573e+11 Pr: 90896(3.38809e+10); Du: 0(6.57882e-13) 17s 8416 3.0915915992e+11 Pr: 90662(1.81352e+10); Du: 0(8.18375e-08) 25s 13115 3.0922510291e+11 Pr: 107375(8.5505e+09); Du: 0(5.29026e-06) 33s 17987 3.0922698178e+11 Pr: 113970(2.4135e+09); Du: 0(1.28277e-05) 42s 22740 3.0924204235e+11 Pr: 115341(1.61926e+09); Du: 0(1.15052e-05) 50s 27154 3.0926595715e+11 Pr: 115775(2.74817e+09); Du: 0(1.23649e-05) 58s 31348 3.0934045509e+11 Pr: 116459(3.81402e+09); Du: 0(1.35052e-05) 65s 35526 3.0937760463e+11 Pr: 116066(4.27698e+09); Du: 0(1.40118e-05) 72s 40117 3.0938539580e+11 Pr: 115009(2.58928e+09); Du: 0(1.51462e-05) 80s 44048 3.0941776157e+11 Pr: 114817(1.61981e+09); Du: 0(1.56678e-05) 87s 48606 3.0942775290e+11 Pr: 115188(2.05915e+09); Du: 0(1.44851e-05) 95s 51487 3.0943590228e+11 Pr: 114711(2.22508e+09); Du: 0(1.44567e-05) 100s 57064 3.0945098691e+11 Pr: 113617(2.09603e+09); Du: 0(1.39369e-05) 110s 62586 3.2271569862e+11 Pr: 113589(1.32798e+09); Du: 0(1.34809e-05) 119s 65762 3.2271881980e+11 Pr: 112283(1.31657e+09); Du: 0(1.40249e-05) 125s 68731 3.2342123851e+11 Pr: 111738(4.6231e+09); Du: 0(1.27348e-05) 130s 71697 3.2342569780e+11 Pr: 110756(3.92603e+09); Du: 0(1.22021e-05) 135s 74646 3.3863408396e+11 Pr: 109830(1.85062e+09); Du: 0(1.2204e-05) 140s 77882 3.4953706221e+11 Pr: 108933(2.01818e+09); Du: 0(1.14844e-05) 146s 80935 3.4953891903e+11 Pr: 108091(2.03458e+09); Du: 0(1.14396e-05) 151s 86693 3.5585250033e+11 Pr: 106258(2.50634e+09); Du: 0(1.23738e-05) 162s 90544 4.0313938577e+11 Pr: 104177(2.51803e+09); Du: 0(1.26828e-05) 168s 94125 4.0314249737e+11 Pr: 102983(3.67182e+09); Du: 0(1.28837e-05) 175s 96868 4.0314410217e+11 Pr: 102313(5.5417e+09); Du: 0(1.32279e-05) 180s 100911 4.1252237992e+11 Pr: 100493(1.01914e+10); Du: 0(1.26498e-05) 188s 104512 4.1252611189e+11 Pr: 100579(6.56678e+09); Du: 0(1.27173e-05) 194s 107547 4.1252737748e+11 Pr: 101469(6.62902e+09); Du: 0(1.21357e-05) 200s 112942 4.1346949004e+11 Pr: 103106(8.68737e+09); Du: 0(1.17417e-05) 210s 116385 4.1347153771e+11 Pr: 103458(1.70209e+10); Du: 0(1.13463e-05) 216s 120718 4.1347366703e+11 Pr: 103186(1.70274e+10); Du: 0(1.13816e-05) 224s 124927 4.1347598018e+11 Pr: 104879(1.60065e+10); Du: 0(1.08756e-05) 231s 128441 4.1386134920e+11 Pr: 105799(2.29403e+10); Du: 0(1.10109e-05) 238s 132365 4.1386248277e+11 Pr: 105657(2.7371e+10); Du: 0(1.13552e-05) 245s 136393 4.1386409512e+11 Pr: 105166(2.31434e+10); Du: 0(1.13849e-05) 252s 139806 4.1386524513e+11 Pr: 105914(1.75352e+10); Du: 0(1.18632e-05) 259s 143079 4.1386610613e+11 Pr: 106741(2.26076e+10); Du: 0(1.23849e-05) 265s 146395 4.1386809886e+11 Pr: 107084(7.6592e+09); Du: 0(1.25917e-05) 272s 149396 4.1386942531e+11 Pr: 106922(6.18176e+09); Du: 0(1.2711e-05) 277s 154752 4.1401853343e+11 Pr: 106870(1.24355e+10); Du: 0(1.2902e-05) 287s 158268 4.1402074126e+11 Pr: 107081(1.42145e+10); Du: 0(1.30716e-05) 294s 162445 4.1402237000e+11 Pr: 106194(1.33352e+10); Du: 0(1.30278e-05) 301s 165907 4.1577378270e+11 Pr: 105925(1.34177e+10); Du: 0(1.33144e-05) 308s 169726 4.1577464108e+11 Pr: 105798(4.88825e+10); Du: 0(1.30599e-05) 315s 173908 4.1577627265e+11 Pr: 105063(2.1454e+11); Du: 0(1.36659e-05) 322s 177353 4.1631442648e+11 Pr: 104305(1.22317e+11); Du: 0(1.36632e-05) 328s 181120 4.1631569446e+11 Pr: 102932(5.86065e+10); Du: 0(1.37198e-05) 335s 184408 4.1631663127e+11 Pr: 102384(5.50424e+10); Du: 0(1.37568e-05) 341s 188202 4.1631732213e+11 Pr: 102055(5.83492e+10); Du: 0(1.37706e-05) 348s 191048 4.1632070004e+11 Pr: 102028(6.22196e+10); Du: 0(1.38831e-05) 354s 194134 4.1632344070e+11 Pr: 102488(9.0224e+11); Du: 0(1.41068e-05) 360s 197408 4.1671546602e+11 Pr: 101461(9.02608e+11); Du: 0(1.44774e-05) 366s 200821 4.1671650854e+11 Pr: 99927(9.38028e+11); Du: 0(1.45397e-05) 372s 204617 4.1832191239e+11 Pr: 98637(9.77306e+11); Du: 0(1.45215e-05) 379s 208993 4.1832350606e+11 Pr: 96799(9.08884e+11); Du: 0(1.40095e-05) 387s 213257 4.1832502644e+11 Pr: 94659(1.03795e+12); Du: 0(1.41599e-05) 395s 217248 4.1832588206e+11 Pr: 93340(1.00667e+12); Du: 0(1.42124e-05) 402s 221052 4.1832780990e+11 Pr: 91897(1.07374e+12); Du: 0(1.3871e-05) 409s 224715 4.1832977942e+11 Pr: 90605(2.29043e+12); Du: 0(1.40442e-05) 416s 228987 4.1833204102e+11 Pr: 89037(2.15559e+12); Du: 0(1.36051e-05) 424s 233201 4.1833508203e+11 Pr: 86959(1.51406e+12); Du: 0(1.31443e-05) 431s 237304 4.1833935349e+11 Pr: 84362(1.32881e+10); Du: 0(1.30708e-05) 439s 239828 4.1834046403e+11 Pr: 84779(1.94792e+10); Du: 0(1.31283e-05) 445s 242079 4.1834255032e+11 Pr: 84872(9.85533e+10); Du: 0(1.30607e-05) 451s 244254 4.1834451574e+11 Pr: 85002(4.01549e+10); Du: 0(1.31603e-05) 457s 247704 4.1834722213e+11 Pr: 84394(4.81237e+10); Du: 0(1.31002e-05) 466s 250131 4.1834883532e+11 Pr: 83740(4.71013e+10); Du: 0(1.30512e-05) 472s 253493 4.1835176754e+11 Pr: 82860(2.4725e+10); Du: 0(1.30927e-05) 478s 257192 4.1835411402e+11 Pr: 81691(1.36421e+10); Du: 0(1.32889e-05) 485s 259648 4.1835837251e+11 Pr: 80788(1.27933e+10); Du: 0(1.28447e-05) 491s 262797 4.1836157175e+11 Pr: 79659(4.46918e+09); Du: 0(1.25464e-05) 497s 265439 4.1836589215e+11 Pr: 80057(2.80734e+10); Du: 0(1.25494e-05) 503s 267980 4.1837001219e+11 Pr: 78442(2.14453e+09); Du: 0(1.24289e-05) 509s 273389 4.1837695882e+11 Pr: 76630(9.38071e+09); Du: 0(1.27889e-05) 518s 278612 4.1838274298e+11 Pr: 74344(1.0729e+10); Du: 0(1.21641e-05) 528s 282607 4.1838609866e+11 Pr: 72792(8.51718e+09); Du: 0(1.22203e-05) 536s 288358 4.1839925625e+11 Pr: 70796(8.72504e+09); Du: 0(1.21655e-05) 546s 294034 4.1840397841e+11 Pr: 68670(1.07535e+10); Du: 0(1.19237e-05) 556s 296866 4.1840731219e+11 Pr: 67950(1.0434e+10); Du: 0(1.16077e-05) 561s 299702 4.1840956442e+11 Pr: 66504(9.58442e+09); Du: 0(1.14208e-05) 566s 302474 4.1841266024e+11 Pr: 65890(1.01092e+10); Du: 0(1.15103e-05) 571s 305327 4.1841478856e+11 Pr: 65075(3.44766e+09); Du: 0(1.12557e-05) 577s 308335 4.1841731885e+11 Pr: 64165(1.62996e+09); Du: 0(1.10489e-05) 583s 313831 4.1842200629e+11 Pr: 62688(5.31507e+09); Du: 0(1.10785e-05) 593s 319237 4.1843766705e+11 Pr: 61420(3.45169e+09); Du: 0(1.10618e-05) 603s 324735 4.1844247545e+11 Pr: 59765(7.07716e+09); Du: 0(1.03111e-05) 613s 330481 4.1844803872e+11 Pr: 57411(3.56661e+08); Du: 0(1.03199e-05) 623s 333208 4.1845040496e+11 Pr: 55985(1.66094e+08); Du: 0(1.03662e-05) 629s 336053 4.1845256163e+11 Pr: 55175(1.53718e+08); Du: 0(1.04367e-05) 634s 338991 4.1845507651e+11 Pr: 54054(1.37845e+08); Du: 0(1.04458e-05) 639s 341880 4.1845673415e+11 Pr: 52796(7.37586e+07); Du: 0(1.05466e-05) 644s 344911 4.1845913879e+11 Pr: 51571(3.95178e+07); Du: 0(1.06623e-05) 650s 347768 4.1846047665e+11 Pr: 50380(3.41628e+07); Du: 0(1.07054e-05) 655s 350942 4.1846230209e+11 Pr: 49377(7.22117e+07); Du: 0(1.07131e-05) 660s 354099 4.1846333776e+11 Pr: 48402(1.85511e+08); Du: 0(1.073e-05) 666s 357702 4.1846509820e+11 Pr: 46730(1.68877e+08); Du: 0(1.05993e-05) 673s 361685 4.1846679763e+11 Pr: 44937(3.73749e+07); Du: 0(1.02416e-05) 680s 365345 4.1846799659e+11 Pr: 43003(4.39791e+07); Du: 0(1.0342e-05) 687s 369219 4.1846929488e+11 Pr: 41264(4.23241e+07); Du: 0(1.05359e-05) 694s 373223 4.1847084964e+11 Pr: 39777(3.8881e+07); Du: 0(1.04102e-05) 701s 377245 4.1847215921e+11 Pr: 37897(2.04493e+07); Du: 0(1.03477e-05) 708s 380895 4.1847293691e+11 Pr: 34886(1.92678e+07); Du: 0(1.02931e-05) 715s 384795 4.1847387505e+11 Pr: 31996(1.82498e+07); Du: 0(1.10981e-05) 722s 388974 4.1847497906e+11 Pr: 29572(2.23558e+07); Du: 0(1.06676e-05) 730s 392995 4.1847578240e+11 Pr: 27063(1.86117e+07); Du: 0(1.04833e-05) 737s 397022 4.1847659238e+11 Pr: 25640(2.2202e+07); Du: 0(1.03746e-05) 744s 400453 4.1847738994e+11 Pr: 24394(2.21763e+07); Du: 0(1.01544e-05) 751s 404536 4.1847811591e+11 Pr: 21643(1.7926e+07); Du: 0(9.90774e-06) 759s 408740 4.1847870983e+11 Pr: 18737(1.77171e+07); Du: 0(9.64864e-06) 766s 413071 4.1847921471e+11 Pr: 15380(9.42481e+06); Du: 0(9.41245e-06) 774s 416611 4.1847948893e+11 Pr: 12734(3.50667e+06); Du: 0(9.38066e-06) 781s 420326 4.1847962494e+11 Pr: 9793(239035); Du: 0(9.3983e-06) 788s 424089 4.1847965796e+11 Pr: 6106(12152.6); Du: 0(9.5543e-06) 795s 428296 4.1847967120e+11 Pr: 1897(3.76768e+06); Du: 0(9.22094e-06) 802s 431979 4.1820295991e+11 Pr: 0(0); Du: 13038(545.571) 828s 434395 4.1820287747e+11 Pr: 0(0); Du: 9724(4.77047) 847s 439395 4.1820281951e+11 Pr: 0(0); Du: 10308(593.899) 874s 444131 4.1820280474e+11 Pr: 0(0); Du: 6910(1.36408) 900s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 914s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 914s 446820 4.1820280416e+11 Pr: 0(0); Du: 0(1.82748e-06) 914s Using EKK primal simplex solver Iteration Objective Infeasibilities num(sum) 446820 4.1820280416e+11 Pr: 0(0); Du: 17(7.94029e-05) 915s 446850 4.1820280416e+11 Pr: 0(0); Du: 0(8.43066e-06) 915s Solving the original LP from the solution after postsolve Model status : Optimal Simplex iterations: 446850 Objective value : 4.1820280416e+11 P-D objective error : 8.1000202251e-15 HiGHS run time : 917.32 --- Results --- Termination Status: OPTIMAL Objective Value: 4.182028041574032e11

I assume the iteration logs are time-based; the intermediary results seem to be more or less the same. This slowdow effect has been happening on my local laptop, on Azure VMs and on Azure DevBoxes.

=================================================================================

versioninfo() for HiGHS.jl 1.17.0:

Julia Version 1.11.5 Commit 760b2e5b73 (2025-04-14 06:53 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: Windows (x86_64-w64-mingw32) CPU: 8 × AMD EPYC 7763 64-Core Processor WORD_SIZE: 64 LLVM: libLLVM-16.0.6 (ORCJIT, znver3) Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores) Environment: JULIA_EDITOR = code JULIA_VSCODE_REPL = 1

versioninfo() for updated HiGHS.jl is identical (same machine)

st -m for HiGHS 1.17.0

(proj) pkg> st -m Project proj v1.0.5 Status C:\work\proj.jl\Manifest.toml [14f7f29c] AMD v0.5.3 [79e6a3ab] Adapt v4.3.0 [13072b0f] AxisAlgorithms v1.1.0 [6e4b80f9] BenchmarkTools v1.6.0 [c3b6d118] BitIntegers v0.3.5 [336ed68f] CSV v0.10.15 [d360d2e6] ChainRulesCore v1.25.2 ⌅ [61c947e1] Clarabel v0.10.0 [da1fd8a2] CodeTracking v1.3.9 [523fee87] CodecBzip2 v0.8.5 [944b1d66] CodecZlib v0.7.8 [a80b9123] CommonMark v0.9.1 [bbf7d656] CommonSubexpressions v0.3.1 [34da2185] Compat v4.17.0 [807dbc54] Compiler v0.1.1 [992eb4ea] CondaPkg v0.2.29 [a8cc5b0e] Crayons v4.1.1 [a10d1c49] DBInterface v2.6.1 [9a962f9c] DataAPI v1.16.0 [a93c6f00] DataFrames v1.7.0 [864edb3b] DataStructures v0.18.22 [e2d170a0] DataValueInterfaces v1.0.0 [31a5f54b] Debugger v0.7.13 [163ba53b] DiffResults v1.1.0 [b552c78f] DiffRules v1.15.1 [ffbed154] DocStringExtensions v0.9.5 [d2f5444f] DuckDB v1.3.2 [e2ba6199] ExprTools v0.1.10 [48062228] FilePathsBase v0.9.24 [fb4d412d] FixedPointDecimals v0.6.3 [f6369f11] ForwardDiff v1.0.1 [60bf3e95] GLPK v1.2.1 [14197337] GenericLinearAlgebra v0.3.17 [c27321d9] Glob v1.3.1 [2e9cd046] Gurobi v1.7.5 ⌅ [34c5aeac] HSL v0.4.3 ⌃ [87dc4568] HiGHS v1.17.0 [eafb193a] Highlights v0.5.3 [842dd82b] InlineStrings v1.4.4 [a98d9a8b] Interpolations v0.16.1 [41ab1584] InvertedIndices v1.3.1 [92d709cd] IrrationalConstants v0.2.4 [82899510] IteratorInterfaceExtensions v1.0.0 [692b3bcd] JLLWrappers v1.7.0 [682c06a0] JSON v0.21.4 [0f8b85d8] JSON3 v1.14.3 [4076af6c] JuMP v1.26.0 [98e50ef6] JuliaFormatter v2.1.2 [aa1ae85d] JuliaInterpreter v0.10.3 ⌅ [70703baa] JuliaSyntax v0.4.10 [b964fa9f] LaTeXStrings v1.4.0 [0e77f7df] LazilyInitializedFields v1.3.0 [89398ba2] LocalRegistry v0.5.7 [2ab3a3ac] LogExpFunctions v0.3.29 [e6f89c97] LoggingExtras v1.1.0 [6f1432cf] LoweredCodeUtils v3.4.1 [1914dd2f] MacroTools v0.5.16 [b8f27783] MathOptInterface v1.42.0 [0b3b1443] MicroMamba v0.1.14 [93f3dd0f] MiniLoggers v0.5.3 [e1d29d7a] Missings v1.2.0 [d8a4904e] MutableArithmetics v1.6.4 [77ba4419] NaNMath v1.1.3 [6fe1bfb0] OffsetArrays v1.17.0 [bac558e1] OrderedCollections v1.8.1 ⌅ [46dd5b70] Pardiso v0.5.0 [69de0a69] Parsers v2.8.3 [fa939f87] Pidfile v1.3.0 [2dfb63ee] PooledArrays v1.4.3 ⌅ [aea7be01] PrecompileTools v1.2.1 [21216c6a] Preferences v1.4.3 [08abe8d2] PrettyTables v2.4.0 [6099a3de] PythonCall v0.9.25 [bfc457fd] QDLDL v0.4.1 [c84ed2f1] Ratios v0.4.5 [189a3867] Reexport v1.2.2 [2792f1a3] RegistryInstances v0.1.0 [d1eb7eb1] RegistryTools v2.3.0 [ae029012] Requires v1.3.1 [295af30f] Revise v3.8.0 [6c6a2e73] Scratch v1.3.0 [91c51154] SentinelArrays v1.4.8 [66db9d55] SnoopPrecompile v1.0.3 [a2af1166] SortingAlgorithms v1.2.1 [276daf66] SpecialFunctions v2.5.1 [90137ffa] StaticArrays v1.9.13 [1e83bf80] StaticArraysCore v1.4.3 [10745b16] Statistics v1.11.1 [892a3eda] StringManipulation v0.4.1 [856f2bd8] StructTypes v1.11.0 [3783bdb8] TableTraits v1.0.1 [bd369af6] Tables v1.12.1 [a759f4b9] TimerOutputs v0.5.29 [3bb67fe8] TranscodingStreams v0.11.3 [e17b2a0c] UnsafePointers v1.0.0 [ea10d353] WeakRefStrings v1.4.2 [efce3f68] WoodburyMatrices v1.0.0 [76eceee3] WorkerUtilities v1.6.1 [6e34b625] Bzip2_jll v1.0.9+0 [2cbbab25] DuckDB_jll v1.3.2+0 [e8aa6df9] GLPK_jll v5.0.1+1 ⌃ [c018c7e6] Gurobi_jll v12.0.1 ⚲ [017b0a0e] HSL_jll v4.0.2+0 ⌅ [8fd58aa0] HiGHS_jll v1.10.0+0 [1d5cc7b8] IntelOpenMP_jll v2025.0.4+0 [856f044c] MKL_jll v2025.0.1+1 [656ef2d0] OpenBLAS32_jll v0.3.29+0 [efe28fd5] OpenSpecFun_jll v0.5.6+0 [f8abcde7] micromamba_jll v1.5.8+0 [1317d2d5] oneTBB_jll v2022.0.0+0 [4d7b5844] pixi_jll v0.41.3+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [8ba89e20] Distributed v1.11.0 [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching v1.11.0 [9fa8497b] Future v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [4af54fe1] LazyArtifacts v1.11.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.11.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.2.0 [44cfe95a] Pkg v1.11.0 [de0858da] Printf v1.11.0 [9abbd945] Profile v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [1a1011a3] SharedArrays v1.11.0 [6462fe0b] Sockets v1.11.0 [2f01184e] SparseArrays v1.11.0 [f489334b] StyledStrings v1.11.0 [4607b0f0] SuiteSparse [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.1.1+0 [781609d7] GMP_jll v6.3.0+0 [deac9b47] LibCURL_jll v8.6.0+0 [e37daf67] LibGit2_jll v1.7.2+0 [29816b5a] LibSSH2_jll v1.11.0+1 [c8ffd9c3] MbedTLS_jll v2.28.6+0 [14a3606d] MozillaCACerts_jll v2023.12.12 [4536629a] OpenBLAS_jll v0.3.27+1 [05823500] OpenLibm_jll v0.8.5+0 [bea87d4a] SuiteSparse_jll v7.7.0+0 [83775a58] Zlib_jll v1.2.13+1 [8e850b90] libblastrampoline_jll v5.11.0+0 [8e850ede] nghttp2_jll v1.59.0+0 [3f19e933] p7zip_jll v17.4.0+2

st -m for updated HiGHS.jl

(proj) pkg> st -m Project proj v1.0.5 Status C:\work\proj.jl\Manifest.toml [14f7f29c] AMD v0.5.3 [79e6a3ab] Adapt v4.3.0 [13072b0f] AxisAlgorithms v1.1.0 [6e4b80f9] BenchmarkTools v1.6.0 [c3b6d118] BitIntegers v0.3.5 [336ed68f] CSV v0.10.15 [d360d2e6] ChainRulesCore v1.25.2 ⌅ [61c947e1] Clarabel v0.10.0 [da1fd8a2] CodeTracking v1.3.9 [523fee87] CodecBzip2 v0.8.5 [944b1d66] CodecZlib v0.7.8 [a80b9123] CommonMark v0.9.1 [bbf7d656] CommonSubexpressions v0.3.1 [34da2185] Compat v4.17.0 [807dbc54] Compiler v0.1.1 [992eb4ea] CondaPkg v0.2.29 [a8cc5b0e] Crayons v4.1.1 [a10d1c49] DBInterface v2.6.1 [9a962f9c] DataAPI v1.16.0 [a93c6f00] DataFrames v1.7.0 [864edb3b] DataStructures v0.18.22 [e2d170a0] DataValueInterfaces v1.0.0 [31a5f54b] Debugger v0.7.13 [163ba53b] DiffResults v1.1.0 [b552c78f] DiffRules v1.15.1 [ffbed154] DocStringExtensions v0.9.5 [d2f5444f] DuckDB v1.3.2 [e2ba6199] ExprTools v0.1.10 [48062228] FilePathsBase v0.9.24 [fb4d412d] FixedPointDecimals v0.6.3 [f6369f11] ForwardDiff v1.0.1 [60bf3e95] GLPK v1.2.1 [14197337] GenericLinearAlgebra v0.3.17 [c27321d9] Glob v1.3.1 [2e9cd046] Gurobi v1.7.5 ⌅ [34c5aeac] HSL v0.4.3 [87dc4568] HiGHS v1.18.1 [eafb193a] Highlights v0.5.3 [842dd82b] InlineStrings v1.4.4 [a98d9a8b] Interpolations v0.16.1 [41ab1584] InvertedIndices v1.3.1 [92d709cd] IrrationalConstants v0.2.4 [82899510] IteratorInterfaceExtensions v1.0.0 [692b3bcd] JLLWrappers v1.7.0 [682c06a0] JSON v0.21.4 [0f8b85d8] JSON3 v1.14.3 [4076af6c] JuMP v1.26.0 [98e50ef6] JuliaFormatter v2.1.2 [aa1ae85d] JuliaInterpreter v0.10.3 ⌅ [70703baa] JuliaSyntax v0.4.10 [b964fa9f] LaTeXStrings v1.4.0 [0e77f7df] LazilyInitializedFields v1.3.0 [89398ba2] LocalRegistry v0.5.7 [2ab3a3ac] LogExpFunctions v0.3.29 [e6f89c97] LoggingExtras v1.1.0 [6f1432cf] LoweredCodeUtils v3.4.1 [1914dd2f] MacroTools v0.5.16 [b8f27783] MathOptInterface v1.42.0 [0b3b1443] MicroMamba v0.1.14 [93f3dd0f] MiniLoggers v0.5.3 [e1d29d7a] Missings v1.2.0 [d8a4904e] MutableArithmetics v1.6.4 [77ba4419] NaNMath v1.1.3 [6fe1bfb0] OffsetArrays v1.17.0 [bac558e1] OrderedCollections v1.8.1 ⌅ [46dd5b70] Pardiso v0.5.0 [69de0a69] Parsers v2.8.3 [fa939f87] Pidfile v1.3.0 [2dfb63ee] PooledArrays v1.4.3 ⌅ [aea7be01] PrecompileTools v1.2.1 [21216c6a] Preferences v1.4.3 [08abe8d2] PrettyTables v2.4.0 [6099a3de] PythonCall v0.9.25 [bfc457fd] QDLDL v0.4.1 [c84ed2f1] Ratios v0.4.5 [189a3867] Reexport v1.2.2 [2792f1a3] RegistryInstances v0.1.0 [d1eb7eb1] RegistryTools v2.3.0 [ae029012] Requires v1.3.1 [295af30f] Revise v3.8.0 [6c6a2e73] Scratch v1.3.0 [91c51154] SentinelArrays v1.4.8 [66db9d55] SnoopPrecompile v1.0.3 [a2af1166] SortingAlgorithms v1.2.1 [276daf66] SpecialFunctions v2.5.1 [90137ffa] StaticArrays v1.9.13 [1e83bf80] StaticArraysCore v1.4.3 [10745b16] Statistics v1.11.1 [892a3eda] StringManipulation v0.4.1 [856f2bd8] StructTypes v1.11.0 [3783bdb8] TableTraits v1.0.1 [bd369af6] Tables v1.12.1 [a759f4b9] TimerOutputs v0.5.29 [3bb67fe8] TranscodingStreams v0.11.3 [e17b2a0c] UnsafePointers v1.0.0 [ea10d353] WeakRefStrings v1.4.2 [efce3f68] WoodburyMatrices v1.0.0 [76eceee3] WorkerUtilities v1.6.1 [6e34b625] Bzip2_jll v1.0.9+0 [2cbbab25] DuckDB_jll v1.3.2+0 [e8aa6df9] GLPK_jll v5.0.1+1 ⌃ [c018c7e6] Gurobi_jll v12.0.1 ⚲ [017b0a0e] HSL_jll v4.0.2+0 [8fd58aa0] HiGHS_jll v1.11.0+1 [1d5cc7b8] IntelOpenMP_jll v2025.0.4+0 [856f044c] MKL_jll v2025.0.1+1 [656ef2d0] OpenBLAS32_jll v0.3.29+0 [efe28fd5] OpenSpecFun_jll v0.5.6+0 [f8abcde7] micromamba_jll v1.5.8+0 [1317d2d5] oneTBB_jll v2022.0.0+0 [4d7b5844] pixi_jll v0.41.3+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [8ba89e20] Distributed v1.11.0 [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching v1.11.0 [9fa8497b] Future v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [4af54fe1] LazyArtifacts v1.11.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.11.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.2.0 [44cfe95a] Pkg v1.11.0 [de0858da] Printf v1.11.0 [9abbd945] Profile v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [1a1011a3] SharedArrays v1.11.0 [6462fe0b] Sockets v1.11.0 [2f01184e] SparseArrays v1.11.0 [f489334b] StyledStrings v1.11.0 [4607b0f0] SuiteSparse [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.1.1+0 [781609d7] GMP_jll v6.3.0+0 [deac9b47] LibCURL_jll v8.6.0+0 [e37daf67] LibGit2_jll v1.7.2+0 [29816b5a] LibSSH2_jll v1.11.0+1 [c8ffd9c3] MbedTLS_jll v2.28.6+0 [14a3606d] MozillaCACerts_jll v2023.12.12 [4536629a] OpenBLAS_jll v0.3.27+1 [05823500] OpenLibm_jll v0.8.5+0 [bea87d4a] SuiteSparse_jll v7.7.0+0 [83775a58] Zlib_jll v1.2.13+1 [8e850b90] libblastrampoline_jll v5.11.0+0 [8e850ede] nghttp2_jll v1.59.0+0 [3f19e933] p7zip_jll v17.4.0+2

jhmenke avatar Jul 15 '25 08:07 jhmenke

Even with the MPS file, I get the same problem dimensions

LP 2466 has 991849 rows; 1097468 cols; 3829286 nonzeros

but I now get (as you)

Objective value : 4.1820280416e+11

Returning to your results, given the identical iteration count between the runs with HiGHS v1.10 and v1.11, the time difference is what might occur with the use of optimized compilation rather than compilation for debugging.

I assume the iteration logs are time-based; the intermediary results seem to be more or less the same.

Yes, so that users don't have to wait too long to know that HiGHS is still running!

jajhall avatar Jul 15 '25 08:07 jajhall

Even with the MPS file, I get the same problem dimensions

LP 2466 has 991849 rows; 1097468 cols; 3829286 nonzeros

It's likely that my initial log was from a slightly different optimization problem. Sorry if that was the case.

Returning to your results, given the identical iteration count between the runs with HiGHS v1.10 and v1.11, the time difference is what might occur with the use of optimized compilation rather than compilation for debugging.

That thought also crossed my mind. Would the fault then be in the HiGHS_jll julia package, @odow ? The commits for HiGHS_jll don't show any compiler flags being different between 1.10.0 and 1.11.0 (1.11.0+1).

jhmenke avatar Jul 15 '25 08:07 jhmenke

I can confirm that using a precompiled highs.exe binary with the .mps file, the solve time is not much slower than previously with 1.10.0. Therefore, it seems to be something HiGHS.jl-related. I have opened an issue there.

jhmenke avatar Jul 15 '25 09:07 jhmenke

In light of https://github.com/jump-dev/HiGHS.jl/issues/291#issuecomment-3076000501 I'm reopening this issue.

There's some overhead in the new HiGHS callback when called from C, and this is exacerbated when called from SimplexInterrupt, which triggers every simplex iteration (potentially hundreds of thousands of times).

I ran this:

using PProf
import JuMP
import HiGHS
function main()
    model = JuMP.read_from_file("debug_model.mps")
    JuMP.set_optimizer(model, HiGHS.Optimizer)
    JuMP.set_attribute(model, "time_limit", 30.0)
    counter = Ref{Int}(0)
    function callback_fn(args...)
        counter[] += 1
        return Cint(0)    
    end
    loc = [HiGHS.kHighsCallbackSimplexInterrupt]
    JuMP.set_attribute(model, HiGHS.CallbackFunction(loc), callback_fn)
    @pprof JuMP.optimize!(model)
    return counter[]
end
main()

Got this flame graph. There's a memmove that couldn't be tied to any particular call and takes up 40% of total runtime:

Image Image

The middle red copy_to is JuMP copying the model into HiGHS.

Of Highs_run, there's a suspicious free_large

Image

So I'm going to call foul with something in the callback routine that is moving memory, and we're doing it every time the callback is hit, which is ~every simplex iteration.

odow avatar Jul 16 '25 00:07 odow