Path issues when running multiple solves at the same time with COIN-OR CBC

[arizzuto]

Summary

I'm running a series of sequential MIP solves with COIN-OR CBC. The process runs a solution and then uses the solution as input to a new problem. This continues in a loop for many iterations. If I simultaneously kick off another series of solves as described above, after some time, one of the two processes throws the following:

Traceback (most recent call last):
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/run_backcast.py", line 197, in <module>
    main()
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/run_backcast.py", line 121, in main
    mip.solve(additional_solver_options={'solu': settings_obj.sol_path}, print_solver_output=False, attempt_tolerance_adjustment=True)
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/venv/lib/python3.12/site-packages/energy_schedule_optimiser/OptimiseEnergyStorage/btm_mip.py", line 708, in solve
    _results = self._run_solve(solver_options=solver_options, print_solver_output=print_solver_output)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/venv/lib/python3.12/site-packages/energy_schedule_optimiser/OptimiseEnergyStorage/btm_mip.py", line 665, in _run_solve
    results = pyo.SolverFactory('cbc').solve(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/venv/lib/python3.12/site-packages/pyomo/opt/base/solvers.py", line 636, in solve
    result = self._postsolve()
             ^^^^^^^^^^^^^^^^^
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/venv/lib/python3.12/site-packages/pyomo/solvers/plugins/solvers/CBCplugin.py", line 1087, in _postsolve
    results = super(CBCSHELL, self)._postsolve()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/aaronrizzuto/code/arbitrageModelling/BackCasting/venv/lib/python3.12/site-packages/pyomo/opt/solver/shellcmd.py", line 286, in _postsolve
    raise IOError(msg % (self._log_file, self.path))
                                         ^^^^^^^^^
AttributeError: 'CBCSHELL' object has no attribute 'path'

Is it possible that I am creating a race condition on intermediary files in some way?

Steps to reproduce the issue

Have to series of sequential solves running at the same time. I will attempt to generate a repo with the problem failure with a simple MIP problem if required.

Error Message

$ # Output message here, including entire stack trace, if available

Information on your system

Pyomo version: 6.9.2 Python version: 3.12 Operating system: MAC OS 15.1 How Pyomo was installed: PyPI Solver: COIN-OR [email protected] via coinbrew script.

[sadavis1]

When you say you are running multiple solves at the same time, do you mean that you are using the threading or the multiprocessing library? (or neither?) We are aware that certain solver interfaces have problems with threading right now. Also, if you are using threading, then you should not use the same solver object multiple times from different threads (because they can contain internal state while working).

[arizzuto]

I'm not running multiprocessing or anything like that. Looking back I see my description is a bit unclear, I'll try make it more direct by writing a step by step:

1. Open a term, cd my/project/dir, and source a python environment.
2. run: python my_workflow.py input1. This does thousands of MIP solves in sequence one after the other, extracting results after every step.
3. Open another term, cd my/project/dir, and source the same environment.
4. run: python my_workflow.py input2. This does the same as step 2 but for different inputs.
5. Watch as they both chug along for a while.
6. After a bit, the trace I posted in the initial issue post happens with the error:

AttributeError: 'CBCSHELL' object has no attribute 'path'

If I run either steps 2 or 4 without running the other, the error doesn't happen.

I'm trying to reproduce with a simpler MIP but I can't seem to make it happen.

[sadavis1]

From looking at the trace, it seems like we are first writing to a solver log file (shellcmd.py:271) and then failing when the path we wrote to doesn't exist immediately afterwards. The AttributeError is an bug with CBCSHELL but it's spurious here.

Is there any chance that your my_workflow.py script is deleting the log files written by the other process? That might be able to cause this issue.