Pyomo
Threading Error
Summary
I ran the code solver = SolverFactory('gurobi').solve(m, tee=False) for many concrete models m. I was attempting to solve a stochastic programming problem using Monte Carlo simulation.
Steps to reproduce the issue
Run this example code. I get the error around iteration 8200.
# example.py
from pyomo.environ import *
import numpy as np
# Parameters
num_iterations = 100000
mean_demand = 50
std_dev_demand = 10
# Monte Carlo Simulation
for i in range(num_iterations):
demand = np.random.normal(mean_demand, std_dev_demand)
demand = max(0, demand)
model = ConcreteModel()
model.x = Var(within=NonNegativeReals)
model.obj = Objective(expr=model.x, sense=minimize)
# Constraint: Production must meet demand
model.demand_constraint = Constraint(expr=model.x >= demand)
solver = SolverFactory('gurobi')
solver.solve(model)
mean_demand = (mean_demand * i + demand) / (i + 1)
# Display results
if i % 50 == 0:
print(f"Iteration {i + 1}: Demand = {demand}, Optimal Production = {model.x.value}, Updated Mean Demand = {mean_demand}")
...
Error Message
$ Traceback (most recent call last):
File , line 44, in <module>
solver.solve(model)
~~~~~~~~~~~~^^^^^^^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\opt\base\solvers.py", line 534, in solve
self.available(exception_flag=True)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\solvers\plugins\solvers\GUROBI.py", line 631, in available
self._check_license()
~~~~~~~~~~~~~~~~~~~^^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\solvers\plugins\solvers\GUROBI.py", line 645, in _check_license
with capture_output(capture_fd=True):
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 310, in __enter__
fd_redirect[i + 1] = self._enter_context(
~~~~~~~~~~~~~~~~~~~^
redirect_fd(i + 1, tee_fd[i], synchronize=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 239, in _enter_context
return cm.__enter__()
~~~~~~~~~~~~^^
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 130, in __enter__
self.original_fd = os.dup(self.fd)
~~~~~~^^^^^^^^^
OSError: [Errno 24] Too many open files
Exception ignored in: <function capture_output.__del__ at 0x000002132DE71BC0>
Traceback (most recent call last):
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 386, in __del__
self.__exit__(None, None, None)
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 368, in __exit__
if self.tee.STDOUT is not sys.stdout:
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 571, in STDOUT
self._stdout = self.open(buffering=b)
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 603, in open
self._start(handle)
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\pyomo\common\tee.py", line 671, in _start
th.start()
File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\threading.py", line 973, in start
_start_joinable_thread(self._bootstrap, handle=self._handle,
PythonFinalizationError: can't create new thread at interpreter shutdown
Information on your system
Pyomo version: 6.9.2 Python version: 3.13.3 and 3.9.21 (I tried both) Operating system: Windows How Pyomo was installed (PyPI, conda, source): PyPI Solver (if applicable): Gurobi
Additional information
If you change the solver to glpk then the code works fine. Additionally, if you move solver = SolverFactory('gurobi') outside of the loop then the code works fine. Thus, the issue seems to be with the way Pyomo is interacting with gurobi when the SolverFactory is instantiated.
Also, changing the gurobi interface to 'gurobi_direct_v2' does not fix the problem.
