pyomo icon indicating copy to clipboard operation
pyomo copied to clipboard

Thread handling in Pyomo

Open ZedongPeng opened this issue 2 years ago • 2 comments

Summary

I am doing some benchmark tests with MindtPy. For the fairness of these benchmarks, I want to limit each benchmark test using only one thread. However, I found a weird situation in Pyomo. I can successfully pass the threads option to solvers like CPLEX and solvers will only use one thread. However, at the stage before CPLEX solves the model. Multiple threads are used. Is there some magic stuff like generating LP inside Pyomo using multiple threads?

Steps to reproduce the issue

  1. Download the unitcommit_200_100_2_mod_7 example.
  2. Use the following Python script to solve it.
# example.py
from pyomo.environ import *
from importlib import import_module

module = import_module("unitcommit_200_100_2_mod_7")
opt = SolverFactory('cplex')
opt.options['threads'] = 1
results = opt.solve(module.m, tee=True)
print(results)

If you run the above code, you will find that multiple threads are involved at the beginning. After that, 'CPLEX' uses only one thread to solve the model.

Information on your system

Pyomo version: 6.6.2.dev0 Python version: 3.10.12 Operating system: macOS How Pyomo was installed (PyPI, conda, source): source Solver (if applicable): CPLEX

Additional information

ZedongPeng avatar Sep 08 '23 23:09 ZedongPeng

I checked again. Multi threads are used when building pyomo models. More exactly, this line. module = import_module("unitcommit_200_100_2_mod_7")

ZedongPeng avatar Sep 11 '23 03:09 ZedongPeng

How are you identifying that multiple threads are being used? While Pyomo does make use of multithreading to manage the console output from subprocess solves (particularly for tee=True), there is nothing in the Pyomo code that should invoke multithreading while importing the model. Further, looking at top on Linux, I don't see the python process exceeding 100% CPU (which again doesn't point to multithreading).

jsiirola avatar Jul 26 '24 20:07 jsiirola