loky
loky copied to clipboard
RFE: start jobs depending on loadavg
Often I want to start many independent processes on a shared server, and I cannot know in advance how many CPUs will be available.
I want to be able to start new process whenever, say, getloadavg1 < 1.5*cpu_count (as opposed to a fixed max_workers ).
cf. https://github.com/joblib/joblib/issues/1006 and https://bugs.python.org/issue39617