keiserlab
Replacing python_utilities.parallel
python_utilities.parallel has always been a bit of a hack to have a unified API for 4 parallelization approaches: serial, distributed, multithreaded, and message passing. It would be better to call that functionality from a better tested and maintained package, but I have until now been unable to find one. It looks like Ray might have this functionality. Their API is quite different, so it would take a little work to switch us over to their API.
Alternatively, mpi4py.futures.MPICommExecutor is a child of concurrent.futures.Executor, so we could perhaps just use Executor API functions to support all 4 modes. I don't see a SequentialExecutor or SerialExecutor anywhere, so we may need to roll our own.