profPlum

@dalcinl Yes I've ask summit IT staff we did a very thorough investigation and I have a lot of experimental data testing what works and what doesn't that I could...

@dalcinl Also there was this funny situation in which I ALMOST got the MPICommExecutor working (with crazy environment variable settings). It was in fact working as long as the MPI...

@stefanozampini I'm not sure how to get a stack trace, because the latest problem is freezing not erroring out. That being said there is some more info to be gained...

@dalcinl Thanks for clarification. I actually realized I could just keyboard interrupt & it would give me a traceback haha. @dalcinl & @stefanozampini Here it is: The command is: `module...

@dalcinl & @stefanozampini: also _when I keyboard interrupt again_ (the first one doesn't kill it for some reason), this happens: ``` /sw/summit/open-ce/anaconda-base/envs/open-ce-1.2.0-py38-0/lib/python3.8/site-packages/mpi4py/futures/_lib.py:142: RuntimeWarning: The level of thread support in MPI...

@jmschrei Was this ever merged?

@hvy The idea is indeed very simple & easy to implement, let me provide pseudo code: ``` import multiprocessing as mp import optuna as ot def objective(cfg_dict): return cfg_dict['x'] def...

@hvy @himkt @divyanshugit So, what do you guys think? P.S. @hvy is this similar to what you mentioned in your PR?

@himkt I think you are too focused on the batch functionality. That was only there so as to emphasize the point that _the worker processes never need to directly interact...

@himkt This should also be possible, just reasoning from first principles: you could use the disk to communicate across nodes. But there could be simpler solutions (e.g. if your job...