Running emcee with mpi and openmp

[sultan-hassan]

I am trying to run emcee with MPI, but my model in lnlike function is a C code runs with openmp. The code works faster if openmp is off. Is there anyway to make use of MPI and openmp together?. Any suggestions to include in the python script or Slurm script is appreciated. Thanks.

[aarchiba]

I can't give you very specific suggestions, but if you can get slurm to run just one MPI job per machine (but treat it as occupying all cores on that machine) then the openmp should handle filling all the cores of the machine.

[manodeep]

@sultanier If you are running on a supercomputer cluster, then you will need to follow the instructions on how to submit a MPI + OpenMP job. For instance, on our local supercomputer running the slurm scheduler, this link explains how to submit such a job.

The key is to convey the OpenMP intent within the MPI script, and then this line export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK, that says to use as many OpenMP threads as you have requested within the slurm script. What happens is that frequently OMP_NUM_THREADS is set to 1, and unless you specifically alter that in the slurm script, you can not use the OpenMP threads.