Brian M. Adams

(Separate discussion with @garlick led me here...) I see what I believe to be a similar issue running on DOE CTS-1 with OpenMPI 4.x applications, notably 4.1.1. Summary: - Assume:...

Nothing special seems needed to run. Here are some abbreviated notes from a clean salloc/direct launch, where the mpirun variant default to running the tests on one node and both...

Good call. Nothing special additional with srun: ``` $ srun -N 1 -n 2 env | egrep 'OMPI_|PMI' | sort -u OMPI_MCA_btl=^openib OMPI_MCA_btl_openib_allow_ib=true OMPI_MCA_btl_openib_ib_retry_count=7 OMPI_MCA_btl_openib_ib_timeout=21 OMPI_MCA_mtl=psm2 OMPI_MCA_pml=cm ``` I also...

Sorry, tried which in our environment? I haven't tried wiping out the system module-set OMPI_* variables in our environment, but if that's what you mean, can give it a try....

> wiping out the system module-set OMPI_* variables in our environment Didn't change any behavior, one rank still hangs when run with -N 1. (I unset any OMPI_* variables both...

Now I caused confusion... All my experiments reported in this issue are with 4.1.1. My team member who asked me to demo this is ultimately aiming to use Dakota +...

**Progress!** It occurred to me that with my previous experiment clearing the OMPI_* env vars, the runtime could still fall-back to the default best transports and maybe pick cm/psm2. If...

Just to verify, I ran my -N1 test case with that increased verbosity with both the default openmpi-4.1.1 environment as well as in a clean one where only those verbosity...

@dmcdougall @roystgnr For the Environment (and other) constructors that accept both input file name and EnvOptionsValues object, what do you feel the precedence should be? On the Dakota side, we're...

Per @dmcdougall, also need to consider where in this workflow a client might programmaticly set options as well, e.g., to override defaults through C++ code.