MPIPreferences + Test

[carstenbauer]

Probably a more general/upstream issue but I'll post it here nonetheless, because this is where I encountered it. (UPDATE: https://github.com/JuliaLang/Pkg.jl/issues/3389)

In short, the issue is that ] test doesn't seem to respect the JULIA_LOAD_PATH variable and thus global preference files.

On our cluster, we provide a global Project.toml (say at /some/global/path/Project.toml) in which we set the MPI preferences, specifically

[extras]
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"

[preferences.MPIPreferences]
_format = "1.0"
abi = "OpenMPI"
binary = "system"
libmpi = "libmpi"
mpiexec = "mpiexec"

We then automatically set JULIA_LOAD_PATH=:/some/gobal/path when the user loads the julia module. This works just fine in a regular Julia session where one gets

LOAD_PATH == ["@", "@v#.#", "@stdlib", "/some/global/path"]

However, when running ] test the JULIA_LOAD_PATH env variable is ignored and one only gets

LOAD_PATH == ["@", "/tmp/jl_KwLRNl"]

This means that the global MPI preferences are not respected, leading to undesired behavior and, in my specific case, an indefinite hang.

[simonbyrne]

Okay, so the advice should generally be that if your package uses MPI in its tests, it should have MPIPreferences in its test dependencies?

[carstenbauer]

@simonbyrne Until it's fixed upstream, yes, that's the recommendation.

[omlins]

@carstenbauer : so putting the following in the project.toml of a package would work?

[targets]
test = ["Test", "MPIPreferences"]

[simonbyrne]

Yes, you also need:

[extras]
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"