JuliaParallel
Unable to locate `LocalPreferences.toml` outside of MPI.jl
I observe a strange behaviour using MPI.jl#master (0.20-dev) while running the MPI.use_system_binary() in a project outside of the MPI.jl project. Namely, the system MPI implementation seems to be correctly selected but the LocalPreferences.toml file that should be generated in the project is not there (only Project.toml and Manifest.toml are present). Do I miss something?
Reproducer:
[lraess@ault dev]$ mkdir tmp_test
[lraess@ault dev]$ cd tmp_test/
[lraess@ault dev]$ julia --project
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.2 (2022-02-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia>
(@v1.7) pkg> activate .
Activating new project at `/scratch/lraess/dev/tmp_test`
(tmp_test) pkg> add MPI#master
(tmp_test) pkg> st
Status `/scratch/lraess/dev/tmp_test/Project.toml`
[da04e1cc] MPI v0.20.0-dev `https://github.com/JuliaParallel/MPI.jl.git#master`
julia> using MPI
julia> MPI.use_system_binary()
┌ Info: MPI implementation
│ libmpi = "libmpi"
│ version_string = "Open MPI v4.0.6rc4, package: Open MPI [email protected] Distribution, ident: 4.0.6rc4, repo rev: v4.0.6-33-gd71f8c2ba8, Unreleased developer copy\0"
│ impl = "OpenMPI"
│ version = v"4.0.6-rc4"
└ abi = "OpenMPI"
┌ Warning: The underlying MPI implementation has changed. You will need to restart Julia for this change to take effect
│ libmpi = "libmpi"
│ abi = "OpenMPI"
│ mpiexec = "mpiexec"
└ @ MPIPreferences ~/.julia/packages/MPIPreferences/uArzO/src/MPIPreferences.jl:119
shell> ls
Manifest.toml Project.toml
shell> cat Project.toml
[deps]
MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
julia> exit()
[lraess@ault dev]$ julia --project
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.2 (2022-02-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> using MPI
julia> MPI.MPI_LIBRARY_VERSION_STRING
"Open MPI v4.0.6rc4, package: Open MPI [email protected] Distribution, ident: 4.0.6rc4, repo rev: v4.0.6-33-gd71f8c2ba8, Unreleased developer copy\0"
You must listen MPIPreferences in the [extras] section of you Project.toml
You can copy LocalPreferences.toml to a different project folder, but you must list MPIPreferences in the [extras] section of the Project.toml for the settings to take effect.
https://juliaparallel.org/MPI.jl/latest/configuration/#Configuration-2
Thanks for pointing to the latest doc @vchuravy. To me it is currently unclear what behaviour to expect. You're suggesting one should manually add MPIPreferences to an [extra] section in the project-local Project.toml. From the doc, I however understand that (with exception of Julia<1.6.5 and 1.7.1) the LocalPreferences.toml file would be created "adjacent to the current Project.toml":
To use the system MPI library, run MPI.use_system_binary(). This will attempt to locate and to identify any available MPI implementation, and create a file called LocalPreferences.toml adjacent to the current Project.toml
Also, using MPI.use_system_binary(export_prefs=true) does not seem to populate the project-local Project.toml with any MPI related prefs.
Ah I see I missed that you did indeed ran the command in your Project. Will investigate
Note that indeed adding manually MPIPreferences to [extra] fixes the issue but may not be the intended action.
~/builds/julia-1.7.2/bin/julia --project=repr
(repr) pkg> add MPI#master
...
[da04e1cc] ~ MPI v0.19.2 ⇒ v0.20.0-dev `https://github.com/JuliaParallel/MPI.jl.git#master`
[3da0fdf6] + MPIPreferences v0.1.1
...
julia> using MPI
julia> MPI.use_system_binary()
┌ Info: MPI implementation
│ libmpi = "libmpi"
│ version_string = "Open MPI v4.1.2, package: Open MPI builduser@arojas Distribution, ident: 4.1.2, repo rev: v4.1.2, Nov 24, 2021\0"
│ impl = "OpenMPI"
│ version = v"4.1.2"
└ abi = "OpenMPI"
┌ Warning: The underlying MPI implementation has changed. You will need to restart Julia for this change to take effect
│ libmpi = "libmpi"
│ abi = "OpenMPI"
│ mpiexec = "mpiexec"
└ @ MPIPreferences ~/.julia/packages/MPIPreferences/uArzO/src/MPIPreferences.jl:119
shell> ls repr/
LocalPreferences.toml Manifest.toml Project.toml
shell> cat repr/Project.toml
[deps]
MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
[extras]
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
shell> cat repr/LocalPreferences.toml
[MPIPreferences]
abi = "OpenMPI"
binary = "system"
libmpi = "libmpi"
mpiexec = "mpiexec"
oO
@staticfloat is this intended:
➜ ~ ls mpiprefs
LocalPreferences.toml Project.toml
➜ ~ ls repr
Manifest.toml Project.toml
cat mpiprefs/LocalPreferences.toml
[MPIPreferences]
abi = "OpenMPI"
binary = "system"
libmpi = "libmpi"
mpiexec = "mpiexec"
JULIA_LOAD_PATH=":$HOME/mpiprefs" ~/builds/julia-1.7.2/bin/julia --project=repr
julia> MPI.MPIPreferences.abi
"OpenMPI"
julia> MPI.use_jll_binary()
shell> ls repr
Manifest.toml Project.toml
shell> cat mpiprefs/LocalPreferences.toml
[MPIPreferences]
__clear__ = ["libmpi", "abi", "mpiexec"]
binary = "MPICH_jll"
E.g. setting the preference is not done in the current Project, but a Project up in the load path?
The issue of course is:
https://github.com/JuliaPackaging/Preferences.jl/blob/80b550904ae5b5615ec4168a2135f87c397a92b8/src/Preferences.jl#L198
MPIPreferences = Base.UUID("3da0fdf6-3ccc-4f1b-acd9-58baa6c99267")
julia> Preferences.find_first_project_with_uuid(MPIPreferences)
("/home/vchuravy/mpiprefs/Project.toml", "MPIPreferences")
shell> cat mpiprefs/Project.toml
[extras]
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
Ah ha
julia> MPIPreferences = Base.UUID("3da0fdf6-3ccc-4f1b-acd9-58baa6c99267")
UUID("3da0fdf6-3ccc-4f1b-acd9-58baa6c99267")
julia> Preferences.find_first_project_with_uuid(MPIPreferences)
("/users/lraess/.julia/environments/v1.7/Project.toml", "MPIPreferences")
shell> cat /users/lraess/.julia/environments/v1.7/Project.toml
[deps]
AMDGPU = "21141c5a-9bdb-4563-92ae-f87d6854732e"
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
MPI = "da04e1cc-30fd-572f-bb4f-1f8673147195"
MPIPreferences = "3da0fdf6-3ccc-4f1b-acd9-58baa6c99267"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"
[extras]
CPUSummary = "2a0fbf3d-bb9c-48f3-b0a9-814d99fd7ab9"
[preferences.MPIPreferences]
abi = "OpenMPI"
binary = "system"
libmpi = "libmpi"
mpiexec = "mpiexec"