openhpc
Set impi default version to match the loaded version of the intel-compiler
Implementation of compiler <-> mpi version matching as discussed in #479 . The only issue I currently see with this is that it still needs some way to deal with the lmod cache. The impi version default changes when the intel module is loaded or swapped but the lmod cache doesn't reflect the change.
I've updated and reworked this, it should work correctly now, i.e., setting the default impi version to match its respective compiler, but still allow explicit combinations of any version to be loaded.
As per the discussion in https://github.com/TACC/Lmod/issues/312, this isn't really possible to do the way I had first attempted with a variable in the impi/.version file because of how the spider cache works. However, I was able to do this through the directory structure. Basically, the idea is to add a versioned folder in the moduledeps that is specific to the actual compiler version, just compiler family. This will then have version files that set the default and then just symlink to the actual modules. The resulting module layout would look like this:
First, the modules created by the intel compiler ohpc package:
# Compiler modules
modules/intel/1 # add moduledeps/versioned/intel/1 to the MODULEPATH
modules/intel/2 # add moduledeps/versioned/intel/2 to the MODULEPATH
modules/intel/3 # add moduledeps/versioned/intel/3 to the MODULEPATH
# A compiler-version specific impi .version file to set the impi default specific
# to the compiler version
moduledeps/versioned/intel/1/impi/.version #set default to "10"
moduledeps/versioned/intel/2/impi/.version #set default to "20"
moduledeps/versioned/intel/3/impi/.version #set default to "30"
And then files created by the impi ohpc package:
# gnu-compiler impi modules are nothing special
moduledeps/gnu/impi/10
moduledeps/gnu/impi/20
moduledeps/gnu/impi/30
# intel-compiler impi modules are created, but hidden in a . folder
moduledeps/intel/.impi/10
moduledeps/intel/.impi/20
moduledeps/intel/.impi/30
# intel-compiler version specific modules, really just symlinks to the actual modules
moduledeps/versioned/intel/1/impi/10 -> moduledeps/intel/.impi/10
moduledeps/versioned/intel/1/impi/20 -> moduledeps/intel/.impi/20
moduledeps/versioned/intel/1/impi/30 -> moduledeps/intel/.impi/30
moduledeps/versioned/intel/2/impi/10 -> moduledeps/intel/.impi/10
moduledeps/versioned/intel/2/impi/20 -> moduledeps/intel/.impi/20
moduledeps/versioned/intel/2/impi/30 -> moduledeps/intel/.impi/30
moduledeps/versioned/intel/3/impi/10 -> moduledeps/intel/.impi/10
moduledeps/versioned/intel/3/impi/20 -> moduledeps/intel/.impi/20
moduledeps/versioned/intel/3/impi/30 -> moduledeps/intel/.impi/30
Many thanks for this detailed explanation. I grok the issue much better now. While thinking about this, I'm also thinking about a scenario which will surely happen at some point where there is ABI incompatibility with a future version of pxse. At that point, we would presumably want to call out the resulting pxse-linked builds as a new variant name - similar to when the gnu7 variant was introduced for gcc builds
That will necessitate the need to introduce unique MODULEPATHs (e.g. something like intel/intel19) which is related to the approach you are trying to take to relate impi to a subset of compiler versions.
I'd like to propose that we leave this open for now and cogitate on a solution that can hopefully deal with your compiler/mpi matching desires and prep for a new intel variant at some point.
We'll include the updated compatibility packages in the next v1.3.2 release (trying to get that out next week) since they do a better job on determining the underlying version and alleviate the dependency on things like intel-compxe-doc which caused some headaches for a couple of users.
