Switch to use find_package(MPI) for MPI compilers and runtime

[bartlettroscoe]

Parents

• #411

Description

This story is to change TriBITS, and all TriBITS projects to use find_package(MPI) (which calls FindMPI.cmake) for finding MPI-compatible compilers (or raw compilers with include dirs and libraries) and the MPI multi-process runtime (e.g. mpiexec). This also defines how mpiexec (or whatever its name is) gets called for adjust for different MPI implementations. (FindMPI.cmake has to do a lot.)

The motivation for doing this is that find_package(MPI) is how modern CMake projects are supposed to use MPI and we want to transition TriBITS to modern CMake (see #411).

Currently, TriBITS uses its own implementation without calling find_package(MPI). This uses different var names and some other incompatibilities. TriBITS assumes the MPI compiler wrappers will be used by find_package(MPI) strips our the raw underlying compilers and then the MPI include dirs, libraries, defines, etc. is put into modern targets MPI::MPI_<lang> like MPI::MPI_C, MPI::MPI_CXX, and MPI::MPI_Fortran. This allows you to have non-MPI binaries and MPI binaries in the same CMake project. Current TriBITS does not cleanly support that.

Also, how mpiexec gets called is a little different an uses different var names. As described at:

• https://tribits.org/doc/TribitsBuildReference.html#configuring-with-mpi-support

The current TriBITS implementation defines MPI tests by calling:

add_test(NAME <testName> COMMAND
  ${MPI_EXEC} ${MPI_EXEC_PRE_NUMPROCS_FLAGS}
  ${MPI_EXEC_NUMPROCS_FLAG} <NP>
  ${MPI_EXEC_POST_NUMPROCS_FLAGS}
  <TEST_EXECUTABLE_PATH> <TEST_ARGS> )

while changing to find_package(MPI) described at:

• https://cmake.org/cmake/help/v3.23/module/FindMPI.html#variables-for-using-mpi

would require this to be updated to:

add_test(NAME <testName> COMMAND
  ${MPIEXEC_EXECUTABLE} ${MPIEXEC_NUMPROC_FLAG} <NP> 
  ${MPIEXEC_PREFLAGS} <TEST_EXECUTABLE_PATH> ${MPIEXEC_POSTFLAGS} <TEST_ARGS>

There are several incompatibilities here that would impact TriBITS itself, TriBITS projects, and TriBITS project users (i.e. who configure TriBITS projects):

• FindMPI.cmake is missing the varaible MPI_EXEC_PRE_NUMPROCS_FLAGS (which would be called something like MPIEXEC_PRE_NUMPROCS_FLAGS if it had it). (That variable was added to TriBITS years ago because there are (or were) systems where some <mpiexec> flags have to be passed before the <numprocs> flag.)
• FindMPI.cmake defines MPIEXEC_POSTFLAGS which is not defined by TriBITS.

Therefore, changing over to find_package(MPI) will involve (at least) the following changes:

• Update Probe and set up the environment in [Full TriBITS Project Configuration](https://tribitspub.github.io/TriBITS/users_guide/index.html#full-tribits-project-configuration) to call find_package(MPI), use that to define the compilers, and find the MPI multi-process runtime (e.g. mpiexec)
• Update tribits/std_tpls/FindTPLMPI.cmake to call find_package(MPI) (which will be a no-op since it has already been found but that is fine).
• Update TriBITS testing support functions tribits_add_test() and tribits_add_advanced_test() to use new variable names.
• Submit PR to CMake repo to add support for MPIEXEC_PRE_NUMPROCS_FLAGS to FindMPI.cmake.

NOTE: A prototype for some of the changes was pushed to Trilinos in the PRs:

• trilinos/Trilinos#13840
• trilinos/Trilinos#13850

See:

• trilinos/Trilinos#13839

Background

When Trilinos first adopted CMake in 2008, the find_package(MPI) module did not work on some platforms. And there were was at least one platform where it was not possible to grab the raw compilers out of the MPI compiler wrappers, so Trilinos could not use the FindMPI.cmake module at that time. But 16+ years have gone by now and likely all of those issues have likely been worked out now given the large increase in the usage of CMake by HPC projects.

Tasks:

• [ ] ???