Potential issue when using PARMETIS column permutation in v8.2.1

[v-dobrev]

I noticed this during testing of mfem-4.7 with SuperLU_DIST v8.2.1 -- one of the mfem examples that uses SuperLU_DIST, examples/ex11p.cpp hangs when run with:

mpirun -np 4 ./ex11p --superlu -no-vis

This seems to happen in both CPU and GPU builds.

If the option superlu::PARMETIS used in ex11p.cpp is changed to superlu::METIS_AT_PLUS_A, the issue goes away.

Also, when using SuperLU_DIST v8.1.2, there is no issue. I have not tried v9.0.0.

The easiest way to reproduce this is probably using Spack -- basically build mfem with superlu-dist and enable the mfem build testing with --test=root:

./bin/spack install --fresh --test=root mfem+superlu-dist ^[email protected]

This install should stall when running ex11p with the superlu-dist option -- just cancel the Spack install at that point with ctrl+c. After the failure, one can go inside the failed build directory and run the failing example (using the mpirun command above) in mfem's examples directory.