update examples, add libfabric

[j-ogas]

Addresses: #1443, possibly others.

Makes #256 moot.

[j-ogas]

 ✓ copy/ls
 ✓ copy/content of regular files
 ✓ copy/symlink targets
 ✓ distroless/hello
 ✓ exhaustive/WORKDIR
 ✓ hello/hello
 ✓ hello/distribution sanity
 ✓ lammps/inject host cray-gni ofi dso
 ✓ lammps/using all cores
 ✓ lammps/crack
 ✓ lammps/dipole
 ✓ lammps/flow
 ✓ lammps/friction
 ✓ lammps/melt
 ✓ lammps/mpi4py simple
 ✓ lammps/revert image
 - mpibench-mpich/pingpong (guest launch) (skipped: openmpi only)
 - mpibench-mpich/sendrecv (guest launch) (skipped: openmpi only)
 - mpibench-mpich/allreduce (guest launch) (skipped: openmpi only)
 ✓ mpibench-mpich/inject libgnix-fi.so provider
 ✓ mpibench-mpich/validate libgnix-fi.so provider
 - mpibench-mpich/using the high-speed network (host launch) (skipped: Cray doesn't support running on tcp)
 ✓ mpibench-mpich/pingpong (host launch)
 ✓ mpibench-mpich/sendrecv (host launch)
 ✓ mpibench-mpich/allreduce (host launch)
 ✓ mpibench-mpich/revert image
 ✓ mpibench-openmpi/pingpong (guest launch)
 ✓ mpibench-openmpi/sendrecv (guest launch)
 ✓ mpibench-openmpi/allreduce (guest launch)
 ✓ mpibench-openmpi/inject libgnix-fi.so provider
 ✓ mpibench-openmpi/validate libgnix-fi.so provider
 - mpibench-openmpi/using the high-speed network (host launch) (skipped: Cray doesn't support running on tcp)
 ✓ mpibench-openmpi/pingpong (host launch)
 ✓ mpibench-openmpi/sendrecv (host launch)
 ✓ mpibench-openmpi/allreduce (host launch)
 ✓ mpibench-openmpi/revert image
 - mpihello-mpich/guest starts ranks (skipped: openmpi only)
 ✓ mpihello-mpich/inject libgnix-fi.so provider
 ✓ mpihello-mpich/validate libgnix-fi.so provider
 - mpihello-mpich/MPI version (skipped: serial launches unsupported on Cray)
 ✓ mpihello-mpich/empty stderr
 - mpihello-mpich/serial (skipped: serial launches unsupported on Cray)
 ✓ mpihello-mpich/host starts ranks
 ✓ mpihello-mpich/Cray bind mounts
 ✓ mpihello-mpich/revert image
 ✓ mpihello-openmpi/guest starts ranks
 ✓ mpihello-openmpi/inject libgnix-fi.so provider
 ✓ mpihello-openmpi/validate libgnix-fi.so provider
 - mpihello-openmpi/MPI version (skipped: serial launches unsupported on Cray)
 ✓ mpihello-openmpi/empty stderr
 - mpihello-openmpi/serial (skipped: serial launches unsupported on Cray)
 ✓ mpihello-openmpi/host starts ranks
 ✓ mpihello-openmpi/Cray bind mounts
 ✓ mpihello-openmpi/revert image
 ✓ multistage/hello
 ✗ multistage/man hello
   (in test file /tmp/ch-test.tmp.jogas/multistage_example.bats, line 19)
     `ch-run "$ch_img" -- man hello > /dev/null' failed with status 16
   man: can't set the locale; make sure $LC_* and $LANG are correct
   No manual entry for hello
 ✓ multistage/files seem OK
 ✓ multistage/no first-stage stuff present
 ✓ obspy/hello
 ✓ obspy/hello PNG
 ✓ paraview/inject host cray-gni ofi dso
 - paraview/cone serial (skipped: serial launches unsupported on Cray)
 - paraview/cone serial PNG (skipped: serial launches unsupported on Cray)
 ✓ paraview/cone ranks=2
 ✓ paraview/cone ranks=2 PNG
 ✓ paraview/cone ranks=N
 ✓ paraview/cone ranks=N PNG
 ✓ paraview/revert image
 ✓ spack/version
 ✓ spack/compilers
 ✓ spack/find
 ✓ spark/configure
 ✓ spark/start
 ✓ spark/worker count
 ✓ spark/pi
 ✓ spark/stop
 ✓ spark/hang

[heasterday]

@j-ogas Is this relatively stable now? If so, I will work to review it soon.

[j-ogas]

@j-ogas Is this relatively stable now? If so, I will work to review it soon.

Mostly. --cray-mpi-cxi needs: 1) to be updated to work like --cray-mpi-gni, e.g., require a variable, rather than search, and 2) have the ldd parse and injection logic for libfrabric replacement fiddled with to work better on LLNL (RHEL with slingshot). That shouldn't hold up a review though, the main logic is here.

[j-ogas]

Addresses #1476.