Add Polaris to GNUMake

[jmsexton03]

Summary

This adds Polaris as an ALCF machine by copying most of the perlmutter section of Make.nersc to Make.alcf .

Additional background

This was tested using a similar strategy to ALCF Getting Started - gpu affinity example with amrex/Tests/GPU/Vector on 1 node of Polaris

module swap PrgEnv-nvhpc PrgEnv-gnu
module load nvhpc-mixed
make USE_CUDA=TRUE USE_MPI=TRUE NVCC_CCBIN=CC
NRANKS_PER_NODE=4
NDEPTH=4
NNODES=1
NTOTRANKS=$(( NNODES * NRANKS_PER_NODE ))
mpiexec -n ${NTOTRANKS} --ppn ${NRANKS_PER_NODE} --depth=${NDEPTH} --cpu-bind depth ./set_affinity_gpu_polaris.sh ./main3d.gnu.MPI.CUDA.ex inputs

and make USE_CUDA=TRUE USE_MPI=FALSE NVCC_CCBIN=CC

Checklist

The proposed changes:

• [ ] fix a bug or incorrect behavior in AMReX
• [ ] add new capabilities to AMReX
• [ ] changes answers in the test suite to more than roundoff level
• [ ] are likely to significantly affect the results of downstream AMReX users
• [ ] include documentation in the code and/or rst files, if appropriate

[zingale]

this works great for me on Polaris