kokkos
Mixed CPU-only and mixed GPU only runs
Hi,
I compiled Kokkos with CUDA support and now I want to do some special case. I have one partition on the cluster with NVIDIA GPUs and one partition with the same CPU but no GPU. So one the GPU partition, I want to use CUDA Kernels only and on the CPU-only partition, I want to use CPU kernels only.
However, all of the CPU-only nodes throw an internal Kokkos error that no CUDA device was found. Is this possible in Kokkos to run one some nodes with a GPU and on some without a GPU?
Thanks,
Patrick
How would you want to decide where to run when using the same executable? It would be far easier to create different executables/Kokkos installations. You could initialize and finalize selective backends yourself, see https://github.com/kokkos/kokkos/blob/develop/core/unit_test/TestLegionInitialization.cpp, but we wouldn't really recommend that if you don't absolutely have to.
Yes, this I what I am trying to get to run right now. I have a Kokkos + CUDA and a Kokkos executable and run them on each partition.
Yeah you can't use a CUDA enabled executable on a system without GPU (not really at least). Do you actually want them to be part of the same MPI run? There are folks who have done that. I.e. use two different executables and have them all part of the same MPI job via MPI launch configurations. If you wrote canonical Kokkos code that isn't actually all that hard since the MPI calls would still match up between the two executables.
Yes, I wanted to combine the Grace Hopper nodes with the Grace Grace nodes to have more nodes. Yes, I am working currently on the approach having exec1 which is compiled with Kokkos and Cuda and exec2 which is compiled with Kokkos without CUDA. I use the same compiler, MPI, and dependencies so this should work.