Warnings when compiling on Jedi

[JoMee]

Issue

On JEDI, when compiling for cuda, the current master branch creates a lot of IPPL-related warnings. This should probably be looked at more closely.

Setup

The setup for compiling is the standard login-node on JEDI. The modules loaded are:

module load Stages/2025  GCC
module load CMake
module load NCCL
module load OpenMPI

Then the CMake command is the one from the README for the HOPPER90 architecture:

cmake .. \
    -DCMAKE_BUILD_TYPE=Release \
    -DKokkos_ARCH_HOPPER90=ON \
    -DCMAKE_CXX_STANDARD=20 \
    -DIPPL_ENABLE_FFT=ON \
    -DIPPL_ENABLE_TESTS=ON \
    -DUSE_ALTERNATIVE_VARIANT=ON \
    -DIPPL_ENABLE_SOLVERS=ON \
    -DIPPL_ENABLE_ALPINE=True \
    -DIPPL_PLATFORMS=cuda

Warnings

For the test/ directory we have

Then a lot of warnings of the following form appear:

warning #20011-D: calling a host function("void ::toss_cookies<int, int> (const char *, const char *, const char *, T1, T2, const char *, int)") from a host device function("ippl::lcm") is not allowed

Some warnings of the following form as well:

warning #611-D: overloaded virtual function "heffte::executor_base::forward" is only partially overridden in class "heffte::real2real_executor<heffte::backend::cufft, heffte::cuda::cos_pre_pos_processor>" struct real2real_executor : public executor_base{ ^ detected during: instantiation of class "heffte::real2real_executor<fft_backend_tag, prepost_processor> [with fft_backend_tag=heffte::backend::cufft, prepost_processor=heffte::cuda::cos_pre_pos_processor]" at line 696 of /p/project1/jureap24/meier5/ippl/build/_deps/heffte-src/include/heffte_common.h instantiation of "std::unique_ptr<heffte::one_dim_backend<backend_tag>::executor, std::default_delete<heffte::one_dim_backend<backend_tag>::executor>> heffte::make_executor<backend_tag,index>(heffte::backend::device_instance<heffte::backend::buffer_traits<backend_tag, nullptr>::location>::stream_type, heffte::box3d) [with backend_tag=heffte::backend::cufft_cos, index=long long]" at line 608 of /p/project1/jureap24/meier5/ippl/build/_deps/heffte-src/include/heffte_fft3d.h instantiation of "void heffte::fft3d<backend_tag, index>::setup(const heffte::logic_plan3d &, MPI_Comm) [with backend_tag=heffte::backend::cufft_cos, index=long long]" at line 582 of /p/project1/jureap24/meier5/ippl/build/_deps/heffte-src/include/heffte_fft3d.h

alpine

With alpine we get both of the above warnings as well.

[aaadelmann]

::toss_cookies

Here we have to distinguish host/device at compile time. Unfortunately Kokkos as far as I know has no compiletime macro avaidable. So I propose in Ippl.h the following

#if defined(__CUDA_ARCH__) || defined(__HIP_DEVICE_COMPILE__) || defined(__SYCL_DEVICE_ONLY__)
  #define IS_DEVICE_CODE 1
#else
  #define IS_DEVICE_CODE 0
#endif

Then we can rewrite https://github.com/IPPL-framework/ippl/blob/c95e65ab00718238c4ded68b9f87d1ebe85a207a/src/Utility/PAssert.cpp#L73

void toss_cookies(const char* cond, const char* file, int line) {
#ifdef   IS_DEVICE_CODE
  return;
#elseif   
    std::string what = "Assertion '" + std::string(cond) + "' failed. \n";
    what += "in \n";
    what += std::string(file) + ", line  " + std::to_string(line);
    throw std::runtime_error(what);
#endif
}

So that will eliminate the warnings. How to deal with the runtime error on the device I do not know yet.