RAJA
RAJA copied to clipboard
Published
20 hours ago •
LLNL
LLNL
why won't CAMP set ENABLE_TARGET_OPENMP?
I must be stupid, because I set -DENABLE_TARGET_OPENMP=On or -DENABLE_TARGET_OPENMP=1 and CAMP does not set it. I'd like some sort of diagnostic here, rather than just having the build fail and having to print-debug the preprocessor.
jehammond@dgx-a100-math:/local/jehammond/RAJA/raja/build$ git clean -dfx ; cmake .. -DCMAKE_C_COMPILER=nvc -DCMAKE_CXX_COMPILER=nvc++ -DCMAKE_CXX_FLAGS="-mp -target=gpu" -DENABLE_TARGET_OPENMP=On && make
warning: failed to remove ./: Invalid argument
Removing ./cmake_install.cmake
Removing ./RAJA.pc
Removing ./tests
Removing ./test
Removing ./CMakeFiles
Removing ./CTestTestfile.cmake
Removing ./tpl
Removing ./lib
Removing ./share
Removing ./Testing
Removing ./exercises
Removing ./bin
Removing ./include
Removing ./DartConfiguration.tcl
Removing ./CMakeCache.txt
Removing ./Makefile
Removing ./examples
Removing ./compile_commands.json
Removing ./blt
-- The CXX compiler identification is NVHPC 21.9.0
-- The C compiler identification is NVHPC 21.9.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/nvidia/hpc_sdk/Linux_x86_64/21.9/compilers/bin/nvc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test COMPILER_SUPPORTS_c++17
-- Performing Test COMPILER_SUPPORTS_c++17 - Success
Using C++ standard: c++17
-- BLT Version: 0.4.1
-- CMake Version: 3.21.1
-- CMake Executable: /usr/local/bin/cmake
CMake Deprecation Warning at blt/SetupBLT.cmake:73 (cmake_policy):
The OLD behavior for policy CMP0076 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
CMakeLists.txt:167 (include)
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Git Support is ON
-- Git Executable: /usr/bin/git
-- Git Version: 2.25.1
-- MPI Support is OFF
-- OpenMP Support is On
-- Found OpenMP_C: -mp
-- Found OpenMP_CXX: -mp
-- Found OpenMP: TRUE
-- OpenMP Compile Flags: -mp
-- OpenMP Link Flags: -mp
-- CUDA Support is Off
-- HIP Support is OFF
-- HCC Support is OFF
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Sphinx support is ON
-- Failed to locate Sphinx executable (missing: SPHINX_EXECUTABLE)
-- Valgrind support is ON
-- Failed to locate Valgrind executable (missing: VALGRIND_EXECUTABLE)
-- AStyle support is ON
-- Failed to locate AStyle executable (missing: ASTYLE_EXECUTABLE)
-- ClangFormat support is ON
-- Failed to locate ClangFormat executable (missing: CLANGFORMAT_EXECUTABLE)
-- Uncrustify support is ON
-- Failed to locate Uncrustify executable (missing: UNCRUSTIFY_EXECUTABLE)
-- Yapf support is ON
-- Failed to locate Yapf executable (missing: YAPF_EXECUTABLE)
-- CMakeFormat support is ON
-- Failed to locate CMakeFormat executable (missing: CMAKEFORMAT_EXECUTABLE)
-- Cppcheck support is ON
-- Failed to locate Cppcheck executable (missing: CPPCHECK_EXECUTABLE)
-- ClangQuery support is ON
-- Failed to locate ClangQuery executable (missing: CLANGQUERY_EXECUTABLE)
-- ClangTidy support is ON
-- Failed to locate ClangTidy executable (missing: CLANGTIDY_EXECUTABLE)
-- C Compiler family not set!!!
-- Adding optional BLT definitions and compiler flags
-- Standard C++17 selected
-- Enabling all compiler warnings on all targets.
-- Fortran support disabled.
-- CMAKE_C_FLAGS flags are: -Wall -Wextra
-- CMAKE_CXX_FLAGS flags are: -mp -target=gpu -Wall -Wextra
-- CMAKE_EXE_LINKER_FLAGS flags are:
-- Google Test Support is ON
-- Google Mock Support is OFF
CMake Deprecation Warning at blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/CMakeLists.txt:56 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
-- Found PythonInterp: /usr/bin/python (found version "3.8.5")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- OpenMP Enabled
-- Desul Atomics support is OFF
-- Using RAJA CAMP submodule.
-- Looking for posix_memalign
-- Looking for posix_memalign - found
-- Looking for std::aligned_alloc
-- Looking for std::aligned_alloc - not found
-- Looking for _mm_malloc
-- Looking for _mm_malloc - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /local/jehammond/RAJA/raja/build
Scanning dependencies of target RAJA
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/AlignedRangeIndexSetBuilders.cpp.o
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/defines.hpp", line 117: warning: #warning directive: NO ENABLE_TARGET_OPENMP
#warning NO ENABLE_TARGET_OPENMP
^
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/resource.hpp", line 41: warning: #warning directive: WTF
#warning WTF
^
"/local/jehammond/RAJA/raja/include/RAJA/util/macros.hpp", line 133: warning: variable "errtemp" was set but never used
const char * errtemp = nullptr;
^
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/DepGraphNode.cpp.o
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/defines.hpp", line 117: warning: #warning directive: NO ENABLE_TARGET_OPENMP
#warning NO ENABLE_TARGET_OPENMP
^
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/LockFreeIndexSetBuilders.cpp.o
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/defines.hpp", line 117: warning: #warning directive: NO ENABLE_TARGET_OPENMP
#warning NO ENABLE_TARGET_OPENMP
^
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/resource.hpp", line 41: warning: #warning directive: WTF
#warning WTF
^
"/local/jehammond/RAJA/raja/include/RAJA/util/macros.hpp", line 133: warning: variable "errtemp" was set but never used
const char * errtemp = nullptr;
^
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/MemUtils_CUDA.cpp.o
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/MemUtils_HIP.cpp.o
[ 0%] Building CXX object CMakeFiles/RAJA.dir/src/MemUtils_SYCL.cpp.o
[ 1%] Building CXX object CMakeFiles/RAJA.dir/src/PluginStrategy.cpp.o
"/local/jehammond/RAJA/raja/include/RAJA/util/macros.hpp", line 133: warning: variable "errtemp" was set but never used
const char * errtemp = nullptr;
^
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/defines.hpp", line 117: warning: #warning directive: NO ENABLE_TARGET_OPENMP
#warning NO ENABLE_TARGET_OPENMP
^
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/resource.hpp", line 41: warning: #warning directive: WTF
#warning WTF
^
[ 1%] Linking CXX static library lib/libRAJA.a
[ 1%] Built target RAJA
Scanning dependencies of target gtest
[ 1%] Building CXX object blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
[ 1%] Linking CXX static library ../../../../lib/libgtest.a
[ 1%] Built target gtest
Scanning dependencies of target gtest_main
[ 1%] Building CXX object blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/CMakeFiles/gtest_main.dir/src/gtest_main.cc.o
[ 1%] Linking CXX static library ../../../../lib/libgtest_main.a
[ 1%] Built target gtest_main
Scanning dependencies of target blt_gtest_smoke
[ 2%] Building CXX object blt/tests/smoke/CMakeFiles/blt_gtest_smoke.dir/blt_gtest_smoke.cpp.o
[ 2%] Linking CXX executable ../../../tests/blt_gtest_smoke
[ 2%] Built target blt_gtest_smoke
Scanning dependencies of target blt_openmp_smoke
[ 2%] Building CXX object blt/tests/smoke/CMakeFiles/blt_openmp_smoke.dir/blt_openmp_smoke.cpp.o
[ 2%] Linking CXX executable ../../../tests/blt_openmp_smoke
[ 2%] Built target blt_openmp_smoke
Scanning dependencies of target test-plugin-forall-OpenMP.exe
[ 2%] Building CXX object test/integration/plugin/CMakeFiles/test-plugin-forall-OpenMP.exe.dir/test-plugin-forall-OpenMP.cpp.o
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/defines.hpp", line 117: warning: #warning directive: NO ENABLE_TARGET_OPENMP
#warning NO ENABLE_TARGET_OPENMP
^
"/local/jehammond/RAJA/raja/include/RAJA/util/macros.hpp", line 133: warning: variable "errtemp" was set but never used
const char * errtemp = nullptr;
^
"/local/jehammond/RAJA/raja/tpl/camp/include/camp/resource.hpp", line 41: warning: #warning directive: WTF
#warning WTF
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 114: error: namespace "camp::resources" has no member "Omp"
using type = camp::resources::Omp;
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 119: error: namespace "camp::resources" has no member "Omp"
using type = camp::resources::Omp;
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 124: error: namespace "camp::resources" has no member "Omp"
using type = camp::resources::Omp;
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 129: error: namespace "camp::resources" has no member "Omp"
using type = camp::resources::Omp;
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 134: error: namespace "camp::resources" has no member "Omp"
using type = camp::resources::Omp;
^
"/local/jehammond/RAJA/raja/include/RAJA/util/resource.hpp", line 154: error: namespace "RAJA::resources" has no member "Omp"
template <> struct is_resource<resources::Omp> : std::true_type {};
^
"/local/jehammond/RAJA/raja/include/RAJA/policy/openmp_target/forall.hpp", line 35: error: namespace "RAJA::resources" has no member "Omp"
RAJA_INLINE resources::EventProxy<resources::Omp> forall_impl(resources::Omp omp_res,
^
"/local/jehammond/RAJA/raja/include/RAJA/policy/openmp_target/forall.hpp", line 35: error: namespace "RAJA::resources" has no member "Omp"
RAJA_INLINE resources::EventProxy<resources::Omp> forall_impl(resources::Omp omp_res,
^
"/local/jehammond/RAJA/raja/include/RAJA/policy/openmp_target/forall.hpp", line 75: error: namespace "RAJA::resources" has no member "Omp"
RAJA_INLINE resources::EventProxy<resources::Omp> forall_impl(resources::Omp omp_res,
^
"/local/jehammond/RAJA/raja/include/RAJA/policy/openmp_target/forall.hpp", line 75: error: namespace "RAJA::resources" has no member "Omp"
RAJA_INLINE resources::EventProxy<resources::Omp> forall_impl(resources::Omp omp_res,
^
"/local/jehammond/RAJA/raja/test/include/RAJA_test-camp.hpp", line 38: error: namespace "camp::resources" has no member "Omp"
using OpenMPTargetResourceList = camp::list<camp::resources::Omp>;
^
11 errors detected in the compilation of "/local/jehammond/RAJA/raja/build/test/integration/plugin/test-plugin-forall-OpenMP.cpp".
make[2]: *** [test/integration/plugin/CMakeFiles/test-plugin-forall-OpenMP.exe.dir/build.make:75: test/integration/plugin/CMakeFiles/test-plugin-forall-OpenMP.exe.dir/test-plugin-forall-OpenMP.cpp.o] Error 2
make[1]: *** [CMakeFiles/Makefile2:3193: test/integration/plugin/CMakeFiles/test-plugin-forall-OpenMP.exe.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
my pathetic attempts to debug this
jehammond@dgx-a100-math:/local/jehammond/RAJA/raja/build$ git diff --submodule=diff
Submodule tpl/camp contains modified content
diff --git a/tpl/camp/include/camp/defines.hpp b/tpl/camp/include/camp/defines.hpp
index 502107b..4ccf3e3 100644
--- a/tpl/camp/include/camp/defines.hpp
+++ b/tpl/camp/include/camp/defines.hpp
@@ -107,11 +107,14 @@ namespace camp
#if defined(ENABLE_TARGET_OPENMP)
#if _OPENMP >= 201511
+#warning HELLO
#define CAMP_HAVE_OMP_OFFLOAD 1
#else
-#define CAMP_HAVE_OMP_OFFLOAD 0
-#warning Compiler does NOT support OpenMP Target Offload even though user has enabled it!
+#error Compiler does NOT support OpenMP Target Offload even though user has enabled it!
+//#define CAMP_HAVE_OMP_OFFLOAD 0
#endif
+#else
+#warning NO ENABLE_TARGET_OPENMP
#endif
// This works for:
diff --git a/tpl/camp/include/camp/resource.hpp b/tpl/camp/include/camp/resource.hpp
index e9dfdc1..8ad9977 100644
--- a/tpl/camp/include/camp/resource.hpp
+++ b/tpl/camp/include/camp/resource.hpp
@@ -31,7 +31,14 @@ http://github.com/llnl/camp
#endif
#if defined(CAMP_HAVE_OMP_OFFLOAD)
+#if CAMP_HAVE_OMP_OFFLOAD
+#warning TRUE
+#else
+#warning FALSE
+#endif
#include "camp/resource/omp_target.hpp"
+#else
+#warning WTF
#endif
Hey @jeffhammond - setting -D on your Cmake line will only set that as a CMake option. Camp is looking for a compiler macro define. Can you try adding -DENABLE_TARGET_OPENMP to CMAKE_CXX_FLAGS?
I just hard-coded it in the header, which is equivalent to the flags suggestion, and that works, but I wonder how RAJA OpenMP Target ever worked, unless everyone who uses it knows that CMake doesn't propagate this token.
@jeffhammond actually, the RAJA OpenMP Target back-end is mostly for cussing at compilers and amusement purposes. Please no wagering.... :smile:
Just seeing this thread now, sorry about this . . . I added that check to prevent OpenMP Target from being built on machines without GPUs. We did not have the #if defined(ENABLE_TARGET_OPENMP) check before, and CAMP would attempt to build omp-target if the compiler was capable.
Hi @jeffhammond, does the latest camp/main (https://github.com/LLNL/camp) solve this problem for you? We put some recent changes into the Camp repo which hopefully have taken care of this, and you're welcome to try it out.