HIPAMD builds successfully but fails to compile tests

[pvelesko]

I installed hipamd using the following CMake options. Note that I had to provide the override for ROCM_PATH since the default /opt/rocm was an old installation and I needed to use /opt/rocm-5.1.0 The use of this override was not documented in the build instructions so I had an
issue open

          cmake .. \
            -DCMAKE_BUILD_TYPE=RelWithDebInfo \
            -DHIP_PLATFORM=amd \
            -DROCM_PATH=/opt/rocm-5.1.0 \
            -DHIP_COMMON_DIR=$HIP_DIR \
            -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX

This finally worked and I was able to build hipamd. I then tried to build the tests but that failed:

[ 10%] Building HIPCC object CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o
cd /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture && /soft/buildtools/cmake/3.22.1/bin/cmake -E make_directory /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/.
cd /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture && /soft/buildtools/cmake/3.22.1/bin/cmake -D verbose:BOOL=1 -D build_configuration:STRING=RELWITHDEBINFO -D generated_file:STRING=/home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/./directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o -P /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o.cmake
-- Removing /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/./directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o
/soft/buildtools/cmake/3.22.1/bin/cmake -E remove /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/./directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o
-- Generating dependency file: /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o.depend.pre
/home/pvelesko/space/HIPAMD/hipamd/build/bin/hipcc -M /home/pvelesko/space/HIPAMD/hip/tests/src/texture/simpleTexture3D.cpp -o /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o.depend.pre --offload-arch=gfx906 --offload-arch=gfx906 --offload-arch=gfx906 --offload-arch=gfx906 -I/home/pvelesko/space/HIPAMD/hipamd/include -I/home/pvelesko/space/HIPAMD/hip/tests/src -I/home/pvelesko/space/HIPAMD/hip/tests/unit -I/home/pvelesko/space/HIPAMD/hip/tests/performance -I/home/pvelesko/space/HIPAMD/hipamd/include -I/home/pvelesko/space/HIPAMD/hip/tests/src
clang (LLVM option parsing): Unknown command line argument '-amdgpu-early-inline-all=true'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--enable-ml-inliner=true'?
clang (LLVM option parsing): Unknown command line argument '-amdgpu-function-calls=false'.  Try: 'clang (LLVM option parsing) --help'
clang (LLVM option parsing): Did you mean '--chr-function-list=false'?
CMake Error at directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o.cmake:146 (message):
  Error generating
  /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/./directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o

Verbose hipcc output:

AMD clang version 14.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-5.1.0 22114 5cba46feb6af367b1cafaa183ec42dbfb8207b14)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm/llvm/bin
Found candidate GCC installation: /usr/lib64/gcc/x86_64-suse-linux/7
Selected GCC installation: /usr/lib64/gcc/x86_64-suse-linux/7
Candidate multilib: .;@m64
Selected multilib: .;@m64
Found HIP installation: /opt/rocm, version 5.1.20531
 (in-process)
 "/opt/rocm-5.1.0/llvm/bin/clang-14" -cc1 -triple x86_64-unknown-linux-gnu -aux-triple amdgcn-amd-amdhsa -Eonly -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name simpleTexture3D.cpp -mrelocation-model static -mframe-pointer=none -fmath-errno -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/pvelesko/space/HIPAMD/hipamd/build -resource-dir /opt/rocm-5.1.0/llvm/lib/clang/14.0.0 -w -dependency-file /home/pvelesko/space/HIPAMD/hipamd/build/CMakeFiles/directed_tests.texture.simpleTexture3D.dir/__/hip/tests/src/texture/directed_tests.texture.simpleTexture3D_generated_simpleTexture3D.cpp.o.depend.pre -MT simpleTexture3D.o -sys-header-deps -internal-isystem /opt/rocm-5.1.0/llvm/lib/clang/14.0.0/include/cuda_wrappers -internal-isystem /opt/rocm/include -include __clang_hip_runtime_wrapper.h -isystem /opt/rocm-5.1.0/llvm/lib/clang/14.0.0/include/.. -isystem /opt/rocm/hsa/include -isystem /gpfs/jlse-fs0/users/pvelesko/HIPAMD/hipamd/build/include -I /home/pvelesko/space/HIPAMD/hipamd/include -I /home/pvelesko/space/HIPAMD/hip/tests/src -I /home/pvelesko/space/HIPAMD/hip/tests/unit -I /home/pvelesko/space/HIPAMD/hip/tests/performance -I /home/pvelesko/space/HIPAMD/hipamd/include -I /home/pvelesko/space/HIPAMD/hip/tests/src -I/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/include -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7 -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7/x86_64-suse-linux -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7/backward -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7 -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7/x86_64-suse-linux -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../include/c++/7/backward -internal-isystem /opt/rocm-5.1.0/llvm/lib/clang/14.0.0/include -internal-isystem /usr/local/include -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -internal-isystem /opt/rocm-5.1.0/llvm/lib/clang/14.0.0/include -internal-isystem /usr/local/include -internal-isystem /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -O3 -std=c++11 -fdeprecated-macro -fdebug-compilation-dir=/home/pvelesko/space/HIPAMD/hipamd/build -ferror-limit 19 -fhip-new-launch-api -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -vectorize-loops -vectorize-slp -mllvm -amdgpu-early-inline-all=true -mllvm -amdgpu-function-calls=false -fcuda-allow-variadic-functions -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x hip /home/pvelesko/space/HIPAMD/hip/tests/src/texture/simpleTexture3D.cpp
clang -cc1 version 14.0.0 based upon LLVM 14.0.0git default target x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"

It's a bit concerning that I see multiple references to /opt/rocm since that directory didn't work for me during CMake.

Again, looking at hipconfig I see multiple references to /opt/rocm even though this should have been inherited from CMake as /opt/rocm-5.1.0 and placed in .hipvars.pm

hipconfig --full
HIP version  : 5.2.21152-4b155a06

== hipconfig
HIP_PATH     : /gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd
ROCM_PATH    : /opt/rocm
HIP_COMPILER : clang
HIP_PLATFORM : amd
HIP_RUNTIME  : rocclr
CPP_CONFIG   :  -D__HIP_PLATFORM_HCC__= -D__HIP_PLATFORM_AMD__= -I/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/include -I/opt/rocm/llvm/bin/../lib/clang/14.0.0 -I/opt/rocm/hsa/include

== hip-clang
HSA_PATH         : /opt/rocm/hsa
HIP_CLANG_PATH   : /opt/rocm/llvm/bin
AMD clang version 14.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-5.1.0 22114 5cba46feb6af367b1cafaa183ec42dbfb8207b14)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm/llvm/bin
AMD LLVM version 14.0.0git
  Optimized build.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: znver2

  Registered Targets:
    amdgcn - AMD GCN GPUs
    r600   - AMD GPUs HD2XXX-HD6XXX
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64
hip-clang-cxxflags :  -std=c++11 -isystem "/opt/rocm-5.1.0/llvm/lib/clang/14.0.0/include/.." -isystem /opt/rocm/hsa/include -isystem "/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/include" -O3
hip-clang-ldflags  :  -L"/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/lib" -O3 -lgcc_s -lgcc -lpthread -lm -lrt

=== Environment Variables
PATH=/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/bin:/home/pvelesko/.pyenv/shims:/home/pvelesko/.pyenv/bin:/home/pvelesko/.pyenv/shims:/home/pvelesko/.pyenv/bin:/home/pvelesko/install/gdb-11.2/bin:/soft/packaging/spack-builds/linux-opensuse_leap15-x86_64/gcc-10.2.0/cmake-3.23.2-luxkr43atenuvdfpcpiwuuqy234kvtil/bin:/home/pvelesko/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/lpp/mmfs/bin:/opt/rocm/bin:/opt/rocm/profiler/bin:/opt/rocm/opencl/bin:/home/pvelesko/install/clang-tools/bin:/home/pvelesko/install/clang-tools/bin:/home/pvelesko/.local/bin:/home/pvelesko/bin
LD_LIBRARY_PATH=/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/lib64:/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd/lib
HIP_DIR=/gpfs/jlse-fs0/users/pvelesko/install/HIP/clang14/hipamd

== Linux Kernel
Hostname     : amdgpu00
Linux amdgpu00 5.3.18-150300.59.54-default #1 SMP Sat Mar 5 10:00:50 UTC 2022 (1d0fa95) x86_64 x86_64 x86_64 GNU/Linux
LSB Version:	n/a
Distributor ID:	openSUSE
Description:	openSUSE Leap 15.3
Release:	15.3
Codename:	n/a

[pvelesko]

I got the tests to compile:

make clean
export ROCM_PATH=/opt/rocm-5.1.0
make -j
make build_tests -j

Seems like the documentation is lacking a bit in this regard.

[abhimeda]

@pvelesko Hi, could we please close this ticket?