[HIP] hipcc cannot compile .cu files on AMD machines(feature request).

[Kaveh01]

Regardless of HIP's ability to compile and on Nvidia machines with a negligible performance penalty on Nvidia machines, some developers prefer to keep both CUDA and HIP code in the same .cu file (with conditional compilation). Unfortunately, currently this is impossible because hipcc cannot compile .cu files on AMD machines.

[dmcdougall]

Do you have a small example that demonstrates the problem?

The file extension should not be important.

[emankov]

Neither hipcc, nor hip-clang is intended to compile CUDA. You may translate CUDA to HIP by hipify-clang and then compile the generated HIP code. Source CUDA code may contain (guarded) HIP code. For a correct hipification corresponding includes to HIP should be presented in such source as long as to CUDA. You may find such examples in https://github.com/pytorch/pytorch.git, for instance.

[Kaveh01]

Do you have a small example that demonstrates the problem?

The file extension should not be important.

@dmcdougall Please see the attached file. I put the error log file there. You can also go to HIP/samples/0_Intro/square folder rename square.hipref.cpp to square.hipref.cu and try to compile that.

Neither hipcc, nor hip-clang is intended to compile CUDA. You may translate CUDA to HIP by hipify-clang and then compile the generated HIP code.

@emankov I don't want to compile CUDA codes with hipcc or hip-clang. My collaborators wish to have both CUDA and HIP codes in the same .cu source file and compile CUDA code (only on Nvidia) with nvcc and HIP code with hipcc (please see the attached example). At the moment, I can do that without any problem on the Nvidia machine. But on AMD I get errors (see LOG.txt).

ex.tar.gz

[emankov]

@Kaveh01 Did you try hip-clang? Compiling hip files with *.cu extension is possible under -x hip option. It is the same clang's trick as compiling *.cpp under -x cuda.

[emankov]

My collaborators wish to have both CUDA and HIP codes in the same .cu source file ...

That is exactly what I wrote above: "Source CUDA code may contain (guarded) HIP code", and actually, guarded CUDA code like in your example.

[Kaveh01]

@Kaveh01 Did you try hip-clang? Compiling hip files with *.cu extension is possible under -x hip option. It is the same clang's trick as compiling *.cpp under -x cuda.

@emankov Thanks for the replay. I did not know the option. I just tried and I got these errors:

hipcc -x hip -o scale_vector_hipcc -DHIPBE scale_vector.cu
clang-10: error: no such file or directory: 'hip.amdgcn.bc'
clang-10: error: no such file or directory: 'opencl.amdgcn.bc'
clang-10: error: no such file or directory: 'ocml.amdgcn.bc'
clang-10: error: no such file or directory: 'ockl.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_finite_only_off.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_daz_opt_off.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_correctly_rounded_sqrt_on.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_unsafe_math_off.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_isa_version_20.amdgcn.bc'
clang-10: error: no such file or directory: 'oclc_wavefrontsize64_on.amdgcn.bc'
make: *** [Makefile:11: scale_vector_hipcc] Error 1

It look like something is wrong with my clang instalation.

[emankov]

Hi @Kaveh01

1. Ensure that AMDGPU is presented in LLVM_TARGETS_TO_BUILD in your LLVM's cmake command line.
2. Here is a guide for building missing AMDGCN bytecodes: https://github.com/RadeonOpenCompute/ROCm-Device-Libs

[jvermaas]

Alright, super dumb question, but how do I get the compiler to recognize existing versions of this set of files? On a standard rocm installation, these files exist in /opt/rocm/lib, and so it would be natural to assume that hipcc can find them and use them. Is there a way of pointing hipcc to these files?

[emankov]

--hip-device-lib-path=$(path_to_bitcode_libs)

[baryluk]

I have same issue:

root@debian:/opt/rocm-3.7.0/hip/samples/0_Intro/square# /opt/rocm-3.7.0/hip/bin/hipcc -v square.cpp -o square.out
clang version 11.0.0 (/src/external/llvm-project/clang ee4e4ebbadcc8ea14ce99e34ed31ab31e94827ac)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm/llvm/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/9
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/10
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
clang-11: error: no such file or directory: 'hip.amdgcn.bc'
clang-11: error: no such file or directory: 'ocml.amdgcn.bc'
clang-11: error: no such file or directory: 'ockl.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_finite_only_off.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_daz_opt_on.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_correctly_rounded_sqrt_on.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_unsafe_math_off.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_isa_version_803.amdgcn.bc'
clang-11: error: no such file or directory: 'oclc_wavefrontsize64_on.amdgcn.bc'

But it looks like installing rocm-device-libs3.7.0 (or just rocm-dev3.7.0 which depends on it) did bring it on.

[ppanchad-amd]

@Kaveh01 Do you still need assistance with this ticket? Thanks!

[yiakwy-xpu-ml-framework-team]

Hi @ppanchad-amd , I am not sure this is right place to disscuss. I have a related question, suppose I have handled a .cu file well with USE_ROCM control to add hip api codes, and enabling HIP language in original cmake with CMAKE_HIP_ARCHITECTURE detected automatically.

How could I make the minimum effort to make a previously defined target (compiled internally by CMAKE_HIP_COMPILER, clang++ or hipcc), to compile with .cu sources ?

Since I found if I rename the .cu to .hip , I can compile successfully, but if I don't , the binary doesn't contain any symbols.

[mangupta]

@yiakwy-xpu-ml-framework-team : cmake determines the language based on source file extensions. It by default recognizes .hip files as HIP language, .cpp files as C++ language, etc. If you want to override this you need to use set_source_files_properties on the source files. Something similar to set_source_files_properties(${SOURCES} PROPERTIES HIP_SOURCE_PROPERTY_FORMAT 1).