justxi
PRs to Gentoo
Can you please submit PRs to Gentoo? (I'm a Gentoo developer and will be happy to help you get them accepted) See https://wiki.gentoo.org/wiki/GitHub_Pull_Requests for reference.
I suggest you start with the OpenCL runtime. ROCT-Thunk-Interface, ROCR-Runtime, and ROCm-OpenCL-Runtime seem to be the base set. You'll need to change the names (as they don't comply with Gentoo standards right now) as well as some other things, please take a gander at https://devmanual.gentoo.org/ebuild-writing/file-format/index.html as a starting point. I think one PR adding all 3 packages (one commit per package - you're not allowed to modify multiple packages in one commit) would be great. I suggest adding one "final" version (ex "1.0") and one live ("9999") version per package (which you're already doing here in this overlay, so keep that up!).
I look forward to working with you and getting all of this into Gentoo for everyone to easily use! Thanks!
Hi, I´m also interested to get this up. The problem with final versions was in the past, that there are not always packaged versions of all components. We have to check this. A solution could be to create archives and upload them to a gentoo server(?). There are also some problems which should be solved, which I have documented in https://bugs.gentoo.org/650804.
I wil read your suggestions in the coming days.
Due to the fact, that ROCm 2.0 shall be release by end of year (https://github.com/RadeonOpenCompute/ROCm/issues/609) we could focus on this version as a first release to the gentoo tree.
Due to the fact, that ROCm 2.0 shall be release by end of year (RadeonOpenCompute/ROCm#609) we could focus on this version as a first release to the gentoo tree.
That sounds good to me - you could start by creating the PR now with 9999 versions only so we can address ebuild format concerns (naming, style, etc).
That's great news, looking forward! I can help out with beta-testing if needed.
Due to the fact, that ROCm 2.0 shall be release by end of year (RadeonOpenCompute/ROCm#609) we could focus on this version as a first release to the gentoo tree.
That sounds good to me - you could start by creating the PR now with 9999 versions only so we can address ebuild format concerns (naming, style, etc).
It seems AMD is rolling out version 2.0. So as suggested I am starting with the three base ebuilds. To get the naming etc. right, I will start with ROCT-Thunk-Interface, which I have to rename to ROCTThunkInterface-9999.ebuild, right? For this package there is also a file release. So it should be easy to create a ROCTThunkInterface-2.0.ebuild.
Are the category names ok?
I will start with ROCT-Thunk-Interface, which I have to rename to ROCTThunkInterface-9999.ebuild, right?
Quoting from https://devmanual.gentoo.org/ebuild-writing/file-format/index.html
The name section should contain only lowercase non-accented letters, the digits 0-9, hyphens, underscores and plus characters. Uppercase characters are strongly discouraged, but technically valid.
Regarding category, I'm not sure if media-libs or dev-libs is better... The proprietary amd OpenCL driver is at dev-libs/amdgpu-pro-opencl, so that makes me think that dev-libs is probably better?
Intel and AMD sdk´s are in dev-util, but ocl-icd is also in dev-libs... (https://wiki.gentoo.org/wiki/OpenCL)
So I will move the ebuilds to dev-libs :).
I created two ebuilds for ROCTThunkInterface -> https://github.com/justxi/rocm/tree/master/dev-libs/ROCTThunkInterface.
Currently the problem is that the git-version (2.0.9999) creates a library with version 2.0.0. The ebuild which downloads the source package (tar.gz) creates a library with version 1.0.0. I think I should clarify this before creating a PR.
It seems that the version string is taken from the git repository otherwise a default string is used. I will submit a issue to the project (https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/issues/26)
In the meantime I could create a patch to correct this. What do you mean?
I created a patch to change the version (https://github.com/justxi/rocm/tree/master/dev-libs/ROCTThunkInterface). I have to configure the repository as suggested and then I will submit my first PR.
Looks good to me. It´s always amazing what further improvements are possible compared to my approach. I will try the ebuilds on my system as soon as possible.
I think we should do the discussion in your thread and reference each new file here. I will keep this open until the basic files are in the official gentoo tree.
Work on installing ROCm according to FHS standard... https://github.com/RadeonOpenCompute/ROCR-Runtime/pull/51
State update: Current work focuses on getting an ebuild for "rocm-opencl-runtime" to work.
Hi, thanks for this repository and great news to move stuff to gentoo. I would gladly help testing / fixing stuff, as I anyway need it myself. Yesterday I spend some time to get more or less the full 2.0 suite compiled, I just opened a PR with several fixes I had to apply, but I understand that you are changing a lot the structure right now. Anyway, If I can help you somehow, please let me know.
Hi, thanks for your PR, I will check it. Currently I am trying to build rocm-opencl-runtime without llvm/clang/lld there are some problems. It seems that parallel building is sometimes a solution and sometimes a problem. That is the point where I currently stuck.
@candrews Due to limited time and the fact that AMD will upstream their changes to llvm/clang in the future I created an ebuild for ROCm-OpenCL-Runtime (dev-lilbs/rocm-opencl-runtime-2.3.0) which uses git-r3 eclass. Is this a possible ebuild which can become part of the Gentoo portage?
@candrews Due to limited time and the fact that AMD will upstream their changes to llvm/clang in the future I created an ebuild for ROCm-OpenCL-Runtime (dev-lilbs/rocm-opencl-runtime-2.3.0) which uses git-r3 eclass. Is this a possible ebuild which can become part of the Gentoo portage?
I can't see version 2.3 of rocm-opencl-runtime in the tree. Is it located somewhere else?
Currently compiling media-libs/ROCm-OpenCL-Runtime-2.3.9999 to see if it gives me a back a working AMD opencl stack.
Yep, with media-libs/ROCm-OpenCL-Runtime-2.3.9999 clinfo does not segfault any more, and my opencl examples are running fine so far.
Oh forgott to upload the new dev-libs/rocm-opencl-runtime. But the result is the same as the one from media-libs category.
I think these could be the next steps:
-
stablize HIP against HCC (?)
-
build HIP aginst clang (HIP-clang)
-
clean up the remaining ebuilds - make them conform to the Gentoo rules
-
if patches are submitted to AMD, check the partly different method to use CMakeLists.txt and make it uniform where possible
-
check the cmake install files if the include paths are correct or if there are paths e.g. "/opt/rocm/" which are not needed / point to non existing directories
-
if ROCm works, add support for nVidia devices and test it
Update of the next steps:
-
how to proceed with rocminfo? - see patch https://github.com/justxi/rocm/blob/master/dev-util/rocminfo/files/rocminfo-2.8.0-add-env-var.patch - This could simplify some sci-libs.
-
clean up ebuilds (e.g. use environment variables instead of local exports)
-
keep on making the ebuilds conform to the Gentoo rules
-
build HIP aginst clang (HIP-clang)
-
if ROCm works, add support for nVidia devices and test it
For reference, in the binary RPMs, AMD added a new package called amdgpu-llvm, which is basically what we have as llvm-roc. Still, their HIP is built against hcc, and there is no HIP-clang yet.
After the upgrade to rocm 2.9, I am getting assertion failures in opencl applications:
$ clinfo
clinfo: /var/tmp/portage/sys-devel/llvm-roc-2.9.0/work/llvm-roc-ocl-2.9.0/include/llvm/Support/CommandLine.h:813: void llvm::cl::parser<DataType>::addLiteralOption(llvm::StringRef, const DT&, llvm::StringRef) [with DT = llvm::FunctionPass* (*)(); DataType = llvm::FunctionPass* (*)()]: Assertion `findOption(Name) == Values.size() && "Option already exists!"' failed.
[1] 48149 abort (core dumped) clinfo
2.7 used to work fine (e.g., could run clinfo, some simple opencl demos without issues). Other pieces of the stack seem to work ok (e.g, rocminfo correctly reports my devices), only opencl seems to be an issue.
Reported here:
https://bugs.gentoo.org/697490
But unfortunately it is not reproducible by @candrews :( Any idea about what might be going on?
@davidrohr Good to know that we are on the same level =).
@bluescarni On my system a simple OpenCL application works. Can you provide your OpenCL application or if not, a reduced version?
@justxi I just tried this simple opencl C program:
http://phycomp.technion.ac.il/~comphy/classfiles/hellocl.c
Compiled with gcc -lOpenCL, same issue when run:
a.out: /var/tmp/portage/sys-devel/llvm-roc-2.9.0/work/llvm-roc-ocl-2.9.0/include/llvm/Support/CommandLine.h:813: void llvm::cl::parser<DataType>::addLiteralOption(llvm::StringRef, const DT&, llvm::StringRef) [with DT = llvm::FunctionPass* (*)(); DataType = llvm::FunctionPass* (*)()]: Assertion `findOption(Name) == Values.size() && "Option already exists!"' failed.
[1] 50450 abort (core dumped) ./a.out
I don't think it has anything to do with specific examples, this smells to me like some issue in the initialisation of the opencl runtime.
I tried to comment out the code for the assertion in the original source code hoping it would be a harmless warning, but nope, after doing that opencl programs just segfault.
I have reported the issue on the ROCm bugtracker as well:
https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime/issues/92
@justxi I just tried this simple opencl C program:
http://phycomp.technion.ac.il/~comphy/classfiles/hellocl.c
Compiled with
gcc -lOpenCL, same issue when run:a.out: /var/tmp/portage/sys-devel/llvm-roc-2.9.0/work/llvm-roc-ocl-2.9.0/include/llvm/Support/CommandLine.h:813: void llvm::cl::parser<DataType>::addLiteralOption(llvm::StringRef, const DT&, llvm::StringRef) [with DT = llvm::FunctionPass* (*)(); DataType = llvm::FunctionPass* (*)()]: Assertion `findOption(Name) == Values.size() && "Option already exists!"' failed. [1] 50450 abort (core dumped) ./a.outI don't think it has anything to do with specific examples, this smells to me like some issue in the initialisation of the opencl runtime.
I tried your example with "OCL_ICD_VENDORS=amdocl64.icd ./hellocl" and it gives
Uryyb, Jbeyq!
Hello, World!
and also with my Mesa and Intel OpenCL installation.
@justxi oh wow, with that env variable set it does work:
$ OCL_ICD_VENDORS=amdocl64.icd clinfo
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
Platform Version OpenCL 2.0 AMD-APP.internal (2973.0)
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_amd_object_metadata cl_amd_event_callback
Platform Max metadata object keys (AMD) 8
Platform Extensions function suffix AMD
Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name gfx803
Device Vendor Advanced Micro Devices, Inc.
Device Vendor ID 0x1002
Device Version OpenCL 1.2
Driver Version 2973.0 (HSA1.1,LC)
Device OpenCL C Version OpenCL C 2.0
Device Type GPU
Device Board Name (AMD) Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
Device Topology (AMD) PCI-E, 26:00.0
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 32
SIMD per compute unit (AMD) 4
SIMD width (AMD) 16
SIMD instruction width (AMD) 1
Max clock frequency 1256MHz
Graphics IP (AMD) 8.3
Device Partition (core)
Max number of sub-devices 32
Supported partition types None
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 256
Preferred work group size (AMD) 256
Max work group size (AMD) 1024
Preferred work group size multiple 64
Wavefront width (AMD) 64
Preferred / native vector sizes
char 4 / 4
short 2 / 2
int 1 / 1
long 1 / 1
half 1 / 1 (cl_khr_fp16)
float 1 / 1
double 1 / 1 (cl_khr_fp64)
Half-precision Floating-point support (cl_khr_fp16)
Denormals No
Infinity and NANs No
Round to nearest No
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Single-precision Floating-point support (core)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 4294967296 (4GiB)
Global free memory (AMD) 4194304 (4GiB)
Global memory channels (AMD) 8
Global memory banks per channel (AMD) 4
Global memory bank width (AMD) 256 bytes
Error Correction support No
Max memory allocation 3650722201 (3.4GiB)
Unified memory for Host and Device No
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Global Memory cache type Read/Write
Global Memory cache size 16384 (16KiB)
Global Memory cache line size 64 bytes
Image support No
Base address alignment for 2D image buffers 0 bytes
Pitch alignment for 2D image buffers 0 pixels
Local memory type Local
Local memory size 65536 (64KiB)
Local memory syze per CU (AMD) 65536 (64KiB)
Local memory banks (AMD) 32
Max number of constant args 8
Max constant buffer size 3650722201 (3.4GiB)
Preferred constant buffer size (AMD) 16384 (16KiB)
Max size of kernel argument 1024
Queue properties
Out-of-order execution No
Profiling Yes
Prefer user sync for interop Yes
Number of P2P devices (AMD) 0
P2P devices (AMD) (n/a)
Profiling timer resolution 1ns
Profiling timer offset since Epoch (AMD) 0ns (Thu Jan 1 01:00:00 1970)
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
Thread trace supported (AMD) No
Number of async queues (AMD) 8
Max real-time compute queues (AMD) 8
Max real-time compute units (AMD) 32
printf() buffer size 4194304 (4MiB)
Built-in kernels (n/a)
Device Extensions cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) AMD Accelerated Parallel Processing
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [AMD]
clCreateContext(NULL, ...) [default] Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx803
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx803
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx803
ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.2.12
ICD loader Profile OpenCL 2.2
Do you know what is going on?
Thanks a lot for the suggestion, at least now there's a workaround.