Jun Liu

> -DMIOPEN_OFFLINE_COMPILER_PATHS_V2=1 > @junliume In the attached logfile I see: `[2024-05-10T20:00:55.207Z] + cmake '-DCMAKE_PREFIX_PATH=/opt/rocm-6.2.0-488/llvm;/opt/rocm-6.2.0-488' '-DCMAKE_SHARED_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN' '-DCMAKE_EXE_LINKER_FLAGS_INIT=-Wl,--enable-new-dtags,--build-id=sha1,--rpath,$ORIGIN/../lib' -DCMAKE_VERBOSE_MAKEFILE=1 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=FALSE -DCMAKE_INSTALL_PREFIX=/opt/rocm-6.2.0-488 -DCMAKE_PACKAGING_INSTALL_PREFIX=/opt/rocm-6.2.0-488 -DBUILD_FILE_REORG_BACKWARD_COMPATIBILITY=OFF -DROCM_SYMLINK_LIBS=OFF -DCPACK_PACKAGING_INSTALL_PREFIX=/opt/rocm-6.2.0-488 -DROCM_DISABLE_LDCONFIG=ON -DROCM_PATH=/opt/rocm-6.2.0-488 -DCPACK_DEBIAN_DEBUGINFO_PACKAGE=FALSE -DCPACK_RPM_DEBUGINFO_PACKAGE=FALSE -DCPACK_RPM_INSTALL_WITH_EXEC=FALSE -DCMAKE_BUILD_TYPE=Release...

@illsilin our CI should have one stage with GPU_TARGETS of "gfx1100;gfx90a;gfx942" :) #1358 and #1372 are both for `GPU_TARGETS="gfx1100;gfx90a;gfx942"` How to reproduce: > CXX=/opt/rocm/bin/amdclang++ cmake -DCMAKE_PREFIX_PATH=/opt/rocm -DCMAKE_BUILD_TYPE=Release -DGPU_TARGETS="gfx1100;gfx90a;gfx942" ..

More problem fixed in https://github.com/ROCm/composable_kernel/pull/1372/commits/4b81c7a5ae42985f3b07df6bfe3c5c9b8ddddb1a Hence `GPU_TARGETS MATCHES` is *very* problematic because it find matches *only*, excluding other targets based on match is very fragile.

FYI: additional issues are found when building `client_example` with multiple targets: ``` cd ${composable_kernel}/client_example/build CXX=/opt/rocm/llvm/bin/clang++ cmake -DCMAKE_PREFIX_PATH="${composable_kernel}/install/;/opt/rocm/" -DCMAKE_BUILD_TYPE=release -DGPU_TARGETS="gfx1100;gfx90a" -DCMAKE_CXX_FLAGS=" -O3 " .. make -j$(nproc) ``` will cause problems ```...

It should be changes in this file which are causing this problem: https://github.com/ROCm/MIOpen/blob/9064d096005af004c64d76a8c4c326a6cfd01c0f/src/solver/conv_direct_naive_conv_fwd.cpp and similar issues might be in other directions too

By removing these lines: https://github.com/ROCm/MIOpen/blob/9064d096005af004c64d76a8c4c326a6cfd01c0f/src/solver/conv_direct_naive_conv.cpp#L441-L442 It starts to work properly again. So the KDB cache does not accept these parameters. However, why the refreshed KDB still does not accept them?...

@atamazov FYI, this issue and the other GPU Target embedded in compiler issues are related. We will branch soon and need to regenerate quite a few KDBs with all fixes...

@BrianHarrisonAMD @atamazov I suspect `-fsanitize=undefined` but need more investigation. It must be one of these: >-DCMAKE_CXX_FLAGS_DEBUG='-g -fno-omit-frame-pointer -fsanitize=undefined -fno-sanitize-recover=undefined -Wno-option-ignored ' Update: confirmed it is due to `-fsanitize=undefined`

> @junliume @amberhassaan @DrizztDoUrden AFAICS, UB is related to hipFree. I recommend checking if reverting #2524 resolves the issue. Unfortunately, in my short experiment reverting #2524 does not resolve this...

Add more information: The issue is found on RHEL 8 where Python version is outdated (Python 2.7.18)