hipBLAS icon indicating copy to clipboard operation
hipBLAS copied to clipboard
verified publisher icon ROCm

[Issue]: hipBLAS install script does not write into /opt/rocm

Open JakeSkelton opened this issue 1 year ago • 3 comments

Problem Description

I feel like I am missing something obvious but when I unpack the tarball, cd into the resulting directory and run ./install.sh -i, with or without sudo permissions, the package appears to be installed (successfully) in my local directory rather than /opt/rocm as the CLI flag instructions suggest

Operating System

Ubuntu 22.04

CPU

AMD Ryzen 5 5600G

GPU

AMD Radeon RX 7900 XTX, AMD Radeon RX 7900 XT, AMD Radeon RX 7900 GRE

ROCm Version

ROCm 6.2.0

ROCm Component

hipBLAS

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

ROCk module version 6.8.5 is loaded

HSA System Attributes

Runtime Version: 1.14 Runtime Ext Version: 1.6 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE
System Endianness: LITTLE
Mwaitx: DISABLED DMAbuf Support: YES

==========
HSA Agents

Agent 1

Name: AMD Ryzen 5 5600G with Radeon Graphics Uuid: CPU-XX
Marketing Name: AMD Ryzen 5 5600G with Radeon Graphics Vendor Name: CPU
Feature: None specified
Profile: FULL_PROFILE
Float Round Mode: NEAR
Max Queue Number: 0(0x0)
Queue Min Size: 0(0x0)
Queue Max Size: 0(0x0)
Queue Type: MULTI
Node: 0
Device Type: CPU
Cache Info:
L1: 32768(0x8000) KB
Chip ID: 0(0x0)
ASIC Revision: 0(0x0)
Cacheline Size: 64(0x40)
Max Clock Freq. (MHz): 3900
BDFID: 0
Internal Node ID: 0
Compute Unit: 12
SIMDs per CU: 0
Shader Engines: 0
Shader Arrs. per Eng.: 0
WatchPts on Addr. Ranges:1
Memory Properties:
Features: None Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: FINE GRAINED
Size: 16158312(0xf68e68) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Recommended Granule:4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
Pool 2
Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 16158312(0xf68e68) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Recommended Granule:4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
Pool 3
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 16158312(0xf68e68) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Recommended Granule:4KB
Alloc Alignment: 4KB
Accessible by all: TRUE
ISA Info:

Agent 2

Name: gfx1032
Uuid: GPU-XX
Marketing Name: AMD Radeon RX 6600
Vendor Name: AMD
Feature: KERNEL_DISPATCH
Profile: BASE_PROFILE
Float Round Mode: NEAR
Max Queue Number: 128(0x80)
Queue Min Size: 64(0x40)
Queue Max Size: 131072(0x20000)
Queue Type: MULTI
Node: 1
Device Type: GPU
Cache Info:
L1: 16(0x10) KB
L2: 2048(0x800) KB
L3: 32768(0x8000) KB
Chip ID: 29695(0x73ff)
ASIC Revision: 0(0x0)
Cacheline Size: 64(0x40)
Max Clock Freq. (MHz): 2750
BDFID: 768
Internal Node ID: 1
Compute Unit: 28
SIMDs per CU: 2
Shader Engines: 2
Shader Arrs. per Eng.: 2
WatchPts on Addr. Ranges:4
Coherent Host Access: FALSE
Memory Properties:
Features: KERNEL_DISPATCH Fast F16 Operation: TRUE
Wavefront Size: 32(0x20)
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Max Waves Per CU: 32(0x20)
Max Work-item Per CU: 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
Max fbarriers/Workgrp: 32
Packet Processor uCode:: 118
SDMA engine uCode:: 76
IOMMU Support:: None
Pool Info:
Pool 1
Segment: GLOBAL; FLAGS: COARSE GRAINED
Size: 8372224(0x7fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Recommended Granule:2048KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 2
Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 8372224(0x7fc000) KB
Allocatable: TRUE
Alloc Granule: 4KB
Alloc Recommended Granule:2048KB
Alloc Alignment: 4KB
Accessible by all: FALSE
Pool 3
Segment: GROUP
Size: 64(0x40) KB
Allocatable: FALSE
Alloc Granule: 0KB
Alloc Recommended Granule:0KB
Alloc Alignment: 0KB
Accessible by all: FALSE
ISA Info:
ISA 1
Name: amdgcn-amd-amdhsa--gfx1032
Machine Models: HSA_MACHINE_MODEL_LARGE
Profiles: HSA_PROFILE_BASE
Default Rounding Mode: NEAR
Default Rounding Mode: NEAR
Fast f16: TRUE
Workgroup Max Size: 1024(0x400)
Workgroup Max Size per Dimension: x 1024(0x400)
y 1024(0x400)
z 1024(0x400)
Grid Max Size: 4294967295(0xffffffff)
Grid Max Size per Dimension: x 4294967295(0xffffffff)
y 4294967295(0xffffffff)
z 4294967295(0xffffffff)
FBarrier Max Size: 32
*** Done ***

Additional Information

My GPU is a Radeon RX 6600 but this was not an item in the drop-down list

JakeSkelton avatar Sep 24 '24 21:09 JakeSkelton

Hi @JakeSkelton,

Would you be able to provide the output of the ./install.sh -i command? Do you have hipBLAS' dependencies installed on your system (rocBLAS, rocSOLVER, etc.)?

Thanks, Daine

daineAMD avatar Sep 26 '24 18:09 daineAMD

Hi, thanks for your response. I have installed rocBLAS and rocSOLVER with the system package manager

$ apt list --installed | grep -E 'roc(blas|solver)
rocblas-dev/jammy,now 4.2.1.60201-112~22.04 amd64 [installed,automatic]
rocblas/jammy,now 4.2.1.60201-112~22.04 amd64 [installed]
rocsolver-dev/jammy,now 3.26.0.60201-112~22.04 amd64 [installed,automatic]
rocsolver/jammy,now 3.26.0.60201-112~22.04 amd64 [installed]

Here is the output of ./install.sh -i. Note that I am running in ~/Downloads/hipBLAS-rocm-6.2.1

/bin/ln: failed to create symbolic link './.git/hooks/': No such file or directory
++ readlink -m ./build
+ build_dir=/home/jake/Downloads/hipBLAS-rocm-6.2.1/build
+ printf '\033[32mCreating project build directory in: \033[33m/home/jake/Downloads/hipBLAS-rocm-6.2.1/build\033[0m\n'
Creating project build directory in: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build
+ [[ true == true ]]
+ rm -rf /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release
+ cmake_executable=cmake
+ cxx=g++
+ cc=gcc
+ fc=gfortran
+ [[ false == true ]]
+ [[ false == true ]]
+ full_build_dir=
+ [[ true == true ]]
+ full_build_dir=/home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release
+ [[ '' != true ]]
++ /opt/rocm/bin/hipconfig --platform
+ export HIP_PLATFORM=amd
+ HIP_PLATFORM=amd
+ [[ false == false ]]
+ pushd .
~/Downloads/hipBLAS-rocm-6.2.1 ~/Downloads/hipBLAS-rocm-6.2.1
+ rm -rf /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release
+ python3 ./rmake.py --install_invoked -i --build_dir=/home/jake/Downloads/hipBLAS-rocm-6.2.1/build --src_path=/home/jake/Downloads/hipBLAS-rocm-6.2.1
OS Info:
{'PRETTY_NAME': 'Ubuntu 22.04.5 LTS', 'NAME': 'Ubuntu', 'VERSION_ID': '22.04', 'VERSION': '22.04.5 LTS (Jammy Jellyfish)', 'VERSION_CODENAME': 'jammy', 'ID': 'ubuntu', 'ID_LIKE': 'debian', 'HOME_URL': 'https://www.ubuntu.com/', 'SUPPORT_URL': 'https://help.ubuntu.com/', 'BUG_REPORT_URL': 'https://bugs.launchpad.net/ubuntu/', 'PRIVACY_POLICY_URL': 'https://www.ubuntu.com/legal/terms-and-policies/privacy-policy', 'UBUNTU_CODENAME': 'jammy', 'NUM_PROC': 12}
Build source path: /home/jake/Downloads/hipBLAS-rocm-6.2.1
cmake -DCMAKE_TOOLCHAIN_FILE=toolchain-linux.cmake -DROCM_DIR:PATH=/opt/rocm -DCPACK_PACKAGING_INSTALL_PREFIX=/opt/rocm -DCMAKE_INSTALL_PREFIX="hipblas-install" -DROCM_PATH=/opt/rocm -DCMAKE_PREFIX_PATH:PATH=/opt/rocm -DCPACK_SET_DESTDIR=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=g++ -DCUSTOM_TARGET=host -DBUILD_WITH_SOLVER=ON -DROCBLAS_PATH="/home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/C:/hipSDK" -DROCSOLVER_PATH="/home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/C:/hipSDK" /home/jake/Downloads/hipBLAS-rocm-6.2.1
-- The CXX compiler identification is GNU 11.4.0
-- The Fortran compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found Git: /usr/bin/git (found version "2.34.1") 
fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
-- *** Building hipBLAS commit: 
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    ROCBLAS_PATH
    ROCSOLVER_PATH


-- Build files have been written to: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release
make -j12 install
Scanning dependencies of target hipblas_fortran
[ 20%] Building CXX object library/src/CMakeFiles/hipblas.dir/amd_detail/hipblas.cpp.o
[ 40%] Building CXX object library/src/CMakeFiles/hipblas.dir/hipblas_auxiliary.cpp.o
[ 60%] Building Fortran object library/src/CMakeFiles/hipblas_fortran.dir/hipblas_module.f90.o
[ 80%] Linking Fortran shared library libhipblas_fortran.so
[ 80%] Built target hipblas_fortran
[100%] Linking CXX shared library libhipblas.so
[100%] Built target hipblas
Install the project...
-- Install configuration: "Release"
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/share/doc/hipblas/LICENSE.md
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include/hipblas
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include/hipblas/hipblas-version.h
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include/hipblas/hipblas-export.h
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include/hipblas/hipblas.h
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/libhipblas.so.2.2
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/libhipblas.so.2
-- Set runtime path of "/home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/libhipblas.so.2.2" to "$ORIGIN/../lib:$ORIGIN/../lib/hipblas/lib"
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/libhipblas.so
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/cmake/hipblas/hipblas-targets.cmake
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/cmake/hipblas/hipblas-targets-release.cmake
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/cmake/hipblas/hipblas-config.cmake
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/lib/cmake/hipblas/hipblas-config-version.cmake
-- Installing: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-install/include/hipblas/hipblas_module.f90
+ check_exit_code 0
+ ((  0 != 0  ))
+ popd
~/Downloads/hipBLAS-rocm-6.2.1
+ pushd .
~/Downloads/hipBLAS-rocm-6.2.1 ~/Downloads/hipBLAS-rocm-6.2.1
+ cd /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release
+ [[ true == true ]]
+ make package
[ 40%] Built target hipblas_fortran
Consolidate compiler generated dependencies of target hipblas
[100%] Built target hipblas
Run CPack packaging tool...
CPack: Create package using TGZ
CPack: Install projects
CPack: - Run preinstall target for: hipblas
CPack: - Install project: hipblas []
CPack: -   Install component: runtime
CPack: -   Install component: devel
CPack: Create package
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-2.2.0-Linux-devel.tar.gz generated.
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-2.2.0-Linux-runtime.tar.gz generated.
CPack: Create package using ZIP
CPack: Install projects
CPack: - Run preinstall target for: hipblas
CPack: - Install project: hipblas []
CPack: -   Install component: runtime
CPack: -   Install component: devel
CPack: Create package
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-2.2.0-Linux-devel.zip generated.
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-2.2.0-Linux-runtime.zip generated.
CPack: Create package using DEB
CPack: Install projects
CPack: - Run preinstall target for: hipblas
CPack: - Install project: hipblas []
CPack: -   Install component: runtime
CPack: -   Install component: devel
CPack: Create package
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas-dev_2.2.0_amd64.deb generated.
CPack: - package: /home/jake/Downloads/hipBLAS-rocm-6.2.1/build/release/hipblas_2.2.0-_amd64.deb generated.
+ check_exit_code 0
+ ((  0 != 0  ))
+ case "${ID}" in
+ elevate_if_not_root dpkg -i hipblas_2.2.0-_amd64.deb hipblas-dev_2.2.0_amd64.deb
++ id -u
+ local uid=1000
+ ((  1000  ))
+ sudo dpkg -i hipblas_2.2.0-_amd64.deb hipblas-dev_2.2.0_amd64.deb
[sudo] password for jake: 
(Reading database ... 290192 files and directories currently installed.)
Preparing to unpack hipblas_2.2.0-_amd64.deb ...
Unpacking hipblas (2.2.0) over (2.2.0) ...
Preparing to unpack hipblas-dev_2.2.0_amd64.deb ...
rm: cannot remove '/etc/ld.so.conf.d/hipblas.conf': No such file or directory
Unpacking hipblas-dev (2.2.0) over (2.2.0) ...
Setting up hipblas (2.2.0) ...
Setting up hipblas-dev (2.2.0) ...
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
+ check_exit_code 0
+ ((  0 != 0  ))
+ check_exit_code 0
+ ((  0 != 0  ))
+ popd
~/Downloads/hipBLAS-rocm-6.2.1

And neither /opt/rocm/ or /opt/rocm-6.2.1/ are modified.

JakeSkelton avatar Sep 27 '24 13:09 JakeSkelton

It looks like hipBLAS may already be installed on your machine if I'm reading the output right. Does dpkg -l hipblas show the package? Does /opt/rocm/lib/libhipblas.so exist?

daineAMD avatar Sep 27 '24 22:09 daineAMD

Hi, I can confirm that the object file does exist in /opt/rocm/lib but when running dpkg -l hipblas it looks like there is some error (I think, I'm not very familiar with the dpkg CLI)

$ dpkg -l hipblas
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-============================================
ii  hipblas        2.2.0        amd64        Radeon Open Compute BLAS marshalling library

JakeSkelton avatar Oct 02 '24 09:10 JakeSkelton

It looks like hipBLAS is installed correctly, the ii in the first column indicates that it's installed without an error. If you'd like to make changes to hipBLAS source code, or just try installing again, you can try removing the hipblas and hipblas-dev packages and then run the install command again. Otherwise I think you should be good to go to use the hipblas packages.

To remove hipblas if you'd like: sudo apt remove hipblas or sudo dpkg -r hipblas hipblas-dev

Then you can try ./install.sh -i again to see if it installs correctly.

Thanks, Daine

daineAMD avatar Oct 02 '24 17:10 daineAMD

Hi @JakeSkelton. Has your issue been resolved? If so, please close the ticket. Thanks!

ppanchad-amd avatar Oct 28 '24 17:10 ppanchad-amd