john
john copied to clipboard
openwall
WSL2-Ubuntu Nvidia GPU not recognize either from CUDA or OpenCL
Checklist
Guide I followed to make JtR:
- https://docs.nvidia.com/cuda/wsl-user-guide/index.html
- https://openwall.info/wiki/john/GPU (NVIDIA part)
- https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
- https://github.com/openwall/john/blob/bleeding-jumbo/doc/INSTALL-UBUNTU
Attach details about your OS and about john, including:
- The output of
./john --list=build-info.
╰─λ /opt/john/run/john --list=build-info
Version: 1.9.0-jumbo-1+bleeding-ff16ecb93 2021-12-08 20:03:10 +0100
Build: linux-gnu 64-bit x86_64 AVX512BW AC OMP
SIMD: AVX512BW, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
CPU tests: AVX512BW
$JOHN is /opt/john/run/
Format interface version: 14
Max. number of reported tunable costs: 4
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
SALT_HASH_SIZE: 1048576
SINGLE_IDX_MAX: 2147483648
SINGLE_BUF_MAX: 4294967295
Effective limit: Number of salts vs. SingleMaxBufferSize
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 9.3.0
GNU libc version: 2.31 (loaded: 2.31)
OpenCL headers version: 1.2
Crypto library: OpenSSL
OpenSSL library version: 01010106f
OpenSSL 1.1.1f 31 Mar 2020
GMP library version: 6.2.0
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
times(2) sysconf(_SC_CLK_TCK) is 100
Using times(2) for timers, resolution 10 ms
HR timer: clock_gettime(), latency 40 ns
Total physical host memory: 31970 MiB
Available physical host memory: 28778 MiB
Terminal locale string: C.UTF-8
Parsed terminal locale: UTF-8
- The command line you are using.
╰─λ nvidia-smi
Fri Dec 10 10:45:46 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.00 Driver Version: 510.06 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA RTX A200... On | 00000000:01:00.0 Off | N/A |
| N/A 45C P8 6W / N/A | 391MiB / 4096MiB | N/A Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
╰─λ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
╰─λ wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin 130 (50.128s)
╰─λ sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
╰─λ wget https://developer.download.nvidia.com/compute/cuda/11.5.1/local_installers/cuda-repo-wsl-ubuntu-11-5-local_11.5.1-1_amd64.deb
╰─λ sudo dpkg -i cuda-repo-wsl-ubuntu-11-5-local_11.5.1-1_amd64.deb
╰─λ sudo apt-key add /var/cuda-repo-wsl-ubuntu-11-5-local/7fa2af80.pub
OK
╰─λ sudo apt-get update
╰─λ sudo apt-get -y install cuda
╰─λ sudo apt-get -y install git build-essential libssl-dev zlib1g-dev
╰─λ sudo apt-get -y install yasm pkg-config libgmp-dev libpcap-dev libbz2-dev
╰─λ sudo apt-get -y install nvidia-opencl-dev
╰─λ sudo apt-get -y install ocl-icd-opencl-dev opencl-headers pocl-opencl-icd
╰─λ sudo git clone https://github.com/openwall/john -b bleeding-jumbo john
╰─λ sudo ./configure
Configured for building John the Ripper jumbo:
Target CPU ......................................... x86_64 AVX512BW, 64-bit LE
AES-NI support ..................................... run-time detection
Target OS .......................................... linux-gnu
Cross compiling .................................... no
Legacy arch header ................................. x86-64.h
Optional libraries/features found:
Memory map (share/page large files) ................ yes
Fork support ....................................... yes
OpenMP support ..................................... yes (not for fast formats)
OpenCL support ..................................... yes
Generic crypt(3) format ............................ yes
OpenSSL (many additional formats) .................. yes
libgmp (PRINCE mode and faster SRP formats) ........ yes
128-bit integer (faster PRINCE mode) ............... yes
libz (7z, pkzip and some other formats) ............ yes
libbz2 (7z and gpg2john bz2 support) ............... yes
libpcap (vncpcap2john and SIPdump) ................. yes
Non-free unrar code (complete RAR support) ......... yes
librexgen (regex mode, see doc/README.librexgen) ... no
OpenMPI support (default disabled) ................. no
Experimental code (default disabled) ............... no
ZTEX USB-FPGA module 1.15y support ................. no
Install missing libraries to get any needed features that were omitted.
╰─λ sudo make -s clean && sudo make -sj4
╰─λ ../run/john --list=opencl-devices
Platform #0 name: Intel(R) OpenCL HD Graphics, version: OpenCL 3.0
Device #0 (1) name: Intel(R) Graphics [0x9a60]
Device vendor: Intel(R) Corporation
Device type: GPU (LE)
Device version: OpenCL 3.0 NEO
Driver version: 21.37.20939
Native vector widths: char 16, short 8, int 4, long 1
Preferred vector width: char 16, short 8, int 4, long 1
Global Memory: 26103 MiB
Global Memory Cache: 512 KiB
Local Memory: 64 KiB (Local)
Constant Buffer size: 1 GiB
Max memory alloc. size: 1 GiB
Max clock (MHz): 1450
Profiling timer res.: 52 ns
Max Work Group Size: 256
Parallel compute cores: 32
Stream processors: 256 (32 x 8)
Speed index: 371200
Platform #1 name: Portable Computing Language, version: OpenCL 1.2 pocl 1.4, None+Asserts, LLVM 9.0.1, RELOC, SLEEF, DISTRO, POCL_DEBUG
Device #0 (2) name: pthread-11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
Device vendor: GenuineIntel
Device type: CPU (LE)
Device version: OpenCL 1.2 pocl HSTR: pthread-x86_64-pc-linux-gnu-icelake-client
Driver version: 1.4
Native vector widths: char 16, short 16, int 16, long 8
Preferred vector width: char 16, short 16, int 16, long 8
Global Memory: 29922 MiB
Global Memory Cache: 24 MiB
Local Memory: 16 MiB (Global)
Constant Buffer size: 16 MiB
Max memory alloc. size: 8 GiB
Max clock (MHz): 2496
Profiling timer res.: 1 ns
Max Work Group Size: 4096
Parallel compute cores: 16
Speed index: 319488
╰─λ ls /dev/dxg
/dev/dxg
Well, this procedure works totally fine on Ubuntu Bare-metal installation. Not exactly the CUDA part, but at least JtR could get the same nvidia RTX A2000 GPU from opencl driver. However, it's not working on WSL2 Ubuntu at all. I tried several times with Windows Nvidia driver installation (guide says WSL2 would not actually need to install nvidia driver like Bare-metal installation), or with actually both Windows Nvidia driver installation along with WSL2 Ubuntu Nvidia-driver-470 installation. Everytime john would only recognize the Intel GPU instead of Nvidia.
May I ask if there's any possibility to do this on WSL2 Ubuntu at all? If yes, which part of me doing wrong please?
Thank you very much. If there's any other information you required, I'm more than happy to keep updating this post.
Cheers, Channing.
@diegoxcn Thank you for providing so much detail.
What files do you have under /etc/OpenCL? Do you have something like this? -
$ cat /etc/OpenCL/vendors/nvidia.icd
libnvidia-opencl.so.1
Hi @solardiz , thanks for quick reply, and yes, I do have it:
╰─λ ls /etc/OpenCL/vendors
intel.icd nvidia.icd pocl.icd
╰─λ cat /etc/OpenCL/vendors/nvidia.icd
libnvidia-opencl.so.1
@solardiz Nope, it only shows Intel OpenCL HD Graphics and Portable Computing Language, no nvidia at all.
Last I read about it, WSL doesn't support GPU Acc/OpenCL.
It is unclear whether this is still the case at https://docs.nvidia.com/cuda/wsl-user-guide/index.html#features-not-yet-supported. However, if clinfo fails, the problem is not related to the JtR binary or the build process.
I would run ldd libnvidia-opencl.so.1 just to keep trying to figure out if I can debug more. And if it's more than an experiment, please use the native binary.
Hi @claudioandre-br Thanks for the infor, I tried ldd libnvidia-opencl.so.1 which shows no such file, seems it's not installed by nvidia-opencl-dev package, will check if I can get it somehow. And update this post later. Thank you very much.
╰─λ find /usr/lib -name libnvidia-opencl.so.1 2>/dev/null
/usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1
Well, apparently this file does exist, just somehow it's not registered correctly by ldd?
@diegoxcn I think @claudioandre-br assumed you'd run this command when in the right directory, so after cd /usr/lib/x86_64-linux-gnu in your case. ldd isn't supposed to find it automatically, so no problem there.
You could want to run ldconfig -v as root and see if its output lists that library or not. Also, this action might help on its own, so retest clinfo after it.
@solardiz Thanks for the instruction.
╰─λ ldd /usr/lib/x86_64-linux-gnu/libnvidia-opencl.so.1 0 (0.000s)
linux-vdso.so.1 (0x00007ffcaafcb000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6575f07000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6575db8000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6575db2000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6575d8f000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6575d84000)
/lib64/ld-linux-x86-64.so.2 (0x00007f65773bf000)
Then it works fine. :)
And ldconfig -v does show this so.1 file:
╰─λ sudo ldconfig -v
....omit output...
libnvidia-opencl.so.1 -> libnvidia-opencl.so.460.106.00
...omit output...
But unfortunately the clinfo still have very same result. Well, not sure if it's the limitation of the WSL2.
Thanks.
@diegoxcn I'm afraid we don't currently have further advice for you.
Guide I followed to make JtR:
2. https://openwall.info/wiki/john/GPU (NVIDIA part)
Ouch. There was nothing supposed to be currently useful in there. I had previously added a note that the content in there was merely historical, but apparently that wasn't enough, so I've just fully dropped the content of that wiki page. If anyone wants to maintain an up-to-date replacement, please feel free to do so, but for now it's better for this page not to exist than to keep it up with content dating so many years back. (Since it's a wiki, the old content can still be seen in page edits history. Hopefully, that's obscure enough that people wouldn't be wasting time on it.)
@solardiz No Worries, I really appreciate all your help. Just wonder if there's any possibility for JtR could run against nvidia opencl GPU on WSL2.
Hope that your team could do some testing on WSL2 would be perfect.
Thanks again.