jan
jan copied to clipboard
janhq
bug: Jan is not using GPU
Describe the bug Looks like Jan is using only CPU, and it is very slow. GMP utilization is 7% on average.
Steps to reproduce Steps to reproduce the behavior: normal operation
Expected behavior to see GPU utilization and work faster
Environment details
- Operating System: PoPOS!
- Jan Version: 4.3
- Processor: Intel i7-9750H
- RAM: 16GB
- Any additional relevant hardware specifics: Nvidia GTX 1050
Same behavior on Windows 11 No usage of GPU Very slow response
Environment details Operating System: Windows 11 Jan Version: 4.3 Processor: Intel i9-13900K RAM: 128GB GPU: Nvidia RTX 4090, RTX 3090TI
By default, we enable GPU acceleration.
If you are experiencing this issue, you may need to download the latest Nvidia CUDA toolkit and turn on the setting to enable Nvidia GPU acceleration. We weren't propagating CUDA issues in the old build.
Steps:
- Download Cuda toolkit (v11.4+): https://developer.nvidia.com/cuda-downloads?target_os=Windows
- Try the latest release to see any other errors: https://github.com/janhq/jan?tab=readme-ov-file#download
@hieu-jan to translate this into docs 🙏
Working after I did reinstall the latest CUDA Toolkit. Seems my old Cuda install was broken
By default, we enable GPU acceleration.
If you are experiencing this issue, you may need to download the latest Nvidia CUDA toolkit and turn on the setting to enable Nvidia GPU acceleration. We weren't propagating CUDA issues in the old build.
Steps:
- Download Cuda toolkit (v11.4+): https://developer.nvidia.com/cuda-downloads?target_os=Windows
- Try the latest release to see any other errors: https://github.com/janhq/jan?tab=readme-ov-file#download
@hieu-jan to translate this into docs 🙏
Reinstalled CUDA, but didn't helped. (I'm using also stable-diffusion for image generation and it is utilizing GPU normally)
Installing the Cuda toolkit worked for me. Interestingly I'm not always seeing an improvement. I guess it depends on the model.
d CUDA, but didn't helped. (I'm using also stable-dif
Do you see more RAM available like this:
d CUDA, but didn't helped. (I'm using also stable-dif
Do you see more RAM available like this:
I have the same issue but have a 6700 xt. It is supposed to be supported as far as I know. Do I have to do something else with an AMD gpu? Also have 100% cpu usage and no gpu usage.
RAM: 32gb DDR4 3200Mhz. CPU: i5 11400 GPU: 6700 xt SSD: 2tb Kingston fury renegade ssd OS: Windows 10
I have the same issue with RTX 4070. Tried re-install CUDA, re-install Jan, and the nightly build. RAM: 32GB DDR4 CPU: i7-12700 GPU: RTX 4070 12GB OS: Windows 11 Jan version: 0.4.3
Hi guys, could you please try our nightly build version available at this link: https://github.com/janhq/jan?tab=readme-ov-file#download, to check if it resolves the issue? Thank you.
We don't have the GPU detection feature in version 0.4.3 yet. It has only been enabled in the nightly build version. This feature will be officially available in Jan's release 0.4.4.
@hiento09 The nightly build does solve the issue.
Thanks!! Looking forward to seeing the 0.4.4 release!
We don't have the GPU detection feature in version 0.4.3 yet. It has only been enabled in the nightly build version. This feature will be officially available in Jan's release 0.4.4.
on Night version I have error:
We don't have the GPU detection feature in version 0.4.3 yet. It has only been enabled in the nightly build version. This feature will be officially available in Jan's release 0.4.4. @hiento09
Should this also work for AMD gpu's? Because I do see the option saying "Enable GPU acceleration for NVidia GPU's", but nothing for AMD GPU's. My CPU is still being used rather than my GPU (6700 xt). Or is the GPU detection feature supposed to be working and I wouldn't have to change anything manually? Because then the feature isn't working in my case.
Hi @lunisahdidouch , we have added a feature to support AMD GPU, you can check it here https://github.com/janhq/jan/issues/1608. Thank you for spending your time with Jan App.