darktable-org
opencl not working with nouveau/mesa/rusticl driver
Describe the bug
I am trying to make darktable opencl work with the nouveau driver on Arch Linux. I have installed opencl-rusticl-mesa and set the environment variable RUSTICL_ENABLE=nouveau. I also activated rusticl in darktable opencl settings. Now I get this when I type darktable-cltest:
[anna@archlinux ~]$ darktable-cltest
darktable 4.6.1
Copyright (C) 2012-2024 Johannes Hanika and other contributors.
Compile options:
Bit depth -> 64 bit
Debug -> DISABLED
SSE2 optimizations -> ENABLED
OpenMP -> ENABLED
OpenCL -> ENABLED
Lua -> ENABLED - API version 9.2.0
Colord -> ENABLED
gPhoto2 -> ENABLED
GMIC -> ENABLED - Compressed LUTs are supported
GraphicsMagick -> ENABLED
ImageMagick -> DISABLED
libavif -> ENABLED
libheif -> ENABLED
libjxl -> ENABLED
OpenJPEG -> ENABLED
OpenEXR -> ENABLED
WebP -> ENABLED
See https://www.darktable.org/resources/ for detailed documentation.
See https://github.com/darktable-org/darktable/issues/new/choose to report bugs.
0.0113 [dt_get_sysresource_level] switched to 1 as `default'
0.0113 total mem: 15915MB
0.0113 mipmap cache: 1989MB
0.0113 available mem: 7957MB
0.0113 singlebuff: 124MB
0.0151 [opencl_init] opencl disabled via darktable preferences
0.0152 [opencl_init] opencl library 'libOpenCL' found on your system and loaded, preference 'default path'
0.1253 [opencl_init] found 1 platform
[opencl_init] found 1 device
[dt_opencl_device_init]
DEVICE: 0: 'NV168'
PLATFORM, VENDOR & ID: rusticl, Mesa/X.org, ID=4318
CANONICAL NAME: rusticlnv168
DRIVER VERSION: 24.0.4-arch1.2
DEVICE VERSION: OpenCL 3.0 , SM_20 SUPPORT
DEVICE_TYPE: GPU, dedicated mem
GLOBAL MEM SIZE: 6134 MB
MAX MEM ALLOC: 2048 MB
MAX IMAGE SIZE: 16384 x 16384
MAX WORK GROUP SIZE: 1024
MAX WORK ITEM DIMENSIONS: 3
MAX WORK ITEM SIZES: [ 1024 1024 64 ]
ASYNC PIXELPIPE: NO
PINNED MEMORY TRANSFER: NO
AVOID ATOMICS: NO
MICRO NAP: 250
ROUNDUP WIDTH & HEIGHT 16x16
CHECK EVENT HANDLES: 128
TILING ADVANTAGE: 0.000
DEFAULT DEVICE: NO
KERNEL BUILD DIRECTORY: /usr/share/darktable/kernels
KERNEL DIRECTORY: /home/anna/.cache/darktable/cached_v3_kernels_for_rusticlNV168_2404arch12
CL COMPILER OPTION:
CL COMPILER COMMAND: -w -DNVIDIA_SM_20=1 -DNVIDIA=1 -I"/usr/share/darktable/kernels"
/usr/include/c++/13.2.1/bits/stl_deque.h:1385: std::deque<_Tp, _Alloc>::const_reference std::deque<_Tp, _Alloc>::operator[](size_type) const [with _Tp = nv50_ir::ValueDef; _Alloc = std::allocator<nv50_ir::ValueDef>; const_reference = const nv50_ir::ValueDef&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.
Aborted (core dumped)
On Ubuntu 23.10, the output is this:
anna@anna-ms7c96:~$ darktable-cltest
0,0143 [dt_get_sysresource_level] switched to 1 as `default'
0,0143 total mem: 15903MB
0,0143 mipmap cache: 1987MB
0,0143 available mem: 7951MB
0,0143 singlebuff: 124MB
0,0143 OpenCL tune mem: OFF
0,0143 OpenCL pinned: OFF
[opencl_init] opencl related configuration options:
[opencl_init] opencl: ON
[opencl_init] opencl_scheduling_profile: 'default'
[opencl_init] opencl_library: 'default path'
[opencl_init] opencl_device_priority: '*/!0,*/*/*/!0,*'
[opencl_init] opencl_mandatory_timeout: 400
0.0168 [dt_dlopencl_init] could not find default opencl runtime library 'libOpenCL'
0.0169 [dt_dlopencl_init] could not find default opencl runtime library 'libOpenCL.so'
[opencl_init] opencl library 'libOpenCL.so.1' found on your system and loaded
[opencl_init] found 2 platforms
[opencl_init] no devices found for Mesa (vendor) - Clover (name)
[opencl_init] found 1 device
[dt_opencl_device_init]
DEVICE: 0: 'NV168', NEW
PLATFORM NAME & VENDOR: rusticl, Mesa/X.org
CANONICAL NAME: rusticlnv168
DRIVER VERSION: 23.2.1-1ubuntu3.1
DEVICE VERSION: OpenCL 3.0 , SM_20 SUPPORT
DEVICE_TYPE: GPU
GLOBAL MEM SIZE: 6134 MB
MAX MEM ALLOC: 2048 MB
MAX IMAGE SIZE: 16384 x 16384
MAX WORK GROUP SIZE: 1024
MAX WORK ITEM DIMENSIONS: 3
MAX WORK ITEM SIZES: [ 1024 1024 64 ]
ASYNC PIXELPIPE: NO
PINNED MEMORY TRANSFER: NO
MEMORY TUNING: NO
FORCED HEADROOM: 400
AVOID ATOMICS: NO
MICRO NAP: 250
ROUNDUP WIDTH: 16
ROUNDUP HEIGHT: 16
CHECK EVENT HANDLES: 128
TILING ADVANTAGE: 0.000
DEFAULT DEVICE: NO
KERNEL BUILD DIRECTORY: /usr/share/darktable/kernels
KERNEL DIRECTORY: /home/anna/.cache/darktable/cached_v1_kernels_for_rusticlNV168_23211ubuntu31
CL COMPILER OPTION: -cl-fast-relaxed-math
1.5749 [opencl_build_program] could not build program: CL_BUILD_PROGRAM_FAILURE
1.5749 [dt_opencl_device_init] failed to compile program `bilateral.cl'!
[opencl_init] no suitable devices found.
[opencl_init] FINALLY: opencl is NOT AVAILABLE and NOT ENABLED.
I guess the problem is the fact that -cl-fast-relaxed-math is missing on Arch. How can it be added?
GPU Hardware is Nvidia GTX 1660 Super
Steps to reproduce
see above
Expected behavior
opencl should work
Logfile | Screenshot | Screencast
No response
Commit
No response
Where did you obtain darktable from?
distro packaging
darktable version
4.6.1
What OS are you using?
Linux
What is the version of your OS?
rolling
Describe your system?
see description
Are you using OpenCL GPU in darktable?
I dont know
If yes, what is the GPU card and driver?
Nvidia GTX 1660 Super + nouveau/mesa/rusticl
Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip
No response
yeah... so that's a mesa bug and the nouveau driver's compiler is in a sorry state atm. It's better to try out zink and nvk here, but that should wait for mesa 24.1 and I haven't check if that works.
yeah... so that's a mesa bug and the nouveau driver's compiler is in a sorry state atm.
Currently dt compiles with -DNVIDIA_SM_20=1 -DNVIDIA=1 when detecting ID=4318. I think that might be wrong? Can you comment please @karolherbst ?
yeah... so that's a mesa bug and the nouveau driver's compiler is in a sorry state atm.
Currently dt compiles with
-DNVIDIA_SM_20=1 -DNVIDIA=1when detectingID=4318. I think that might be wrong? Can you comment please @karolherbst ?
what's ID=4318? the PCI device id? But yeah, the NV168/TU118 is SM75, but not sure what SM levels you care about here.
How can zink and nvk be used for darktable/opencl?
through rusticl. It should work but I don't know if nvk implements all the required extensions yet. It's something on my todo list to figure out, but I think it works. Just do RUSTICL_ENABLE=zink instead. Might also have to point to a specific vulkan impl if you have multiple installed.
do
RUSTICL_ENABLE=zinkinstead
Ok. Now I installed vulkan-nouveau-git and set RUSTICL_ENABLE=zink and I get this (opencl available and NOT enabled):
[anna@archlinux ~]$ darktable-cltest
darktable 4.6.1
Copyright (C) 2012-2024 Johannes Hanika and other contributors.
Compile options:
Bit depth -> 64 bit
Debug -> DISABLED
SSE2 optimizations -> ENABLED
OpenMP -> ENABLED
OpenCL -> ENABLED
Lua -> ENABLED - API version 9.2.0
Colord -> ENABLED
gPhoto2 -> ENABLED
GMIC -> ENABLED - Compressed LUTs are supported
GraphicsMagick -> ENABLED
ImageMagick -> DISABLED
libavif -> ENABLED
libheif -> ENABLED
libjxl -> ENABLED
OpenJPEG -> ENABLED
OpenEXR -> ENABLED
WebP -> ENABLED
See https://www.darktable.org/resources/ for detailed documentation.
See https://github.com/darktable-org/darktable/issues/new/choose to report bugs.
0.0113 [dt_get_sysresource_level] switched to 1 as `default'
0.0113 total mem: 15915MB
0.0113 mipmap cache: 1989MB
0.0113 available mem: 7957MB
0.0113 singlebuff: 124MB
0.0152 [opencl_init] opencl disabled via darktable preferences
0.0153 [opencl_init] opencl library 'libOpenCL' found on your system and loaded, preference 'default path'
0.4250 [opencl_init] found 2 platforms
0.4250 [opencl_init] no devices found for Mesa (vendor) - Clover (name)
[opencl_init] found 1 device
[dt_opencl_device_init]
DEVICE: 0: 'zink Vulkan 1.3(NVIDIA GeForce GTX 1660 SUPER (NVK TU116) (MESA_NVK))'
PLATFORM, VENDOR & ID: rusticl, Mesa/X.org, ID=4318
CANONICAL NAME: rusticlzinkvulkan13
DRIVER VERSION: 24.0.4-arch1.2
DEVICE VERSION: OpenCL 3.0
DEVICE_TYPE: GPU, dedicated mem
GLOBAL MEM SIZE: 6400 MB
MAX MEM ALLOC: 2048 MB
MAX IMAGE SIZE: 32768 x 32768
MAX WORK GROUP SIZE: 1024
MAX WORK ITEM DIMENSIONS: 3
MAX WORK ITEM SIZES: [ 1024 1024 64 ]
ASYNC PIXELPIPE: NO
PINNED MEMORY TRANSFER: NO
AVOID ATOMICS: NO
MICRO NAP: 250
ROUNDUP WIDTH & HEIGHT 16x16
CHECK EVENT HANDLES: 128
TILING ADVANTAGE: 0.000
DEFAULT DEVICE: NO
KERNEL BUILD DIRECTORY: /usr/share/darktable/kernels
KERNEL DIRECTORY: /home/anna/.cache/darktable/cached_v3_kernels_for_rusticlzinkVulkan13_2404arch12
CL COMPILER OPTION:
CL COMPILER COMMAND: -w -DNVIDIA=1 -I"/usr/share/darktable/kernels"
KERNEL LOADING TIME: 0.1882 sec
[opencl_init] OpenCL successfully initialized. internal numbers and names of available devices:
[opencl_init] 0 'rusticl zink Vulkan 1.3'
0.6145 [opencl_init] FINALLY: opencl is AVAILABLE and NOT ENABLED.
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities] image preview export thumbs preview2
[dt_opencl_update_priorities] 0 -1 0 0 -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities] image preview export thumbs preview2
[dt_opencl_update_priorities] 0 0 0 0 0
[opencl_synchronization_timeout] synchronization timeout set to 200
[dt_opencl_update_priorities] these are your device priorities:
[dt_opencl_update_priorities] image preview export thumbs preview2
[dt_opencl_update_priorities] 0 -1 0 0 -1
[dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe:
[dt_opencl_update_priorities] image preview export thumbs preview2
[dt_opencl_update_priorities] 0 0 0 0 0
[opencl_synchronization_timeout] synchronization timeout set to 200
do
RUSTICL_ENABLE=zinkinsteadOk. Now I installed vulkan-nouveau-git and set RUSTICL_ENABLE=zink and I get this (opencl available and NOT enabled):
[anna@archlinux ~]$ darktable-cltest darktable 4.6.1 Copyright (C) 2012-2024 Johannes Hanika and other contributors. Compile options: Bit depth -> 64 bit Debug -> DISABLED SSE2 optimizations -> ENABLED OpenMP -> ENABLED OpenCL -> ENABLED Lua -> ENABLED - API version 9.2.0 Colord -> ENABLED gPhoto2 -> ENABLED GMIC -> ENABLED - Compressed LUTs are supported GraphicsMagick -> ENABLED ImageMagick -> DISABLED libavif -> ENABLED libheif -> ENABLED libjxl -> ENABLED OpenJPEG -> ENABLED OpenEXR -> ENABLED WebP -> ENABLED See https://www.darktable.org/resources/ for detailed documentation. See https://github.com/darktable-org/darktable/issues/new/choose to report bugs. 0.0113 [dt_get_sysresource_level] switched to 1 as `default' 0.0113 total mem: 15915MB 0.0113 mipmap cache: 1989MB 0.0113 available mem: 7957MB 0.0113 singlebuff: 124MB 0.0152 [opencl_init] opencl disabled via darktable preferences 0.0153 [opencl_init] opencl library 'libOpenCL' found on your system and loaded, preference 'default path' 0.4250 [opencl_init] found 2 platforms 0.4250 [opencl_init] no devices found for Mesa (vendor) - Clover (name) [opencl_init] found 1 device [dt_opencl_device_init] DEVICE: 0: 'zink Vulkan 1.3(NVIDIA GeForce GTX 1660 SUPER (NVK TU116) (MESA_NVK))' PLATFORM, VENDOR & ID: rusticl, Mesa/X.org, ID=4318 CANONICAL NAME: rusticlzinkvulkan13 DRIVER VERSION: 24.0.4-arch1.2 DEVICE VERSION: OpenCL 3.0 DEVICE_TYPE: GPU, dedicated mem GLOBAL MEM SIZE: 6400 MB MAX MEM ALLOC: 2048 MB MAX IMAGE SIZE: 32768 x 32768 MAX WORK GROUP SIZE: 1024 MAX WORK ITEM DIMENSIONS: 3 MAX WORK ITEM SIZES: [ 1024 1024 64 ] ASYNC PIXELPIPE: NO PINNED MEMORY TRANSFER: NO AVOID ATOMICS: NO MICRO NAP: 250 ROUNDUP WIDTH & HEIGHT 16x16 CHECK EVENT HANDLES: 128 TILING ADVANTAGE: 0.000 DEFAULT DEVICE: NO KERNEL BUILD DIRECTORY: /usr/share/darktable/kernels KERNEL DIRECTORY: /home/anna/.cache/darktable/cached_v3_kernels_for_rusticlzinkVulkan13_2404arch12 CL COMPILER OPTION: CL COMPILER COMMAND: -w -DNVIDIA=1 -I"/usr/share/darktable/kernels" KERNEL LOADING TIME: 0.1882 sec [opencl_init] OpenCL successfully initialized. internal numbers and names of available devices: [opencl_init] 0 'rusticl zink Vulkan 1.3' 0.6145 [opencl_init] FINALLY: opencl is AVAILABLE and NOT ENABLED. [dt_opencl_update_priorities] these are your device priorities: [dt_opencl_update_priorities] image preview export thumbs preview2 [dt_opencl_update_priorities] 0 -1 0 0 -1 [dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe: [dt_opencl_update_priorities] image preview export thumbs preview2 [dt_opencl_update_priorities] 0 0 0 0 0 [opencl_synchronization_timeout] synchronization timeout set to 200 [dt_opencl_update_priorities] these are your device priorities: [dt_opencl_update_priorities] image preview export thumbs preview2 [dt_opencl_update_priorities] 0 -1 0 0 -1 [dt_opencl_update_priorities] show if opencl use is mandatory for a given pixelpipe: [dt_opencl_update_priorities] image preview export thumbs preview2 [dt_opencl_update_priorities] 0 0 0 0 0 [opencl_synchronization_timeout] synchronization timeout set to 200
I think you just need to enable it again or soething? but yeah, please let me know how that works, though you might run into bugs. In any case, not much we can do here as bug reports against 24.0 are futile as nvk is still experimental. So you should wait until 24.1 in case you run into any issues and file bugs against mesa there.
@karolherbst yea, it kind does work. Turns out, I just had to start darktable from the very same terminal window where I set the rusticl environment variable. However, it seems to crash often and sometimes the preview is strange. When it crashes, I get outputs like this in the terminal:
[anna@archlinux ~]$ darktable
SPIR-V parsing FAILED:
In file ../mesa/src/compiler/spirv/spirv_to_nir.c:369
Type mismatch for SPIR-V value %622
23268 bytes into the SPIR-V binary
Trace/breakpoint trap (core dumped)
Well and it seems that it works better on X11, apparently the strange behavior is more frequent on Wayland/Xwayland.
Performance-wise it's comparable to the proprietary Nvidia driver, though I didn't measure with darktable -d perf.
Edit: as far as the preview is concerned, a frequent issue is the fact that it just turns black when I move a slider or switch on a module. If I then change the zoom level or hide/unhide a panel, it comes back. Pretty annoying. Well in theory it works but not really usable...
@karolherbst yea, it kind does work. Turns out, I just had to start darktable from the very same terminal window where I set the rusticl environment variable. However, it seems to crash often and sometimes the preview is strange. When it crashes, I get outputs like this in the terminal:
[anna@archlinux ~]$ darktable SPIR-V parsing FAILED: In file ../mesa/src/compiler/spirv/spirv_to_nir.c:369 Type mismatch for SPIR-V value %622 23268 bytes into the SPIR-V binary Trace/breakpoint trap (core dumped)Well and it seems that it works better on X11, apparently the strange behavior is more frequent on Wayland/Xwayland.
Performance-wise it's comparable to the proprietary Nvidia driver, though I didn't measure with darktable -d perf.
Edit: as far as the preview is concerned, a frequent issue is the fact that it just turns black when I move a slider or switch on a module. If I then change the zoom level or hide/unhide a panel, it comes back. Pretty annoying. Well in theory it works but not really usable...
yeah.. I doubt it would work on any version before 24.1 as zink is still kinda experimental with rusticl. But anyway, you should probably file bugs against mesa running into any issues with rusticl.
This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@jenshannoschwalm re tested about 2 or 3 weeks ago. Seems there is progress. My impression was that the strange behaviour/previews are gone, but there are still quite a few crashes when newer scene referred modules are used, such as diffuse/sharpen or color balance rgb.
This is on master or 4.8.1 ?
Maybe worth to have a fresh -d pipe -d opencl' log but no haste ...
@jenshannoschwalm I guess the version that's in the arch repo, so probably 4.8 or 4.8.1.
As you seem to be testing rusticl from time to time i would be glad if you could report if you find it to be stable, ok?
@jenshannoschwalm ok. Don't forget to remind me though. Rightnow I have some newphotos and I guess I could test again. But in general a lot of things are going on right now in my life so I might need some reminders.
@jenshannoschwalm I just tried to use the Appimage dev build with Rusticl and I get this:
╭─anna@anna in ~/Downloads as 🧙 took 16s
╰─λ ./Darktable-4.9.0.AppImage
Gtk-Message: 08:31:59.559: Failed to load module "window-decorations-gtk-module"
Gtk-Message: 08:31:59.560: Failed to load module "colorreload-gtk-module"
MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)
Would it make sense to compile dt 4.9?
you might have to set VK_ICD_FILENAMES and point it to wherever nouveau_icd.x86_64.json is installed (e.g. VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nouveau_icd.x86_64.json. Device selection with zink atm is quite janky and I'll need to rework it.
Also this all really depends on what version of mesa is getting used, and for nouveau/nvk specifically it's usually best to try with the latest as it's still in heavy development. Best to use 24.2 or newer.
I've done some CL CTS testing recently and there still a few bugs to figure out and darktable seems to work fine with recent mesa main on my Turing GPU, but random filters can still be buggy I guess.
@karolherbst I think this is an issue with dt 4.9 (dev), since 4.8.x can use zink/vulkan, at least many modules.
@jenshannoschwalm Do I need an amd gpu for that? Because I don't have one. I think I need precise instructions for that.
Ok, I thought you have one.
I would love to make rusticl the default for AMD in darktable if it is stable...
This issue has been marked as stale due to inactivity for the last 60 days. It will be automatically closed in 300 days if no update occurs. Please check if the master branch has fixed it and report again or close the issue.
Ok, I thought you have one.
I would love to make rusticl the default for AMD in darktable if it is stable...
It seems to work fine with Darktable-4.9.0+1271.g0acaaab617-x86_64.AppImage on my RX 6950 XT, although it did need RUSTICL_ENABLE=radeonsi for now. I would personally also be in favour of making rusticl the default, at least for AMD going forward :) I did some testing on RAW and JPEG images and found no problems whatsoever.
Ok, I thought you have one. I would love to make rusticl the default for AMD in darktable if it is stable...
It seems to work fine with
Darktable-4.9.0+1271.g0acaaab617-x86_64.AppImageon my RX 6950 XT, although it did needRUSTICL_ENABLE=radeonsifor now. I would personally also be in favour of making rusticl the default, at least for AMD going forward :) I did some testing on RAW and JPEG images and found no problems whatsoever.
fyi, it's up to the drivers maintainers to opt in having it enabled by default. So far it's only enabled by default on asahi Linux (apple silicon M1/M2) systems and the asahi maintainers are officially supporting it.
Ok, I thought you have one. I would love to make rusticl the default for AMD in darktable if it is stable...
It seems to work fine with
Darktable-4.9.0+1271.g0acaaab617-x86_64.AppImageon my RX 6950 XT, although it did needRUSTICL_ENABLE=radeonsifor now. I would personally also be in favour of making rusticl the default, at least for AMD going forward :) I did some testing on RAW and JPEG images and found no problems whatsoever.fyi, it's up to the drivers maintainers to opt in having it enabled by default. So far it's only enabled by default on asahi Linux (apple silicon M1/M2) systems and the asahi maintainers are officially supporting it.
Yup, I'm aware of that. I thought enabling it by default on Darktable's side s.t. it would be preferred over other OpenCL drivers when available.
seems to work ok now with nvidia hardware, no strange behavior or crashed, the only issue is that it seems a bit slow
Arch Linux, dt 5.0 appimage
FYI: Clover has officially been deprecated and will most likely be removed in Mesa 25.2 due out by Q3-2025. I think now would be as good of a time to prefer Rusticl by default as any other :) Otherwise Darktable risks losing out-of-the box OpenCL support due to its rather slow release cycle.