obs-vkcapture icon indicating copy to clipboard operation
obs-vkcapture copied to clipboard

obs-glcapture doesn't work on the proprietary nvidia driver

Open dariasc opened this issue 2 years ago • 39 comments

Running obs-glcapture glxgears on the proprietary driver leads to the following output:

Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
[obs-vkcapture] Init GLX 1.0.0
[obs-vkcapture] Texture GL_RGBA 841x444
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
X connection to :0 broken (explicit kill or server shutdown).
X connection to :0 broken (explicit kill or server shutdown).

I tried following the debug steps outlined in #16 but running with that patch led to a segmentation fault, not sure why.

The output for xdpyinfo | grep DRI is only DRI2 after a bit of trying I was unable to enable DRI3 on my system (i think it's because the driver doesn't support it).

I am also unable to capture egl, running obs-glcapture eglgears_x11 leads to a segmentation fault.

Not sure what other information is needed.

And thanks for writing an awesome piece of software :D.

dariasc avatar Sep 17 '21 01:09 dariasc

I have completely misunderstood something, I thought the VK_EXT_external_memory_dma_buf limitation only applied when using obs-vkcapture, I am guessing this missing extension is related to the issue I am having, will close this issue.

I hope I can use this in the future whenever nvidia adds support :/.

dariasc avatar Sep 17 '21 02:09 dariasc

Yes, on Vulkan it requires VK_EXT_external_memory_dma_buf, on GLX it requires DRI3 and on EGL it requires EGL_MESA_image_dma_buf_export (it should probably query this extension). But it only works on opensource drivers currently.

Nvidia may be adding support for this with next beta driver as they said they have Wayland working.

nowrep avatar Sep 17 '21 06:09 nowrep

@nowrep they have implemented GBM and therefore have added DMA-BUF. But they have done it some other way because the driver still doesn't report VK_EXT_external_memory_dma_buf (but they do support EGL_MESA_image_dma_buf_export now):

GBM platform:
EGL API version: 1.5
EGL vendor string: NVIDIA
EGL version string: 1.5
EGL client APIs: OpenGL_ES OpenGL
EGL extensions string:
    EGL_EXT_image_dma_buf_import_modifiers EGL_MESA_image_dma_buf_export

(there's a ton more extensions in the GBM platform section but that's the important one).

vulkaninfo doesn't report VK_EXT_external_memory_dma_buf though. But obviously they have DMA-BUF support (they have to have), so is there any other way to hook vkcapture into the driver or whatever?

Also obs-glcapture still doesn't work even with that extension supported, eglgears_x11 segfaults w/ obs-glcapture

gardotd426 avatar Oct 28 '21 06:10 gardotd426

so is there any other way to hook vkcapture into the driver or whatever?

No, ask (or wait for) nvidia to add this extension. It may be possible to use gbm directly, but I'm not doing that.

Also obs-glcapture still doesn't work even with that extension supported, eglgears_x11 segfaults w/ obs-glcapture

Can you please get backtrace?

nowrep avatar Oct 28 '21 06:10 nowrep

Here's strace obs-glcapture eglgears_x11 2>&1 | tee capture.txt:

capture.txt

Here's the systemd coredump:

core.eglgears_x11.1000.bfdd815c203d48bda24e785a57fbc61a.104740.1635403592000000.zst.gz

gardotd426 avatar Oct 28 '21 07:10 gardotd426

Please do coredumpctl gdb on the core (it's useless outside of your system), then type bt and enter.

nowrep avatar Oct 28 '21 07:10 nowrep

warning: Section `.reg-xstate/104740' in core file too small.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `eglgears_x11'.
Program terminated with signal SIGSEGV, Segmentation fault.

warning: Section `.reg-xstate/104740' in core file too small.
#0  0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007f588a26885d in ?? () from /usr/lib/libEGL_nvidia.so.0
#2  0x00007f588a24c908 in ?? () from /usr/lib/libEGL_nvidia.so.0
#3  0x00007f588a24cdef in ?? () from /usr/lib/libEGL_nvidia.so.0
#4  0x00007f588a19ac27 in __pthread_once_slow () from /usr/lib/libpthread.so.0
#5  0x00007f588a24cb73 in ?? () from /usr/lib/libEGL_nvidia.so.0
#6  0x00007f588a24e805 in ?? () from /usr/lib/libEGL_nvidia.so.0
#7  0x00007f588a1de593 in ?? () from /usr/lib/libEGL_nvidia.so.0
#8  0x00007f588a1f8ebd in ?? () from /usr/lib/libEGL_nvidia.so.0
#9  0x00007f588a7f8def in ?? () from /usr/lib/libEGL.so.1
#10 0x00007f588a7f953e in ?? () from /usr/lib/libEGL.so.1
#11 0x00007f588a7f1e2d in ?? () from /usr/lib/libEGL.so.1
#12 0x00007f588a7f495d in eglGetDisplay () from /usr/lib/libEGL.so.1
#13 0x000055a0692f4487 in ?? ()
#14 0x000055a0692f3052 in ?? ()
#15 0x00007f588a649b25 in __libc_start_main () from /usr/lib/libc.so.6
#16 0x000055a0692f322e in ?? ()

gardotd426 avatar Oct 28 '21 07:10 gardotd426

Is that with obs-vkcapture debug build?

I would say it just incorrectly resolves EGL functions, but it seems to crash too deep inside the driver for it to be the case. Unfortunately I have no idea, and no nvidia card to test myself.

nowrep avatar Oct 28 '21 07:10 nowrep

Is that with obs-vkcapture debug build?

I don't see anything about building a debug build of obs-vkcapture in any of the source code. It's just a standard build (from the AUR).

I can easily compile it from source but idk how to get a debug build of obs-vkcapture

gardotd426 avatar Oct 28 '21 09:10 gardotd426

This would be a debug build

diff --git a/dist/PKGBUILD b/dist/PKGBUILD
index a5d7301..e7487bf 100644
--- a/dist/PKGBUILD
+++ b/dist/PKGBUILD
@@ -10,6 +10,7 @@ depends=('vulkan-icd-loader' 'obs-studio>=27')
 makedepends=('git' 'gcc' 'cmake' 'vulkan-headers' 'lib32-gcc-libs' 'lib32-vulkan-icd-loader')
 source=("$_pkgbase::git+$url")
 sha512sums=('SKIP')
+options=(!strip)
 
 pkgver() {
     cd "$_pkgbase"
@@ -20,7 +21,7 @@ build() {
     cmake -B build -S "$_pkgbase" -DCMAKE_INSTALL_PREFIX=/usr \
         -DCMAKE_INSTALL_LIBDIR=lib \
         -DCMAKE_INSTALL_DATADIR=/share \
-        -DCMAKE_BUILD_TYPE=Release
+        -DCMAKE_BUILD_TYPE=Debug
     make -C build
 
     export CFLAGS="-m32 ${CFLAGS}"
@@ -31,7 +32,7 @@ build() {
         -DCMAKE_INSTALL_LIBDIR=lib32 \
         -DCMAKE_INSTALL_DATADIR=/share \
         -DCMAKE_LIBRARY_PATH=/usr/lib32 \
-        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_BUILD_TYPE=Debug \
         -DBUILD_PLUGIN=OFF
     make -C build32
 }

nowrep avatar Oct 28 '21 10:10 nowrep

It's not obs-vkcapture that needs a debug build:

Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/eglgears_x11...
(No debugging symbols found in /usr/bin/eglgears_x11)
[New LWP 104740]

(this is the same both with and without a debug build of obs-vkcapture)

It was mesa-demos that needed it (though obviously I kept the debug build of obs-vkcapture), so I went ahead and built it with debug symbols and now coredumpctl gdb ###### shows:

Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/eglgears_x11...
[New LWP 334388]

Anyhow, with debug symbols:

(gdb) bt\
#0  0x0000000000000000 in ?? ()
#1  0x00007f85df50f85d in ?? () from /usr/lib/libEGL_nvidia.so.0
#2  0x00007f85df4f3908 in ?? () from /usr/lib/libEGL_nvidia.so.0
#3  0x00007f85df4f3def in ?? () from /usr/lib/libEGL_nvidia.so.0
#4  0x00007f85df441c27 in __pthread_once_slow () from /usr/lib/libpthread.so.0
#5  0x00007f85df4f3b73 in ?? () from /usr/lib/libEGL_nvidia.so.0
#6  0x00007f85df4f5805 in ?? () from /usr/lib/libEGL_nvidia.so.0
#7  0x00007f85df485593 in ?? () from /usr/lib/libEGL_nvidia.so.0
#8  0x00007f85df49febd in ?? () from /usr/lib/libEGL_nvidia.so.0
#9  0x00007f85dfa9fdef in ?? () from /usr/lib/libEGL.so.1
#10 0x00007f85dfaa053e in ?? () from /usr/lib/libEGL.so.1
#11 0x00007f85dfa98e2d in ?? () from /usr/lib/libEGL.so.1
#12 0x00007f85dfa9b95d in eglGetDisplay () from /usr/lib/libEGL.so.1
#13 0x0000558251f2f8e8 in eglutInit (argc=1, argv=0x7ffd3c1cfbc8) at eglut.c:215
#14 0x0000558251f2f095 in main (argc=1, argv=0x7ffd3c1cfbc8) at eglgears.c:287

Unfortunately looks the same

gardotd426 avatar Oct 28 '21 10:10 gardotd426

Hey @nowrep!

It seems that recently NVIDIA has enabled DRI3 on their proprietary driver. I am on driver 495.44, though not sure when this was activated.

xdpyinfo | grep DRI
    DRI2
    DRI3

But when I go and run it, it fails at the Failed to get buffer from pixmap with error code 4, same as #16. Yet I don't actually run XWayland as described in the issue, any idea what might be causing this?

dariasc avatar Dec 11 '21 19:12 dariasc

That would seem the EGL and GLX stuff necessary is there, but yeah vulkan is still a no-go as it doesn't have the required extension

gardotd426 avatar Dec 11 '21 19:12 gardotd426

Did you have DRI3 when doing your testing last month?

dariasc avatar Dec 11 '21 19:12 dariasc

Well I've had this driver (495.44) installed since then. It came out Oct. 26. So it's the same driver. But I mean, there shouldn't be anything stopping obs-glcapture now.

And oh shit, it works now! Screenshot_20211211_150948

I didn't have to do anything, I mean I already had OBS_USE_EGL=1 in my /etc/environment but I don't think that's necessary (I'm using X11 I don't even think that does anything on X11).

I can't get it to do anything other than launch glxgears and capture that (idk how to get it to attach to anything already running), but it's working!

gardotd426 avatar Dec 11 '21 20:12 gardotd426

You only need OBS_USE_EGL when you are on X11, it's not needed on Wayland. You also have to run any OpenGL application with obs-glcapture, it can't attach to running process.

This can be now closed again, as it does work with nvidia proprietary driver.

nowrep avatar Dec 11 '21 21:12 nowrep

Nice!

But it's not working on my end :( I'm having that Failed to get buffer from pixmap error, any pointers on how I could find out what's happening?

Thanks in advance.

dariasc avatar Dec 11 '21 21:12 dariasc

I actually can't get it to do anything other than glxgears, I just tried lutris, I ran obs-glcapture lutris from the command line, and it launched, and seemed to initialize properly:

obs-glcapture lutris
[obs-vkcapture] Init GLX 1.0.0
2021-12-11 16:16:14,323: Starting Lutris 0.5.9.1
2021-12-11 16:16:15,800: Startup complete
2021-12-11 16:16:15,800: Using NVIDIA drivers 495.44 for x86_64
2021-12-11 16:16:15,800: GPU: Unknown
2021-12-11 16:16:15,801: GPU: 10DE:2204 3842:3975 (nvidia drivers)
[obs-vkcapture] Init Vulkan 1.0.0

But if I actually go into obs, and try to add a GL Source, it does nothing. I can only start a program to capture from the plugin, and the only thing that works so far is eglgears_x11 and glxgears (and like I said, I have to start them from the plugin, using the terminal doesn't work).

gardotd426 avatar Dec 11 '21 21:12 gardotd426

Shouldn't it be a Game Capture source, not a GL Source? I have a feeling that GL Source is from another gl capture obs plugin

dariasc avatar Dec 11 '21 21:12 dariasc

Well if that's it, lol then yeah it doesn't work for me either.

When I try to run obs-glcapture glxgears I get the same pixmap error as you, when I try obs-glcapture eglgears_x11 I get a segfault. But when I try obs-glcapture lutris, like I said, it loads without error (but OBS doesn't pick it up):

obs-glcapture lutris      
[obs-vkcapture] Init GLX 1.0.0
2021-12-11 16:34:43,181: Starting Lutris 0.5.9.1
2021-12-11 16:34:44,632: Failed to read content length on response from https://api.github.com/repos/lutris/vkd3d/releases
2021-12-11 16:34:44,824: Startup complete
2021-12-11 16:34:44,825: Using NVIDIA drivers 495.44 for x86_64
2021-12-11 16:34:44,825: GPU: Unknown
2021-12-11 16:34:44,825: GPU: 10DE:2204 3842:3975 (nvidia drivers)
[obs-vkcapture] Init Vulkan 1.0.0

gardotd426 avatar Dec 11 '21 21:12 gardotd426

I'm having that Failed to get buffer from pixmap error, any pointers on how I could find out what's happening?

There's not an easy way to find out what's happening as it's all in the nvidia driver.

But when I try obs-glcapture lutris, like I said, it loads without error (but OBS doesn't pick it up):

You shouldn't run lutris with glcapture. You should only use it for wine/proton, just like you would use mangohud. You can try to run eg. supertuxkart or other opengl native games.

nowrep avatar Dec 12 '21 15:12 nowrep

This isn't working for me with the proprietary driver. I have all the apparent pre-requesites (DRI3, EGL etc) but it's crashing exactly as others note in this thread. a GDB backtrace shows very little:


bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff74aa85d in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#2  0x00007ffff748e908 in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#3  0x00007ffff748edef in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#4  0x00007ffff73dc610 in __pthread_once_slow (once_control=0x7ffff7733468, init_routine=0x7ffff748edd0) at pthread_once.c:116
        __cancel_routine = 0x7ffff73dc670 <clear_once_control>
        __clframe = {
          __cancel_routine = 0x7ffff73dc670 <clear_once_control>,
          __cancel_arg = 0x7ffff7733468,
          __do_it = 1,
          __buffer = {
            __routine = 0x7ffff73dc6b0 <__pthread_cleanup_combined_routine_voidptr>,
            __arg = 0x7fffffffdb30,
            __canceltype = 0,
            __prev = 0x0
          }
        }
        val = <optimized out>
        newval = <optimized out>
#5  0x00007ffff748eb73 in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#6  0x00007ffff7490805 in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#7  0x00007ffff7420593 in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#8  0x00007ffff743aebd in ?? () from /lib/x86_64-linux-gnu/libEGL_nvidia.so.0
No symbol table info available.
#9  0x00007ffff7ce1abd in ?? () from /lib/x86_64-linux-gnu/libEGL.so.1
No symbol table info available.
#10 0x00007ffff7ce220a in ?? () from /lib/x86_64-linux-gnu/libEGL.so.1
No symbol table info available.
#11 0x00007ffff7ce377d in ?? () from /lib/x86_64-linux-gnu/libEGL.so.1
No symbol table info available.
#12 0x00007ffff7ce23df in ?? () from /lib/x86_64-linux-gnu/libEGL.so.1
No symbol table info available.
#13 0x00007ffff7cdf68c in eglGetDisplay () from /lib/x86_64-linux-gnu/libEGL.so.1
No symbol table info available.
#14 0x0000555555558334 in ?? ()
No symbol table info available.
#15 0x000055555555644f in ?? ()
No symbol table info available.
#16 0x00007ffff7b387ed in __libc_start_main (main=0x555555556420, argc=1, argv=0x7fffffffdf58, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdf48) at ../csu/libc-start.c:332
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {
          cancel_jmp_buf = {[0] = {
              jmp_buf = {[0] = 93824992251440, [1] = 2128271466033075961, [2] = 93824992240800, [3] = 0, [4] = 0, [5] = 0, [6] = -2128271464944609543, [7] = -2128288407258114311},
              mask_was_saved = 0
            }},
          priv = {
            pad = {[0] = 0x0, [1] = 0x0, [2] = 0x1, [3] = 0x7fffffffdf58},
            data = {
              prev = 0x0,
              cleanup = 0x0,
              canceltype = 1
            }
          }
        }
        not_first_call = <optimized out>
#17 0x00005555555564ca in ?? ()
No symbol table info available.

cpw avatar Jan 12 '22 21:01 cpw

Just chiming in here, I saw that NVIDIA released the 510.39.01 beta driver: https://www.nvidia.com/Download/driverResults.aspx/184911/en-us

Seems like a lot of work on VK_EXT extensions and official GBM API support. Although I don't see the VK_EXT_external_memory_dma_buf extension, but perhaps the GBM API support is actually implemented, seems to be usable with Mesa 21.2 and above (Ubuntu 21.10 is currently on 21.2.2 stable).

I'm definitely keen on trying this plugin out, but looking at the README and this thread, it seemed like I wasn't going to get very far. Has anyone tried this driver yet, perhaps with the GBM loader it might work better with the EGL support in OBS? rampant speculation intensifies

Edit: I just wanted to add, I've noticed with the X Window capture in OBS, it disables G-SYNC for me when playing via Proton. I've always wondered what the minor screen tearing in my recorded video was; and I'm pretty sure it's to do with the fact G-SYNC is being disabled for me when running OBS on Linux. I verified this by enabling the visual indicator for the NVIDIA driver, as soon as I close OBS the G-SYNC is indicator shows it is enabled. I was hoping that this plugin might function in a way that doesn't break G-SYNC.

berglh avatar Jan 26 '22 04:01 berglh

Seems like a lot of work on VK_EXT extensions and official GBM API support

Sorry to disappoint you, but literally the entire GBM section is copy and pasted from two whole driver versions ago (the very first beta 495 driver).

Release notes for 495.44:

Added support for the GBM API. This adds the new symlink nvidia-drm_gbm.so pointing to the file libnvidia-allocator.so.VERSION to implement a GBM backend driver usable with the GBM loader from the Mesa project version 21.2 and above, as well as the files libnvidia-egl-gbm.so.1.1.0 and 15_nvidia_gbm.json, which implement EGL support for the GBM platform (EGL_KHR_platform_gbm).

Release notes from 495.46:

Added support for the GBM API. This adds the new symlink nvidia-drm_gbm.so pointing to the file libnvidia-allocator.so.VERSION to implement a GBM backend driver usable with the GBM loader from the Mesa project version 21.2 and above, as well as the files libnvidia-egl-gbm.so.1.1.0 and 15_nvidia_gbm.json, which implement EGL support for the GBM platform (EGL_KHR_platform_gbm).

Release notes from the 510 driver you're referencing:

Added support for the GBM API. This adds the new symlink nvidia-drm_gbm.so pointing to the file libnvidia-allocator.so.VERSION to implement a GBM backend driver usable with the GBM loader from the Mesa project version 21.2 and above, as well as the files libnvidia-egl-gbm.so.1.1.0 and 15_nvidia_gbm.json, which implement EGL support for the GBM platform (EGL_KHR_platform_gbm).

As I said, it's just copy and pasted. GBM support came to the Nvidia driver in 495.44 back in October. Each Wayland Compositor/Window Manager had to enable it/make it default for Nvidia, but both KDE and GNOME have all already done that, also months ago.

Obviously the release notes referencing GBM for some reason keep accidentally getting copied and pasted over (or rather maybe aren't being removed from some template) for subsequent driver releases. There's nothing else referencing GBM other than the literal exact same paragraph that was in 495.46 and 495.44, so I don't believe any additional work has been done on GBM, and GBM has been usable (and default) for Nvidia users on Plasma and GNOME for a while now.

EDIT: Just to illustrate, opening up the release notes for all three releases and pasting the entire paragraph into the CTRL+F find window:

Screenshot_20220125_232329

gardotd426 avatar Jan 26 '22 04:01 gardotd426

@gardotd426 Of course, you're right. I made the mistake of assuming this article was correct in claiming refined GBM suppprt

NVIDIA's 510 Linux beta also has a ReBAR indicator, updated Linux kernel support, refined GBM API support, and other updates.

Off topic, I'm still experiencing issues with my display flickering on GPU accelerated applications/windows coming into the foreground with flipping option enabled, they keep showing regression fixes here (this occurred in older kernel version and was subsequently fixed), nevertheless it's still misbehaving. There are a few items still preventing me from going to Wayland also, mainly to do with barrier.

Either way, I'd be still interested to confirm if this issue is still present in the beta driver, but it's not sounding promising as you point out. I will probably give this driver a go to see if my screen flicker issue is still present, but I'm not liking my chances on that either. If I do give it a go, I'll try to confirm the bug persists with this plugin, which seems likely on X11.

berglh avatar Jan 26 '22 05:01 berglh

Update Tried 510.39.01 using the NVIDIA installer, made obs-vkcapture and receiving the same errors as OP when running glxgears in X11.

berglh avatar Jan 26 '22 07:01 berglh

Trying to run Open Noteblock Studio through Wine through Bottles through Flatpak with NVIDIA 510.54, with the latest stable everything from flatpak Screenshot_20220414_220349 I was able to launch the exe with obs-glcapture (I see that vkcapture doesn't work on nvidia so I'm trying with wine's builtin directx-to-opengl instead of dxvk), and it does show up in obs Game Capture thing Screenshot_20220414_220623 and it doesn't crash, but nothing actually appears in obs

4321ba avatar Apr 14 '22 20:04 4321ba

Can someone please test with the latest driver?

nowrep avatar May 12 '22 07:05 nowrep

Vulkan capture now works with 515.43.04 driver, opengl capture still doesn't work properly, but eglgears_x11 is capturing ok(since 495 driver i guess).

qwert0p avatar May 12 '22 13:05 qwert0p

i'm on 515.43.04 and obs seems to crash if I run it with OBS_USE_EGL=1 obs and doesn't start. but obs-glcapture glxgears and obs-glcapture eglgears_x11 works with no problems

mohad12211 avatar May 14 '22 17:05 mohad12211

I'm on 515.43.04 as well and I have the same behavior as @qwert0p. vkcube works nice, eglgears_x11 also works fine, but glxgears gives

[obs-vkcapture] Texture GL_RGBA 300x300
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed

Cherenkov11 avatar May 14 '22 21:05 Cherenkov11

Can you try this patch?

diff --git a/src/glinject.c b/src/glinject.c
index d4339ac..0166fa4 100644
--- a/src/glinject.c
+++ b/src/glinject.c
@@ -144,6 +144,9 @@ static bool gl_init_funcs(bool glx)
             hlog("Failed to open libxcb-dri3.so.0");
             return false;
         }
+        GETXADDR(xcb_dri3_buffer_from_pixmap);
+        GETXADDR(xcb_dri3_buffer_from_pixmap_reply);
+        GETXADDR(xcb_dri3_buffer_from_pixmap_reply_fds);
         GETXADDR(xcb_dri3_buffers_from_pixmap);
         GETXADDR(xcb_dri3_buffers_from_pixmap_reply);
         GETXADDR(xcb_dri3_buffers_from_pixmap_reply_fds);
@@ -326,19 +329,34 @@ static bool gl_shtex_init()
         void *xcb_con = x11_f.XGetXCBConnection(data.display);
         P_xcb_dri3_buffers_from_pixmap_cookie_t cookie = x11_f.xcb_dri3_buffers_from_pixmap(xcb_con, data.xpixmap);
         P_xcb_dri3_buffers_from_pixmap_reply_t *reply = x11_f.xcb_dri3_buffers_from_pixmap_reply(xcb_con, cookie, NULL);
-        if (!reply) {
-            hlog("Failed to get buffer from pixmap");
-            return false;
-        }
-        data.nfd = reply->nfd;
-        for (uint8_t i = 0; i < reply->nfd; ++i) {
-            data.buf_fds[i] = x11_f.xcb_dri3_buffers_from_pixmap_reply_fds(xcb_con, reply)[i];
-            data.buf_strides[i] = x11_f.xcb_dri3_buffers_from_pixmap_strides(reply)[i];
-            data.buf_offsets[i] = x11_f.xcb_dri3_buffers_from_pixmap_offsets(reply)[i];
+        if (reply) {
+            data.nfd = reply->nfd;
+            for (uint8_t i = 0; i < reply->nfd; ++i) {
+                data.buf_fds[i] = x11_f.xcb_dri3_buffers_from_pixmap_reply_fds(xcb_con, reply)[i];
+                data.buf_strides[i] = x11_f.xcb_dri3_buffers_from_pixmap_strides(reply)[i];
+                data.buf_offsets[i] = x11_f.xcb_dri3_buffers_from_pixmap_offsets(reply)[i];
+            }
+            data.buf_fourcc = DRM_FORMAT_ARGB8888;
+            data.buf_modifier = reply->modifier;
+            free(reply);
+        } else {
+            hlog("try xcb_dri3_buffer_from_pixmap");
+            P_xcb_dri3_buffers_from_pixmap_cookie_t cookie = x11_f.xcb_dri3_buffer_from_pixmap(xcb_con, data.xpixmap);
+            P_xcb_dri3_buffer_from_pixmap_reply_t *reply = x11_f.xcb_dri3_buffer_from_pixmap_reply(xcb_con, cookie, NULL);
+            if (!reply) {
+                hlog("Failed to get buffer from pixmap");
+                return false;
+            }
+            data.nfd = reply->nfd;
+            for (uint8_t i = 0; i < reply->nfd; ++i) {
+                data.buf_fds[i] = x11_f.xcb_dri3_buffer_from_pixmap_reply_fds(xcb_con, reply)[i];
+                data.buf_strides[i] = reply->stride;
+                data.buf_offsets[i] = 0; // ??
+            }
+            data.buf_fourcc = DRM_FORMAT_ARGB8888;
+            data.buf_modifier = DRM_FORMAT_MOD_INVALID;
+            free(reply);
         }
-        data.buf_fourcc = DRM_FORMAT_ARGB8888;
-        data.buf_modifier = reply->modifier;
-        free(reply);
     } else {
         data.image = egl_f.CreateImage(data.display, egl_f.GetCurrentContext(), P_EGL_GL_TEXTURE_2D, data.texture, NULL);
         if (!data.image) {
diff --git a/src/glinject.h b/src/glinject.h
index 07e2c93..137999c 100644
--- a/src/glinject.h
+++ b/src/glinject.h
@@ -126,6 +126,20 @@ typedef struct
 }
 *P_XPrivDisplay;
 
+typedef struct P_xcb_dri3_buffer_from_pixmap_reply_t {
+    uint8_t  response_type;
+    uint8_t  nfd;
+    uint16_t sequence;
+    uint32_t length;
+    uint32_t size;
+    uint16_t width;
+    uint16_t height;
+    uint16_t stride;
+    uint8_t  depth;
+    uint8_t  bpp;
+    uint8_t  pad0[12];
+} P_xcb_dri3_buffer_from_pixmap_reply_t;
+
 typedef struct P_xcb_dri3_buffers_from_pixmap_reply_t {
     uint8_t  response_type;
     uint8_t  nfd;
@@ -150,6 +164,9 @@ struct x11_funcs {
     int (*XFree)(void *data);
 
     void *(*XGetXCBConnection)(void *display);
+    P_xcb_dri3_buffers_from_pixmap_cookie_t (*xcb_dri3_buffer_from_pixmap)(void *c, unsigned long pixmap);
+    P_xcb_dri3_buffer_from_pixmap_reply_t *(*xcb_dri3_buffer_from_pixmap_reply)(void *c, P_xcb_dri3_buffers_from_pixmap_cookie_t cookie, void *error);
+    int *(*xcb_dri3_buffer_from_pixmap_reply_fds)(void *c, P_xcb_dri3_buffer_from_pixmap_reply_t *reply);
     P_xcb_dri3_buffers_from_pixmap_cookie_t (*xcb_dri3_buffers_from_pixmap)(void *c, unsigned long pixmap);
     P_xcb_dri3_buffers_from_pixmap_reply_t *(*xcb_dri3_buffers_from_pixmap_reply)(void *c, P_xcb_dri3_buffers_from_pixmap_cookie_t cookie, void *error);
     int *(*xcb_dri3_buffers_from_pixmap_reply_fds)(void *c, P_xcb_dri3_buffers_from_pixmap_reply_t *reply);


nowrep avatar May 15 '22 05:05 nowrep

I tried the patch above and this is the output I receive from obs-glcapture glxgears

[obs-vkcapture] Texture GL_RGBA 300x300
[obs-vkcapture] try xcb_dri3_buffer_from_pixmap
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed

Cherenkov11 avatar May 15 '22 15:05 Cherenkov11

Thanks for testing.

No idea what's wrong then. Maybe it doesn't like the parameters it creates the GLX pixmap, but hard to say.

nowrep avatar May 16 '22 04:05 nowrep

No idea what's wrong then. Maybe it doesn't like the parameters it creates the GLX pixmap, but hard to say.

I would say that nvidia just didn't made a complete support for dri3 calls, as always we just need to wait until they fix it.

qwert0p avatar May 16 '22 09:05 qwert0p

i'm on 515.43.04 and obs seems to crash if I run it with OBS_USE_EGL=1 obs and doesn't start. but obs-glcapture glxgears and obs-glcapture eglgears_x11 works with no problems

I had the same issue, it was because of some conflicting plugin i had installed, unfortunately i don't know what plugin was conflicting, because i just deleted all i had installed.

qwert0p avatar May 16 '22 09:05 qwert0p

i'm on 515.43.04 and obs seems to crash if I run it with OBS_USE_EGL=1 obs and doesn't start. but obs-glcapture glxgears and obs-glcapture eglgears_x11 works with no problems

I had the same issue, it was because of some conflicting plugin i had installed, unfortunately i don't know what plugin was conflicting, because i just deleted all i had installed.

yeah it was the nvfbc obs plugin. thank you

mohad12211 avatar May 16 '22 10:05 mohad12211

is there any chance this could be made to work on DRI2? I am on nvidia proprietary jetson drivers where we don't have the luxury of frequent driver updates

theofficialgman avatar Jul 20 '22 03:07 theofficialgman

No, because there is no dmabuf support in dri2. Also, glx capture doesn't currently work with up-to-date nvidia drivers anyway, so you are not missing out :D

nowrep avatar Jul 20 '22 07:07 nowrep