[28.0.0-beta1] nvidia prime offloading breaks preview on EGL backend

[Francexi]

Operating System Info

Other

Other OS

Slackware-current

OBS Studio Version

28.0.0-beta1

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/ahBcfAE2IjlY3Le1

OBS Studio Crash Log URL

No response

Expected Behavior

You expect the Preview to be correctly rendered adn showed.

Current Behavior

Preview does fails to be rendered. Same does the interact window of browser sources. Browser panels, thought, works correctly

Steps to Reproduce

1. Start OBS with PRIME render offloading

Anything else we should know?

By the log, the error is the following: 20:11:58.173: Cannot get window EGL surface: EGL_BAD_NATIVE_WINDOW 20:11:58.173: gl_platform_init_swapchain failed 20:11:58.173: obs_display_init: Failed to create swap chain

the command line used to run OBS is the following: __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia obs --ignore-certificate-errors

the error appear even if you remove the __GLX_VENDOR_LIBRARY_NAME option

the following is the list of shared library module loaded obtained via gdb: https://pastebin.com/0MejUSP6

The preview runs correctly when running OBS without offloading (so, on integrated GPU).

Here is the output of eglinfo: https://pastebin.com/PQSEzfjP

I also noticed that another user on Discord had the same problem, but on Windows, so probably is not a O.S. related issue.

[Francexi]

seems like the failure happen at line 424 of gl-x11-egl.c

eglCreateWindowSurface(plat->edisplay, plat->config, wid, 0);

for some reason this fails, but i have no idea how to debug this

[Francexi]

another piece of information:

xcb_create_window(xcb_conn, 24 /* Hardcoded? */, wid, parent, 0, 0, geometry->width, geometry->height, 0, 0, visual, mask, mask_values);

this fails returing the following error

{response_type = 0 '\000', error_code = 8 '\b', sequence = 2372, resource_id = 146800655, minor_code = 0, major_code = 1 '\001', pad0 = 0 '\000', pad = {0, 0, 0, 0, 0}, full_sequence = 2372}

googling around seems like error_code 8 is BAD_MATCH error. I couldn't find any other useful information

since that fails, the following eglCreateWindowSurface in turn fails because wid is not linked to any window

EDIT: seems like this happens also in 27.2.4 when OBS_USE_EGL is set. So there is something wrong with EGL.

[Francexi]

I found the probable root cause.

NVIDIA PRIME configuration does not support EGL_NATIVE_RENDERABLE flag.

EGL needs that flag to create the surface, which currently NVIDIA drivers for headless configurations doesn't support. I strongly suggest to offer GLX for those configuration until a better alterative come out

[kkartaltepe]

The new and corrected error in rc1 is

Swapchain window creation failed: BadMatch (invalid parameter attributes), Major opcode:1, Minor opcode: 0

[kkartaltepe]

Please provide the full output of eglinfo with and without prime offloading please. Please also provide the output of xwininfo, from the x11-utils package on ubuntu, when following these instructions:

• Start obs with offloading __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia obs, and observe the broken preview
• Add a source and open the source settings, confirm this preview is also broken.
• Run xwininfo -tree and click on the source settings window
• From the output you will see a list of windows like

xwininfo: Window id: 0x3e00eba "Properties for 'Video Capture Device (V4L2)'"

  Root window id: 0x1e5 (the root window) "i3"
  Parent window id: 0x4b0f3d "[i3 con] container around 0x55d5d4c6fec0"
     1 child:
     0x3e00ebc "obs": ("obs" "obs")  1021x980+12+12  +312+139
        1 child:
        0x3e00ec7 (has no name): ()  1021x980+0+0  +312+139

• run xwininfo -id 0x3e00ec7 using the id from the child that has no name.
• Provide the full output from this.

Then do the same steps for without offloading, and with offloading on the old release with glx, where we see previews working correctly.

[kkartaltepe]

If you can test the linked PR and report if it fixes the issue for you that would be great.

[Francexi]

I am not able to test this out, since I resolved the problem some time ago with a local configuration workaround (I inverted the Reverse PRIME configuration of X11 server). I hope someone else is able to do it