nvidia-vaapi-driver icon indicating copy to clipboard operation
nvidia-vaapi-driver copied to clipboard

[HELP] Prime offloading, can't add modeset=1 and this happens

Open ManuLinares opened this issue 3 years ago • 7 comments

opening firefox and playing a video stdout:

env NVD_LOG=1 MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia /usr/lib/firefox/firefox %u

[GFX1-]: glxtest: VA-API test failed: failed to initialise VAAPI connection.
ATTENTION: default value of option mesa_glthread overridden by environment.
ATTENTION: default value of option mesa_glthread overridden by environment.
ATTENTION: default value of option mesa_glthread overridden by environment.
ATTENTION: default value of option mesa_glthread overridden by environment.
Missing chrome or resource URL: resource://gre/modules/UpdateListener.jsm
Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs
       762.693595332 [5524-5797] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e020 31
       762.693623191 [5524-5797] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.182910504 [5524-5830] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e5c0 31
       763.182924525 [5524-5830] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
[2022-08-02T21:07:08Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
[2022-08-02T21:07:08Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
[2022-08-02T21:07:08Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
[2022-08-02T21:07:08Z ERROR mp4parse] Found 2 nul bytes in "\0\0"
       763.395038400 [5524-5845] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522f2e0 31
       763.395050125 [5524-5845] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.489993395 [5524-5850] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e5c0 31
       763.490013462 [5524-5850] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.531776883 [5524-5857] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522fc40 31
       763.531800528 [5524-5857] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.550963724 [5524-5861] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e5c0 31
       763.550991208 [5524-5861] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.582247032 [5524-5868] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522f2e0 31
       763.582271035 [5524-5868] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       763.584631584 [5524-5872] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522ed40 31
       763.584650985 [5524-5872] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       764.215076037 [5524-5886] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e7a0 31
       764.215160354 [5524-5886] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       991.610334788 [5524-6184] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522fa60 31
       991.610405514 [5524-6184] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed
       991.612651791 [5524-6189] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7f29e522e7a0 31
       991.612662894 [5524-6189] ../src/vabackend.c:1795       __vaDriverInit_1_0 Invalid driver for DRM device: i915
libva error: /usr/lib/dri/nvidia_drv_video.so init failed

---

I was trying to put nvidia-drm.modeset=1 in /etc/default/grub, or options nvidia_drm modeset=1 in /etc/modprobe.d/nvidia-drm.conf

but when I restart the system, lightdm fails to init.


My system

Manjaro XFCE Using intel display (i915) with nvidia prime offloading (driver video-hybrid-intel-nvidia-prime)

ManuLinares avatar Aug 02 '22 21:08 ManuLinares

edit: I'm an idiot, forgot to add "prime-run" or these environment variables to firefox start.

__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia

So the command to start firefox ended like this:

env NVD_LOG=1 MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia /usr/lib/firefox/firefox

edit 2: Jumped the gun on closing the issue... adding prime-run adds these errors to firefox start

[GFX1-]: Failed to create EGLSurface!: 0x3009
[GFX1-]: Failed to create EGLSurface
[GFX1-]: Handling webrender error 3
[GFX1-]: Fallback WR to SW-WR

ManuLinares avatar Aug 02 '22 21:08 ManuLinares

Can you try setting the MOZ_DRM_DEVICE environment variable to the correct /dev/dri/renderD??? device for your NVIDIA card, and see if that helps?

elFarto avatar Aug 03 '22 07:08 elFarto

Can you try setting the MOZ_DRM_DEVICE environment variable to the correct /dev/dri/renderD??? device for your NVIDIA card, and see if that helps?

I tried setting to /dev/driver/renderD129 that is the nvidia card to no avail.

env MOZ_DRM_DEVICE=/dev/dri/renderD129 NVD_LOG=1 MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia /usr/lib/firefox/firefox %u

[GFX1-]: glxtest: VA-API test failed: failed to initialise VAAPI connection.
ATTENTION: default value of option mesa_glthread overridden by environment.
ATTENTION: default value of option mesa_glthread overridden by environment.
ATTENTION: default value of option mesa_glthread overridden by environment.
Missing chrome or resource URL: resource://gre/modules/UpdateListener.jsm
Missing chrome or resource URL: resource://gre/modules/UpdateListener.sys.mjs
    262910.283498075 [135031-135404] ../src/vabackend.c:1780       __vaDriverInit_1_0 Initialising NVIDIA VA-API Driver: 0x7ff809a2e5c0 31
    262910.283512281 [135031-135404] ../src/vabackend.c:1801       __vaDriverInit_1_0 Now have 0 (0 max) instances
    262910.289282231 [135031-135404] ../src/export-buf.c: 159       findGPUIndexFromFd Looking for DRM device index: 1
    262910.291076168 [135031-135404] ../src/export-buf.c: 172       findGPUIndexFromFd Found 4 EGL devices
    262910.291220145 [135031-135404] ../src/export-buf.c: 181       findGPUIndexFromFd Got EGL_CUDA_DEVICE_NV value '0' for EGLDevice 0
    262910.291228503 [135031-135404] ../src/export-buf.c: 187       findGPUIndexFromFd Found drmDeviceIndex: 1
    262910.291244276 [135031-135404] ../src/export-buf.c: 129 checkModesetParameterFromFd ERROR: This driver requires the nvidia_drm.modeset kernel module parameter set to 1
    262910.291250677 [135031-135404] ../src/export-buf.c: 209       findGPUIndexFromFd No EGL_CUDA_DEVICE_NV support for EGLDevice 1
    262910.291254662 [135031-135404] ../src/export-buf.c: 209       findGPUIndexFromFd No EGL_CUDA_DEVICE_NV support for EGLDevice 2
    262910.291258084 [135031-135404] ../src/export-buf.c: 212       findGPUIndexFromFd No DRM device file for EGLDevice 3
    262910.291259567 [135031-135404] ../src/export-buf.c: 215       findGPUIndexFromFd No match found, falling back to default device
    262910.377078574 [135031-135404] ../src/export-buf.c: 267             initExporter Driver doesn't support 16-bit surfaces
    262910.377094159 [135031-135404] ../src/export-buf.c:  99                reconnect Reconnecting to stream
    262910.377282012 [135031-135404] ../src/vabackend.c:1129      nvQueryImageFormats In nvQueryImageFormats
    262910.396494770 [135031-135404] ../src/vabackend.c: 557           nvCreateConfig got profile: 6 with 0 attributes
    262910.396577531 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 (nil) 0
    262910.396597821 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 0x7ff7b89afc80 5
    262910.398197422 [135031-135404] ../src/vabackend.c: 557           nvCreateConfig got profile: 7 with 0 attributes
    262910.398248309 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 (nil) 0
    262910.398266439 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 0x7ff7b89af280 5
    262910.399822221 [135031-135404] ../src/vabackend.c: 557           nvCreateConfig got profile: 13 with 0 attributes
    262910.399869840 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 (nil) 0
    262910.399887619 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 4 0x7ff7b89afc80 5
    262910.401254303 [135031-135404] ../src/vabackend.c: 557           nvCreateConfig got profile: 19 with 0 attributes
    262910.401292954 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 10 (nil) 0
    262910.401310293 [135031-135404] ../src/vabackend.c:1497 nvQuerySurfaceAttributes with 10 0x7ff7b89afc80 5
    262910.403137264 [135031-135404] ../src/vabackend.c:1757              nvTerminate Terminating 0x7ff809a2e5c0
    262910.403146843 [135031-135404] ../src/export-buf.c:  65          releaseExporter Releasing exporter, 0 outstanding frames
    262910.404213388 [135031-135404] ../src/export-buf.c:  82          releaseExporter Done releasing frames
    262910.439717591 [135031-135404] ../src/vabackend.c:1771              nvTerminate Now have 0 (0 max) instances

relevant line: 262910.291244276 [135031-135404] ../src/export-buf.c: 129 checkModesetParameterFromFd ERROR: This driver requires the nvidia_drm.modeset kernel module parameter set to 1

But if I set the kernel parameter nvidia_drm.modeset=1 the system doesn't start. Lightdm fails.

ManuLinares avatar Aug 05 '22 22:08 ManuLinares

You would need to investigate why lightdm fails to start with that parameter set, as this driver requires it set to work.

elFarto avatar Aug 06 '22 07:08 elFarto

I'm getting the same Failed to create EGLSurface!: 0x3009 error along with the other three lines as the OP mentioned in the 2nd post, and I've confirmed that the nvidia drm modeset is set properly, so it can't be the cause. I'm running Xubuntu 20.04 and all the compiling worked as intended with no errors. I can get Kodi working (it appears to use Chromium as a backend, judging by the logs from NVD_LOG=1) but Firefox throws that error for some reason.

FaeAlicia avatar Aug 11 '22 09:08 FaeAlicia

@Yowlen I assume you've also have multiple GPUs?

Firefox is throwing the error because it's attempting to use the wrong EGL implementation. I haven't managed to understand exactly what options are needed to get Firefox to pick the correct DRM device/EGL implementations in these circumstances.

elFarto avatar Aug 15 '22 19:08 elFarto

Correct. I don't know how much it matters, but I'm using an AMD GPU as primary and nvidia as secondary with the proprietary drivers set to offload on demand. Specifically, an R9 290x and GTX 1050 respectively, if it helps.

FaeAlicia avatar Aug 16 '22 01:08 FaeAlicia

Closing issue, as we require modeset=1 set in order to work.

elFarto avatar Dec 07 '22 08:12 elFarto

Closing issue, as we require modeset=1 set in order to work.

I have nvidia-drm.modeset=1 working:

$ sudo cat /sys/module/nvidia_drm/parameters/modeset
Y

I'm trying the new direct backend but this is still an issue.

$ env MOZ_DRM_DEVICE=/dev/dri/renderD129 NVD_LOG=1 NVD_BACKEND=direct MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia firefox

[GFX1-]: glxtest: VA-API test failed: failed to initialise VAAPI connection.
ATTENTION: default value of option mesa_glthread overridden by environment.

or with prime offloading

$ env MOZ_DRM_DEVICE=/dev/dri/renderD129 NVD_LOG=1 NVD_BACKEND=direct MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia firefox

[GFX1-]: Failed to create EGLSurface!: 0x3009
[GFX1-]: Failed to create EGLSurface. 1 renderers, 0 active.
[GFX1-]: Handling webrender error 3
[GFX1-]: Fallback WR to SW-WR

ManuLinares avatar Dec 07 '22 19:12 ManuLinares

Can you paste the output of eglinfo with and without the __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia variables set?

elFarto avatar Dec 07 '22 19:12 elFarto

Can you also try eglinfo and firefox with this set aswell: "__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json"

elFarto avatar Dec 07 '22 19:12 elFarto

no prime

$ env MOZ_DRM_DEVICE=/dev/dri/renderD129 NVD_LOG=1 NVD_BACKEND=direct MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia firefox

[GFX1-]: glxtest: libEGL no display

__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json eglinfo &> eglinfo_noprime_vendorlibraryfilenames.log eglinfo_noprime_vendorlibraryfilenames.log

with prime

$ env __EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json MOZ_DRM_DEVICE=/dev/dri/renderD129 NVD_LOG=1 NVD_BACKEND=direct MOZ_X11_EGL=1 MOZ_DISABLE_RDD_SANDBOX=1 LIBVA_DRIVER_NAME=nvidia __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia firefox

[GFX1-]: Failed to create EGLSurface!: 0x3009
[GFX1-]: Failed to create EGLSurface. 1 renderers, 0 active.
[GFX1-]: Handling webrender error 3
[GFX1-]: Fallback WR to SW-WR

__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia __EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json eglinfo &> eglinfo_withprime_vendorlibraryfilenames.log eglinfo_withprime_vendorlibraryfilenames.log

ManuLinares avatar Dec 07 '22 19:12 ManuLinares

I assume you've got X11 running on the Intel card in your setup?

elFarto avatar Dec 07 '22 21:12 elFarto

Indeed

System:
  Host: think Kernel: 6.0.11-1-cachyos arch: x86_64 bits: 64 Desktop: Xfce
    v: 4.16.1 Distro: EndeavourOS
Graphics:
  Device-1: Intel Xeon E3-1200 v3 Processor Integrated Graphics driver: i915
    v: kernel
  Device-2: NVIDIA GP107 [GeForce GTX 1050 Ti] driver: nvidia v: 525.60.11
  Display: x11 server: X.Org v: 21.1.4 driver: X: loaded: intel,nvidia
    unloaded: modesetting dri: i965 gpu: i915 resolution: 2048x1152~60Hz
  API: OpenGL v: 4.6 Mesa 22.3.0 renderer: Mesa Intel HD Graphics
    P4600/P4700 (HSW GT2)

ManuLinares avatar Dec 07 '22 21:12 ManuLinares

I've been looking over the Firefox code again, specifically the glxtest part. It seems that when it asks for the EGLDisplay, it passes in an X11 Display object from the default display (the one on the Intel card)[1]. That seems to be restricting the list of EGL implementations to ones that are compatible with Intel. However, when Firefox actually goes to use EGL for real, it asks EGL for the default implementation, and doesn't pass in anything related to X11.

This is all to say, I'm not sure it can ever work with how Firefox is written at the moment. You may have more luck raising an issue in Bugzilla.

[1] https://searchfox.org/mozilla-central/rev/ef0aa879e94534ffd067a3748d034540a9fc10b0/toolkit/xre/glxtest.cpp#672

elFarto avatar Dec 07 '22 21:12 elFarto

Mine was also this same issue, and likewise with modeset=1 properly set, although I forgot to upgrade to new versions so I'm still running the same version of the driver I was back then. I can try to provide the same logs to help confirm if it's desired, but I don't know how necessary it will be given the issue you found within Firefox itself. It doesn't seem like new versions of the driver will be helpful until that's solved first.

That said, given mine is a rather unique setup with two discreet GPUs, the primary being AMD and the secondary Nvidia, it might give some different results and I'd be happy to test things to help ensure this project would work in a variety of setups.

FaeAlicia avatar Dec 08 '22 01:12 FaeAlicia

I'm going to close this issue in favour of #11. I'll post an update there.

elFarto avatar Dec 19 '22 17:12 elFarto