NvFBC patch doesn't work on driver >=560

[ALVAROPING1]

Describe the bug The FBC patch doesn't work on the 565.77 driver

To Reproduce Steps to reproduce the behavior:

1. Install the 565.77 driver
2. Apply FBC patch (sudo ./patch-fbc.sh)
3. Try using the NvFBC source plugin in OBS

Expected behavior OBS is able to record the screen using FBC

Output listings GPU and driver version:

$ nvidia-smi
Sun Jan 19 20:04:35 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.77                 Driver Version: 565.77         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 2070        Off |   00000000:1C:00.0  On |                  N/A |
|  0%   45C    P8              8W /  185W |     851MiB /   8192MiB |      8%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      3825      G   /usr/lib/xorg/Xorg                            413MiB |
+-----------------------------------------------------------------------------------------+

FBC check with NvFBCCUDAAsync:

$ cd Capture_Linux_v8.0.8/NvFBC/samples/NvFBCCUDAAsync/
$ make
gcc -m64 -Wall -I../../inc -I../inc  -c NvFBCCUDAAsync.c -o NvFBCCUDAAsync.o
gcc -m64 -Wall -I../../inc -I../inc  -c ../common/NvFBCUtils.c -o NvFBCUtils.o
gcc -m64 NvFBCCUDAAsync.o NvFBCUtils.o -ldl -o NvFBCCUDAAsync
$ ./NvFBCCUDAAsync
Application version: 4
NvFBC API version: 1.8

This hardware does not support NvFBC

Environment:

• OS: Pop_Os 22.04 (based on Ubuntu 22.04)
• GPU model: RTX 2070
• Patch commit used: 0837f46dfe25b6e750abc7e601032bdd12c70be0
• Nvidia driver version: 565.77

Additional context The patch worked before I recently upgraded the driver from 555 to 565.77. Usually re-applying the patch is enough to get it working again, but it didn't work this time

[Gelmo]

I'm not running the driver, so I can't test it, but the regex query does work against 565.77; I just downloaded the driver to do a quick test. FWIW, the same regex is working on 570.86.16; just tested with NvFBCCUDAAsync

[PancakeTAS]

The original poster updated from 555 to 565, skipping 560. I just tested, NvFBC does not work on 560 either, please rename the issue. (This was also when NVIDIA added the pipewire backend, likely reshuffling a bunch of the internal code structure)

[Gelmo]

Just want to point out that I had a typo in my last comment here; I wrote 560.86.16 but what I actually tested successfully was 570.86.16.

The original poster updated from 555 to 565, skipping 560. I just tested, NvFBC does not work on 560 either, please rename the issue. (This was also when NVIDIA added the pipewire backend, likely reshuffling a bunch of the internal code structure)

I just checked and the patch itself does work on 560.35.03:

❯ rafind2 -x "85c00f849b00000048" libnvidia-fbc.so.560.35.03
0x379cd

Which exact driver are you using, and are you certain the patch applied successfully before you tested?

[PancakeTAS]

The patch might work but that doesn't mean it was effective. Try downloading the NvFBC sample and running that. Or this untested bit of code might be sufficient for testing as well:

NVFBC_API_FUNCTION_LIST fbc = { .dwVersion = NVFBC_VERSION };
NvFBCCreateInstance(&fbc);

NVFBC_SESSION_HANDLE handle;
fbc.nvFBCCreateHandle(&handle, &(NVFBC_CREATE_HANDLE_PARAMS) { .dwVersion = NVFBC_CREATE_HANDLE_PARAMS_VER });

This latter call returns 7, which when looking at the docs means Unsupported. That's what it returns on old unpatched drivers too and it disappears after you apply the patch.

[Schramme2021]

I reviewed the assembly of the CUDA sample and managed to get it working again, at least for version 570.124.06.

Here’s the fork: https://github.com/Schramme2021/nvidia-patch

I suspect the fix might also be compatible with other versions.

update:

• The original bytecode remains in the binary, so while the patch seems to apply successfully, it has no effect during runtime execution.

• The modified bytecode is also present in versions 570.86.16, 570.124.04, and 570.133.07. The fork has been updated accordingly, but remains to be tested.

[jailuthra]

@ALVAROPING1 can you test NVFBC with the latest patches (master branch) for >= 570.xx driver?

[ALVAROPING1]

@ALVAROPING1 can you test NVFBC with the latest patches (master branch) for >= 570.xx driver?

i can confirm that the patch works in 570.133.07

[PancakeTAS]

560.X and 565.X still do not work.