graphics-hook crashing Vulkan applications while VSync is enabled.

[Charlese2]

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

30.2.3

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/oLTp1JeQf4Yvi4PT

OBS Studio Crash Log URL

No response

Expected Behavior

Expected graphics-hook not the crash the application it is trying to capture.

Current Behavior

graphics-hook dll crashes due to an unexpected state due to NVidia drivers and VSync.

Steps to Reproduce

1. Launch Baldur's Gate 3 in Vulkan mode.
2. Set VSync to Double buffering in the in-game options.
3. Capture Baldur's Gate 3 using game capture.
4. Toggle visibility of the game capture source.
5. Baldur's Gate 3 should crash.

Anything else we should know?

Only NVidia drivers 555.85 and later suffer from this issue. I have not tested AMD.

Exception thrown at 0x00007FFE52512C27 (graphics-hook64.dll) in bg3.exe: 0xC0000005: Access violation reading location 0x0000000000000000.

graphics-hook64.dll!d3d12_capture() graphics-hook64.dll!hook_present() GameOverlayRenderer64.dll!00007ffe822be5c2() nvoglv64.dll!00007ffe14dcfa60() nvoglv64.dll!00007ffe14a0a0a8() kernel32.dll!BaseThreadInitThunk() ntdll.dll!RtlUserThreadStart()

After some debugging it crashes due to the data struct of type d3d12_data being null. https://github.com/obsproject/obs-studio/blob/edd7a387a4f278e004a5a40b1dc72916cc061421/plugins/win-capture/graphics-hook/d3d12-capture.cpp#L288

I also have a screenshot of the crash while running a debug build of graphics-hook64.dll in a different Vulkan application (Dolphin Emulator)

While VSync is active the user-mode part of the NVidia driver directly sends a Present call captured by graphics-hook, in addition to the normal vkQueuePresentKHR call from the application.

Related to #11003

[Fenrirthviti]

Potentially related to: https://github.com/obsproject/obs-studio/issues/11003 https://github.com/obsproject/obs-studio/issues/10540 https://github.com/obsproject/obs-studio/issues/9168 https://github.com/obsproject/obs-studio/issues/6149

Not very clear what's going on, but they all seem to be related to some kind of mixed context. There's a good chance this is a driver bug given how it's only recently started to crop up.

[dreamsyntax]

I wanted to confirm I can reproduce as well. Thanks for tracking it down.

[Charlese2]

I noticed that when I activate Double Buffering in Baldur's Gate 3 the NVidia Nsight Graphics overlay indicates that Independent Flip is Engaged.

There is also a D3D12 Queue added to the process (probably by the driver).

That is the state that would lead to a crash if I was capturing the game normally (with VSync) and toggled off and on the game capture.