[Bug]: Recording gameplay using H265 will semi-regularly crash the GPU driver; using OBS Studio and ReLive

[cs9kc]

Describe the bug

Context: It is known from a reddit thread and from my own and others' experiences, that recording with the H265 encoder, in both OBS Studio and ReLive, has the chance to crash the display driver. In my experience, I can not record H265 for more than about 30 minutes without a driver crash.

https://www.reddit.com/r/AMDHelp/comments/v6nn68/recent_drivers_cause_driver_and_game_crashes/

Originally I thought this issue was limited to only AMF 1.4.24 and Display Driver 22.3.1 and newer. While the encoder features do behave properly on 22.2.3 (AMF 1.4.23), H265 will still crash the GPU drivers.

Related to Bug 326: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/326 Semi-related to bug 325: https://github.com/GPUOpen-LibrariesAndSDKs/AMF/issues/325

To Reproduce

Steps to reproduce the behavior:

1. Start a recording with H265/HEVC using the new OBS Studio w/AMF texture-based encoder, or AMD ReLive
2. Play the 3D program/game
3. Usually, a driver crash will occur in under 30 minutes. Crash is random and encoder usage looks nominal, and GPU memory shows no signs of being overloaded.

Setup

OS version: Windows 11 21H2 22000.739 Adrenalin driver version: non-WHQL 22.2.3 GPU: AMD Reference RX 6900 XT LC Which component has the issue: H265 Encoder

Debug Log:

Nothing out of the ordinary occurs in the OBS Studio logs when the crash occurs... the encoder goes unresponsive, and if one clicks "stop recording", the button changes to "stopping recording..." but does nothing else. Encoder usage in Task Manager drops to 0% once the driver crash occurs, and remains that way.

Expected behavior

It was known from a reddit thread that recording while using the H265 encoder could cause a driver crash, but it seems that it is both more-easily repeatable, and more-widespread than I previously thought. And while the new OBS Studio w/AMF texture-based encoder does perform properly using all encoder features that are available in AMF 1.4.23, the H265 crash does still occur.

Additional context

Hopefully there is an internal ticket created for this issue already, be it with you or with the AMD driver team at large. I have submitted crash data from the bug report tool through my channels, in hopes that multiple examples across many different driver versions will help you all out. Thanks again for hearing me out!

[AndreDVJ]

Instant Replay enabled with HEVC encoding, recording Desktop and everything, will trigger a TDR even faster.

[cs9kc]

This appears to have been resolved in 22.8.1. Closing this for now, will re-open if it comes back up.

Thanks again, AMF team! <3