obsproject
Enabling Hardware Decoding Causes Media Source to Freeze and Crash OBS on Exit (Intel Mac)
Operating System Info
macOS 13
Other OS
No response
OBS Studio Version
28.1.2
OBS Studio Version (Other)
No response
OBS Studio Log URL
https://obsproject.com/logs/XpJQmNGjezSWcdJW
OBS Studio Crash Log URL
No response
Expected Behavior
Be able to use hardware decoding without resulting in a black screen.
Current Behavior
Enabling hardware decoding causes media source to freeze and crash OBS on exit
Steps to Reproduce
- Add Media Source
- Select .webm file
- Select Use hardware decoding when available
- Click OK
- Black screen appears
Anything else we should know?
No response
Could you try this on the beta of OBS 29 as well? (This sounds like a dependency issue to me and we updated dependencies for 29). Also, could you attach the webm file you're using?
Via the OBS Discord, last week I was asked to try it using the beta and I got the same result. I've attached a .webm file that I've tried to add to the scene collection.
Looking into this bug, I was able to recreate it using the provided .webm file but instead of a black screen, my app would crash continuously upon restarting until I removed the file as a media source.
When testing the bug on an Apple Silicon (M2) Mac, there was no crash. Switching to an Intel Mac and taking the same steps to reproduce the bug does result in a crash after starting and stopping recording. Upon further inspection of the OBS Studio Log, a line mentions that "CBR support for VideoToolbox encoder requires Apple Silicon. Will use ABR instead." However, in the lines under, rate_control is set to CBR in the [VideoToolbox advanced_video_stream: 'h264']: settings, and CBR is set to on in the [CoreAudio AAC: 'Track1']: settings. Changing the rate_control variable in the encoder.c file leads to a change in behavior of the crash: the app no longer crashes after starting and stopping recording. Instead, the app freezes when exiting, and the user must force quit. Given the above statements, the bug is potentially connected to the fact that on Intel Macs, CBR is still being used rather than an alternative encoding method.
By default, selecting CBR in the VideoToolbox encoder results in this path: https://github.com/obsproject/obs-studio/blob/0fc3cb1962063d95666ce0bef5a46fd06e14c59f/plugins/mac-videotoolbox/encoder.c#L296-L299
That is ABR. You can see that the properties set are identical to if the user had explicitly selected ABR. https://github.com/obsproject/obs-studio/blob/0fc3cb1962063d95666ce0bef5a46fd06e14c59f/plugins/mac-videotoolbox/encoder.c#L326-L329
If on macOS 13+ and on Apple Silicon, this path runs: https://github.com/obsproject/obs-studio/blob/0fc3cb1962063d95666ce0bef5a46fd06e14c59f/plugins/mac-videotoolbox/encoder.c#L301-L310
That is CBR.
We simply don't change the logged value of rate_control.
That said, the Steps To Reproduce do not mention recording as a required step to crash, so I do not see how this is related.
I have the exact same issue on an AMD GPU using Windows 11. When I use the nVidia GPU it works without any problems.
