OBS cannot find correct Apple VT encoder after computer is rebooted

[Piipperi]

Operating System Info

macOS 11.4

Other OS

No response

OBS Studio Version

27.0.1

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/5kWRKi_Zp7Qq_Tuy

OBS Studio Crash Log URL

No response

Expected Behavior

I would expect the thing to be able to find the right encoder after the computer is rebooted...haha

Current Behavior

Since the 27.0 update, OBS has given me two Apple VT hardware encoders in the encoder settings. The other one is my RX 5700 XT eGPU, and the other one is the Intel iGPU.

After rebooting the computer, and trying to record with OBS, "Starting the output failed. Please check the log for details." error comes up. When I go to the Output settings, I see Apple VT H264 Hardware Encoder as the encoder, but it says below "No properties available". Then if I open the encoder dropdown menu, I see the 2 Apple VT hardware encoders there, no way to know which one is my eGPU and which one is iGPU. Usually, the first one on the list seems to be my eGPU thought (pointed out by Activity Monitor when recording).

After choosing the right encoder and configuring the encoding settings the recording works as expected, even between closing and opening OBS, as soon as I reboot my computer thought this all happens again.

Steps to Reproduce

1. Open OBS Output settings, choose the right encoder as explained in the current behavior.
2. Test the recording works properly and that it's being encoded by the eGPU.
3. Close OBS and restart the computer
4. Open OBS, try to start a recording... you should now see the "Starting the output failed. Please check the log for details." error.
5. Congrats, now you can go back into settings to reconfig the encoder and try this again. Bug successfully reproduced.

Anything else we should know?

This may also apply for Macs with an iGPU and a dGPU but will require further testing.

[PatTheMav]

If possible, please clone and run the program from this repository and attach the output as a text file:

https://github.com/SharkyRawr/videotoolbox-list

Usually Video Toolbox tries to select the best possible encoder available and we just address it using a single encoder ID. We are aware that Macs with a T2 chip have an additional encoder ID available and that M1 Macs have only that second ID.

We are not aware of other encoder IDs so far, which is why we'd need to see output from systems with e.g. eGPUs to check for these IDs.

[Piipperi]

Note: I am running a non-T2 Mac, more precisely a Late 2013 MacBook Pro

Terminal Saved Output.txt

[PatTheMav]

Thanks, the relevant parts of that log are:

Name: Apple H.263 (SW)
Display Name: H.263
Id: com.apple.videotoolbox.videoencoder.h263
=========================
Name: Apple H.264 (HW)
Display Name: Apple H.264 (HW)
Id: com.apple.videotoolbox.videoencoder.h264.gva.100000983
=========================
Name: Apple H.264 (HW)
Display Name: Apple H.264 (HW)
Id: com.apple.videotoolbox.videoencoder.h264.gva
=========================
Name: Apple H.264 (SW)
Display Name: Apple H.264 (SW)
Id: com.apple.videotoolbox.videoencoder.h264

So as per your comments the eGPU seems to be reported as a separate available encoder next to the built-in one.

Interestingly the ID detected by the tool is com.apple.videotoolbox.videoencoder.h264.gva.100000983 while the provided OBS logs mentions com.apple.videotoolbox.videoencoder.h264.gva.10000088e.

What I can imagine to be happening is that this ID is dynamic, so you can configure OBS to use your eGPU for encoding, but after a reboot VideoToolbox will assign a new suffix to your eGPU and thus the ID - as stored in OBS' config - becomes invalid.

@Piipperi - to confirm this, could you pls reboot your computer at your convenience and run the tool again, checking if the ID changed between both runs?

[Piipperi]

Your hypothesis seems to be correct.

Terminal Saved Output 2.txt

[PatTheMav]

Thank you - that's unfortunate as there is not really much we can do out of the box. We'd have to think about how to solve this, e.g. detecting an eGPU by checking for a alphanumeric suffix after the .gva part and notify the user about this.

As this ID changes per reboot, there would be no surefire way to just apply the settings of the "old" ID to the new one as we cannot be sure that it's actually the same eGPU.

I'd prefer something more explicit like a dialog asking you whether the old settings should be applied to another detected eGPU on first launch.

[Piipperi]

Yeah too bad there doesn't really seem to be a way to clearly tell what is what since macOS doesn't report the source of the encoder.

[PatTheMav]

Thank you nonetheless, we will report this to Apple directly.

[MisutaaAsriel]

Can I ask if you have thought to associate the GPURegistryID instead of the Encoder ID for eGPUs?

External GPUs return an additional "GPURegistryID" which is associated with the GPU itself. This can be seen in hexidecimal via safeejectgpu gpusin the terminal, or in decimal via this sample code in Playgrounds with an eGPU connected:

import Foundation
import VideoToolbox
import MetalKit
import PlaygroundSupport

var gpuList = MTLCopyAllDevices()

var encoder_list: CFArray?
VTCopyVideoEncoderList(nil, &encoder_list)

var encoders = encoder_list as? Array<Any>
var encoderResults = Set<Dictionary<String, AnyHashable>>()

for item in encoders ?? [] {
    var encoder = item as! Dictionary<AnyHashable, AnyHashable>
    var result = Dictionary<String, AnyHashable>()
    var codec: String = encoder["CodecName"] as? String ?? ""
    
    if let registryID = encoder["GPURegistryID"] {
        for gpu in gpuList {
            
            var gpuID: Int = registryID as! Int
            if (gpu.registryID == gpuID ?? 0) {
                result[gpu.name] = encoder
            }
        }
    } else {
        result["Internal"] = encoder
    }
    
    print(result as! AnyObject)
}

If OBS were to check for a GPURegistryID, and store a reference to it along with the encoder, it could catch changes to the EncoderID and select the closest match by GPURegistryID.

[MisutaaAsriel]

I'm trying to look into this but I have to admit perhaps fixing this is "above my grade" as I am not used to Objective-C.

However, I can lay out how a conceptual fix would work, which would avoid breaking setups. Likewise, it would be working off #5424 as that already uses GPU IDs to get the hardware name, and is a nice QoL improvement that is needed for multi-GPU Macs in general.

1. If a GPURegistryID exists for the specified encoder, the encoder OBS ID will be vthardware.[GPURegistryID].[CodecType] (e.g. registeredgpu.4294968629.1635148593 for my RX570 H.264), as opposed to [EncoderID] — Software encoders do not have hardware associated, so relying on CodecType is not an issue if it is restricted to being paired with encoders containing a GPURegistryID.

2. On launch, VT encoders will be validated; if the user selected encoder cannot be found, it will alert the user that "The selected encoder ([Encoder Name]) cannot be found. Please select an encoder to continue.", with the option to either:

   • "Choose Encoder..." to map to an existing encoder (i.e. a "Select a matching encoder:" dialog with a dropdown displaying only encoders starting with vthardware. will be listed; existing encoder configurations will be transfered 1:1)
   • "Dismiss", in which a message stating "An encoder may be selected at any time via Output options in OBS Settings.", and the encoder option will remain invalid.
     • This behavior attempts to mimic the missing assets dialog, informing the user of the broken link, but allowing them to manually correct the issue at a later time, or prompting them again on next launch if unresolved.
     • This will allow resolving broken references from previous versions, without losing settings; Currently OBS instead silently fails until a stream/record option is attempted, at which an erroneous error about AMD/NVidia is displayed.

[Fenrirthviti]

eGPUs are not currently supported by us, and Apple has dropped support in newer version of macOS, so it's unlikely we will fix this.