mediadevices icon indicating copy to clipboard operation
mediadevices copied to clipboard

Add darwin runtime device observer support

Open hexbabe opened this issue 1 month ago • 1 comments

Summary

Before, we got a static snapshot of connected devices via initialize, now if you call SetupObserver/StartObserver on your downstream program's main thread, the observer will update the connected devices state so the program can appropriately handle hot unplug/replug events and new devices connected. DestroyObserver must be during closure to clean up Go and C observer resources.

This brings darwin camera support to feature parity with linux.

Details

C code (DeviceObserver.m) State machine

[ Uninitialized ]
       |
       | (DeviceObserverInit)
       v
[    Idle     ]<---------------------------------------------------+
       |                          |
       |    (DeviceObserverStart)                                  | (DeviceObserverStop)
       v                          |
[  Monitoring (pumped by DeviceObserverRunFor) ]-------------------+
       |
       | (Hardware Event: USB plug/unplug)
       v
  (Calculate Device Set Diffs -> Fire Callback)

It is not thread-safe, but the Go code makes sure that there is a singleton entity/goroutine talking to it in its lifecycle.

Events are passed from C to Go through the bridge via mCallback the DeviceEventCallback.

Go observer that manages the above

[ observerInitial ]
       |
       | (SetupObserver)
       v
[ observerSetup (Sets up C state machine in idle state) ] <~~~~(Waiting for Signal to start, StartObserver)~~~~|
       |                                       
       |
       v                                       
[ observerStartup ]                           
       |                                       
       | (Replay initial events, set up concurrency channels etc.)               
       v                                       
[ observerRunning (C.DeviceObserverRunFor, pumps C state machine in running state) ]  
       | 
       |
       | (DestroyObserver signals 'destroyObserver')
       v
[ observerDestroyed ] (Terminal)

We make SetupObserver and StartObserver distinct because not all downstream programs will want to start pumping the NSRunLoop to handle events immediately.

Additional caveat

SetupObserver must run on the main thread of the program process, so it should be run in the init func or early in the main func of the Go program's entrypoint. This is a known pattern: https://stackoverflow.com/questions/25361831/benefits-of-runtime-lockosthread-in-golang. Hence, we use runtime.LockOSThread to achieve this. After consulting docs here, I decided to use the main thread's NSRunLoop as opposed to spinning up our own to avoid the complexity of having to manage its lifecycle manually. See section on "secondary threads" https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/RunLoopManagement/RunLoopManagement.html

How to use?

Read over the new example I added in examples/device_observer_darwin!

hexbabe avatar Nov 25 '25 16:11 hexbabe

Codecov Report

:x: Patch coverage is 18.96552% with 235 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 42.13%. Comparing base (7d8cbdb) to head (68455a3).

Files with missing lines Patch % Lines
pkg/avfoundation/device_observer_darwin.go 17.73% 167 Missing :warning:
pkg/driver/camera/camera_darwin.go 16.04% 61 Missing and 7 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #670      +/-   ##
==========================================
- Coverage   43.41%   42.13%   -1.29%     
==========================================
  Files          85       86       +1     
  Lines        4899     5186     +287     
==========================================
+ Hits         2127     2185      +58     
- Misses       2617     2839     +222     
- Partials      155      162       +7     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Nov 25 '25 16:11 codecov[bot]

Thank you @hexbabe this PR is really cool.

JoTurk avatar Dec 12 '25 22:12 JoTurk

The feature looks nice!

It might be better to define a DeviceObserver interface and expose the driver's observer as a struct instance for the API consistency among drivers.

For example like:

// pkg/driver/observer.go
type DeviceObserver interface {
  SetupObserver() error
  StartObserver() error
  DestroyObserver() error
}

// pkg/driver/camera/observer.go
type deviceObserver struct {} // implements driver.DeviceObserver

var defaultDeviceObserver = &deviceObserver{}

func DeviceObserver() driver.DeviceObserver {
  return defaultDeviceObserver
}

@at-wat I think mac is the only platform where we need an active bg observer goroutine to handle realtime device connections/disconnections using the event handler model. On linux and windows, poll based device enumeration during runtime is supported (i.e. checking /dev/video* and Media Foundation enumeration API), so they will not need this interface since they do not need a long-lived observer.

Are there other drivers that I may be missing in my mental model that you feel might want this interface?

hexbabe avatar Dec 16 '25 17:12 hexbabe