Add bidirectional audio support for HomeKit cameras
Summary
- Add bidirectional audio support for HomeKit cameras (speaker/talkback functionality)
- Audio path: WebRTC/Browser mic → FFmpeg transcoding (OPUS→AAC-ELD) → HomeKit speaker via SRTP
- Implements connection tracking to prevent duplicate consumer-track pairs
- Adds
accept-audioparameter to FFmpeg for waiting on external audio sources - Thread-safe SRTP writes with mutex protection
Requirements
-
FFmpeg must be compiled with
libfdk_aacfor AAC-ELD encoding (required for HomeKit speaker audio)
Test plan
- [x] Tested with Aqara G4 doorbell - microphone works in Home app
- [x] Verified audio latency is optimal for the device (~3s jitter buffer is device firmware, not go2rtc)
- [ ] Test with other HomeKit cameras that support bidirectional audio
Implemented by Anders Baekgaard in cooperation with Anthropic Claude Opus 4.5 https://claude.ai/
Thanks! The implementation is quite complex and introduces a lot of confusion into the core. I won't accept it in this form, but I will try to understand the idea and adapt it when I have time.
And I really don't like the idea of using custom ffmpeg. This is one of the block factors for implementing two-way for HomeKit cameras.
Thank you for the prompt feedback.
I understand your concern about complexity, the PR got much more complex than I expected when first looking into it. I could not get streams matching to work for the aac-eld transcoder producer, due to archtiectural limitation. With your insight, you could maybe make it simpler.
Note that there is also a simple bug fix.