Sunshine feat(macOS): Capture audio on macOS using Tap API

Description

This PR adds system-wide audio tap support for macOS. The implementation introduces:

System-wide audio tap functionality to capture audio from all system sources
- Uses an audio converter to handle varying client audio requirements
Unit tests covering the new system tap methods, along with additional coverage for the existing microphone path
Updated UI with a macOS-specific toggle (see screenshots)
Updated configuration options to support the new audio tap feature
Added Doxygen documentation for new APIs

Additional changes

Adjusted cmake files for compatibility with Homebrew-based setup.
- Tweaked dependency detection so that openssl and opus are found automatically. This replaces the need to run manual ln commands, but I’m not sure if this is the best long-term approach. Feedback welcome.
- Updated cmake/compile_definitions/unix.cmake to ensure SUNSHINE_ASSETS_DIR resolves correctly.
Updated src_assets/macos/assets/Info.plist to prepare for required macOS permission prompts.
Added a macos_system_wide_audio_tap config option to the audio_t struct.

Testing

Verified functionality with multiple (Moonlight) clients requiring mono, stereo, 5.1, and 7.1 audio configurations.
Confirmed audio conversion works as expected across varying client setups.
Stress-tested with multiple concurrent clients for several hours without memory leaks or race conditions observed.
Host ran an arm64 build during testing

Notes

My background is primarily in .NET, with some experience in C, C++, and Rust. Objective-C is new to me, but I carefully reviewed and tested the bits on memory management and synchronization.
I leaned on GitHub Copilot and AI assistance heavily for the Objective-C parts, especially syntax and boilerplate; but I reviewed, debugged and tested everything myself a bunch of times.
Some of the unit tests were bordering on integration tests, so I tried to keep them focused and not blur the lines too much.
I developed this feature to make Sunshine more accessible on macOS, especially for less technical users. I have to admit that I fumbled quite a bit getting BlackHole to work. 🙂 But I also wanted to make sure to not break the existing setupMicrophone functionality as it is also a viable option!
I am not entirely convinced on the naming or location of the macos_system_wide_audio_tap setting.
- Open to suggestions here!
Built this on arm64 macOS 15.6.1 (24G90)

If the AI involvement is a blocker for merging, no worries. I can totally understand! This was also a learning project for me and I had a lot of fun building it.

Screenshot

Web UI – Audio/Video Configuration New option for enabling system-wide audio recording on macOS. Disables the audio sink option when checked.

macOS Permission Prompt System permission request when Sunshine first tries to access system audio:

System Settings – Screen & System Audio Recording macOS privacy settings showing Sunshine/Terminal access to "System Audio Recording Only":

Issues Fixed or Closed

Roadmap Issues

Closes https://github.com/LizardByte/roadmap/issues/27

Type of Change

[x] feat: New feature (non-breaking change which adds functionality)
[x] fix: Bug fix (non-breaking change which fixes an issue)
[x] docs: Documentation only changes
[ ] style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
[x] refactor: Code change that neither fixes a bug nor adds a feature
[ ] perf: Code change that improves performance
[x] test: Adding missing tests or correcting existing tests
[x] build: Changes that affect the build system or external dependencies
[ ] ci: Changes to CI configuration files and scripts
[ ] chore: Other changes that don't modify src or test files
[ ] revert: Reverts a previous commit
[ ] BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

[x] Code follows the style guidelines of this project
[x] Code has been self-reviewed
[x] Code has been commented, particularly in hard-to-understand areas
[x] Code docstring/documentation-blocks for new or existing methods/components have been added or updated
[x] Unit tests have been added or updated for any new or modified functionality

AI Usage

[ ] None: No AI tools were used in creating this PR
[ ] Light: AI provided minor assistance (formatting, simple suggestions)
[ ] Moderate: AI helped with code generation or debugging specific parts
[X] Heavy: AI generated most or all of the code changes

Aug 29 '25 19:08 ThomVanL

Thank you for the PR!

If the AI involvement is a blocker for merging, no worries. I can totally understand! This was also a learning project for me and I had a lot of fun building it.

This isn't a blocker, we just want to understand how much it was used when developing the PR.

I've went ahead and approved the workflow run so the basic linting and whatnot can be taken care of before full reviews take place.

Looks like there's an issue with the tests. I don't see an error, just kind of looks like it crashes out. https://github.com/LizardByte/Sunshine/actions/runs/17332988293/job/49216925169?pr=4209#step:8:3666

Linting:

https://github.com/LizardByte/Sunshine/actions/runs/17332988295/job/49216903453?pr=4209#step:11:19
https://github.com/LizardByte/Sunshine/actions/runs/17332988295/job/49216903453?pr=4209#step:14:87

We'll also need another way to compile the tests without the .mm file on non macOS platforms: https://github.com/LizardByte/Sunshine/actions/runs/17332988293/job/49216934161?pr=4209#step:11:1886

Aug 29 '25 21:08 ReenigneArcher

Hey @ReenigneArcher, thanks for kicking things off and for clarifying the AI usage policy!

I’ll dig into those CI issues next.

Got one question regarding the cross-platform test compilation issue: would you be okay with me updating tests/CMakeLists.txt so that those .mm files are only included when building on macOS? That seems like the cleanest way to keep non-macOS builds happy while still testing the new audio tap functionality where it applies.

I’ll push updates once I have fixes in place. And thanks again for the feedback! 🙂

Aug 29 '25 21:08 ThomVanL

Got one question regarding the cross-platform test compilation issue: would you be okay with me updating tests/CMakeLists.txt so that those .mm files are only included when building on macOS?

Sounds fine to me and I also believe that's the cleanest and easiest approach.

Aug 29 '25 22:08 ReenigneArcher

https://github.com/LizardByte/Sunshine/actions/runs/17337137215/job/49225275832?pr=4209#step:8:635

I don't think this error has anything to do with your changes, must be something changed in homebrew recently. We use a custom python based action from https://github.com/LizardByte/actions/tree/master/actions/release_homebrew to validate the homebrew builds. I don't see anything obvious in the release notes for what would cause this: https://github.com/Homebrew/brew/releases

Aug 30 '25 00:08 ReenigneArcher

Bundle Report

Changes will increase total bundle size by 121 bytes (0.01%) :arrow_up:. This is within the configured threshold :white_check_mark:

Detailed changes

Bundle name	Size	Change
sunshine-esm	965.15kB	121 bytes (0.01%) :arrow_up:

Affected Assets, Files, and Routes:

view changes for bundle: sunshine-esm

Assets Changed:

Asset Name	Size Change	Total Size	Change (%)
`assets/_plugin-*.js`	121 bytes	343.43kB	0.04%

Files in assets/_plugin-*.js:

./src_assets/common/assets/web/public/assets/locale/en.json → Total Size: 34.91kB

Aug 30 '25 01:08 codecov[bot]

Codecov Report

:x: Patch coverage is 0% with 5 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 12.09%. Comparing base (eb72930) to head (f667865). :warning: Report is 8 commits behind head on master. :white_check_mark: All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/audio.cpp	0.00%	3 Missing :warning:
src/platform/linux/audio.cpp	0.00%	1 Missing :warning:
src/platform/windows/audio.cpp	0.00%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4209      +/-   ##
==========================================
- Coverage   12.09%   12.09%   -0.01%     
==========================================
  Files          87       87              
  Lines       17612    17613       +1     
  Branches     8097     8097              
==========================================
  Hits         2131     2131              
- Misses      14579    14580       +1     
  Partials      902      902

Flag	Coverage Δ
Linux-AppImage	`11.62% <0.00%> (-0.01%)`	:arrow_down:
Windows-AMD64	`13.41% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/config.h	`0.00% <ø> (ø)`
src/platform/common.h	`24.13% <ø> (ø)`
src/platform/linux/audio.cpp	`10.63% <0.00%> (ø)`
src/platform/windows/audio.cpp	`25.13% <0.00%> (ø)`
src/audio.cpp	`21.89% <0.00%> (-0.17%)`	:arrow_down:

Aug 30 '25 01:08 codecov[bot]

Thanks for the PR. I have built this on my M1 Pro MBP and have been trying to get it to work, but haven't had much success so far. I am not able to get any sound to be captured and/or sent. So far I am only testing with the simplest use case of audio playing out of my MBP speakers. I edited the code to make the tap and aggregate non-private, so I could try to view the tap/aggregate with Apple's sample app I can see the tap, but not the aggregate. I'm not sure what's wrong. Can you detail your testing process?

I do seem to be getting OK log entries and since I'm running from iTerm, all my permissions seem to be in order (iTerm has access to many things).

[2025-08-29 23:56:39.783]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2025-08-29 23:56:39.783]: Info: Configuring selected display (1) to stream
[2025-08-29 23:56:39.841]: Info: Creating encoder [hevc_videotoolbox]
[2025-08-29 23:56:39.841]: Info: Color coding: SDR (Rec. 601)
[2025-08-29 23:56:39.841]: Info: Color depth: 10-bit
[2025-08-29 23:56:39.841]: Info: Color range: MPEG
[2025-08-29 23:56:39.841]: Info: Streaming bitrate is 55987000
[2025-08-29 23:56:39.850]: Info: [hevc_videotoolbox @ 0x143e60e70] This device does not support the max_ref_frames option. Value ignored.
[2025-08-29 23:56:40.712]: Info: Using macOS system audio tap for capture.
[2025-08-29 23:56:40.712]: Info: Sample rate: 48000, Frame size: 240, Channels: 2
[2025-08-29 23:56:40.738]: Info: Aggregate device created with ID: 203
[2025-08-29 23:56:40.738]: Info: Aggregate device created and configured successfully
[2025-08-29 23:56:40.739]: Info: No conversion needed - formats match (device: 48000Hz/2ch)
[2025-08-29 23:56:40.739]: Info: Device properties and converter configuration completed
[2025-08-29 23:56:40.793]: Info: System tap IO proc created and started successfully
[2025-08-29 23:56:40.793]: Info: Audio buffer initialized successfully with size: 8192 bytes
[2025-08-29 23:56:40.793]: Info: System tap setup completed successfully!
[2025-08-29 23:56:40.793]: Info: macOS system audio tap capturing.
[2025-08-29 23:56:40.794]: Info: Opus initialized: 48 kHz, 2 channels, 512 kbps (total), LOWDELAY

I had to make the following changes to get it to build:

-#pragma clang diagnostic push
-#pragma clang diagnostic ignored "-Wunguarded-availability-new"
-    AVCaptureDeviceDiscoverySession *discoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeBuiltInMicrophone, AVCaptureDeviceTypeExternalUnknown]
+    AVCaptureDeviceDiscoverySession *discoverySession = [AVCaptureDeviceDiscoverySession discoverySessionWithDeviceTypes:@[AVCaptureDeviceTypeMicrophone, AVCaptureDeviceTypeExternal]
                                                                                                                mediaType:AVMediaTypeAudio
                                                                                                                 position:AVCaptureDevicePositionUnspecified];
     NSArray *devices = discoverySession.devices;
     BOOST_LOG(debug) << "Found "sv << [devices count] << " devices using discovery session"sv;
     return devices;
-#pragma clang diagnostic pop

plus a fix in our input.cpp that breaks the latest Xcode 16.4, I'm surprised if you didn't run into this one. I am running Xcode 16.4 clang-1700.0.13.5).

--- a/src/platform/macos/input.cpp
+++ b/src/platform/macos/input.cpp
@@ -534,7 +534,7 @@ const KeyCodeMap kKeyCodesMap[] = {
     if (!output_name.empty()) {
       uint32_t max_display = 32;
       uint32_t display_count;
-      CGDirectDisplayID displays[max_display];
+      CGDirectDisplayID displays[32];

Aug 30 '25 03:08 andygrundman

Posting this here for reference in case it's needed for permissions of unit tests. We used to need something like this for macports, but it was never necessary for homebrew.

https://github.com/LizardByte/Sunshine/blob/b662b8e7c2aa23a4673edb1e601f7337458dcc7e/.github/workflows/CI.yml#L841-L898

Aug 30 '25 04:08 ReenigneArcher

Hey @andygrundman, thanks for taking the time to test the PR. Sorry to hear it’s not working correctly.

Here’s my setup on an M4 MBP and what I did.

clang --version
Apple clang version 17.0.0 (clang-1700.0.13.5)
Target: arm64-apple-darwin24.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I made the changes in VS Code and followed the steps in building.md:

mkdir build
cmake -B build -G Ninja -S . 
ninja -C build

For testing I did the following steps:

In the VS Code terminal: ./build/sunshine
Accept the allow connections prompt
Connect with multiple Moonlight clients with the following sample rate/frame size/channel combos:
- iPad @ 1080p → 48000Hz/240/2ch audio → works
- Android TV @ 1080p → 48000Hz/240/8ch audio → works
- iOS @ 360p → 48000Hz/480/2ch audio → works

So at one point I'm running three streams simultaneously and the audio plays through the devices.

My sunshine.conf looks like:

audio_sink = Steam Streaming Speakers
macos_system_wide_audio_tap = true
origin_web_ui_allowed = pc
stream_audio = true
wan_encryption_mode = 2

With min_log_level = debug, here’s the output when connecting from the 360p device (trimmed for brevity).

[2025-08-30 10:42:27.456]: Info: CLIENT CONNECTED
[2025-08-30 10:42:27.463]: Debug: Start capturing Video
[2025-08-30 10:42:27.463]: Info: Detecting displays
[2025-08-30 10:42:27.463]: Info: Detected display: Built-in Retina Display (id: 1) connected: true
[2025-08-30 10:42:27.463]: Info: Configuring selected display (1) to stream
[2025-08-30 10:42:27.541]: Info: Creating encoder [hevc_videotoolbox]
[2025-08-30 10:42:27.541]: Info: Color coding: SDR (Rec. 601)
[2025-08-30 10:42:27.541]: Info: Color depth: 8-bit
[2025-08-30 10:42:27.541]: Info: Color range: MPEG
[2025-08-30 10:42:27.541]: Info: Streaming bitrate is 1268000
[2025-08-30 10:42:27.550]: Info: [hevc_videotoolbox @ 0x13d01de00] This device does not support the max_ref_frames option. Value ignored.
[2025-08-30 10:42:27.945]: Debug: Start capturing Audio
[2025-08-30 10:42:27.946]: Warning: audio_control_t::set_sink() unimplemented: Steam Streaming Speakers
[2025-08-30 10:42:27.946]: Info: Using macOS system audio tap for capture.
[2025-08-30 10:42:27.946]: Info: Sample rate: 48000, Frame size: 480, Channels: 2
[2025-08-30 10:42:27.946]: Debug: setupSystemTap called with sampleRate:48000 frameSize:480 channels:2
[2025-08-30 10:42:27.946]: Debug: macOS version check passed (running 15.6.1)
[2025-08-30 10:42:27.946]: Debug: System tap initialization completed
[2025-08-30 10:42:27.946]: Debug: Creating tap description for 2 channels
[2025-08-30 10:42:27.946]: Debug: Creating process tap with name: SunshineAVAudio-Tap-0x6000007d00c0
[2025-08-30 10:42:27.949]: Debug: AudioHardwareCreateProcessTap returned status: 0
[2025-08-30 10:42:27.949]: Debug: Process tap created successfully with ID: 140
[2025-08-30 10:42:27.949]: Debug: Creating aggregate device with tap UID: 675EC59C-D7CC-4A25-A093-F2B4B4227895
[2025-08-30 10:42:27.957]: Debug: AudioHardwareCreateAggregateDevice returned status: 0
[2025-08-30 10:42:27.957]: Info: Aggregate device created with ID: 141
[2025-08-30 10:42:27.959]: Debug: Set aggregate device sample rate to 48000Hz
[2025-08-30 10:42:27.959]: Debug: Set aggregate device buffer size to 480 frames
[2025-08-30 10:42:27.959]: Info: Aggregate device created and configured successfully
[2025-08-30 10:42:27.960]: Debug: Device reports 2 input channels
[2025-08-30 10:42:27.960]: Debug: Device properties - Sample Rate: 48000Hz, Channels: 2
[2025-08-30 10:42:27.960]: Debug: needsConversion: NO (device: 48000Hz/2ch -> client: 48000Hz/2ch)
[2025-08-30 10:42:27.960]: Info: No conversion needed - formats match (device: 48000Hz/2ch)
[2025-08-30 10:42:27.960]: Info: Device properties and converter configuration completed
[2025-08-30 10:42:27.960]: Debug: Creating IOProc for aggregate device ID: 141
[2025-08-30 10:42:27.977]: Debug: AudioDeviceCreateIOProcID returned status: 0
[2025-08-30 10:42:27.978]: Debug: Starting IOProc for aggregate device
[2025-08-30 10:42:27.995]: Debug: AudioDeviceStart returned status: 0
[2025-08-30 10:42:27.995]: Info: System tap IO proc created and started successfully
[2025-08-30 10:42:27.995]: Debug: Initializing audio buffer for 2 channels
[2025-08-30 10:42:27.995]: Info: Audio buffer initialized successfully with size: 8192 bytes
[2025-08-30 10:42:27.995]: Info: System tap setup completed successfully!
[2025-08-30 10:42:27.995]: Info: macOS system audio tap capturing.
[2025-08-30 10:42:27.995]: Info: Opus initialized: 48 kHz, 2 channels, 96 kbps (total), LOWDELAY

I’m familiar with the sample app! It might be the case that the aggregate device settings are probably still marked as private. You’ll need to flip those to NO in two places, once for the tap description and once for the aggregate device.

https://github.com/LizardByte/Sunshine/blob/64047055b86c378fb79835101a78e55b2e33f6e2/src/platform/macos/av_audio.mm#L604

https://github.com/LizardByte/Sunshine/blob/64047055b86c378fb79835101a78e55b2e33f6e2/src/platform/macos/av_audio.mm#L649

Then the tap will show up in Apple's sample app's UI.

And the aggregate device shows up as well.

But even with those values flipped to YES, I can still hear audio on the 360p device!

Aug 30 '25 09:08 ThomVanL

Thanks for the detailed info. I forgot my log only had Info level, when using Debug my log does look exactly like yours. I feel like I must just have a permission issue, maybe using iTerm as the permission "owner" isn't correct.

Do you see both of these items? What process names are they using? I only get a System Audio item when recording a test file in AudioTapSample. permissions

Here's basically what my Screen & System Audio Recording settings look like: privacy

If this is the issue, I wonder how Sunshine can detect that it doesn't actually have permission, hmm.

Aug 30 '25 11:08 andygrundman

You're right, because I ran into a similar issue with VS Code. Granting it "Screen & System Audio Recording" wasn’t enough; I had to explicitly allow "System Audio Recording Only." From what I found online, the VS Code app bundle itself might be causing the problem. I also tried running tccutil reset All to clear permissions, but the behavior stayed the same. I had to explicitly add permissions, but only for VS Code.

I did not even notice the little privacy notice at the top until just now, thanks for that. Here's what it's like on my end.

When I launched ./build/sunshine directly from the macOS Terminal (not iTerm), I did get the prompt mentioned in my initial message on this PR.

Edit: just to be clear, my dev loop consists of launching sunshine builds through the VS Code integrated terminal.

Aug 30 '25 11:08 ThomVanL

Great, adding usr/local/bin/sushine manually worked. (I run sudo make install probably because I used to have problems with the assets directory or something.)

I worry that this is a nightmare for the average user. I think we'll probably have to ship a proper dmg package so that only the actual app gets the permission and not whatever terminal the user ran brew install from. But I'm getting ahead of myself... I will put in some other review notes I have, after I do some more testing.

Aug 30 '25 16:08 andygrundman

I completely agree that the user experience shouldn’t be tied to a specific terminal. At some point, you’ll likely want to create an app bundle. I've also added an additional property list key in the Info.plist file to help facilitate this a little:

https://github.com/LizardByte/Sunshine/blob/64047055b86c378fb79835101a78e55b2e33f6e2/src_assets/macos/assets/Info.plist#L11-L12

I will check for a way to properly check those required permissions because right now this is missing.

By the way, I ran into the same issue with loading the assets. I initially worked around it by creating a symlink:

ln -s /Users/<username>/Sunshine/build/assets/web /usr/local/assets/web

However, I didn’t like having that lingering around so I updated the unix.cmake file to allow passing in the absolute path via -DSUNSHINE_ASSETS_DIR. But we could revert this if needed.

https://github.com/LizardByte/Sunshine/blob/64047055b86c378fb79835101a78e55b2e33f6e2/cmake/compile_definitions/unix.cmake#L1-L11

Aug 31 '25 01:08 ThomVanL

By the way, I ran into the same issue with loading the assets. I initially worked around it by creating a symlink:

a make install is required after compiling

Aug 31 '25 01:08 ReenigneArcher

I spent a bit of time looking about making a proper app bundle with cmake. It's certainly doable, and mostly just a matter of refactoring the maze of all the include files. App bundle would give us easier permissions, asset storage, signing, much easier for users.

The more I run the Mac version the more I realize the video side of things needs a lot of work too: host latency stat is missing, min/max framerate support, HDR doesn't work, frametime is uneven, it's using BGRA textures so probably using much more memory/cpu/gpu than it should be. It's one of the only apps that runs my MBP fan... Lots of fun stuff to work on I guess.

What's your use case with Mac Sunshine, if you don't mind me asking? We should assume controller support will also be a lot of work. Apple's "Screen Sharing" app already covers the remote desktop use cases, and no one plays games on their Mac (yet).

Aug 31 '25 02:08 andygrundman

I spent a bit of time looking about making a proper app bundle with cmake. It's certainly doable, and mostly just a matter of refactoring the maze of all the include files.

I assume static linking would also be needed? We had a pkg or dmg a while back but I removed it because the size was huge with dynamic linking and having to include every single dependency and sub dependency. We may not have been creating the package properly though. I think we were relying on macports to create it for us back then.

Edit: There's a roadmap entry for this here (https://github.com/LizardByte/roadmap/issues/18). I closed it as I was planning to do this via homebrew with a homebrew bottle, but they were not willing to accept all of our required dependencies. If it'll be a dmg instead that's fine, I can re-open it. dmg seems like the better option for macOS anyway.

Aug 31 '25 02:08 ReenigneArcher

I'll have to read up on how cmake handles it but I don't think the size will be that bad. We could probably remove all the software encoders to save space, for example.

The most annoying part will probably be that someone will need to use their Apple dev account to sign & notarize official builds.

Aug 31 '25 03:08 andygrundman

Overall I think this is a well done patch, and is a good example of how to do an AI-assisted PR.

I originally was worried about the frequent use of manual release, e.g.
[audioInput release];
[audioOutput release];
but then I learned that ObjC built by cmake doesn't use the newer ARC memory management model that I was more familiar with from developing in Xcode. ARC dates from 2012 in OSX Lion if you can believe it, but since devs had to opt into using ARC, nobody ever did for this code. So, I guess that's a point for AI here. I would have probably not thought about this and been quite confused at all the memory leaks I was creating. Or more likely it would just result in a lot of compiler errors. (In -fobjc-arc mode, these kinds of release calls are compiler errors.)

Yeah, I wondered about ARC too and did a quick search for -fobjc-arc and @autoreleasepool usage. Everything pointed to ARC not being enabled, so I followed the existing pattern of manual release calls.

Good to hear that lines up with your findings as well.

Aug 31 '25 21:08 ThomVanL