[SDL3][audio] time to drop hardcoded channel maps for good?

[sylware]

Hi,

I have been coding a bit of audio lately (SDL3 audio alsa backend), and I humbly think it may be time to drop presumed hardcoded channel maps.

Audio systems without channel maps would be given the old hardcoded ones, but the SDL3 audio programing model would not presume the user code to be hardcoded for such channel maps and channel positions/specifications.

That would mean SDL3 would require a full set of channel specifications/positions (see ffmpeg resample library, which has specifications/positions to even handle the few cases of sparse channel maps).

Since some generic channel map swizzling code is being implemented in SDL3, that would mean for audio backends to only map their channel specifications/positions to SDL3 ones.

( and then, finally, it will be the user code to be carefull of the 6channels surround positions, which could be "Side" or "Rear" ).

[sylware]

Additionnally, the channel number may not be appropriate anymore as a major configuration parameter. For a number of channels, you could have different channel positions/specifications (for instance the 6channels surround is either "side" or "rear" which is important if going into a DSP).

Time to depart from this programing model for SDL3?

[slouken]

@icculus, this is related to our discussion about number of channels vs speaker layout

[icculus]

But (almost) everything wants exactly that hardcoded channel layout, and we added channel maps for cases that don't.

This doesn't feel like a limitation we're actually running up against, to make it worth making this more complex for everyone.

But maybe I'm misunderstanding the situation...?

[sylware]

Allright, you are already currently having this conversation!

My intent was to bring that forward for this conversation to happen.

It is a significant departure from SDL2 audio programming model, but I am not experienced enough from the user code perspective to predict how much more complex it would end up. It feels hardly more complex, but I don't think I am the right person to talk about it.

While coding for the alsa backend and skimming thru ffmpeg resample library headers, channel positions/specificatiosn centric felt so much "cleaner". That's only my take.

Keep us in touch with the results of arbitration on that matter!

[icculus]

Okay, we've talked about this further over here, and here's the current thinking:

• Audio will continue to work as it is, but...
• (in the future) we will add APIs to let you find out what speakers the elements of the PCM array apply to, and how to map between what you have and what you need in some form.

This doesn't change the ABI, so we'll be adding it later. Stay tuned!

[sylware]

@icculus @slouken

BTW, I did an update on the alsa patch, if you could update the pull request. See the last comment, I uploaded the replacement code on paste.c-net.org.

https://github.com/libsdl-org/SDL/pull/9753

[slouken]

@icculus, this is not just theoretical. I just ran into a case where Vorbis channel definition and SDL channel definition are different.

For 3 channel audio, SDL defines the channels as:

• Left
• Right
• LFE

For 3 channel audio Vorbis defines the channels as:

• Left
• Center
• Right

We can't just remap the channel indices, we need to actually provide the correct content for the expected speaker. Which means if we're mapping from 5.1 surround, we'd map channel 3 (LFE) to channel 2 for SDL output and channel 2 (FC) to channel 1 for Vorbis output.

[slouken]

Another case in point, if you're mapping 7.1 audio to 5.1, you need to know which of these 5.1 layouts you're using:

• AAUDIO_CHANNEL_5POINT1 = AAUDIO_CHANNEL_FRONT_LEFT |AAUDIO_CHANNEL_FRONT_RIGHT |AAUDIO_CHANNEL_FRONT_CENTER |AAUDIO_CHANNEL_LOW_FREQUENCY |AAUDIO_CHANNEL_BACK_LEFT |AAUDIO_CHANNEL_BACK_RIGHT,
• AAUDIO_CHANNEL_5POINT1_SIDE = AAUDIO_CHANNEL_FRONT_LEFT |AAUDIO_CHANNEL_FRONT_RIGHT |AAUDIO_CHANNEL_FRONT_CENTER |AAUDIO_CHANNEL_LOW_FREQUENCY |AAUDIO_CHANNEL_SIDE_LEFT |AAUDIO_CHANNEL_SIDE_RIGHT,

We even have a note in the SDL audio documentation that says 5.1 audio can be either of these layouts, with no way to define or query which is being used.

[icculus]

Kicking this into 3.2.0, but if this turns out to be a massive or disruptive change, I'm kicking it back to 3.x.

[slouken]

I don't know that you need to do that, I'm just running into it a lot right now. It's not blocking me, but boy it would be nice if we could express the speaker arrangement. Three channel audio could either be 2.1 with LFE, or 3 channel left-right-center, etc.

It looks like the standard way to do this is have a predefined order of speakers and then a mask that has bits set for the speakers that are enabled. If you then need to permute the speaker order you can do that with a channel map.

Here's how Android AAudio does it, for example: https://developer.android.com/ndk/reference/group/audio#group___audio_1gac36f475ca5b446f4fde4c9b90bec77c8