mpv does not map channels correctly on devices with more than 16 outputs

[Arve]

mpv version and platform

MacOS, latest release (Issues are visible in code on current master branch, see links below)

Reproduction steps

1. Use audio device with more than 16 channels, and a surround layout
2. Start mpv
3. Play file with surround content

Expected behavior

1. That mpv maps the channel layout correctly

Actual behavior

mpv spits out the following error message in the log

[ 233.167][v][ao/coreaudio] layout has too many descriptions (76, max: 16)

This is due to the following define in https://github.com/mpv-player/mpv/blob/ae8b531207b3206b9ff1ab7d17092899bd67c565/audio/chmap.h#L25

#define MP_NUM_CHANNELS 16

This is then checked here: https://github.com/mpv-player/mpv/blob/ae8b531207b3206b9ff1ab7d17092899bd67c565/audio/out/ao_coreaudio_chmap.c#L172

After this, mpv falls back to either of tfl-tfr layout or mono (seemingly depending on how the channel mapping is set up in Audio MIDI Setup).

While I realize that this code is in as a fix for "Problem with external audio interface on OS X" (Bug #1249) - it causes issues with a number of relatively commonplace multichannel audio interfaces that have channel counts that extend beyond 16.

[Arve]

I mentioned this on the mpv-devel channel, but just thought I'd log it here as well. Things I've attempted to see if the issue could be mitigated

1. Removing the check from 1249 - that merely causes mpv to crash when it tries to play audio with the 76-channel device with Abort trap: 6 on the command line
2. Tried upping MP_NUM_CHANNELS to 80, while keeping the check in place. mpv doesn't crash, but downmixes everything to stereo.

The relevant lines in the log from attempt 2 are found here: https://pastebin.com/qmcsVmna

[Arve]

The full log: https://pastebin.com/hCbwefqp

[hmedia1]

I mentioned this on the mpv-devel channel, but just thought I'd log it here as well. Things I've attempted to see if the issue could be mitigated

1. Removing the check from 1249 - that merely causes mpv to crash when it tries to play audio with the 76-channel device with Abort trap: 6 on the command line
2. Tried upping MP_NUM_CHANNELS to 80, while keeping the check in place. mpv doesn't crash, but downmixes everything to stereo.

The relevant lines in the log from attempt 2 are found here: https://pastebin.com/qmcsVmna

I basically tried the same hacks as you, and got either no end result change, or the Abort trap: 6.

Non tampered log trying to output a 16 channel stream to a 64 channel CoreAudio device:

[ao/coreaudio] input channel layout:
[ao/coreaudio] layout: tag: <0>, bitmap: <0>, descriptions <2>
[ao/coreaudio]  - description 0: label <1, 0>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio]  - description 1: label <2, 1>,  flags: <0>, coords: <0.000000, 0.000000, 0.000000>
[ao/coreaudio] mp chmap: stereo
[ao/coreaudio] Channel layouts:
[ao/coreaudio]  - stereo
[ao/coreaudio]  - mono
[ao/coreaudio] result: stereo
[ao/coreaudio] Latency property cntl: 0 frames
[ao/coreaudio] Latency property zisf: 512 frames
[ao/coreaudio] Latency property tfas: 0 frames
[ao/coreaudio] audiounit latency [us]: 333
[ao/coreaudio] device latency [us]: 10666
[ao/coreaudio] using soft-buffer of 9600 samples.
[cplayer] AO: [coreaudio] 48000Hz stereo 2ch floatp
[cplayer] AO: Description: CoreAudio AudioUnit
[af] Adding filter lavrresample 
[lavrresample] Remix: fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl -> stereo
[af] Audio filter chain:
[af]   [in] 48000Hz unknown16 (empty) 16ch floatp
[af]   [lavrresample] 48000Hz stereo 2ch floatp [a]
[af]   [out] 48000Hz stereo 2ch floatp
[af]   [ao] 48000Hz stereo 2ch floatp```

[hmedia1]

@Arve did you end up solving this for yourself ?

[Arve]

I basically never found a good fix, no, and is basically a tad beyond my knowledge of CoreAudio

[hmedia1]

Further information and findings:

mpv version and platform

• 865627d
• Mac OS X 10.13.1 High Sierra
• Mac OS X 10.14 Mojave

Reproduction steps

Launch a newly compiled (or very old version or anything in between) mpv with a multi channel audio file using the following command:

./mpv --audio-device=coreaudio/AppleUSBAudioEngine:KLARK TEKNIK:DN32-USB:0A9D35E6:1,2' --ad-lavc-downmix=no /Users/user/testmulti.wav --video=no --coreaudio-change-physical-format=yes --audio-channels=auto -v

Expected behavior

Audio channels would be mapped 1:1 to the coreaudio inputs.

Actual behavior

If the interface has more than 16 channels (any coreaudio interface - even virtual ones), it either downmixes to stereo, or fails.

Specific example below of a coreaudio hardware card that is switchable between 16 channel and 32 channel IO:

---

When card is in 16 channel mode (Log File #1):

Playback is fine:

---

When card is in 32 channel mode (Log File #2):

Playback is stereo:

---

When using a recompiled version of mpv (changing #define MP_NUM_CHANNELS 16 to #define MP_NUM_CHANNELS 32 in ./audio/chmap.h), in an attempt to fix (Log File #3):

Playback fails:

[   0.086][i][cplayer] AO: [coreaudio] 48000Hz fl-fr-bl-br-fc-bc-wl-wr-na-na-na-na-na-na-na-na-na-na-na-na-na-na-na-na
[   0.086][v][cplayer] AO: Description: CoreAudio AudioUnit
[   0.086][v][autoconvert] inserting resampler
[   0.086][v][swresample] format change, reinitializing resampler
[   0.086][v][swresample] 48000Hz unknown16 s16 -> 48000Hz fl-fr-bl-br-fc-bc-wl-wr-na-na-na-na-na-na-na-na-na-na-na-na-na- s16
[   0.086][v][swresample] Remix: fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl -> fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl-tbc-tbr-
[   0.086][e][ffmpeg] SWR: Output channel layout '32 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC+BC+SL+SR+TC+TFL+TFC+TFR+TBL+TBC+TBR+DL+DR+WL)' is not supported
[   0.086][e][swresample] Cannot open Libavresample context.
[   0.086][f][swresample] libswresample failed to initialize.
[   0.086][f][af] Cannot convert decoder/filter output to any format supported by the output.
[   0.086][v][af] filter output EOF
[   0.086][v][ad] Uninit decoder.
[   0.086][v][af] dropping frame due to pin disconnect

---

Incidentally - afplay works with the card in 32 channel mode just fine.

---

Log file

Links above in behavior headings

Sample files

https://0x0.st/sIr_.zip 16 channels of the same drum overhead mic 10 second file that looks like this:

Input #0, wav, from '/Users/user/testmulti.wav':
  Metadata:
    encoder         : Lavf58.19.102
  Duration: 00:00:10.00, bitrate: 12288 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 16 channels, s16, 12288 kb/s

All of the above behavior can be replicated using a virtual interface - i.e. Soundflower (64ch) plays as stereo, but recompiling it to Soundflower (16ch), plays fine.

[hmedia1]

It's not just that mpv can't play more than 16 channels, it's that it can't map above 16 channels.

Even by running two processes, there is no way to offset the first channel to start at 17.

[oxnatun]

@hmedia1 @Arve Hello! Can you explain how to compile mpv version with "#define MP_NUM_CHANNELS 80" option, or just share the app. I am novice in programming, but I have same problem with mpv. Thanks!

[hmedia1]

@hmedia1 @Arve Hello! Can you explain how to compile mpv version with "#define MP_NUM_CHANNELS 80" option, or just share the app. I am novice in programming, but I have same problem with mpv. Thanks!

Simply changing the directive doesn't do anything except crash the application.

The reason this thread is still open is because there's no solution for outputting more than 16 channels in mpv to coreaudio devices.

[hmedia1]

Trying to return to a state before #1249 - this is as far as I get;

[swresample] Remix: fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl-tbc-tbr- -> stereo

[ao/coreaudio] input channel layout:
[ao/coreaudio] layout: tag: , bitmap: , descriptions 
[ao/coreaudio]  - description 0: label ,  flags: , coords: 
[ao/coreaudio]  - description 1: label ,  flags: , coords: 
[ao/coreaudio]  - description 2: label ,  flags: , coords: 
[ao/coreaudio]  - description 3: label ,  flags: , coords: 
[ao/coreaudio]  - description 4: label ,  flags: , coords: 
[ao/coreaudio]  - description 5: label ,  flags: , coords: 
[ao/coreaudio]  - description 6: label ,  flags: , coords: 
[ao/coreaudio]  - description 7: label ,  flags: , coords: 
[ao/coreaudio]  - description 8: label ,  flags: , coords: 
[ao/coreaudio]  - description 9: label ,  flags: , coords: 
[ao/coreaudio]  - description 10: label ,  flags: , coords: 
[ao/coreaudio]  - description 11: label ,  flags: , coords: 
[ao/coreaudio]  - description 12: label ,  flags: , coords: 
[ao/coreaudio]  - description 13: label ,  flags: , coords: 
[ao/coreaudio]  - description 14: label ,  flags: , coords: 
[ao/coreaudio]  - description 15: label ,  flags: , coords: 
[ao/coreaudio]  - description 16: label ,  flags: , coords: 
[ao/coreaudio]  - description 17: label ,  flags: , coords: 
[ao/coreaudio]  - description 18: label ,  flags: , coords: 
[ao/coreaudio]  - description 19: label ,  flags: , coords: 
[ao/coreaudio]  - description 20: label ,  flags: , coords: 
[ao/coreaudio]  - description 21: label ,  flags: , coords: 
[ao/coreaudio]  - description 22: label ,  flags: , coords: 
[ao/coreaudio]  - description 23: label ,  flags: , coords: 
[ao/coreaudio]  - description 24: label ,  flags: , coords: 
[ao/coreaudio]  - description 25: label ,  flags: , coords: 
[ao/coreaudio]  - description 26: label ,  flags: , coords: 
[ao/coreaudio]  - description 27: label ,  flags: , coords: 
[ao/coreaudio]  - description 28: label ,  flags: , coords: 
[ao/coreaudio]  - description 29: label ,  flags: , coords: 
[ao/coreaudio]  - description 30: label ,  flags: , coords: 
[ao/coreaudio]  - description 31: label ,  flags: , coords: 
[ao/coreaudio] mp chmap: unknown32
[ao/coreaudio] input channel layout:
[ao/coreaudio] layout: tag: , bitmap: , descriptions 
[ao/coreaudio]  - description 0: label ,  flags: , coords: 
[ao/coreaudio]  - description 1: label ,  flags: , coords: 
[ao/coreaudio] mp chmap: stereo
[ao/coreaudio] Channel layouts:
[ao/coreaudio]  - stereo
[ao/coreaudio]  - mono
[ao/coreaudio] result: stereo
[ao/coreaudio] Latency property cntl: 192 frames
[ao/coreaudio] Latency property zisf: 512 frames
[ao/coreaudio] Latency property tfas: 0 frames
[ao/coreaudio] audiounit latency [us]: 0
[ao/coreaudio] device latency [us]: 14666
[ao/coreaudio] using soft-buffer of 9600 samples.
AO: [coreaudio] 48000Hz stereo 2ch float
[cplayer] AO: Description: CoreAudio AudioUnit
[autoconvert] inserting resampler
[swresample] format change, reinitializing resampler
[swresample] 48000Hz unknown32 float -> 48000Hz stereo float
[swresample] Remix: fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl-tbc-tbr- -> stereo
[ffmpeg] SWR: Input channel layout '32 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC+BC+SL+SR+TC+TFL+TFC+TFR+TBL+TBC+TBR+DL+DR+WL)' is not supported
[swresample] Cannot open Libavresample context.
[swresample] libswresample failed to initialize.
Cannot convert decoder/filter output to any format supported by the output.
[af] filter output EOF
[ad] Uninit decoder.
[af] dropping frame due to pin disconnect
[lavf] deselect track 0
Audio: no audio
[cplayer] playback restart complete
[cplayer] EOF code: 6  
[cplayer] finished playback, no audio or video data played (reason 4)


Exiting... (Errors when loading file)

The FFmpeg that mpv is compiled with, on it's own can handle the same file;

 Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, 32 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC+BC+SL+SR+TC+TFL+TFC+TFR+TBL+TBC+TBR+DL+DR+WL), s16, 24576 kb/s (default)

As can the included afplay utility on OSX:

Playing file: file.wav
Playing format: AudioStreamBasicDescription: 32 ch,  48000 Hz, 'lpcm' (0x0000000C) 16-bit little-endian signed integer
Buffer Byte Size: 1536000, Num Packets to Read: 24000

I'll keep debugging. If anyone has insight on whether or not this is likely a waste of time and a realized limitation based on the current implementation of the audio system as a whole, would be appreciated

[zc62]

I have a commit that makes mpv print the complete channel layout: https://github.com/zc62/mpv/commit/bb04a7f7a3280b0f5c590764ca9ff978721314e2

On my system it shows:

[autoconvert] inserting resampler
[swresample] format change, reinitializing resampler
[swresample] 48000Hz mono u8 -> 48000Hz fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl-tbc-tbr-sp18-sp19-sp20-sp21-sp22-sp23-sp24-sp25-sp26-sp27-sp28-dl-dr-wl u8
[swresample] Remix: mono -> fl-fr-fc-lfe-bl-br-flc-frc-bc-sl-sr-tc-tfl-tfc-tfr-tbl-tbc-tbr-sp18-sp19-sp20-sp21-sp22-sp23-sp24-sp25-sp26-sp27-sp28-dl-dr-wl
[ffmpeg] SWR: Output channel layout '32 channels (FL+FR+FC+LFE+BL+BR+FLC+FRC+BC+SL+SR+TC+TFL+TFC+TFR+TBL+TBC+TBR+DL+DR+WL)' is not supported
[swresample] Cannot open Libavresample context.
[swresample] libswresample failed to initialize.
[af] Cannot convert decoder/filter output to any format supported by the output.

sp18 to sp28 are unassigned speakers, since the assigned speakers defined in https://github.com/mpv-player/mpv/blob/8b563a034604ff5ab2ad92d12c63e806f45d1bb6/audio/chmap.h#L30-L57 range from 0 to 17, then from 29 we have a few unofficial extensions.

mpv generates the layout code that will be fed to FFmpeg in this way: https://github.com/mpv-player/mpv/blob/8b563a034604ff5ab2ad92d12c63e806f45d1bb6/audio/chmap.c#L247 So for 32 channels it is (1ULL << 32) - 1.

Then from the code defined in FFmpeg: https://github.com/FFmpeg/FFmpeg/blob/a899b3b3c5f55e6c527f8aa81dbe15edb9fab63f/libavutil/channel_layout.h#L49-L73 we can understand why the maximum number of channels was limited to 16. Actually the maximum number allowed is 18. Since (1ULL << 18) - 1 produces a layout that all assigned speakers are active. While (1ULL << 32) - 1 does not produce a meaningful layout, FFmpeg tries to ignore the speakers from 18 to 28, so it says the layout is FL+FR+FC+LFE+BL+BR+FLC+FRC+BC+SL+SR+TC+TFL+TFC+TFR+TBL+TBC+TBR+DL+DR+WL. Probably the next thing happens is the layout cannot pass the sanity check in libswresample: https://github.com/FFmpeg/FFmpeg/blob/a899b3b3c5f55e6c527f8aa81dbe15edb9fab63f/libswresample/rematrix.c#L144-L154

FFmpeg itself can work with this layout probably because they just have this sanity check in libswresample, while not in other parts.

[Nu7ron]

Hi @Akemi @pigoz @wm4 Is there an update for this issue? I'm facing the same problems with a Symphony I/O 8x8 ... too many channels -> stereo fallback.

[hmedia1]

I have a commit that makes mpv print the complete channel layout: zc62@bb04a7f ...

Hey @zc62 - I must have missed this post. Does this commit fix the issues described in your testing, or is your post about the behavior after compiling with this commit ?

[zc62]

Hey @zc62 - I must have missed this post. Does this commit fix the issues described in your testing, or is your post about the behavior after compiling with this commit ?

The behavior after compiling with this commit. My commit does not fix the issue, it just produces more information to see what could be the cause. If I remember correctly, I believe it is because FFmpeg does not support huge channel numbers such as 32 (18 is the upper limit I guess), though in some parts of FFmpeg, 64 channels are supported, but only internally (inside the code logic of FFmpeg itself, but not intended for FFmpeg users).

[braymullo]

Hi, coming across this issue on my Scarlett 18i20 (20 Outputs). Auto down mixes to 2chan Stereo. Outputting to a virtual device like BlackHole (16chan) works, as hmedia1 confirms with their similar SoundFlower test.

Any fixes or help?

[kasper93]

Will be fixed by https://github.com/mpv-player/mpv/pull/11174