preferredAudioChannelCount not filtering variants as expected

[willdharris]

Have you read the FAQ and checked for duplicate open issues? yes

If the problem is related to FairPlay, have you read the tutorial?

not related

What version of Shaka Player are you using?

4.8.0

Can you reproduce the issue with our latest release version? yes

Can you reproduce the issue with the latest code from main? yes

Are you using the demo app or your own custom app? custom

If custom app, can you reproduce the issue using our demo app? yes

What browser and OS are you using? LG TV WebOS / Chrome 87

For embedded devices (smart TVs, etc.), what model and firmware version are you using? LG UQ7590PUB 03.34.00

What are the manifest and license server URIs?

https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8?93443

What configuration are you using? What is the output of player.getConfiguration()?

Default config.

What did you do?

On a device that supports surround sound:

1. Load a stream that contains both stereo and surround audio renditions.

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a1",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",URI="a1/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a2",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="6",URI="a2/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="a3",NAME="English",LANGUAGE="en-US",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="6",URI="a3/prog_index.m3u8"

3. Keep default player config, or explicitly set preferredAudioChannelCount = 2 which is the default value.
4. Call player.getVariantTracks().

What did you expect to happen? Expected the tracks array to only contain variants with 2 audio channels, as it does in Shaka v4.5 and previous versions.

What actually happened?

The tracks array contains variants of all audio channel counts that are in the manifest.

[willdharris]

This causes an issue when the player tries to switch to variant with a different audio codec. The player freezes and does not resume; cannot seek or pause/play out. No error is thrown. Last log emitted is Change Type: audio/mp4; codecs="mp4a.40.2" -> audio/mp4; codecs="ec-3".

[avelad]

preferredAudioChannelCount is not a variant filter, what it does is indicate to the ABR which channel we should start with.

[willdharris]

@avelad It looks like preferredAudioChannelCount was considered in the variant filtering up until Shaka v4.6, and was removed with https://github.com/shaka-project/shaka-player/pull/5859.

If this is working as intended, when a manifest has variants with 2 channels of audio and variants with 6 channels of audio, is there a way I can now configure Shaka to provide me with the variants that only have 2 channels of audio? preferredAudioChannelCount used to do that for us, now we get variants with mixed audio channel counts.

[willdharris]

@avelad Here is the change where preferredAudioChannelCount was removed from variant filtering. https://github.com/shaka-project/shaka-player/pull/5859/files#diff-2e884949ae9f1e9761d33eae703f9ec3795cbe6288c5b79ab2b488daafaaacc2

We previously relied on that filtering to give us a variant track list with only stereo audio. Without it we can end up with a variant track list of differing audio codecs and channel counts like this.

In this scenario, if I try switching to a variant with a different audio codec and channel count, playback either permanently freezes or the audio just completely drops after the switch.

I have not seen this be an issue when auto abr switching is enabled, I'm assuming auto switching only considers variants of the same codec or channel count. But we have some use cases where we provide users the option to manually switch variants and that's when the issue occurs.

I noticed the https://github.com/shaka-project/shaka-player/pull/5859 fix where this change was introduced, was ultimately not the necessary fix for it's parent issue: https://github.com/shaka-project/shaka-player/issues/5858. It looks like another fix was needed: https://github.com/shaka-project/shaka-player/pull/5869. I'm wondering if https://github.com/shaka-project/shaka-player/pull/5859 could be reverted or reimplemented?

But if that change is there as intended, what would now be the recommended way to configure the player to filter for variants with 2 audio channels?

[avelad]

@willdharris I think this should be implemented as a restriction and create minChannelsCount and macChannelsCount (as we have minFrameRate and maxFrameRate for example). If you agree with this, I can implement it next week.

[willdharris]

@avelad That sounds good. A restrictive minChannelsCount and maxChannelsCount makes sense and is semantically more clear than preferredAudioChannelCount for this case. The initial concern was that there did seem to be a breaking change with https://github.com/shaka-project/shaka-player/pull/5859, which we could workaround by setting preferredAudioCodecs to specific 2 channel codecs. But these new configs would be more flexible. A question to consider for the implementation, would preferredAudioCodecs take precedence over minChannelsCount and maxChannelsCount as it did with preferredAudioChannelCount previously?

[avelad]

No, the most restrictive is the new config, so this new config will have more priority.