Player fails based on media playlists' referencing order in HLS

[mariocynicys]

Can you reproduce the issue with our latest release version? Yes

What browser and OS are you using? Chrome on Ubuntu 18.04

What did you do?

Used an HLS playlist that has 2 audio streams encoded in two different codecs (aac, ac3). What i did is that i flipped the order of their media playlists referencing in the master hls playlist. say it was like this:

#EXT-X-MEDIA:TYPE:AUDIO,~~~ <-- this one in ac3
#EXT-X-MEDIA:TYPE:AUDIO,~~~ <-- this one in aac

this one would play fine. I flipped them so that the aac codec is referenced first:

#EXT-X-MEDIA:TYPE:AUDIO,~~~ <-- this one in aac
#EXT-X-MEDIA:TYPE:AUDIO,~~~ <-- this one in ac3

What did you expect to happen? I expect the media to play disregarding which codec's media playlist was mentioned first in the master playlist.

What actually happened? When the aac codec's media playlist is mentioned first, the player produces these error(in order):

• code: 3016 message: "Shaka Error MEDIA.VIDEO_ERROR (4,,CHUNK_DEMUXER_ERROR_APPEND_FAILED: Unsupported audio format 0x61632d33 in stsd box.)"
• code: 3014 message: "Shaka Error MEDIA.MEDIA_SOURCE_OPERATION_FAILED (0)"
• code: 3015 message: "Shaka Error MEDIA.MEDIA_SOURCE_OPERATION_THREW (InvalidStateError: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.)"

Another example that will fail in whichever order the two codec are referenced(note1: produced by Shaka Packager. note2: this is an audio-only playlist):

#EXTM3U
## Generated with https://github.com/google/shaka-packager version c1f64e5-release

#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,URI="stream_0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="sv",NAME="stream_0",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="stream_1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="sv",NAME="stream_1",CHANNELS="2"

#EXT-X-STREAM-INF:BANDWIDTH=192500,AVERAGE-BANDWIDTH=192500,CODECS="ac-3",AUDIO="default-audio-group"
stream_0.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=131619,AVERAGE-BANDWIDTH=131619,CODECS="mp4a.40.2",AUDIO="default-audio-group"
stream_1.m3u8

Trying to switch the order of the media playlists and stream variant playlists will produce different type of error.

I have sent an email with the two test cases attached.

[TheModMaker]

The order here doesn't matter. What is happening is which audio stream gets selected by default will be the first one. You could reproduce this by manually selecting the other track. The problem is that we don't mark one of the streams as being ac-3 and treat both as being aac, which causes playback problems.

In the case of output_files.zip, you put both streams in the same audio group. We then see a CODECS attribute listing both codecs. This causes us to ignore the second codec and just use the first one. In this case, there is no way to tell (from the manifest alone) which stream uses which codec. We could just throw an error in this case saying we can't play it. But it would be better to put different codecs in different audio groups so we can tell which codec it uses.

In the case of output_files_audio_only.zip, the two streams are stored in different stream descriptors, so we can tell which is which. We still treat the two streams as using the same codec for some reason; it may be cause by using the same audio group again. We'll look into this case since we can fix it.

[mariocynicys]

@TheModMaker Aha got you. May you please take a look at this(the last example paragraph).

This definition of the rendition groups actually complies with the playlist generated by Shaka Packager, so I believe there would be a way to know what sample format each individual stream is without the need to guess it using the stream's group. Is that possible?

[TheModMaker]

I recognize this is valid content, but we need to know which stream uses which codec to be able to filter out streams and initialize MediaSource. We could figure out which is which by parsing the media segments, but that would require fetching them during initialization. We want to only use the info in the manifest.

Here is all the info we have in your case:

#EXT-X-MEDIA:TYPE=AUDIO,URI="stream_0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="sv",NAME="stream_0",AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="stream_1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="sv",NAME="stream_1",CHANNELS="2"

#EXT-X-STREAM-INF:BANDWIDTH=192500,AVERAGE-BANDWIDTH=192500,CODECS="avc1.4d481e,ac-3,mp4a.40.2",AUDIO="default-audio-group"
stream_3.m3u8

You should be able to see why we can't tell which is which. We assume stream_3.m3u8 is the video one since there is an audio group. But we have no way to know what is in the other streams. We have both stream_0.m3u8 and stream_1.m3u8 in the audio group, but we don't know which of them is the ac-3 and which is mp4a.40.2.

[mariocynicys]

This could be fixed by checking the other renditions if the one we picked turned out to be non-playable, right?? I can work on this if you find it worthy.

[joeyparrish]

I think it might be best fixed in Shaka Packager, to put incompatible audio codecs into different groups.

[TheModMaker]

Looking at the content again, the output_files_audio_only.zip content is actually invalid. The CODECS attribute should specify ALL the codecs for the streams. Since the audio group contains both codecs, then the two EXT-X-STREAM-INF tags should list both codecs.

Since this is either bad content or something we can't detect, I'm moving this to Shaka Packager and we can handle generating better content there.