Playing representations from multiple adaptation sets

[BucherTomas]

We aim to start providing content encrypted with different content keys based on quality groups. I was trying to follow the DASH-IF-IOP documentation which stipulates in its "7.7.8. Protection of Media Presentations that Include SD, HD and UHD Adaptation Sets" paragraph that:

• "Per DASH IF interop points, Representations with separate keys, licenses, and license policy are contained in different Adaptation Sets."

• "A player may download licenses for multiple Adaptation Sets in a Group, and seamlessly switch between them if it is able."

Dash.js, however, ignores in this scenario representations from the first adaptation set and plays only representations from the other, even though they all have the same encoding parameters that should matter when grouping playable tracks from the mpd manifest. I then removed the encryption and attempted it with clear content and experienced the same behavior, this is the stream URL https://utils.ssl.cdn.cra.cz/dash/1/manifest.mpd . All tracks in this test are encoded intentionally with letterbox in standard resolutions to ensure that they have absolutely the same PAR and SAR values.

I tried to group both video adaptation sets with the same group attribute without success and also tried on a whim the new, not finalized, approach mentioned in "3.8. Switching across Adaptation Sets" with schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016", again obviously without success as it is still unimplemented. This scheme method can now be seen in the test manifest.

The result I gleaned from this is that dash.js so far does not include the means to select tracks from multiple adaptation sets or I am doing something wrong. On the other hand, Shaka player can play my stream correctly with representations selected from all adaptation sets even without using group attribute or the new scheme. True, Shaka seems to be at the forefront supporting this, because for instance Bitmovin and THEOplayer are in the same boat with dash.js.

I couldn't find in the specs anything that should prevent this, so can dash.js start supporting this feature as well?

[BucherTomas]

Some feedback would be appreciated. At the very least please respond with something akin to:

1. makes sense, we have it on backlog and plan to add it soonish
2. makes sense, but we do not have the resources now
3. the whole approach is not finalized yet and until it is clarified, we do not wish to spend any time on it

Thank you

[epiclabsDASH]

@BucherTomas, sorry for the late response.

Your request makes sense but this is not something we are prioritizing right now. We are stabilizing some base stuff before starting with more complex scenarios like this one. For example, we are reviewing DRM support to be able to have different keys/schemas across different adaptations or representations that is currently not supported.

@BucherTomas, I keep this issue open so we can all track its progress. I will keep you updated regarding any progress regarding it.

Thanks for the patience.

[BucherTomas]

Thanks for the update.

We are on the same page then, we also want to have different keys across different adaptation sets, the ability to group everything from these different adaptation sets and play as single adaptive stream is just a logical outcome, it is not any additional complex scenario.

[tdaines]

Any update on this issue? Is it planned for a future release?

[BucherTomas]

Have you managed to make any progress in the DRM review please or is still in the pipeline?

It is a bit confusing to have reference content such as the modern Axinom assets with multiple keys for so long and yet still being unable to play them properly since dash.js picks Representations from single Adaptation Set only.

The streaming industry and content licensors have been leaning towards using different security rules for SD and HD content and thus using multiple keys for some time now, I am very hopeful that dash.js is going to catch up soon.

It would be great to hear some positive news about this topic. Thank you.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[BucherTomas]

@dsilhavy It seems that the maintainer I was in contact with here is no longer active in the project.

Can you please share some update on the matter? I asked about it also in the newly opened discussions thread and coincidentally the stale bot a couple of minutes later warned here that this issue might be closed. I'm striving to keep it open for now by commenting.

Thank you.

[dsilhavy]

We added support for switching between Representations of different AdaptationSets in 4.0, nigthly version is available here: https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html

@BucherTomas Can you do some tests on your side? When switching between content with different keys I sometimes run into a decode error. Have to investigate further what causes this

[BucherTomas]

@dsilhavy Thank you for working on this. I did test with a couple of streams on Nightly build, but cannot see any difference. The player still picks video tracks from a single adaptation set.

As an example, this is reproducible also with dash.js's own test vector "DRM (Modern]/[Axinom] 1080p with PlayReady and Widevine DRM, multiple keys" even though the player does request multiple licenses since it sees that there are multiple keys. EME keystatus reports that all keys are usable, so all tracks should be usable, but the player picks only 720p and 1080p renditions with one key and ignores SD renditions with another key.

[dsilhavy]

@BucherTomas My description wasnt precise enough. dash.js will not switch between different AdaptationSets when doing ABR. However, you can switch AdaptationSets in the application which should not cause a reset of the MSE anymore. Once the new AdaptationSet is played ABR will performed with the "new" Representations.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. However, it might still be relevant so please leave a short comment if it should remain open. Otherwise the issue will be closed automatically after two weeks. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because no further activity occurred. If you think this issue is still relevant please reopen it or contact @dsilhavy. Thank you for your contributions.

[synologic]

+1

The multi key samples are just using separate keys for audio tracks, but all video tracks use the same key.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. However, it might still be relevant so please leave a short comment if it should remain open. Otherwise the issue will be closed automatically after two weeks. Thank you for your contributions.

[dsilhavy]

unstale

[TJ-Perez]

Hey, is there any plans to implement switching between representations of different adaptation sets while doing ABR? I am not certain if this is what is meant by "seamless switching" in section 3.8 of the DASH-IF guidelines, however this is how I interpreted the document.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. However, it might still be relevant so please leave a short comment if it should remain open. Otherwise the issue will be closed automatically after two weeks. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because no further activity occurred. If you think this issue is still relevant please reopen it or contact @dsilhavy. Thank you for your contributions.

[dsilhavy]

Duplicate of #3908. Support for schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016 was added in #4276. The work on dash.js v.5 is still ongoing and in an early phase but the current build of the v5 branch is automatically deployed here: https://reference.dashif.org/dash.js/v5/samples/dash-if-reference-player/index.html