Support client-side ads in DASH multi period

[dejan-korchev-deltatre]

Hello,

We having a case where we set Client-side Ads on DASH Multi period video, but video stop after add and we receive the error.

If possible, can you provide an example for the right implementation or some kind of help?

The version of ExoPlayer that I am using is Media3 1.4.0 Thank you

Custom Test Case:

> {
  "name": "Client-side Ads on DASH Multi period video",
  "uri": "https://dash.akamaized.net/dash264/TestCases/5a/nomor/1.mpd",
  "ad_tag_uri": "https://pubads.g.doubleclick.net/gampad/ads?sz=640x480&iu=/124319096/external/single_ad_samples&ciu_szs=300x250&impl=s&gdfp_req=1&env=vp&output=vast&unviewed_position_start=1&cust_params=deployment%3Ddevsite%26sample_ct%3Dlinear&correlator="
}

Error:

Caused by: java.lang.IllegalArgumentException
                                                                                                          at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:40)
                                                                                                          at androidx.media3.exoplayer.source.ads.AdsMediaSource.onChildSourceInfoRefreshed(AdsMediaSource.java:289)
                                                                                                          at androidx.media3.exoplayer.source.ads.AdsMediaSource.onChildSourceInfoRefreshed(AdsMediaSource.java:63)
                                                                                                          at androidx.media3.exoplayer.source.CompositeMediaSource.lambda$prepareChildSource$0$androidx-media3-exoplayer-source-CompositeMediaSource(CompositeMediaSource.java:117)
                                                                                                          at androidx.media3.exoplayer.source.CompositeMediaSource$$ExternalSyntheticLambda0.onSourceInfoRefreshed(D8$$SyntheticClass:0)
                                                                                                          at androidx.media3.exoplayer.source.BaseMediaSource.refreshSourceInfo(BaseMediaSource.java:90)
                                                                                                          at androidx.media3.exoplayer.source.MaskingMediaSource.onChildSourceInfoRefreshed(MaskingMediaSource.java:211)
                                                                                                          at androidx.media3.exoplayer.source.WrappingMediaSource.onChildSourceInfoRefreshed(WrappingMediaSource.java:154)
                                                                                                          at androidx.media3.exoplayer.source.WrappingMediaSource.onChildSourceInfoRefreshed(WrappingMediaSource.java:49)
                                                                                                          at androidx.media3.exoplayer.source.CompositeMediaSource.lambda$prepareChildSource$0$androidx-media3-exoplayer-source-CompositeMediaSource(CompositeMediaSource.java:117)
                                                                                                          at androidx.media3.exoplayer.source.CompositeMediaSource$$ExternalSyntheticLambda0.onSourceInfoRefreshed(D8$$SyntheticClass:0)
                                                                                                          at androidx.media3.exoplayer.source.BaseMediaSource.refreshSourceInfo(BaseMediaSource.java:90)
                                                                                                          at androidx.media3.exoplayer.dash.DashMediaSource.processManifest(DashMediaSource.java:937)
                                                                                                          at androidx.media3.exoplayer.dash.DashMediaSource.onManifestLoadCompleted(DashMediaSource.java:718)
                                                                                                          at androidx.media3.exoplayer.dash.DashMediaSource$ManifestCallback.onLoadCompleted(DashMediaSource.java:1395)
                                                                                                          at androidx.media3.exoplayer.dash.DashMediaSource$ManifestCallback.onLoadCompleted(DashMediaSource.java:1390)

[tonihei]

Client-side ad insertion in multi-period DASH streams is not yet supported I'm afraid. There is an existing issue in https://github.com/google/ExoPlayer/issues/3693 to track this enhancement. I'll close the other one to move the request over to the Media3 issue tracker (and also because the other issue mixed multiple things together that have since been addressed).

[mayurk2]

@tonihei I believe it is still not supported. Do you have any workaround / suggestions on how can we support this use-case?

[tonihei]

Don't really know a workaround that would maintain the IMA integration nicely I'm afraid.

[mayurk2]

:( @tonihei We are not using IMA SDK but using AdsMediaSource directly. Let us know if there is any possible way to support the multi period DASH with ads. Thanks.

[stevemayhew]

@mayurk2 the issue is AdsMediaSource itself, the doc comment says it all:

/**
 * A {@link MediaSource} that inserts ads linearly into a provided content media source.
 *
 * <p>The wrapped content media source must contain a single {@link Timeline.Period}.
 */
@UnstableApi
public final class AdsMediaSource extends CompositeMediaSource<MediaPeriodId> {

A DASH playlist with multiple DASH Period's will have multiple Timeline.Period's.

@tonihei would no better if the limitation is with AdsMediaSource or CompositeMediaSource (which looks pretty painful to even think about changing).

It looks to me like it's AdsMediaSource which only has a single member Timeline.Period and single period id to call CompositeMediaSource.prepareChildSource() with.

This is a pretty fundamental requirement, when there is key/license rotation in a DASH playlist there will be multiple periods.

[marcbaechinger]

The limitation obvious to me is in AdsMediaSource indeed. First off, this would be about supporting multi-period as the content source, while the inserted ad streams still must be single period ads sources like ProgressiveMediaSource, HlsMediaSource or single-period DashMediaSource. I haven't thought it trough, but I think supporting multi-period as ad sources would add another layer of complexity on top.

As a first step, the major complexity to solve is that AdsLoader offers an interface AdsLoader.EventListener.onAdPlaybackState(AdPlaybackState) that is based on a single AdPlaybackState. In the Timeline the AdPlaybackState has an 1:1 relationship with Timeline.Period that is having a field adPlaybackState. Hence the current AdsLoader.EventListener.onAdPlaybackState(adPlaybackState) isn't simply reusable if we use a multi-period streams that requires an AdPlaybackState for each period. Facing a similar situation ServerSideAdsMediaSource offers an setAdPlaybackState(Map<Object, AdPlaybackState> that can handle a map with periodUid as the key to provide multiple `AdPlyabckStates. This allows to support multi-period streams but makes usage and implementation on either side of the interface more difficult and fiddly.

For AdsLoader of the client side AdsMediaSource introducing multiple ad playback states seems to be a disruptive change that would possibly break existing AdsLoader implementations out there and make it more difficult to implement it even for single period. An option would be to keep the AdsLoader as it is and to make AdsMediaSource map the single AdPlaybackState that the user passes into EventListener.onAdPlayabckState(adPlaybackState) to multiple periods/AdPlaybacKStates that are existent in the internal content media source. We don't know yet how feasible this approach would be, and whether it is natural for a user to think of a multi-period stream in a single AdPlaybackState. I think this is an option though that we should look into. At least for multi-period VOD this seems doable.

Besides these basic thoughts we haven't worked on this yet and we don't have concrete plans to start this by now.

@mayurk2 @stevemayhew

Given AdsMediaSource is currently only supporting VOD streams, I assume you are asking this for multi-period VOD streams. Am I correct with this assumption? Supporting multi-period VOD seems to be the next step. Live would be a bit more challenging than what is described above due to the nature of live streams.

(aside: With the above I haven't thought about DRM as in, I have assumed it just works the same way as non-DRM streams)

[stevemayhew]

The limitation obvious to me is in AdsMediaSource indeed. First off, this would be about supporting multi-period as the content source, while the inserted ad streams still must be single period ads sources like ProgressiveMediaSource, HlsMediaSource or single-period DashMediaSource. I haven't thought it trough, but I think supporting multi-period as ad sources would add another layer of complexity on top.

I don't see any use-case for the ads themselves to be multi-period. The ads we see generated by GAM are typically a single ProgressiveMediaSource or pod of multiple ProgressiveMediaSource 30 - 60 seconds long.

The ask is simply to support an AdsMediaSource that wraps a multi-period DASH content MediaSource child.

One use-case we have is ads inserted pre-roll on VOD or SoCu (Start-over Catch-up is a feature to allow playback of live streams in the past, restarting an offering at the beginning of the show). Multi-period DASH VOD might be possible, if ads are also inserted by DAI in the MPD. It is very common for live streams (and thus SoCu) to support key rotation.

As a first step, the major complexity to solve is that AdsLoader offers an interface AdsLoader.EventListener.onAdPlaybackState(AdPlaybackState) that is based on a single AdPlaybackState. In the Timeline the AdPlaybackState has an 1:1 relationship with Timeline.Period that is having a field adPlaybackState. Hence the current AdsLoader.EventListener.onAdPlaybackState(adPlaybackState) isn't simply reusable if we use a multi-period streams that requires an AdPlaybackState for each period. Facing a similar situation ServerSideAdsMediaSource offers an setAdPlaybackState(Map<Object, AdPlaybackState> that can handle a map with periodUid as the key to provide multiple `AdPlyabckStates. This allows to support multi-period streams but makes usage and implementation on either side of the interface more difficult and fiddly.

For AdsLoader of the client side AdsMediaSource introducing multiple ad playback states seems to be a disruptive change that would possibly break existing AdsLoader implementations out there and make it more difficult to implement it even for single period. An option would be to keep the AdsLoader as it is and to make AdsMediaSource map the single AdPlaybackState that the user passes into EventListener.onAdPlayabckState(adPlaybackState) to multiple periods/AdPlaybacKStates that are existent in the internal content media source. We don't know yet how feasible this approach would be, and whether it is natural for a user to think of a multi-period stream in a single AdPlaybackState. I think this is an option though that we should look into. At least for multi-period VOD this seems doable.

Assuming only pre or post roll ads might simplify this, unless I'm missing something, as the ad playback should not care at all about the content other than it's total duration (for the post-roll case). Mid-roll, on the other hand is a timed break into the content. At any rate the AdPlaybackState.AdGroup.timeUs would still remain a period time, logic would simply need to determine which Timeline.Period in the content would place the ad pre or post roll as required. So the actual ad playback should only care about the specific Timeline.Period is is inserted into.

I'm sure there are many subtle architectural elements I'm missing or glossing over as I only see the code not the discussions behind creating it.

Given AdsMediaSource is currently only supporting VOD streams, I assume you are asking this for multi-period VOD streams. Am I correct with this assumption? Supporting multi-period VOD seems to be the next step. Live would be a bit more challenging than what is described above due to the nature of live streams.

Live is almost entirely dominated by SSAI, the only use-case I see for semi-live is a SoCu assest that is in-progress (at least one Period in the MPD is not yet closed). These ads would always be pre-roll so again the insertion point would always be in the first period.

(aside: With the above I haven't thought about DRM as in, I have assumed it just works the same way as non-DRM streams)

Interestingly, when I simply removed the assertions that Andrew put in that check period count playback failed because licenses were not fetched properly.

So, the main question is were the asserts and constraints added only because having multiple periods in the content makes locating the insertion point more complex?

?

[tonihei]

So, the main question is were the asserts and constraints added only because having multiple periods in the content makes locating the insertion point more complex?

Sorry, I never replied back here: this is a very good summary of the current state of affairs.

Supporting multi-period likely means the AdsLoader needs to provider a "global" AdPlaybackState (across all periods) and AdsMediaSource then needs to figure out how to distribute the ads correctly to each period, creating a reduced-scope and adjusted AdPlaybackState for each period. There may be a few more complications though. If the AdsLoader doesn't expect multi-period content, it will be surprised when it retrieves the AdPlaybackState back from the Player directly and it doesn't match its internal state. So there needs to be a reverse mapping as well to let ads loaders operate across all of the periods.

An alternative (which @marcbaechinger described above already) is to let the ads loader to the splitting and merging internally. Then AdsMediaSource needs to have an API that accepts a List/Map of AdPlaybackStates instead of just a single one.

It also sounds to me that we would move this splitting and merging logic to some centralized util to make it easily reusable for custom ads loaders as all. I think we already have something like this for the server-side ad insertion media sources, but not sure how reusable the logic is for other cases.