Approaches to lazily loading content and building a playlist

[sampengilly]

When using a MediaService and a MediaController for background playback, there are certain responsibilities handled by either side of that arrangement. But I'm finding that things feel a little inconsistent on the Service side when dealing with lots of different use cases, and I want to check to see if there are any recommended ways to go about dealing with these inconsistencies.

The first thing to cover is populating media items in the Service, since there are limitations carrying some data over the process boundary. We have the MediaSession.Callback.onAddMediaItems() function for this, and it works fine on its own, but feels like it might start to fall over when coupled with other use cases. For some added context in my application I'm making use of the MediaItem.RequestMetadata.uri to define the URI of a piece of media in our backend, and in onAddMediaItems I'm performing a lookup of our URI in order to fetch the actually playable stream URI as well as other metadata to populate.

The two related use cases I'm looking at where this starts to get messy is lazily building up a playlist of related items, and Android Audio/MediaLibrarySession support.

Firstly, queuing. My initial solution to this was to create a ForwardingPlayer which I wrapped around the MediaController on the app side, which would intercept calls to setMediaItem, set the item (at this point just a MediaItem with a URI in RequestMetadata), and asynchronously load a list of additional items to add around that first item (again, just request URIs).

This worked but had two shortcomings:

• It resulted in downcasting MediaController to a Player and removed access to certain other features like custom commands
• adding the rest of the items to the playlist seemed to need chunking as if you tried to add too many at once the operation just wouldn't happen (possibly due to bundle transfer size limits?). I've seen some other tickets about main thread issues when adding hundreds or thousands of items, we never experienced that, we just never saw the items get added at all.

Secondly, Android Auto also won't perform any queuing/playlist building on its own as described in #156, so obviously the app-side is the wrong place to put the playlist logic if it should be applied uniformly across all clients.

It seems like the correct approach would be to move the building of the playlist onto the service side. But it starts to fall down there as there doesn't appear to be a good mechanism for doing so which is consistent with the other systems. For instance, I could override MediaSession.Callback.onSetMediaItems() and build the whole playlist in the Future and it'll all just work, but this will create an unacceptably long delay between launching playback and the first track playing. Ideally it would be better to immediately add the first media item for playback and then asynchronously add the rest afterwards.

However to do this in onSetMediaItems() would require launching a new asynchronous process that outlives the returned Future which seems rather hacky. You would also then need to use the onAddMediaItems() callback function to add the rest of the queue items in order to take advantage of the local configuration URL and metadata population that occurs there. If you were to call addMediaItem on the Player instance it would completely bypass the MediaSession.Callback as that class is looking outwardly at the MediaController connection.

This approach using the MediaSession.Callback also assumes that the full playlist is known at that point in time. It doesn't really cater for cases like filling the playlist up perpetually with related content as the user continues listening.

Another option could be using a ConcatenatingMediaSource2 but it's a little unclear how to use it when you want everything to otherwise go through the DefaultMediaSourceFactory (e.g. how to create a factory for it which can be supplied to the player builder). It would also completely bypass the MediaSession.Callback and would require either duplicating or sharing the logic for populating MediaItems that need to be added to the playlist. It's also unclear whether asynchronous operations can be performed from within ConcatenatingMediaSource2. I assume one could inject a CoroutineScope tied to the lifecycle of the Service into it? Other than that it seems from the documentation and articles to be the most appropriate place to dynamically manipulate the playlist from within the service.

Is the concatenating media source the way to go for lazy playlist management on the MediaSession/Service side?

[sampengilly]

From reading the docs for both the old ConcatenatingMediaSource and the new ConcatenatingMediaSource2, it seems that maybe with media3 the approach for something like this might just be:

• Add a listener to the player
• Observe change to Timeline
• Perform background lookup for queue items
• call addMediaItems on Player

Though we'd need to make sure that the timeline change is due to the timeline being cleared and set rather than just being due to a new item being added (as it will be triggered again when addMediaItems is called). Not sure there is a way to determine that?

[marcbaechinger]

Thanks for your question and the detailed raisoning! I don't think there is an API that covers your use case 1:1 I'm afraid.

Is the concatenating media source the way to go for lazy playlist management on the MediaSession/Service side?

I don't think so. ConcatenatingMediaSource is deprecated in favor of the playlist API (addMediaItem/setMediaItem and friends). ConcatenatinMediaSource2 makes a single window out of all media items contained in it. From what you write above, this isn't what you intend.

cases like filling the playlist up perpetually with related content as the user continues listening

I think the approach you are describing with starting an async operation that outlives the Future is what I would consider.

Though we'd need to make sure that the timeline change is due to the timeline being cleared and set rather than just being due to a new item being added (as it will be triggered again when addMediaItems is called).

Just checking this is understood: Setting the first item A with a controller would trigger Callback.onAddMedidaItems to add A. That's where I would trigger the async task to fetch related items. When this tasks returns you'd call mediaSession.getPlayer.addMediaItems(), which doesn't trigger Callback.onAddMediaItems() again. Only methods called on a controller trigger the callback.

However, you are right that you still need to increment a sequence or something each time you start such a new task. This way you can drop a stale update that is coming in after the controller has called setMediaItem(item) again and incremented the sequence some further.

filling the playlist up perpetually

You can also listen to Listener.onMediaItemTransitioned to check whether playback has automatically advanced to a given mediaItemIndex or the user sought across a given mediaItemIndex. This could be a signal to continue loading the next N items once. Like fill 1 + 100 items and append the next 50 when the user arrived at mediaItemIndex >= mediaItemCount - 25.