Choose MediaSource based on content-type header or sniffing data

[every-time]

If the client hasn't explicitly provided a MIME type, and ExoPlayer couldn't find any suitable extractor based on the source URI, then fall back to consider the content-type header from the response to the source.

i.e. I have a MediaItem who's URI looks like https://example.com/path_without_extension. I don't explicitly set the MIME type. Despite making a request to the source and the source responding with "content-type: application/vnd.apple.mpegurl", ExoPlayer still throws Caused by: androidx.media3.exoplayer.source.UnrecognizedInputFormatException: None of the available extractors (FlvExtractor, FlacExtractor, WavExtractor, FragmentedMp4Extractor, Mp4Extractor, AmrExtractor, PsExtractor, OggExtractor, TsExtractor, MatroskaExtractor, AdtsExtractor, Ac3Extractor, Ac4Extractor, Mp3Extractor, AviExtractor, JpegExtractor, PngExtractor, WebpExtractor, BmpExtractor, HeifExtractor, AvifExtractor) could read the stream. {contentIsMalformed=false, dataType=1}

[marcbaechinger]

Thanks for your suggestion.

We have had several such requests since ever. I know it's not obvious that this isn't straightforward to implement.

I think the main difficulty is that for ExoPlayer with its flexible design, data is loaded from a DataSource that just delivers bytes. There might be a network or even HTTP stream behind it, but assuming this is always the case is a bit of a simplification. There are various non-network DataSource implementation and a DataSource can be wrapped by another DataSource like for instance a ResolvingDataSource or a custom data source we even don't know. The decision which media source to use and create is taken earlier. When a media item is passed to the player in a playlist, a source is created without even loading data at all. While I think this felxible design is one of the strengths of ExoPlayer that our users appreciate, it has some drawback, like this problem being not easy to solve.

Our users have thought about this. #2573 is an example where a user appreciates that design and provides a proposal how that can be upfront. Their code shows the sniff method which detects the format for those streams that can't be recognized as HLS/Dash/Smooth and then are given to the progressive data source as a fallback.

We also thought about this. There are various approaches we have evaluated. We haven't come to the conclusion that the problem isn't as painful for users as that we would go with a solution that doesn't convince us. That's why we still have this situation.

Thanks either way for your report. We take this as a further data point.

[every-time]

Sorry for the simplification, but why couldn't DataSource be refactored such that it can also return a MIME type, or have access to the MediaItem and can set the MIME type, etc.? It wouldn't really be breaking separation of concerns because as we can see, sometimes only the DataSource has this information. I think it's a big miss to not treat content-type headers for HTTP sources as a default fall back option.

[marcbaechinger]

Given we need to decide on a MediaSource earlier, I don't think the DataSource knowing the content type is sufficient, because we would need to read data for each of the items in the playlist first. For users with large playlist it wouldn't be a good solution to start reading data before the media source is prepared.

Lemme assign to data source experts so they can comment if I'm wrong. Fine to just reassign back to me if there's nothing to add.

[every-time]

The MIME type doesn't really need to be known way in advance before data is read, right? It would open up use cases like this if the data could alter/set the MIME type, but I admittedly don't know how major of a lift to the codebase that would be.

[icbaker]

I agree with Marc, the problem is not lack of info from DataSource [1], it is the combination of the two constraints he described:

1. ExoPlayer needs to use a different MediaSource implementation for HLS vs DASH vs progressive media.
2. In ExoPlayer's architecture the MediaSource "owns" the DataSource.
   • Meaning we need to decide which MediaSource impl to use before we have a DataSource available (current behaviour).
   • Or you need a magic MediaSource which opens a DataSource, sniffs a bit, and then delegates to the right sub-MediaSource.

I think this is basically a duplicate of https://github.com/google/ExoPlayer/issues/3165. I will close that one as a duplicate of this (since this is on our newer issue tracker), and keep this issue open - but I'm afraid we have no plans to work on this.

---

[1] We already use MIME type from HTTP headers to decide which Extractor is likely to work for a given stream, but this logic takes place after we have decided to use ProgressiveMediaSource:

https://github.com/androidx/media/blob/414c260619f7104000148409e9554673bd89704f/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java#L441