WebVtt captions display for HLS Live stream with 10s segments, but not for stream with 3.8s segments

[jrw95]

The setup: Our customer has a live HLS stream, with 10-segment duration, with VTT subtitles. When playing this stream in the Androidx Media demo, the captions can be selected and displayed. The customer wants to use a shorter segment duration (3.84s), however, for the stream that plays these shorter durations in the demo, captions can be selected, and show in the tracks log output as selected, but do not display.

There are no errors written to the logcat.

I've also observed this behavior with ExoPlayer 2.19.1 (which our SDK is currently using; we are planning to migrate to AndroidX Media later this spring).

I've noticed with the customer's 10s segment feed, that the captions displayed appear to have some minor sync issues with the spoken dialog. (both 10s and 3.84s URLs will be shared through the dev email)

Questions:

1. Considering that there are no errors seen in the demo app's logcat, it does not appear that this is a bug, would you agree?
2. Considering that we're still using ExoPlayer with our SDK, are there any ExoPlayer configuration parameters that might help with this behavior?
3. Based on this behavior only occurring with the shorter segment duration, we're going to suggest to the customer to test with somewhat longer segment durations, is there a segment duration (less than 10s) that you may have seen elsewhere, where this behavior did not occur?

Thanks in advance for your assistance.

[icbaker]

Considering that there are no errors seen in the demo app's logcat, it does not appear that this is a bug, would you agree?

If you expect to see subtitles on screen and they don't show up, that seems like a bug to me...

---

Can you try parsing subtitles during extraction instead to see if it resolves the issue (this option was only added recently, so you'll need to try this in the media3 demo app): https://developer.android.com/reference/androidx/media3/exoplayer/source/DefaultMediaSourceFactory#experimentalParseSubtitlesDuringExtraction(boolean)

---

Hard to comment on the rest of your questions without understanding why the problem is happening I'm afraid.

[jrw95]

The only class that I could find that has that method is DashMediaSource.

[icbaker]

The method is also present on DefaultMediaSourceFactory in the latest release (as I linked above), or on the main branch.

[jrw95]

I added that call in the demo's PlayerActivity, line 318:

    return new DefaultMediaSourceFactory(/* context= */ this)
        .experimentalParseSubtitlesDuringExtraction(true)
        .setDataSourceFactory(dataSourceFactory)

and set a breakpoint in DefaultMediaSourceFactory, line 511, since this conditional wraps the conditional for parseSubtitlesDuringExtraction (which evaluates to true, even though it's not reached with either stream):

    List<MediaItem.SubtitleConfiguration> subtitleConfigurations =
        castNonNull(mediaItem.localConfiguration).subtitleConfigurations;
    if (!subtitleConfigurations.isEmpty()) {
      MediaSource[] mediaSources = new MediaSource[subtitleConfigurations.size() + 1];

I'm seeing that for both streams, 10s and 3.84s segments, that the subtitleConfigurations list is empty, however, the captions are displayed when selected in the Media3 demo for the 10s segment stream. For the 3.84s segment stream, they are not displayed when selected.

Looking at logcats for tests with each stream, I'm seeing that on the same device (Samsung S20, Android 12), there is an additional codec listed the text group for the 3.84s segment stream: For the 10s segment stream:

2024-03-08 11:18:47.327 21093-21093 EventLogger             androidx.media3.demo.main            D    group [
2024-03-08 11:18:47.327 21093-21093 EventLogger             androidx.media3.demo.main            D      [X] Track:0, id=subs_wvtt:ENG_80_888, mimeType=application/x-media3-cues, container=application/x-mpegURL, codecs=text/vtt, language=en, label=ENG_80_888, selectionFlags=[auto], supported=YES
2024-03-08 11:18:47.327 21093-21093 EventLogger             androidx.media3.demo.main            D    ]

For the 3.84s stream:

2024-03-08 11:24:58.584 21093-21093 EventLogger             androidx.media3.demo.main            D    group [
2024-03-08 11:24:58.584 21093-21093 EventLogger             androidx.media3.demo.main            D      [X] Track:0, id=subs_wvtt:ENG_80_888, mimeType=application/x-media3-cues, container=application/x-mpegURL, codecs=text/vtt wvtt, language=en, label=ENG_80_888, selectionFlags=[auto], supported=YES
2024-03-08 11:24:58.584 21093-21093 EventLogger             androidx.media3.demo.main            D    ]

[icbaker]

Which version of the library did you use to generate the logs/behaviour in https://github.com/androidx/media/issues/1147#issuecomment-1986189636?

[jrw95]

I was testing on the main branch. Just re-checked it to be sure, on both main and the latest release tag, and the same observations apply.

[jrw95]

@icbaker Can we transition this to a bug, or should I open a new ticket?

[jrw95]

@icbaker I've opened a bug for this, since I could not transition it directly (and the bug needed a bugreport as well). Please see #1457.