media showing caption causes exoplayer.ExoPlaybackException: Unexpected runtime error

Version

Media3 1.4.0

More version details

No response

Devices that reproduce the issue

Android 9, Android 14

Devices that do not reproduce the issue

none

Reproducible in the demo app?

Yes

Reproduction steps

run app: https://github.com/XilinJia/NewPipeX
play any youtube video (plays fine actually)
turn on caption,
video stops playing, Logcat error messages attached below.
when it happens, the app doesn't have a way so far to turn off caption., so, to repeat, the app has to be re-installed unfortunately.

Media3 1.3.1 works fine, BTW.

The error seems complaining about "Legacy decoding is disabled". I checked the release notes of 1.4,0, which has some text about legacy content (quoted below), so this exception is intended? If that's the case, please help on how to work around it. Unfortunately I can't find further documentation on that.

"Change default subtitle parsing behavior so it happens during extraction instead of during rendering (see ExoPlayer's architecture diagram for the difference between extraction and rendering). This change can be overridden by calling both MediaSource.Factory.experimentalParseSubtitlesDuringExtraction(false) and TextRenderer.experimentalSetLegacyDecodingEnabled(true). See the docs on customization for how to plumb these components into an ExoPlayer instance. These methods (and all support for legacy subtitle decoding) will be removed in a future release. Apps with custom SubtitleDecoder implementations need to update them to implement SubtitleParser instead (and SubtitleParser.Factory instead of SubtitleDecoderFactory)"

Expected result

should continue playing and show caption

Actual result

got an exception:

androidx.media3.exoplayer.ExoPlaybackException: Unexpected runtime error
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:720)
at android.os.Handler.dispatchMessage(Handler.java:108)
at android.os.Looper.loop(Looper.java:216)
at android.os.HandlerThread.run(HandlerThread.java:65)
Caused by: java.lang.IllegalStateException: Legacy decoding is disabled, can't handle application/ttml+xml samples (expected application/x-media3-cues).
at androidx.media3.common.util.Assertions.checkState(Assertions.java:100)
at androidx.media3.exoplayer.text.TextRenderer.assertLegacyDecodingEnabledIfRequired(TextRenderer.java:589)
at androidx.media3.exoplayer.text.TextRenderer.onStreamChanged(TextRenderer.java:212)
at androidx.media3.exoplayer.BaseRenderer.replaceStream(BaseRenderer.java:151)
at androidx.media3.exoplayer.BaseRenderer.enable(BaseRenderer.java:125)
at androidx.media3.exoplayer.ExoPlayerImplInternal.enableRenderer(ExoPlayerImplInternal.java:2770)
at androidx.media3.exoplayer.ExoPlayerImplInternal.enableRenderers(ExoPlayerImplInternal.java:2744)
at androidx.media3.exoplayer.ExoPlayerImplInternal.reselectTracksInternal(ExoPlayerImplInternal.java:1926)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:586)
at android.os.Handler.dispatchMessage(Handler.java:108) 
at android.os.Looper.loop(Looper.java:216) 
at android.os.HandlerThread.run(HandlerThread.java:65)

Media

any youtube video from within the NewPipeX app.

Bug Report

[ ] You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

Aug 22 '24 09:08 XilinJia

You pasted the documentation that describes the 2 necessary steps: "This change can be overridden by calling both

MediaSource.Factory.experimentalParseSubtitlesDuringExtraction(false) and
TextRenderer.experimentalSetLegacyDecodingEnabled(true)."

In your project, you have:

fun buildLiveMediaSource(dataSource: PlayerDataSource,
                         sourceUrl: String?,
                         type: @C.ContentType Int,
                         metadata: MediaItemTag?
): MediaSource? {
  val factory: MediaSource.Factory = when (type) {
    C.CONTENT_TYPE_SS -> dataSource.liveSsMediaSourceFactory
    C.CONTENT_TYPE_DASH -> dataSource.liveDashMediaSourceFactory
    C.CONTENT_TYPE_HLS -> dataSource.liveHlsMediaSourceFactory
    C.CONTENT_TYPE_OTHER, C.CONTENT_TYPE_RTSP -> throw ResolverException("Unsupported type: $type")
    else -> throw ResolverException("Unsupported type: $type")
  }
  return factory.createMediaSource(

So you know where you are creating your MediaSource.Factory

You also have

private fun initPlayer(playOnReady: Boolean) {
  Logd(TAG, "initPlayer() called with: playOnReady = [$playOnReady]")

  exoPlayer = ExoPlayer.Builder(context, renderFactory)
      .setTrackSelector(trackSelector)
      .setLoadControl(loadController)
      .setUsePlatformDiagnostics(false)
      .build()

Which means you have renderFactory which you can customise with the second call. If you were using DefaultMediaSourceFactory, you could do both like:

ExoPlayer.Builder(context)
        .setRenderersFactory(
            new DefaultRenderersFactory(context) {

              @Override
              protected void buildTextRenderers(
                  Context context,
                  TextOutput output,
                  Looper outputLooper,
                  @ExtensionRendererMode int extensionRendererMode,
                  ArrayList<Renderer> out) {
                super.buildTextRenderers(context, output, outputLooper, extensionRendererMode, out);
                ((TextRenderer) Iterables.getLast(out))
                    .experimentalSetLegacyDecodingEnabled(!parseSubtitlesDuringExtraction);
              }
            })
        .setMediaSourceFactory(
            new DefaultMediaSourceFactory(context)
                .experimentalParseSubtitlesDuringExtraction(parseSubtitlesDuringExtraction))
        .build();

Aug 22 '24 12:08 oceanjules

The reason your app is using "legacy subtitle decoding" is due to directly using SingleSampleMediaSource here: https://github.com/XilinJia/NewPipeX/blob/a1dfde029f5f86a59c019792e7b6901b763b8508/app/src/main/java/org/schabi/newpipe/player/PlayerManager.kt#L1771

I can't find any custom SubtitleDecoder implementations in your project, so I don't think you need to use legacy decoding for any reason.

You can either:

Enable support for legacy decoding, as @oceanjules has described above. This option will go away in a future version of media3.
Switch to using DefaultMediaSourceFactory instead of manually trying to create your MediaSource for each content type. This will enable the new form of subtitle handling automatically (assuming you don't opt out), by using ProgressiveMediaSource for subtitles internally instead of SingleSampleMediaSource.
Directly use ProgressiveMediaSource for subtitles instead of SingleSampleMediaSource in your app. You will need to set it up similar to how it's done inside DefaultMediaSourceFactory: https://github.com/androidx/media/blob/b01c6ffcb3fca3d038476dab5d3bc9c9f2010781/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactory.java#L514-L538

I would strongly recommend you look into option (2).