androidx
TS stream pixelating when subtitles are parsed during extraction
Version
Media3 1.4.0
More version details
The first commit with this issue is 0352db9. We weren't able to reproduce with 5c6f48e.
Devices that reproduce the issue
- Changhong GIU3A00 (Android 9)
- Changhong GIU6770 (Android 9)
- SEI Robotics Box R Lite 4K (Android 11)
- Amlogic TwinKey2 (Android 7.1.1)
- Amino Amigo7x (Android 12)
Devices that do not reproduce the issue
- Pixel 3a (Android 12)
- Lenovo TB-X606F (Android 10)
Reproducible in the demo app?
Yes
Reproduction steps
Play the file sent via email in Media3 demo app (either from local storage or via UDP).
Parsing subtitles during rendering fixes the issue:
textRenderer.experimentalSetLegacyDecodingEnabled(true)
mediaSourceFactory.experimentalParseSubtitlesDuringExtraction(false)
Expected result
The media plays without pixelating even when subtitles are parsed during extraction
Actual result
The media is pixelating with the following error in the logcat every time there is a pixelation in the video
Audio sink error
androidx.media3.exoplayer.audio.AudioSink$UnexpectedDiscontinuityException: Unexpected audio track timestamp discontinuity: expected 1000029511400, got 1000029799400
at androidx.media3.exoplayer.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:997)
at androidx.media3.exoplayer.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:773)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.bypassRender(MediaCodecRenderer.java:2358)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:868)
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1136)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:561)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:193)
at android.os.HandlerThread.run(HandlerThread.java:65)
Media
Sent via email
Bug Report
- [ ] You will email the zip file produced by
adb bugreportto [email protected] after filing this issue.
Thank you for reporting. I'm afraid I'm not able to reproduce the issue on my Pixel 7a.
I see the subtitles in your media are application/dvbsubs which is a bitmap-based subtitle format. When subtitles are parsed during rendering we decode to a Bitmap object on a standalone thread. When they are parsed during extraction we originally decode to a Bitmap on the loading thread (which is OK to do heavy work on) and then serialize them to bytes in the SampleQueue, before parsing them back to a Bitmap on the playback thread in the renderer. I'm wondering if the issue you're seeing could be related to time spent decoding the Bitmap on the playback thread.
Do you have a more powerful device you could try on (maybe a phone rather than a TV/STB?), to confirm that you're also unable to repro in that case? This would give us some signal that this only occurs on low-powered devices.
In addition to the question above, could you also describe in more detail what exactly "pixelated" means in this case? That probably helps to understand where the problem is exactly.
- Is the video output of a lower resolution than expected?
- Does the video output have artefacts (for example green pixels)?
- Is something wrong with the timing of the video (e.g. dropped frames, delayed frames, wrong A/V sync)?
- Is audio playback working correctly (given your error log about audio discontinuities)?
Do you have a more powerful device you could try on (maybe a phone rather than a TV/STB?), to confirm that you're also unable to repro in that case? This would give us some signal that this only occurs on low-powered devices.
I've just tested on a Pixel 3a (Android 12) and a Lenovo TB-X606F (Android 10) and both devices played without issue. In both cases, I played the content from local storage.
In addition to the question above, could you also describe in more detail what exactly "pixelated" means in this case? That probably helps to understand where the problem is exactly.
Is the video output of a lower resolution than expected?
No
Does the video output have artefacts (for example green pixels)?
Yes, but not green pixels. Please see the attached video https://github.com/user-attachments/assets/aed91369-8509-45e3-8cab-95ae53bd5102
Is something wrong with the timing of the video (e.g. dropped frames, delayed frames, wrong A/V sync)?
There are often dropped frames following the audio sink errors, EventLogger also reports them. A/V sync seems to be correct when the video output has no artefacts.
Is audio playback working correctly (given your error log about audio discontinuities)?
Audio seems to be playing without glitches despite the audio sink errors