Is ScrubbingMode possible with ImageOuptut?

[StaehliJ]

I tried to use ExoPlayer.setScrubbingModelEnabled with an ImageOutput and a stream with tiled thumbnail inside the stream and when scrubbing with the slider, only the thumbnail of the position when scrubbing occurs is displayed and never updated after.

Is it intended to work only with video tracks ?

My compose slider view with the logics, in my real codes I only set the ImageOutput when the streams has an image track.

Slider(
    value = sliderValue,
    onValueChange = {
        if (!isScrubbing) {
                player.trackSelectionParameters = player.trackSelectionParameters.buildUpon()
                        .setPrioritizeImageOverVideoEnabled(true)
                        .build()
                player.setImageOutput(imageOutput)
                    player.isScrubbingModeEnabled = true
        }
        sliderValue = it
        player.seekTo((it * duration).toLong())
            },
    onValueChangeFinished = {
        isScrubbing = false
        player.setImageOutput(null)
        player.isScrubbingModeEnabled = false
        player.trackSelectionParameters = player.trackSelectionParameters.buildUpon()
                .setPrioritizeImageOverVideoEnabled(false)
                .build()
            }
)

[microkatz]

@StaehliJ

Thanks for reporting your issue!

Updating this thread with the debug analysis. The problem is linked to that ExoPlayer, in scrubbing mode, prevents incoming seeks from pre-empting in-progress seeks, allowing more seeks to complete when the scrubber bar is dragged quickly back and forth. ExoPlayer resets the seekIsPendingWhileScrubbing field when the onVideoFrameAboutToBeRendered event is received by its VideoFrameMetadataListener. Since the ImageRenderer does not invoke these events, ExoPlayer pre-empts all the seeks as the player scrubs and that's why the thumbnail never gets updated until scrubbing stops.

We will work on a solution to this issue and update this thread accordingly.

[microkatz]

@StaehliJ

A fix has been pushed to the main branch, https://github.com/androidx/media/commit/fe2f062b7f4b1899b9a9c772a8a01bfc70711a0e. This addresses the issue that the thumbnail never seemed to update while in scrubbing mode. However, updates will seem to only take effect when the user "stops scrubbing". The full fix will require skipping of intermittent seeks and will be added in a follow-up. You may implement a version in the meantime in your app layer to prevent calling seekTo until the pending operation has provided an onImageAvailable callback.

I'll update this thread when the full fix has been pushed.